feat(reports): one safe restart-retry in the cycle before alerting

If the post-restart DOD check shows the meter isn't measuring, retry once with start_recording (a plain start that does NOT re-index, unlike start_cycle) and re-verify before raising the schedule-failed alert. Retry fires only on a confident not-measuring reading — never on a failed/inconclusive DOD read — so a flaky read can't disrupt an already-running measurement or split the night across two store folders. Turns a transient restart hiccup into a self-heal instead of a meter left stopped overnight.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.gitignore

@@ -220,7 +220,6 @@ marimo/_static/
 marimo/_lsp/
 __marimo__/
 
-<<<<<<< HEAD
 # Seismo Fleet Manager
 # SQLite database files
 *.db
@@ -228,6 +227,3 @@ __marimo__/
 /data/
 /data-dev/
 .aider*
-.aider*
-=======
->>>>>>> 0c2186f5d89d948b0357d674c0773a67a67d8027

CHANGELOG.md

@@ -105,14 +105,6 @@ SLMM feed; every route resolves the client through one swappable
 
 ---
 
-### Portal authentication (Phase 1)
-- Each project's client portal is now gated by a **secure per-project link + shared password** (argon2-hashed). Operators manage it from the project page's **Portal access** panel (enable, generate password, copy link).
-- Per-project session isolation (a session for one project can't read another's data); brute-force lockout (5 tries / 15 min) on the password gate.
-- Retired the interim magic-link / `PORTAL_OPEN_LINKS` open links and the `portal_admin.py mint-link` command.
-- **Upgrade:** run `python3 backend/migrate_add_project_portal_auth.py` per DB. Set `COOKIE_SECURE=true` once served over HTTPS.
-
----
-
 ## [0.13.3] - 2026-06-05
 
 Calibration sync from SFM events.  Closes the manual data-entry loop on calibration dates — Terra-View now pulls `device.calibration_date` from each seismograph's most recent event sidecar once a day and updates `RosterUnit.last_calibrated` when the device reports something fresher than what's stored.  Manual edits still win when they're newer than the latest event; a fresh event arriving later supersedes the manual edit.  Adds a "Sync now" button under Settings → Advanced → Calibration Defaults for on-demand runs, and a `docs/ROADMAP.md` to track in-flight + deferred work.

REPORT_PIPELINE_BRIEF.md

@@ -0,0 +1,59 @@
+# FTP Report Pipeline — session brief
+
+**Branch:** `feat/ftp-report-pipeline` (off `dev`), worktree `/home/serversdown/terra-view-reports`.
+**Scope:** Terra-View only. Do NOT touch SLMM — the SLMM alert/monitor work is live in a
+parallel session on `slmm` branch `feat/drd-fix`. Pull device data through the **existing**
+SLMM FTP proxy endpoints; add no SLMM code (for v1).
+
+See memory note `client_sound_monitoring_job_2026-07` for the client requirements + timeline.
+
+## Goal
+Automated **daily morning report** for the John Myler 3-location sound job: each AM, last
+night's noise levels vs the **baseline week**, per location. Data pulled from the meters via
+FTP (the meter records 24/7 to SD regardless of TCP wedges). Alerts are a *separate* workstream
+(SLMM, real-time DOD) — not in scope here.
+
+## The big realization (why this is small)
+The hard parts already exist:
+- **SLMM (use as-is, via the `/api/slmm/...` proxy):**
+  - `GET  /api/slmm/{unit}/ftp/files?path=/NL-43` → list files/folders
+  - `POST /api/slmm/{unit}/ftp/download-folder` → returns the `Auto_####` folder as a **ZIP**
+- **Terra-View ingest (reuse):** `backend/routers/project_locations.py:1743` `upload_nrl_data`
+  already accepts a **ZIP**, extracts, keeps `.rnh` + `_Leq_ .rnd` (drops `_Lp_`/junk via
+  `_is_wanted`), runs `_parse_rnh` (line 1687) → creates `MonitoringSession` + `DataFile`.
+- **Report generator (reuse, source-agnostic):** `backend/routers/projects.py`. The `.rnd`
+  file reads funnel through 3 helpers — `_peek_rnd_headers` (~135), `_is_leq_file` (~147),
+  `_read_rnd_file_rows` (~256). `.rnd` files live on disk under `data/{file_path}` (DataFile
+  holds the path, not a BLOB). The stats/Excel/formatting logic doesn't care where bytes come from.
+
+## Build (Terra-View)
+1. **Refactor** `upload_nrl_data`'s core into a callable `ingest_nrl_zip(location_id, zip_bytes, db)`
+   so it can be invoked programmatically (not only via HTTP UploadFile).
+2. **Scheduled pull job** (reuse the existing scheduler): per project location/unit →
+   `GET /ftp/files` to find new `Auto_####` folders → `POST /ftp/download-folder` (zip) →
+   `ingest_nrl_zip(...)`. **Dedup** so repeated pulls don't duplicate sessions/files
+   (track ingested folder names per location).
+3. **Baseline aggregation:** aggregate the baseline-week `_Leq_` intervals per location →
+   reference values (nighttime Leq, L90 floor, typical Lmax).
+4. **Nightly report + email:** compute last night's metrics per location, compare to baseline
+   (deltas), render (reuse the Excel/report machinery), email each morning.
+
+## Data-location decision (light version, agreed)
+Keep `MonitoringSession`/`DataFile` **metadata in TV** for now; reuse the existing on-disk file
+store. Optional refinement (later): have SLMM keep the pulled files and TV read them through a
+SLMM file-serve endpoint (avoids the copy-into-TV step). Don't do that refinement under the
+deadline unless trivial — the report logic is identical either way.
+
+## Open questions to resolve early
+1. **What's actually in a `_Leq_ .rnd`** — Leq only, or Leq + Lmax + Ln per 15-min interval?
+   Decides whether the night-vs-baseline report can show L90/Lmax or just Leq. Inspect a real file.
+2. **Session rollover / dedup** — does a 2-week run write one growing `Auto_####` folder or new
+   folders? Drives the "what's new" logic.
+3. **`download-folder` over a multi-day run** — confirm it zips cleanly (size/time).
+
+## Client params (confirm with Dave before locking)
+Threshold/metric + their "night" window; report recipients + format (email body vs PDF/Excel).
+
+## Timeline
+Setup ~7/1–7/2 (baseline week), shutdown week through ~7/17. Reports needed by ~7/8 (before
+shutdown). Today is ~3 weeks out — reliability > features.

backend/auth_passwords.py

@@ -1,26 +0,0 @@
-"""Password hashing for the client portal — argon2id via argon2-cffi.
-
-Kept separate from portal_auth (cookie signing) so the future operator auth can
-reuse the same hasher. Never store or log raw passwords."""
-import secrets
-from argon2 import PasswordHasher
-
-_ph = PasswordHasher()
-
-
-def hash_password(raw: str) -> str:
-    """Return an argon2id hash string for a raw password."""
-    return _ph.hash(raw)
-
-
-def verify_password(raw: str, hashed: str) -> bool:
-    """True iff raw matches the stored hash. Never raises."""
-    try:
-        return _ph.verify(hashed, raw)
-    except Exception:  # argon2 raises on mismatch/garbage; treat all as "no match"
-        return False
-
-
-def generate_password(n_bytes: int = 12) -> str:
-    """A strong, URL-safe shareable password (~16 chars for n_bytes=12)."""
-    return secrets.token_urlsafe(n_bytes)

backend/main.py

@@ -69,7 +69,7 @@ from backend.templates_config import templates
 # Client-portal auth: an unauthenticated portal request renders the access page
 # (HTML routes) or returns 401 JSON (/portal/api/* routes). Centralized so every
 # portal route can simply Depends(get_current_client).
-from backend.portal_auth import PortalAuthError
+from backend.portal_auth import PortalAuthError, PORTAL_OPEN_LINKS
 
 @app.exception_handler(PortalAuthError)
 async def portal_auth_handler(request: Request, exc: PortalAuthError):
@@ -167,6 +167,10 @@ app.include_router(deployments.router)
 from backend.routers import calibration
 app.include_router(calibration.router)
 
+# Nightly sound-report pipeline (manual triggers; scheduled tick reuses run_nightly_report)
+from backend.routers import reports
+app.include_router(reports.router)
+
 # Start scheduler service and device status monitor on application startup
 from backend.services.scheduler import start_scheduler, stop_scheduler
 from backend.services.device_status_monitor import start_device_status_monitor, stop_device_status_monitor
@@ -410,79 +414,81 @@ async def project_detail_page(request: Request, project_id: str):
     return templates.TemplateResponse("projects/detail.html", {
         "request": request,
         "project_id": project_id,
+        "portal_open_links": PORTAL_OPEN_LINKS,
     })
 
 
 @app.get("/projects/{project_id}/portal-preview")
 async def project_portal_preview(project_id: str, db: Session = Depends(get_db)):
-    """Operator testing shortcut: open this project's client portal (no CLI)."""
+    """Operator testing shortcut: log into the client portal scoped to this project
+    (auto-provisioning a client/link if needed), no CLI. Lives under /projects (not
+    /portal), so a public proxy that exposes only /portal/* won't expose this."""
     from backend.models import Project
-    from backend.portal_auth import mint_portal_session, make_session_cookie, COOKIE_NAME, COOKIE_MAX_AGE, COOKIE_SECURE
+    from backend.portal_auth import (
+        provision_preview_session, make_session_cookie, COOKIE_NAME, COOKIE_MAX_AGE,
+    )
     project = db.query(Project).filter_by(id=project_id).first()
     if not project:
         return JSONResponse(status_code=404, content={"detail": "Project not found"})
-    token_id = mint_portal_session(project, db)
+    token_id = provision_preview_session(project, db)
     resp = RedirectResponse(url="/portal", status_code=303)
     resp.set_cookie(COOKIE_NAME, make_session_cookie(token_id),
-                    max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax", secure=COOKIE_SECURE)
+                    max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax")
     return resp
 
 
-@app.get("/projects/{project_id}/portal-access")
-async def project_portal_access_state(project_id: str, request: Request, db: Session = Depends(get_db)):
-    """Current portal-access state for the operator panel."""
+@app.post("/projects/{project_id}/portal-link")
+async def project_portal_link_create(project_id: str, request: Request, db: Session = Depends(get_db)):
+    """Mint a fresh shareable client link for this project's client. Returns the
+    full /portal/enter/<token> URL (shown once). Operator-only (internal app)."""
     from backend.models import Project
-    p = db.query(Project).filter_by(id=project_id).first()
-    if not p:
+    from backend.portal_auth import ensure_project_client, mint_link_token
+    project = db.query(Project).filter_by(id=project_id).first()
+    if not project:
         return JSONResponse(status_code=404, content={"detail": "Project not found"})
-    link_url = (str(request.base_url).rstrip("/") + f"/portal/p/{p.portal_link_token}") \
-        if (p.portal_enabled and p.portal_link_token) else None
-    return {"enabled": bool(p.portal_enabled), "has_password": bool(p.portal_password_hash),
-            "link_url": link_url}
+    client = ensure_project_client(project, db)
+    raw = mint_link_token(client, db, label="shared link")
+    url = str(request.base_url).rstrip("/") + f"/portal/enter/{raw}"
+    return {"url": url, "client_name": client.name}
 
 
-@app.post("/projects/{project_id}/portal-access/enable")
-async def project_portal_access_enable(project_id: str, request: Request, db: Session = Depends(get_db)):
-    """Turn the portal on; mint a link token if one doesn't exist yet."""
-    import secrets
-    from backend.models import Project
-    p = db.query(Project).filter_by(id=project_id).first()
-    if not p:
-        return JSONResponse(status_code=404, content={"detail": "Project not found"})
-    if not p.portal_link_token:
-        p.portal_link_token = secrets.token_urlsafe(24)
-    p.portal_enabled = True
+@app.get("/projects/{project_id}/portal-links")
+async def project_portal_links_list(project_id: str, db: Session = Depends(get_db)):
+    """List active (non-revoked) shareable links for this project's client."""
+    from backend.models import Project, ClientAccessToken, Client
+    project = db.query(Project).filter_by(id=project_id).first()
+    if not project or not project.client_id:
+        return {"client_name": None, "links": []}
+    client = db.query(Client).filter_by(id=project.client_id).first()
+    toks = (db.query(ClientAccessToken)
+            .filter_by(client_id=project.client_id, revoked_at=None)
+            .order_by(ClientAccessToken.created_at.desc()).all())
+    return {
+        "client_name": client.name if client else None,
+        "links": [{
+            "id": t.id, "label": t.label,
+            "created_at": t.created_at.isoformat() if t.created_at else None,
+            "last_used_at": t.last_used_at.isoformat() if t.last_used_at else None,
+        } for t in toks],
+    }
+
+
+@app.post("/projects/{project_id}/portal-link/{token_id}/revoke")
+async def project_portal_link_revoke(project_id: str, token_id: str, db: Session = Depends(get_db)):
+    """Revoke one shareable link (scoped to this project's client). Kills the link
+    and any live session minted from it on the next request."""
+    from datetime import datetime as _dt
+    from backend.models import Project, ClientAccessToken
+    project = db.query(Project).filter_by(id=project_id).first()
+    if not project or not project.client_id:
+        return JSONResponse(status_code=404, content={"detail": "Not found"})
+    tok = db.query(ClientAccessToken).filter_by(id=token_id, client_id=project.client_id).first()
+    if not tok:
+        return JSONResponse(status_code=404, content={"detail": "Link not found"})
+    if not tok.revoked_at:
+        tok.revoked_at = _dt.utcnow()
         db.commit()
-    link_url = str(request.base_url).rstrip("/") + f"/portal/p/{p.portal_link_token}"
-    return {"enabled": True, "has_password": bool(p.portal_password_hash), "link_url": link_url}
-
-
-@app.post("/projects/{project_id}/portal-access/password")
-async def project_portal_access_password(project_id: str, db: Session = Depends(get_db)):
-    """Generate a fresh strong password, store its hash, return the raw once."""
-    from backend.models import Project
-    from backend.auth_passwords import hash_password, generate_password
-    p = db.query(Project).filter_by(id=project_id).first()
-    if not p:
-        return JSONResponse(status_code=404, content={"detail": "Project not found"})
-    raw = generate_password()
-    p.portal_password_hash = hash_password(raw)
-    db.commit()
-    return {"password": raw}
-
-
-@app.post("/projects/{project_id}/portal-access/disable")
-async def project_portal_access_disable(project_id: str, db: Session = Depends(get_db)):
-    """Turn the portal off and rotate the link token (kills the old link)."""
-    import secrets
-    from backend.models import Project
-    p = db.query(Project).filter_by(id=project_id).first()
-    if not p:
-        return JSONResponse(status_code=404, content={"detail": "Project not found"})
-    p.portal_enabled = False
-    p.portal_link_token = secrets.token_urlsafe(24)  # rotate so the old link 404s
-    db.commit()
-    return {"enabled": False}
+    return {"ok": True}
 
 
 @app.get("/projects/{project_id}/nrl/{location_id}", response_class=HTMLResponse)

backend/migrate_add_project_portal_auth.py

@@ -1,61 +0,0 @@
-#!/usr/bin/env python3
-"""
-Database migration: Project portal auth (Phase 1).
-
-Adds the per-project portal gate columns to `projects`:
-  - portal_enabled        (BOOLEAN, default 0)
-  - portal_password_hash  (TEXT, nullable)
-  - portal_link_token     (TEXT, nullable)   [+ unique index]
-
-Idempotent. Run once per existing DB:
-    docker exec terra-view-terra-view-1 python3 backend/migrate_add_project_portal_auth.py
-"""
-import sqlite3
-from pathlib import Path
-
-_COLUMNS = {
-    "portal_enabled": "BOOLEAN DEFAULT 0",
-    "portal_password_hash": "TEXT",
-    "portal_link_token": "TEXT",
-}
-
-
-def migrate():
-    possible_paths = [Path("data/seismo_fleet.db"), Path("data/sfm.db"), Path("data/seismo.db")]
-    db_path = next((p for p in possible_paths if p.exists()), None)
-    if db_path is None:
-        print(f"Database not found in any of: {[str(p) for p in possible_paths]}")
-        print("A fresh DB created via models.py will include these columns automatically.")
-        return
-
-    print(f"Using database: {db_path}")
-    conn = sqlite3.connect(db_path)
-    cursor = conn.cursor()
-    cursor.execute("PRAGMA table_info(projects)")
-    existing = {row[1] for row in cursor.fetchall()}
-
-    for col, ddl in _COLUMNS.items():
-        if col in existing:
-            print(f"○ Column already exists: projects.{col}")
-            continue
-        try:
-            cursor.execute(f"ALTER TABLE projects ADD COLUMN {col} {ddl}")
-            print(f"✓ Added column: projects.{col} ({ddl})")
-        except sqlite3.OperationalError as e:
-            print(f"✗ Failed to add projects.{col}: {e}")
-
-    # Unique index on the link token (separate from ADD COLUMN; idempotent via IF NOT EXISTS).
-    try:
-        cursor.execute("CREATE UNIQUE INDEX IF NOT EXISTS ix_projects_portal_link_token "
-                       "ON projects (portal_link_token)")
-        print("✓ Ensured unique index: ix_projects_portal_link_token")
-    except sqlite3.OperationalError as e:
-        print(f"✗ Failed to create index: {e}")
-
-    conn.commit()
-    conn.close()
-    print("\n✓ Project portal-auth migration complete.")
-
-
-if __name__ == "__main__":
-    migrate()

backend/models.py

@@ -193,10 +193,6 @@ class Project(Base):
     # Project metadata
     client_name = Column(String, nullable=True, index=True)  # Client name (e.g., "PJ Dick")
     client_id = Column(String, nullable=True, index=True)    # FK -> clients.id; authoritative portal link (client_name kept for display)
-    # --- Client portal (Phase 1: per-project link + password gate) ---
-    portal_enabled = Column(Boolean, default=False)          # is the portal open for this project
-    portal_password_hash = Column(String, nullable=True)     # argon2 hash of the shared password
-    portal_link_token = Column(String, nullable=True, unique=True, index=True)  # unguessable token in the secure link
     site_address = Column(String, nullable=True)
     site_coordinates = Column(String, nullable=True)  # "lat,lon"
     start_date = Column(Date, nullable=True)
@@ -223,6 +219,35 @@ class ProjectModule(Base):
     __table_args__ = (UniqueConstraint("project_id", "module_type", name="uq_project_module"),)
 
 
+class SoundReportConfig(Base):
+    """
+    Per-project configuration for the automated nightly sound report
+    (FTP report pipeline). One row per project. Read by the morning tick in
+    SchedulerService and by the manual /reports endpoints (as defaults).
+
+    New table → created by Base.metadata.create_all() on startup; no migration
+    needed (only a rebuild/restart).
+    """
+    __tablename__ = "sound_report_configs"
+
+    id = Column(String, primary_key=True, default=lambda: __import__('uuid').uuid4().__str__())
+    project_id = Column(String, nullable=False, index=True, unique=True)  # FK to projects.id
+
+    enabled = Column(Boolean, default=False, nullable=False)        # run the daily report?
+    report_time = Column(String, default="08:00", nullable=False)   # local HH:MM to run/send
+    metric_keys = Column(String, default="lmax,l01,l10,l90", nullable=False)  # csv of metric keys
+    # Baseline source: "captured" = compute from recorded nights in the date range below;
+    # "reference" = use fixed values typed per location (old-report averages or a spec limit).
+    baseline_mode = Column(String, default="captured", nullable=False)
+    baseline_start = Column(Date, nullable=True)                    # captured-mode range
+    baseline_end = Column(Date, nullable=True)
+    recipients = Column(Text, nullable=True)                        # csv; falls back to REPORT_SMTP_RECIPIENTS env
+    last_run_date = Column(Date, nullable=True)                     # evening-date of the last reported night (dedup)
+
+    created_at = Column(DateTime, default=datetime.utcnow)
+    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
+
+
 class MonitoringLocation(Base):
     """
     Monitoring locations: generic location for monitoring activities.

backend/portal_admin.py

@@ -12,22 +12,22 @@ only its hash is stored.
   python3 backend/portal_admin.py link-project --slug myler --project-number 2567-23
   python3 backend/portal_admin.py link-project --slug myler --project-name "RKM Hall"
 
-  # mint-link is RETIRED — per-client magic URLs (/portal/enter) no longer exist.
-  #   Client access is now per-PROJECT + password: open the project's page in
-  #   Terra-View → "Portal access" to enable it, generate a password, and copy
-  #   the /portal/p/<token> link. (create-client / link-project / list / revoke
-  #   still operate on the underlying Client/token rows.)
+  # mint a magic access link (FULL URL PRINTED ONCE — copy it now)
+  python3 backend/portal_admin.py mint-link --slug myler [--label "Dave's link"]
 
   # list clients, their projects, and active links
   python3 backend/portal_admin.py list
 
   # revoke a link (stops the link AND any live session it minted)
   python3 backend/portal_admin.py revoke --token-id <TID>
+
+The printed URL base comes from PORTAL_BASE_URL (default http://localhost:8001).
 """
 
 import os
 import sys
 import uuid
+import secrets
 import argparse
 from datetime import datetime
 
@@ -37,6 +37,9 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 
 from backend.database import SessionLocal
 from backend.models import Client, ClientAccessToken, Project
+from backend.portal_auth import hash_token
+
+PORTAL_BASE_URL = os.getenv("PORTAL_BASE_URL", "http://localhost:8001").rstrip("/")
 
 
 def _get_client(db, slug):
@@ -84,15 +87,20 @@ def link_project(args):
 
 
 def mint_link(args):
-    # Retired: the per-client magic URL (/portal/enter/...) was removed when the
-    # portal moved to per-project + password access. Minting a token here would
-    # only produce a dead link.
-    sys.exit(
-        "mint-link is retired: per-client magic URLs (/portal/enter/...) no longer exist.\n"
-        "Client access is now per-project + password. In Terra-View, open the project's page →\n"
-        "'Portal access' to enable the portal, generate a password, and copy the /portal/p/<token>\n"
-        "link to send the client."
-    )
+    db = SessionLocal()
+    try:
+        c = _get_client(db, args.slug)
+        raw = secrets.token_urlsafe(32)
+        tok = ClientAccessToken(id=str(uuid.uuid4()), client_id=c.id,
+                                token_hash=hash_token(raw), label=args.label)
+        db.add(tok)
+        db.commit()
+        print(f"✓ Minted access link for '{c.name}'"
+              f"{f' ({args.label})' if args.label else ''} — token id {tok.id}")
+        print("\n  COPY THIS NOW (shown only once):\n")
+        print(f"      {PORTAL_BASE_URL}/portal/enter/{raw}\n")
+    finally:
+        db.close()
 
 
 def revoke(args):

backend/portal_auth.py

@@ -21,12 +21,13 @@ import base64
 import hashlib
 import logging
 import secrets
+from datetime import datetime
 
 from fastapi import Request, Depends
 from sqlalchemy.orm import Session
 
 from backend.database import get_db
-from backend.models import Client, ClientAccessToken, Project
+from backend.models import Client, ClientAccessToken
 
 logger = logging.getLogger(__name__)
 
@@ -38,9 +39,15 @@ if SECRET_KEY == "dev-insecure-change-me":
 
 COOKIE_NAME = "portal_session"
 COOKIE_MAX_AGE = 60 * 60 * 24 * 30  # 30 days
-# Set COOKIE_SECURE=true once the portal is served over HTTPS (TLS terminates at
-# the Synology reverse proxy). Default false so plain-HTTP dev still works.
-COOKIE_SECURE = os.getenv("COOKIE_SECURE", "false").lower() in ("1", "true", "yes")
+
+# Plain, no-token portal links (/portal/open/{project_id}). These are an
+# UNAUTHENTICATED, proxy-reachable session-minting path (and a linked project's
+# open link grants the *whole* client's scope), so they default OFF and must be
+# explicitly enabled — set PORTAL_OPEN_LINKS=true only in a dev/prototype env.
+PORTAL_OPEN_LINKS = os.getenv("PORTAL_OPEN_LINKS", "false").lower() in ("1", "true", "yes")
+if PORTAL_OPEN_LINKS:
+    logger.warning("[PORTAL] open links ENABLED — no-token /portal/open/* shareable links. "
+                   "Keep this OFF in any internet-facing / production deployment.")
 
 
 class PortalAuthError(Exception):
@@ -113,81 +120,64 @@ def get_current_client(request: Request, db: Session = Depends(get_db)) -> Clien
     return client
 
 
-# --- Phase-1 per-project password gate -------------------------------------------
-# A portal-enabled project gets its OWN dedicated client (slug "portal-<project.id>")
-# owning exactly that project. The project is linked to it via project.client_id so
-# the existing client-scoped routes (which resolve projects by Project.client_id ==
-# client.id) surface exactly this one project for the portal session — per-project
-# isolation with no route changes. (Phase 1 repurposes project.client_id for this; a
-# real per-client model is the deferred multi-tenant work.)
+def resolve_token(raw_token: str, db: Session):
+    """Validate a raw magic-URL token. Returns (ClientAccessToken, Client) on
+    success, or (None, None). Also stamps last_used_at."""
+    tok = db.query(ClientAccessToken).filter_by(
+        token_hash=hash_token(raw_token), revoked_at=None
+    ).first()
+    if not tok:
+        return None, None
+    client = db.query(Client).filter_by(id=tok.client_id, active=True).first()
+    if not client:
+        return None, None
+    tok.last_used_at = datetime.utcnow()
+    db.commit()
+    return tok, client
 
 
-def portal_client_for_project(project, db) -> Client:
-    """Get-or-create the dedicated 1:1 portal client for a project, and link the
-    project to it so the client-scoped routes resolve exactly this project."""
-    slug = f"portal-{project.id}"
+def ensure_project_client(project, db) -> Client:
+    """Find or create the Client for a project. Reuses the project's linked client
+    if it has one; otherwise creates/uses a per-project 'preview-<id>' client and
+    sets project.client_id (only when unset, so it never clobbers a real link)."""
+    client = None
+    if project.client_id:
+        client = db.query(Client).filter_by(id=project.client_id, active=True).first()
+    if client is None:
+        slug = f"preview-{project.id}"   # full id — an 8-char prefix can collide across projects
         client = db.query(Client).filter_by(slug=slug).first()
         if client is None:
             client = Client(id=str(uuid.uuid4()),
-                        name=(project.client_name or project.name or "Client"),
+                            name=(project.client_name or project.name or "Preview"),
                             slug=slug, active=True)
             db.add(client)
             db.flush()
-    if project.client_id != client.id:
-        project.client_id = client.id   # without this, the client owns no projects
-        db.flush()
+        if not project.client_id:
+            project.client_id = client.id
     return client
 
 
-def mint_portal_session(project, db) -> str:
-    """Ensure the project's portal client + an access token exist; return the token
-    id to seal into a session cookie. Reuses an existing token to avoid clutter."""
-    client = portal_client_for_project(project, db)
+def mint_link_token(client, db, label=None) -> str:
+    """Mint a fresh access token for a client and return the RAW secret (caller
+    builds the /portal/enter/<raw> URL and shows it once). Only the hash is stored."""
+    raw = secrets.token_urlsafe(32)
+    db.add(ClientAccessToken(id=str(uuid.uuid4()), client_id=client.id,
+                             token_hash=hash_token(raw), label=label))
+    db.commit()
+    return raw
+
+
+def provision_preview_session(project, db) -> str:
+    """Operator preview shortcut: ensure a Client + access token exist for a project
+    and return a token id to seal into a session cookie (no shared link). Reuses an
+    existing token so repeat previews don't accumulate clutter; the raw secret is
+    discarded (preview rides the cookie)."""
+    client = ensure_project_client(project, db)
     tok = db.query(ClientAccessToken).filter_by(client_id=client.id, revoked_at=None).first()
     if tok is None:
         tok = ClientAccessToken(id=str(uuid.uuid4()), client_id=client.id,
                                 token_hash=hash_token(secrets.token_urlsafe(32)),
-                                label="portal")
+                                label="preview")
         db.add(tok)
     db.commit()
     return tok.id
-
-
-def resolve_project_by_link_token(link_token: str, db):
-    """Return the portal-enabled Project for a link token, or None."""
-    if not link_token:
-        return None
-    return db.query(Project).filter_by(
-        portal_link_token=link_token, portal_enabled=True).first()
-
-
-# In-memory brute-force lockout, keyed per link_token (the password is shared per
-# project, so per-IP granularity buys nothing and an IP term only lets an attacker
-# reset the budget by rotating source IPs). Resets on restart; adequate for a
-# read-only surface behind the UniFi edge. Single-worker dev; multi-worker would
-# need a shared store.
-MAX_ATTEMPTS = 5
-LOCK_SECONDS = 15 * 60
-_failures: dict = {}  # key -> (count, first_failure_epoch)
-
-
-def is_locked(key: str) -> bool:
-    rec = _failures.get(key)
-    if not rec:
-        return False
-    count, first = rec
-    if count < MAX_ATTEMPTS:
-        return False
-    if (time.time() - first) > LOCK_SECONDS:
-        _failures.pop(key, None)  # window expired
-        return False
-    return True
-
-
-def register_failure(key: str) -> None:
-    count, first = _failures.get(key, (0, time.time()))
-    _failures[key] = (count + 1, first)
-
-
-def clear_failures(key: str) -> None:
-    _failures.pop(key, None)

backend/routers/portal.py

@@ -1,10 +1,9 @@
 """
 Client portal — read-only, scoped client view (see docs/CLIENT_PORTAL.md).
 
-A client opens a per-project secure link (/portal/p/{link_token}), enters the
-shared password, and gets a signed session cookie scoped to that project; they
-then see that project's locations (overview) and per-location read-only live
-data sourced from SLMM's cache. Every data route re-checks ownership.
+M1: a client opens a magic URL (/portal/enter/{token}) which mints a signed
+session cookie, then sees their locations (overview) and per-location read-only
+live data sourced from SLMM's cache. Every data route re-checks ownership.
 """
 
 import os
@@ -15,7 +14,7 @@ from datetime import datetime
 
 import httpx
 import websockets
-from fastapi import APIRouter, Request, Depends, HTTPException, WebSocket, Form
+from fastapi import APIRouter, Request, Depends, HTTPException, WebSocket
 from fastapi.responses import RedirectResponse
 from sqlalchemy import or_
 from sqlalchemy.orm import Session
@@ -24,12 +23,10 @@ from backend.database import get_db, SessionLocal
 from backend.models import Client, MonitoringLocation, Project, UnitAssignment
 from backend.templates_config import templates
 from backend.portal_auth import (
-    get_current_client, client_from_cookie, make_session_cookie,
-    COOKIE_NAME, COOKIE_MAX_AGE, COOKIE_SECURE,
-    resolve_project_by_link_token, mint_portal_session,
-    is_locked, register_failure, clear_failures,
+    get_current_client, client_from_cookie, make_session_cookie, resolve_token,
+    provision_preview_session, PORTAL_OPEN_LINKS,
+    COOKIE_NAME, COOKIE_MAX_AGE,
 )
-from backend.auth_passwords import verify_password
 
 logger = logging.getLogger(__name__)
 router = APIRouter(prefix="/portal", tags=["portal"])
@@ -94,6 +91,46 @@ def _client_locations(client: Client, db: Session) -> list:
     } for loc in locs]
 
 
+@router.get("/enter/{token}")
+def portal_enter(token: str, request: Request, db: Session = Depends(get_db)):
+    """Magic-URL entry: validate the token, mint a session cookie, land on /portal."""
+    tok, client = resolve_token(token, db)
+    if not client:
+        return templates.TemplateResponse(
+            "portal/access_required.html",
+            {"request": request, "reason": "invalid"},
+            status_code=403,
+        )
+    resp = RedirectResponse(url="/portal", status_code=303)
+    resp.set_cookie(
+        COOKIE_NAME, make_session_cookie(tok.id),
+        max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax",
+    )
+    logger.info(f"[PORTAL] {client.slug}: session opened via token {tok.id[:8]}")
+    return resp
+
+
+@router.get("/open/{project_id}")
+def portal_open(project_id: str, request: Request, db: Session = Depends(get_db)):
+    """Dev-only plain shareable link: open a project's client portal with no token
+    (gated by PORTAL_OPEN_LINKS). Lets anyone with the URL view it for feedback —
+    sets the session cookie and lands on /portal. Lives under /portal so it works
+    through a reverse proxy that exposes only /portal/*."""
+    if not PORTAL_OPEN_LINKS:
+        return templates.TemplateResponse(
+            "portal/access_required.html", {"request": request, "reason": "required"},
+            status_code=404)
+    project = db.query(Project).filter_by(id=project_id).first()
+    if not project:
+        return templates.TemplateResponse(
+            "portal/access_required.html", {"request": request, "reason": "invalid"},
+            status_code=404)
+    token_id = provision_preview_session(project, db)
+    resp = RedirectResponse(url="/portal", status_code=303)
+    resp.set_cookie(COOKIE_NAME, make_session_cookie(token_id),
+                    max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax")
+    return resp
+
 
 @router.get("/logout")
 def portal_logout():
@@ -110,60 +147,6 @@ def portal_access(request: Request):
     )
 
 
-@router.get("/p/{link_token}")
-def portal_password_prompt(link_token: str, request: Request, db: Session = Depends(get_db)):
-    """Secure per-project link: resolve the project from the token, prompt for the
-    shared password. Generic page if the token is unknown/disabled (no leak)."""
-    project = resolve_project_by_link_token(link_token, db)
-    if not project or not project.portal_password_hash:
-        # unknown token, disabled portal, or enabled-but-no-password-set — all look
-        # identical to a client (no existence/config leak, no self-lockout on a
-        # passwordless project).
-        return templates.TemplateResponse(
-            "portal/access_required.html", {"request": request, "reason": "invalid"},
-            status_code=404)
-    return templates.TemplateResponse("portal/password.html", {
-        "request": request, "link_token": link_token,
-        "project_name": project.name, "error": None})
-
-
-@router.post("/p/{link_token}")
-def portal_password_submit(link_token: str, request: Request,
-                           password: str = Form(...), db: Session = Depends(get_db)):
-    """Verify the shared password; on success mint a project-scoped session cookie."""
-    project = resolve_project_by_link_token(link_token, db)
-    if not project or not project.portal_password_hash:
-        # unknown token, disabled portal, or enabled-but-no-password-set — all look
-        # identical to a client (no existence/config leak, no self-lockout on a
-        # passwordless project).
-        return templates.TemplateResponse(
-            "portal/access_required.html", {"request": request, "reason": "invalid"},
-            status_code=404)
-
-    # Shared per-project password → lock per token. (Keying on IP too only enabled a
-    # bypass via source-IP rotation, and behind the reverse proxy every client shares
-    # one IP anyway.)
-    lock_key = link_token
-    if is_locked(lock_key):
-        return templates.TemplateResponse("portal/password.html", {
-            "request": request, "link_token": link_token, "project_name": project.name,
-            "error": "Too many attempts. Try again in 15 minutes."}, status_code=200)
-
-    if not verify_password(password, project.portal_password_hash):
-        register_failure(lock_key)
-        return templates.TemplateResponse("portal/password.html", {
-            "request": request, "link_token": link_token, "project_name": project.name,
-            "error": "Incorrect password."}, status_code=200)
-
-    clear_failures(lock_key)
-    token_id = mint_portal_session(project, db)
-    resp = RedirectResponse(url="/portal", status_code=303)
-    resp.set_cookie(COOKIE_NAME, make_session_cookie(token_id),
-                    max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax", secure=COOKIE_SECURE)
-    logger.info(f"[PORTAL] password ok for project {project.id[:8]} → session opened")
-    return resp
-
-
 @router.get("")
 def portal_home(request: Request, client: Client = Depends(get_current_client),
                 db: Session = Depends(get_db)):

backend/routers/project_locations.py

@@ -9,7 +9,7 @@ from fastapi import APIRouter, Request, Depends, HTTPException, Query
 from fastapi.responses import HTMLResponse, JSONResponse
 from sqlalchemy.orm import Session
 from sqlalchemy import and_, or_
-from datetime import datetime
+from datetime import datetime, timedelta
 from zoneinfo import ZoneInfo
 from typing import Optional
 import uuid
@@ -1712,6 +1712,19 @@ def _parse_rnh(content: bytes) -> dict:
                     result["stop_time_str"] = value
                 elif key == "Total Measurement Time":
                     result["total_time_str"] = value
+                elif key == "Frequency Weighting (Main)":
+                    result["frequency_weighting"] = value
+                elif key == "Time Weighting (Main)":
+                    result["time_weighting"] = value
+                elif key == "Leq Calculation Interval":
+                    result["leq_interval"] = value
+                elif key.startswith("Percentile "):
+                    # e.g. "Percentile 4,90.0" → percentiles["4"] = "90.0".
+                    # Lets the report label the LN slots (here LN4 = L90) from the
+                    # device's own config instead of hardcoding which slot is which —
+                    # the percentile assignment is reconfigurable per job.
+                    slot = key[len("Percentile "):].strip()
+                    result.setdefault("percentiles", {})[slot] = value
     except Exception:
         pass
     return result
@@ -1740,6 +1753,347 @@ def _classify_file(filename: str) -> str:
     return "data"
 
 
+def _rnd_interval_seconds(s: Optional[str]) -> Optional[int]:
+    """Parse an NL-43 interval string ('15m' / '1s' / '1h') into seconds."""
+    import re
+    m = re.match(r"\s*(\d+)\s*([smh])", (s or "").strip().lower())
+    if not m:
+        return None
+    return int(m.group(1)) * {"s": 1, "m": 60, "h": 3600}[m.group(2)]
+
+
+def _leq_window_local(leq_bytes: bytes):
+    """Recording window from a Leq .rnd's 'Start Time' column (meter-local time).
+
+    Returns (first_start, last_start, row_count, inferred_interval_seconds).
+    This is the source of truth for the recording window on NL-43 units, whose
+    .rnh carries no measurement timestamps.  Reuses the report's AU2 normaliser
+    so NL-43 and AU2 files parse identically.
+    """
+    import csv as _csv
+    from backend.routers.projects import _normalize_rnd_rows  # lazy: avoid import cycle
+    try:
+        text = leq_bytes.decode("utf-8", errors="replace")
+        rows = list(_csv.DictReader(io.StringIO(text)))
+    except Exception:
+        return None, None, 0, None
+    try:
+        rows, _ = _normalize_rnd_rows(rows)
+    except Exception:
+        pass
+    times = []
+    for r in rows:
+        v = (r.get("Start Time") or "").strip()
+        try:
+            times.append(datetime.strptime(v, "%Y/%m/%d %H:%M:%S"))
+        except (ValueError, TypeError):
+            continue
+    if not times:
+        return None, None, 0, None
+    times.sort()
+    inferred = int((times[1] - times[0]).total_seconds()) if len(times) >= 2 else None
+    return times[0], times[-1], len(times), inferred
+
+
+def _is_wanted_nrl_file(fname: str) -> bool:
+    """Keep only the files an NRL ingest cares about: .rnh metadata + the
+    averaged Leq .rnd.  Drops the 1-second _Lp_ files and everything else.
+
+      - NL-43 writes two .rnd types: _Leq_ (15-min averages, wanted) and
+        _Lp_ (1-second granular, skipped).
+      - AU2 (NL-23/older Rion) writes a single Au2_####.rnd — always keep.
+
+    Note this is purely about which *files* to store, not which *metrics* to
+    report: the kept Leq file carries every column (Leq, Lmax, L1/L10/L50/
+    L90/L95, Lpeak, …), so the report layer can select any metric later.
+    """
+    n = fname.lower()
+    if n.endswith(".rnh"):
+        return True
+    if n.endswith(".rnd"):
+        if "_leq_" in n:          # NL-43 Leq file
+            return True
+        if n.startswith("au2_"):  # AU2 format (NL-23) — Leq equivalent
+            return True
+        if "_lp" not in n and "_leq_" not in n:
+            # Unknown .rnd format — include it so we don't silently drop data
+            return True
+    return False
+
+
+class IngestError(Exception):
+    """Raised when an NRL upload/ZIP has no usable data or an invalid target.
+
+    Kept HTTP-agnostic so the ingest core can be driven programmatically (the
+    scheduled FTP pull) as well as from the HTTP upload endpoint.  Callers
+    translate it: the endpoint → HTTP 400, the scheduler → logged failure.
+    """
+    pass
+
+
+def _find_existing_session(
+    db: Session,
+    location_id: str,
+    store_name: str,
+    started_at,
+    start_time_str: str,
+):
+    """Return an already-ingested session for this location that represents the
+    same measurement, or None.
+
+    Used to make FTP re-pulls idempotent: a daily cycle closes one Auto_####
+    folder per day, so a session is uniquely identified within a location by
+    (store_name + measurement start time).  Store names recycle across jobs, so
+    we always match on start time too.
+    """
+    if not store_name and not started_at:
+        return None
+    candidates = db.query(MonitoringSession).filter(
+        MonitoringSession.location_id == location_id,
+        MonitoringSession.session_type == "sound",
+    ).all()
+    for s in candidates:
+        try:
+            meta = json.loads(s.session_metadata or "{}")
+        except (json.JSONDecodeError, TypeError):
+            meta = {}
+        if store_name and meta.get("store_name") != store_name:
+            continue
+        # Same store_name — confirm it's the same measurement by start time.
+        if start_time_str and meta.get("start_time_str") == start_time_str:
+            return s
+        if not meta.get("start_time_str") and started_at and s.started_at == started_at:
+            return s
+    return None
+
+
+def _ingest_file_entries(
+    location: MonitoringLocation,
+    file_entries: list[tuple[str, bytes]],
+    db: Session,
+    *,
+    source: str = "manual_upload",
+    dedupe: bool = False,
+    unit_id: Optional[str] = None,
+) -> dict:
+    """Core NRL ingest, shared by the HTTP upload and the programmatic FTP pull.
+
+    Takes already-normalized (filename, bytes) entries, keeps the wanted files,
+    parses the .rnh, and creates a MonitoringSession + DataFile rows under the
+    location's project.  Metric-agnostic: the full Leq file is written to disk
+    and every column preserved; metric selection happens in the report layer.
+
+    `unit_id` attributes the session to the recording unit when the caller knows
+    it (manual FTP download / SD upload from a known unit).  Left None for paths
+    that link the unit afterwards (the scheduler's `_ingest_and_link`).
+
+    Raises IngestError if no usable files are present.
+    """
+    # --- Filter to the files we keep (.rnh + Leq .rnd) ---
+    file_entries = [(f, b) for f, b in file_entries if _is_wanted_nrl_file(f)]
+    if not file_entries:
+        raise IngestError(
+            "No usable .rnd or .rnh files found. Expected NL-43 _Leq_ files or AU2 format .rnd files."
+        )
+
+    # --- Parse .rnh metadata (first one wins) ---
+    rnh_meta = {}
+    for fname, fbytes in file_entries:
+        if fname.lower().endswith(".rnh"):
+            rnh_meta = _parse_rnh(fbytes)
+            break
+
+    # RNH stores local time (no UTC offset).  Use local for period/label, then
+    # convert to UTC for storage so the local_datetime filter displays correctly.
+    started_at_local = _parse_rnh_datetime(rnh_meta.get("start_time_str")) or datetime.utcnow()
+    stopped_at_local = _parse_rnh_datetime(rnh_meta.get("stop_time_str"))
+    started_at = local_to_utc(started_at_local)
+    stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
+    duration_seconds = (
+        int((stopped_at - started_at).total_seconds())
+        if (started_at and stopped_at) else None
+    )
+
+    store_name = rnh_meta.get("store_name", "")
+    serial_number = rnh_meta.get("serial_number", "")
+    index_number = rnh_meta.get("index_number", "")
+    start_time_str = rnh_meta.get("start_time_str", "")
+
+    # The NL-43 .rnh has NO measurement timestamps — the real recording window
+    # lives in the Leq .rnd's "Start Time" column.  Whenever the header didn't
+    # give us a start (and/or stop), derive it from the Leq rows so the session
+    # gets the true window + duration (and a stable start_time_str for dedupe).
+    if not start_time_str or stopped_at_local is None:
+        leq_entry = next(
+            ((f, b) for f, b in file_entries
+             if f.lower().endswith(".rnd") and ("_leq_" in f.lower() or f.lower().startswith("au2_"))),
+            None,
+        )
+        if leq_entry is not None:
+            first_dt, last_dt, _n, inferred = _leq_window_local(leq_entry[1])
+            interval_s = _rnd_interval_seconds(rnh_meta.get("leq_interval")) or inferred or 0
+            if first_dt and not start_time_str:
+                started_at_local = first_dt
+                start_time_str = first_dt.strftime("%Y/%m/%d %H:%M:%S")
+            if last_dt and stopped_at_local is None:
+                stopped_at_local = last_dt + timedelta(seconds=interval_s)
+            # Recompute UTC + duration from the resolved window.
+            started_at = local_to_utc(started_at_local)
+            stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
+            duration_seconds = (
+                int((stopped_at - started_at).total_seconds())
+                if (started_at and stopped_at) else duration_seconds
+            )
+
+    # --- Dedupe: skip if this exact measurement is already ingested ---
+    if dedupe:
+        existing = _find_existing_session(db, location.id, store_name, started_at, start_time_str)
+        if existing:
+            return {
+                "success": True,
+                "deduped": True,
+                "session_id": existing.id,
+                "files_imported": 0,
+                "leq_files": 0,
+                "lp_files": 0,
+                "metadata_files": 0,
+                "store_name": store_name,
+                "started_at": started_at.isoformat() if started_at else None,
+                "stopped_at": stopped_at.isoformat() if stopped_at else None,
+                "duration_seconds": duration_seconds,
+            }
+
+    # --- Create MonitoringSession (local times drive period/label) ---
+    period_type = _derive_period_type(started_at_local) if started_at_local else None
+    session_label = (
+        _build_session_label(started_at_local, location.name, period_type)
+        if started_at_local else None
+    )
+
+    session_id = str(uuid.uuid4())
+    monitoring_session = MonitoringSession(
+        id=session_id,
+        project_id=location.project_id,
+        location_id=location.id,
+        unit_id=unit_id,
+        session_type="sound",
+        started_at=started_at,
+        stopped_at=stopped_at,
+        duration_seconds=duration_seconds,
+        status="completed",
+        session_label=session_label,
+        period_type=period_type,
+        session_metadata=json.dumps({
+            "source": source,
+            "store_name": store_name,
+            "serial_number": serial_number,
+            "index_number": index_number,
+            "start_time_str": start_time_str,
+            # Captured from the .rnh so the report can label metrics from the
+            # device's own config (which LN slot is L90, the weightings, etc.).
+            "percentiles": rnh_meta.get("percentiles", {}),
+            "frequency_weighting": rnh_meta.get("frequency_weighting", ""),
+            "time_weighting": rnh_meta.get("time_weighting", ""),
+            "leq_interval": rnh_meta.get("leq_interval", ""),
+        }),
+    )
+    db.add(monitoring_session)
+    db.commit()
+    db.refresh(monitoring_session)
+
+    # --- Write files to disk + create DataFile records ---
+    output_dir = Path("data/Projects") / location.project_id / session_id
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    leq_count = lp_count = metadata_count = files_imported = 0
+    for fname, fbytes in file_entries:
+        fname_lower = fname.lower()
+        if fname_lower.endswith(".rnd"):
+            if "_leq_" in fname_lower:
+                leq_count += 1
+            elif "_lp" in fname_lower:
+                lp_count += 1
+        elif fname_lower.endswith(".rnh"):
+            metadata_count += 1
+
+        dest = output_dir / fname
+        dest.write_bytes(fbytes)
+        checksum = hashlib.sha256(fbytes).hexdigest()
+        rel_path = str(dest.relative_to("data"))
+
+        db.add(DataFile(
+            id=str(uuid.uuid4()),
+            session_id=session_id,
+            file_path=rel_path,
+            file_type=_classify_file(fname),
+            file_size_bytes=len(fbytes),
+            downloaded_at=datetime.utcnow(),
+            checksum=checksum,
+            file_metadata=json.dumps({
+                "source": source,
+                "original_filename": fname,
+                "store_name": store_name,
+            }),
+        ))
+        files_imported += 1
+
+    db.commit()
+
+    return {
+        "success": True,
+        "deduped": False,
+        "session_id": session_id,
+        "files_imported": files_imported,
+        "leq_files": leq_count,
+        "lp_files": lp_count,
+        "metadata_files": metadata_count,
+        "store_name": store_name,
+        "started_at": started_at.isoformat() if started_at else None,
+        "stopped_at": stopped_at.isoformat() if stopped_at else None,
+        "duration_seconds": duration_seconds,
+    }
+
+
+def ingest_nrl_zip(
+    location_id: str,
+    zip_bytes: bytes,
+    db: Session,
+    *,
+    source: str = "ftp_pull",
+    dedupe: bool = True,
+    unit_id: Optional[str] = None,
+) -> dict:
+    """Programmatically ingest an Auto_#### ZIP (e.g. a scheduled FTP pull).
+
+    Extracts the ZIP (flattening any nested Auto_Leq/Auto_Lp_ folders), keeps
+    the .rnh + Leq .rnd, parses the header, and creates a MonitoringSession +
+    DataFile rows for `location_id`.  Defaults to dedupe=True so repeated daily
+    pulls of the same closed folder don't create duplicate sessions.  Pass
+    `unit_id` to attribute the session to the recording unit at creation.
+
+    Returns the same dict shape as the HTTP upload, plus a `deduped` flag.
+    Raises IngestError on a bad ZIP, no usable files, or unknown location.
+    """
+    location = db.query(MonitoringLocation).filter_by(id=location_id).first()
+    if not location:
+        raise IngestError(f"Location {location_id} not found")
+
+    try:
+        with zipfile.ZipFile(io.BytesIO(zip_bytes)) as zf:
+            file_entries: list[tuple[str, bytes]] = []
+            for info in zf.infolist():
+                if info.is_dir():
+                    continue
+                name = Path(info.filename).name  # strip nested folder paths
+                if not name:
+                    continue
+                file_entries.append((name, zf.read(info)))
+    except zipfile.BadZipFile:
+        raise IngestError("Downloaded data is not a valid ZIP archive.")
+
+    return _ingest_file_entries(location, file_entries, db, source=source, dedupe=dedupe, unit_id=unit_id)
+
+
 @router.post("/nrl/{location_id}/upload-data")
 async def upload_nrl_data(
     project_id: str,
@@ -1754,11 +2108,13 @@ async def upload_nrl_data(
     - A single .zip file (the Auto_#### folder zipped) — auto-extracted
     - Multiple .rnd / .rnh files selected directly from the SD card folder
 
-    Creates a MonitoringSession from .rnh metadata and DataFile records
-    for each measurement file. No unit assignment required.
+    Normalizes the upload to (filename, bytes) entries, then hands off to the
+    shared ingest core (`_ingest_file_entries`) — the same path the scheduled
+    FTP pull uses via `ingest_nrl_zip`.  Creates a MonitoringSession from the
+    .rnh metadata and DataFile records for each measurement file.  No unit
+    assignment required.  dedupe=False here preserves the prior manual-upload
+    behaviour (re-uploading creates a fresh session).
     """
-    from datetime import datetime
-
     # Verify project and location exist
     project = db.query(Project).filter_by(id=project_id).first()
     _require_module(project, "sound_monitoring", db)
@@ -1769,7 +2125,7 @@ async def upload_nrl_data(
     if not location:
         raise HTTPException(status_code=404, detail="Location not found")
 
-    # --- Step 1: Normalize to (filename, bytes) list ---
+    # --- Normalize upload to (filename, bytes) entries ---
     file_entries: list[tuple[str, bytes]] = []
 
     if len(files) == 1 and files[0].filename.lower().endswith(".zip"):
@@ -1793,145 +2149,11 @@ async def upload_nrl_data(
     if not file_entries:
         raise HTTPException(status_code=400, detail="No usable files found in upload.")
 
-    # --- Step 1b: Filter to only relevant files ---
-    # Keep: .rnh (metadata) and measurement .rnd files
-    #   NL-43 generates two .rnd types: _Leq_ (15-min averages, wanted) and _Lp_ (1-sec granular, skip)
-    #   AU2 (NL-23/older Rion) generates a single Au2_####.rnd per session — always keep those
-    # Drop: _Lp_ .rnd, .xlsx, .mp3, and anything else
-    def _is_wanted(fname: str) -> bool:
-        n = fname.lower()
-        if n.endswith(".rnh"):
-            return True
-        if n.endswith(".rnd"):
-            if "_leq_" in n:        # NL-43 Leq file
-                return True
-            if n.startswith("au2_"): # AU2 format (NL-23) — always Leq equivalent
-                return True
-            if "_lp" not in n and "_leq_" not in n:
-                # Unknown .rnd format — include it so we don't silently drop data
-                return True
-        return False
-
-    file_entries = [(fname, fbytes) for fname, fbytes in file_entries if _is_wanted(fname)]
-
-    if not file_entries:
-        raise HTTPException(status_code=400, detail="No usable .rnd or .rnh files found. Expected NL-43 _Leq_ files or AU2 format .rnd files.")
-
-    # --- Step 2: Find and parse .rnh metadata ---
-    rnh_meta = {}
-    for fname, fbytes in file_entries:
-        if fname.lower().endswith(".rnh"):
-            rnh_meta = _parse_rnh(fbytes)
-            break
-
-    # RNH files store local time (no UTC offset).  Use local values for period
-    # classification / label generation, then convert to UTC for DB storage so
-    # the local_datetime Jinja filter displays the correct time.
-    started_at_local = _parse_rnh_datetime(rnh_meta.get("start_time_str")) or datetime.utcnow()
-    stopped_at_local = _parse_rnh_datetime(rnh_meta.get("stop_time_str"))
-
-    started_at = local_to_utc(started_at_local)
-    stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
-
-    duration_seconds = None
-    if started_at and stopped_at:
-        duration_seconds = int((stopped_at - started_at).total_seconds())
-
-    store_name = rnh_meta.get("store_name", "")
-    serial_number = rnh_meta.get("serial_number", "")
-    index_number = rnh_meta.get("index_number", "")
-
-    # --- Step 3: Create MonitoringSession ---
-    # Use local times for period/label so classification reflects the clock at the site.
-    period_type = _derive_period_type(started_at_local) if started_at_local else None
-    session_label = _build_session_label(started_at_local, location.name, period_type) if started_at_local else None
-
-    session_id = str(uuid.uuid4())
-    monitoring_session = MonitoringSession(
-        id=session_id,
-        project_id=project_id,
-        location_id=location_id,
-        unit_id=None,
-        session_type="sound",
-        started_at=started_at,
-        stopped_at=stopped_at,
-        duration_seconds=duration_seconds,
-        status="completed",
-        session_label=session_label,
-        period_type=period_type,
-        session_metadata=json.dumps({
-            "source": "manual_upload",
-            "store_name": store_name,
-            "serial_number": serial_number,
-            "index_number": index_number,
-        }),
-    )
-    db.add(monitoring_session)
-    db.commit()
-    db.refresh(monitoring_session)
-
-    # --- Step 4: Write files to disk and create DataFile records ---
-    output_dir = Path("data/Projects") / project_id / session_id
-    output_dir.mkdir(parents=True, exist_ok=True)
-
-    leq_count = 0
-    lp_count = 0
-    metadata_count = 0
-    files_imported = 0
-
-    for fname, fbytes in file_entries:
-        file_type = _classify_file(fname)
-        fname_lower = fname.lower()
-
-        # Track counts for summary
-        if fname_lower.endswith(".rnd"):
-            if "_leq_" in fname_lower:
-                leq_count += 1
-            elif "_lp" in fname_lower:
-                lp_count += 1
-        elif fname_lower.endswith(".rnh"):
-            metadata_count += 1
-
-        # Write to disk
-        dest = output_dir / fname
-        dest.write_bytes(fbytes)
-
-        # Compute checksum
-        checksum = hashlib.sha256(fbytes).hexdigest()
-
-        # Store relative path from data/ dir
-        rel_path = str(dest.relative_to("data"))
-
-        data_file = DataFile(
-            id=str(uuid.uuid4()),
-            session_id=session_id,
-            file_path=rel_path,
-            file_type=file_type,
-            file_size_bytes=len(fbytes),
-            downloaded_at=datetime.utcnow(),
-            checksum=checksum,
-            file_metadata=json.dumps({
-                "source": "manual_upload",
-                "original_filename": fname,
-                "store_name": store_name,
-            }),
-        )
-        db.add(data_file)
-        files_imported += 1
-
-    db.commit()
-
-    return {
-        "success": True,
-        "session_id": session_id,
-        "files_imported": files_imported,
-        "leq_files": leq_count,
-        "lp_files": lp_count,
-        "metadata_files": metadata_count,
-        "store_name": store_name,
-        "started_at": started_at.isoformat() if started_at else None,
-        "stopped_at": stopped_at.isoformat() if stopped_at else None,
-    }
+    # --- Hand off to the shared ingest core ---
+    try:
+        return _ingest_file_entries(location, file_entries, db, source="manual_upload", dedupe=False)
+    except IngestError as e:
+        raise HTTPException(status_code=400, detail=str(e))
 
 
 # ============================================================================

backend/routers/projects.py

@@ -1591,24 +1591,32 @@ async def get_sessions_calendar(
 async def get_ftp_browser(
     project_id: str,
     request: Request,
+    location_id: Optional[str] = None,
     db: Session = Depends(get_db),
 ):
     """
     Get FTP browser interface for downloading files from assigned SLMs.
     Returns HTML partial with FTP browser. Sound Monitoring projects only.
+
+    When `location_id` is given, scope to just the unit(s) assigned to that NRL
+    (used by the per-NRL Data Files tab, which mirrors the project-wide tab).
     """
     from backend.models import DataFile
 
     project = db.query(Project).filter_by(id=project_id).first()
     _require_module(project, "sound_monitoring", db)
 
-    # Get all assignments for this project (active = assigned_until IS NULL)
-    assignments = db.query(UnitAssignment).filter(
+    # Active assignments for this project (active = assigned_until IS NULL),
+    # optionally scoped to a single NRL/location.
+    q = db.query(UnitAssignment).filter(
         and_(
             UnitAssignment.project_id == project_id,
             UnitAssignment.assigned_until == None,
         )
-    ).all()
+    )
+    if location_id:
+        q = q.filter(UnitAssignment.location_id == location_id)
+    assignments = q.all()
 
     # Enrich with unit and location details
     units_data = []
@@ -1638,9 +1646,13 @@ async def ftp_download_to_server(
     db: Session = Depends(get_db),
 ):
     """
-    Download a file from an SLM to the server via FTP.
-    Creates a DataFile record and stores the file in data/Projects/{project_id}/
-    Sound Monitoring projects only.
+    Download a single file from an SLM to the server via FTP.
+
+    NRL measurement files (.rnh / _Leq_ .rnd) are routed through the shared NRL
+    ingest so the session is parsed and attributed to the unit (a lone .rnh still
+    yields the real recording window + duration).  Any other file type — or a
+    unit with no location — falls back to a generic stored DataFile, preserving
+    the original behaviour.  Sound Monitoring projects only.
     """
     import httpx
     import os
@@ -1658,7 +1670,55 @@ async def ftp_download_to_server(
     if not unit_id or not remote_path:
         raise HTTPException(status_code=400, detail="Missing unit_id or remote_path")
 
-    # Get or create active session for this location/unit
+    filename = os.path.basename(remote_path)
+
+    # Download the file from SLMM
+    SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
+    try:
+        async with httpx.AsyncClient(timeout=300.0) as client:
+            response = await client.post(
+                f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download",
+                json={"remote_path": remote_path}
+            )
+    except httpx.TimeoutException:
+        raise HTTPException(status_code=504, detail="Timeout downloading file from SLM")
+    except Exception as e:
+        logger.error(f"Error reaching SLMM for file download: {e}")
+        raise HTTPException(status_code=502, detail=f"Failed to reach SLMM: {str(e)}")
+
+    if not response.is_success:
+        raise HTTPException(
+            status_code=response.status_code,
+            detail=f"Failed to download from SLMM: {response.text}",
+        )
+    file_content = response.content
+
+    # NRL measurement file + known location → shared ingest (parsed + attributed).
+    from backend.routers.project_locations import (
+        _ingest_file_entries, IngestError, _is_wanted_nrl_file,
+    )
+    if location_id and _is_wanted_nrl_file(filename):
+        location = db.query(MonitoringLocation).filter_by(id=location_id).first()
+        if location:
+            try:
+                result = _ingest_file_entries(
+                    location, [(filename, file_content)], db,
+                    source="ftp_manual", dedupe=False, unit_id=unit_id,
+                )
+            except IngestError as e:
+                raise HTTPException(status_code=400, detail=str(e))
+            return {
+                "success": True,
+                "message": f"Imported {filename} as NRL measurement data",
+                "ingested": True,
+                "session_id": result["session_id"],
+                "file_size": len(file_content),
+                "started_at": result["started_at"],
+                "stopped_at": result["stopped_at"],
+                "duration_seconds": result["duration_seconds"],
+            }
+
+    # --- Generic path: any other file type (or no location) — store as-is ---
     session = db.query(MonitoringSession).filter(
         and_(
             MonitoringSession.project_id == project_id,
@@ -1668,7 +1728,6 @@ async def ftp_download_to_server(
         )
     ).first()
 
-    # If no active session, create one
     if not session:
         _ftp_unit = db.query(RosterUnit).filter_by(id=unit_id).first()
         session = MonitoringSession(
@@ -1687,77 +1746,25 @@ async def ftp_download_to_server(
         db.commit()
         db.refresh(session)
 
-    # Download file from SLMM
-    SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
-
-    try:
-        async with httpx.AsyncClient(timeout=300.0) as client:
-            response = await client.post(
-                f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download",
-                json={"remote_path": remote_path}
-            )
-
-            if not response.is_success:
-                raise HTTPException(
-                    status_code=response.status_code,
-                    detail=f"Failed to download from SLMM: {response.text}"
-                )
-
-            # Extract filename from remote_path
-            filename = os.path.basename(remote_path)
-
-            # Determine file type from extension
     ext = os.path.splitext(filename)[1].lower()
     file_type_map = {
-                # Audio files
-                '.wav': 'audio',
-                '.mp3': 'audio',
-                '.flac': 'audio',
-                '.m4a': 'audio',
-                '.aac': 'audio',
-                # Sound level meter measurement files
+        '.wav': 'audio', '.mp3': 'audio', '.flac': 'audio', '.m4a': 'audio', '.aac': 'audio',
         '.rnd': 'measurement',
-                # Data files
-                '.csv': 'data',
-                '.txt': 'data',
-                '.json': 'data',
-                '.xml': 'data',
-                '.dat': 'data',
-                # Log files
+        '.csv': 'data', '.txt': 'data', '.json': 'data', '.xml': 'data', '.dat': 'data',
         '.log': 'log',
-                # Archives
-                '.zip': 'archive',
-                '.tar': 'archive',
-                '.gz': 'archive',
-                '.7z': 'archive',
-                '.rar': 'archive',
-                # Images
-                '.jpg': 'image',
-                '.jpeg': 'image',
-                '.png': 'image',
-                '.gif': 'image',
-                # Documents
-                '.pdf': 'document',
-                '.doc': 'document',
-                '.docx': 'document',
+        '.zip': 'archive', '.tar': 'archive', '.gz': 'archive', '.7z': 'archive', '.rar': 'archive',
+        '.jpg': 'image', '.jpeg': 'image', '.png': 'image', '.gif': 'image',
+        '.pdf': 'document', '.doc': 'document', '.docx': 'document',
     }
     file_type = file_type_map.get(ext, 'data')
 
-            # Create directory structure: data/Projects/{project_id}/{session_id}/
     project_dir = Path(f"data/Projects/{project_id}/{session.id}")
     project_dir.mkdir(parents=True, exist_ok=True)
-
-            # Save file to disk
     file_path = project_dir / filename
-            file_content = response.content
-
     with open(file_path, 'wb') as f:
         f.write(file_content)
-
-            # Calculate checksum
     checksum = hashlib.sha256(file_content).hexdigest()
 
-            # Create DataFile record
     data_file = DataFile(
         id=str(uuid.uuid4()),
         session_id=session.id,
@@ -1773,7 +1780,6 @@ async def ftp_download_to_server(
             "location_id": location_id,
         })
     )
-
     db.add(data_file)
     db.commit()
 
@@ -1785,18 +1791,6 @@ async def ftp_download_to_server(
         "file_size": len(file_content),
     }
 
-    except httpx.TimeoutException:
-        raise HTTPException(
-            status_code=504,
-            detail="Timeout downloading file from SLM"
-        )
-    except Exception as e:
-        logger.error(f"Error downloading file to server: {e}")
-        raise HTTPException(
-            status_code=500,
-            detail=f"Failed to download file to server: {str(e)}"
-        )
-
 
 @router.post("/{project_id}/ftp-download-folder-to-server")
 async def ftp_download_folder_to_server(
@@ -1805,20 +1799,20 @@ async def ftp_download_folder_to_server(
     db: Session = Depends(get_db),
 ):
     """
-    Download an entire folder from an SLM to the server via FTP.
-    Extracts all files from the ZIP and preserves folder structure.
-    Creates individual DataFile records for each file.
-    Sound Monitoring projects only.
+    Download an entire Auto_#### measurement folder from an SLM to the server.
+
+    Routes the downloaded ZIP through the shared NRL ingest — the same path the
+    scheduled FTP pull, the daily cycle, and the manual SD-card upload use.  That
+    means: keep the .rnh + Leq .rnd, parse the header (real recording start/stop
+    + duration, percentile slot map, weightings), drop the 1-second _Lp_ files,
+    and create one clean MonitoringSession attributed to the unit.  Sound
+    Monitoring projects only.
     """
     import httpx
     import os
-    import hashlib
-    import zipfile
-    import io
 
     _require_module(db.query(Project).filter_by(id=project_id).first(), "sound_monitoring", db)
-    from pathlib import Path
-    from backend.models import DataFile
+    from backend.routers.project_locations import ingest_nrl_zip, IngestError
 
     data = await request.json()
     unit_id = data.get("unit_id")
@@ -1827,160 +1821,66 @@ async def ftp_download_folder_to_server(
 
     if not unit_id or not remote_path:
         raise HTTPException(status_code=400, detail="Missing unit_id or remote_path")
-
-    # Get or create active session for this location/unit
-    session = db.query(MonitoringSession).filter(
-        and_(
-            MonitoringSession.project_id == project_id,
-            MonitoringSession.location_id == location_id,
-            MonitoringSession.unit_id == unit_id,
-            MonitoringSession.status.in_(["recording", "paused"])
+    if not location_id:
+        raise HTTPException(
+            status_code=400,
+            detail=("This unit isn't assigned to a monitoring location. Assign it to an "
+                    "NRL first so the downloaded measurement attaches to the right location."),
         )
-    ).first()
 
-    # If no active session, create one
-    if not session:
-        _ftp_unit = db.query(RosterUnit).filter_by(id=unit_id).first()
-        session = MonitoringSession(
-            id=str(uuid.uuid4()),
-            project_id=project_id,
-            location_id=location_id,
-            unit_id=unit_id,
-            session_type="sound",  # SLMs are sound monitoring devices
-            status="completed",
-            started_at=datetime.utcnow(),
-            stopped_at=datetime.utcnow(),
-            device_model=_ftp_unit.slm_model if _ftp_unit else None,
-            session_metadata='{"source": "ftp_folder_download", "note": "Auto-created for FTP folder download"}'
-        )
-        db.add(session)
-        db.commit()
-        db.refresh(session)
-
-    # Download folder from SLMM (returns ZIP)
+    # Download the folder from SLMM (returns a ZIP of the Auto_#### folder)
     SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
-
     try:
-        async with httpx.AsyncClient(timeout=600.0) as client:  # Longer timeout for folders
+        async with httpx.AsyncClient(timeout=600.0) as client:  # longer timeout for folders
             response = await client.post(
                 f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download-folder",
                 json={"remote_path": remote_path}
             )
+    except httpx.TimeoutException:
+        raise HTTPException(
+            status_code=504,
+            detail="Timeout downloading folder from SLM (large folders may take a while)",
+        )
+    except Exception as e:
+        logger.error(f"Error reaching SLMM for folder download: {e}")
+        raise HTTPException(status_code=502, detail=f"Failed to reach SLMM: {str(e)}")
 
     if not response.is_success:
         raise HTTPException(
             status_code=response.status_code,
-                    detail=f"Failed to download folder from SLMM: {response.text}"
+            detail=f"Failed to download folder from SLMM: {response.text}",
         )
 
-            # Extract folder name from remote_path
+    # Ingest through the shared NRL core.  dedupe=False so a re-download of a
+    # still-growing folder captures the latest intervals (matches manual upload).
+    try:
+        result = ingest_nrl_zip(
+            location_id, response.content, db,
+            source="ftp_manual", dedupe=False, unit_id=unit_id,
+        )
+    except IngestError as e:
+        # No usable .rnd/.rnh in the folder, or unknown location.
+        raise HTTPException(status_code=400, detail=str(e))
+    except Exception as e:
+        logger.error(f"Error ingesting downloaded folder: {e}")
+        raise HTTPException(status_code=500, detail=f"Failed to ingest downloaded folder: {str(e)}")
+
     folder_name = os.path.basename(remote_path.rstrip('/'))
-
-            # Create base directory: data/Projects/{project_id}/{session_id}/{folder_name}/
-            base_dir = Path(f"data/Projects/{project_id}/{session.id}/{folder_name}")
-            base_dir.mkdir(parents=True, exist_ok=True)
-
-            # Extract ZIP and save individual files
-            zip_content = response.content
-            created_files = []
-            total_size = 0
-
-            # File type mapping for classification
-            file_type_map = {
-                # Audio files
-                '.wav': 'audio', '.mp3': 'audio', '.flac': 'audio', '.m4a': 'audio', '.aac': 'audio',
-                # Data files
-                '.csv': 'data', '.txt': 'data', '.json': 'data', '.xml': 'data', '.dat': 'data',
-                # Log files
-                '.log': 'log',
-                # Archives
-                '.zip': 'archive', '.tar': 'archive', '.gz': 'archive', '.7z': 'archive', '.rar': 'archive',
-                # Images
-                '.jpg': 'image', '.jpeg': 'image', '.png': 'image', '.gif': 'image',
-                # Documents
-                '.pdf': 'document', '.doc': 'document', '.docx': 'document',
-            }
-
-            with zipfile.ZipFile(io.BytesIO(zip_content)) as zf:
-                for zip_info in zf.filelist:
-                    # Skip directories
-                    if zip_info.is_dir():
-                        continue
-
-                    # Read file from ZIP
-                    file_data = zf.read(zip_info.filename)
-
-                    # Determine file path (preserve structure within folder)
-                    # zip_info.filename might be like "Auto_0001/measurement.wav"
-                    file_path = base_dir / zip_info.filename
-                    file_path.parent.mkdir(parents=True, exist_ok=True)
-
-                    # Write file to disk
-                    with open(file_path, 'wb') as f:
-                        f.write(file_data)
-
-                    # Calculate checksum
-                    checksum = hashlib.sha256(file_data).hexdigest()
-
-                    # Determine file type
-                    ext = os.path.splitext(zip_info.filename)[1].lower()
-                    file_type = file_type_map.get(ext, 'data')
-
-                    # Create DataFile record
-                    data_file = DataFile(
-                        id=str(uuid.uuid4()),
-                        session_id=session.id,
-                        file_path=str(file_path.relative_to("data")),
-                        file_type=file_type,
-                        file_size_bytes=len(file_data),
-                        downloaded_at=datetime.utcnow(),
-                        checksum=checksum,
-                        file_metadata=json.dumps({
-                            "source": "ftp_folder",
-                            "remote_path": remote_path,
-                            "unit_id": unit_id,
-                            "location_id": location_id,
-                            "folder_name": folder_name,
-                            "relative_path": zip_info.filename,
-                        })
-                    )
-
-                    db.add(data_file)
-                    created_files.append({
-                        "filename": zip_info.filename,
-                        "size": len(file_data),
-                        "type": file_type
-                    })
-                    total_size += len(file_data)
-
-            db.commit()
-
     return {
         "success": True,
-                "message": f"Downloaded folder {folder_name} with {len(created_files)} files",
+        "message": (
+            f"Imported {result['leq_files']} Leq file(s) from {folder_name} "
+            f"({result['files_imported']} stored; 1-second _Lp_ data skipped)"
+        ),
         "folder_name": folder_name,
-                "file_count": len(created_files),
-                "total_size": total_size,
-                "files": created_files,
+        "session_id": result["session_id"],
+        "file_count": result["files_imported"],
+        "leq_files": result["leq_files"],
+        "started_at": result["started_at"],
+        "stopped_at": result["stopped_at"],
+        "duration_seconds": result["duration_seconds"],
     }
 
-    except httpx.TimeoutException:
-        raise HTTPException(
-            status_code=504,
-            detail="Timeout downloading folder from SLM (large folders may take a while)"
-        )
-    except zipfile.BadZipFile:
-        raise HTTPException(
-            status_code=500,
-            detail="Downloaded file is not a valid ZIP archive"
-        )
-    except Exception as e:
-        logger.error(f"Error downloading folder to server: {e}")
-        raise HTTPException(
-            status_code=500,
-            detail=f"Failed to download folder to server: {str(e)}"
-        )
-
 
 # ============================================================================
 # Project Types
@@ -1990,21 +1890,26 @@ async def ftp_download_folder_to_server(
 async def get_unified_files(
     project_id: str,
     request: Request,
+    location_id: Optional[str] = None,
     db: Session = Depends(get_db),
 ):
     """
     Get unified view of all files in this project.
     Groups files by recording session with full metadata.
     Returns HTML partial with hierarchical file listing.
+
+    When `location_id` is given, scope to a single NRL/location (used by the
+    per-NRL Data Files tab so it mirrors the project-wide tab).
     """
     from backend.models import DataFile
     from pathlib import Path
     import json
 
-    # Get all sessions for this project
-    sessions = db.query(MonitoringSession).filter_by(
-        project_id=project_id
-    ).order_by(MonitoringSession.started_at.desc()).all()
+    # Sessions for this project (optionally scoped to one NRL/location)
+    q = db.query(MonitoringSession).filter_by(project_id=project_id)
+    if location_id:
+        q = q.filter(MonitoringSession.location_id == location_id)
+    sessions = q.order_by(MonitoringSession.started_at.desc()).all()
 
     sessions_data = []
     for session in sessions:

backend/routers/reports.py

@@ -0,0 +1,434 @@
+"""
+Nightly Report Router.
+
+Manual triggers for the night-vs-baseline sound report — the same entry point
+the scheduled morning tick will reuse.  Two endpoints:
+
+  GET  …/reports/nightly/view  → render and return the HTML inline (preview).
+                                 No write, no email.  Browser-friendly.
+  POST …/reports/nightly/run   → full run: build → write report.html/json to
+                                 disk → (dry-run) email.  Returns JSON result.
+
+Dates are the *evening* date of the night being reported (the 7/7 in "night of
+7/7 → morning 7/8").  Defaults to last night.  Baseline is optional; pass the
+baseline-week range to populate the comparison.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+import uuid
+from datetime import datetime, timedelta, date
+from html import escape
+from pathlib import Path
+from typing import Optional
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Request
+from fastapi.responses import HTMLResponse
+from sqlalchemy.orm import Session
+
+from backend.database import get_db
+from backend.models import Project, SoundReportConfig, MonitoringLocation
+from backend.services.report_pipeline import (
+    METRIC_REGISTRY, DEFAULT_METRICS, DEFAULT_WINDOWS, _location_reference_baseline,
+)
+from backend.services.report_orchestrator import run_nightly_report
+from backend.utils.timezone import utc_to_local
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/projects/{project_id}/reports", tags=["reports"])
+
+
+def _default_night_date() -> date:
+    """Last night = yesterday in the user's local timezone."""
+    return (utc_to_local(datetime.utcnow()) - timedelta(days=1)).date()
+
+
+def _parse_date(s: Optional[str], field: str) -> Optional[date]:
+    if not s:
+        return None
+    try:
+        return datetime.strptime(s, "%Y-%m-%d").date()
+    except ValueError:
+        raise HTTPException(status_code=400, detail=f"{field} must be YYYY-MM-DD (got {s!r})")
+
+
+def _parse_metrics(s: Optional[str]) -> list[str]:
+    if not s:
+        return list(DEFAULT_METRICS)
+    keys = [k.strip().lower() for k in s.split(",") if k.strip()]
+    unknown = [k for k in keys if k not in METRIC_REGISTRY]
+    if unknown:
+        raise HTTPException(
+            status_code=400,
+            detail=f"Unknown metric(s): {unknown}. Known: {sorted(METRIC_REGISTRY)}",
+        )
+    return keys or list(DEFAULT_METRICS)
+
+
+def _validate_hhmm(s) -> str:
+    """Validate a local HH:MM (24h) time string."""
+    try:
+        hh, mm = str(s).split(":")
+        h, m = int(hh), int(mm)
+        if 0 <= h < 24 and 0 <= m < 60:
+            return f"{h:02d}:{m:02d}"
+    except (ValueError, AttributeError):
+        pass
+    raise HTTPException(status_code=400, detail=f"report_time must be HH:MM 24-hour (got {s!r})")
+
+
+def _config_dict(cfg: Optional[SoundReportConfig], project_id: str) -> dict:
+    """Serialise a config row (or defaults if none yet) to JSON."""
+    return {
+        "project_id": project_id,
+        "exists": cfg is not None,
+        "enabled": cfg.enabled if cfg else False,
+        "report_time": cfg.report_time if cfg else "08:00",
+        "metric_keys": cfg.metric_keys if cfg else ",".join(DEFAULT_METRICS),
+        "baseline_mode": cfg.baseline_mode if cfg else "captured",
+        "baseline_start": cfg.baseline_start.isoformat() if cfg and cfg.baseline_start else None,
+        "baseline_end": cfg.baseline_end.isoformat() if cfg and cfg.baseline_end else None,
+        "recipients": (cfg.recipients if cfg and cfg.recipients else ""),
+        "last_run_date": cfg.last_run_date.isoformat() if cfg and cfg.last_run_date else None,
+    }
+
+
+@router.get("/config")
+async def get_report_config(project_id: str, db: Session = Depends(get_db)):
+    """Return the project's nightly-report config (or defaults if not set yet)."""
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+    return _config_dict(cfg, project_id)
+
+
+@router.put("/config")
+async def put_report_config(project_id: str, request: Request, db: Session = Depends(get_db)):
+    """Create or update the project's nightly-report config (JSON body)."""
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    data = await request.json()
+
+    cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+    created = cfg is None
+    if cfg is None:
+        cfg = SoundReportConfig(id=str(uuid.uuid4()), project_id=project_id)
+        db.add(cfg)
+
+    if "enabled" in data:
+        cfg.enabled = bool(data["enabled"])
+    if "report_time" in data:
+        cfg.report_time = _validate_hhmm(data["report_time"])
+    if "metric_keys" in data:
+        mk = data["metric_keys"]
+        mk = mk if isinstance(mk, str) else ",".join(mk or [])
+        cfg.metric_keys = ",".join(_parse_metrics(mk))
+    if "baseline_mode" in data:
+        bm = str(data["baseline_mode"]).lower()
+        if bm not in ("captured", "reference"):
+            raise HTTPException(status_code=400, detail="baseline_mode must be 'captured' or 'reference'")
+        cfg.baseline_mode = bm
+    if "baseline_start" in data or "baseline_end" in data:
+        bs = _parse_date(data.get("baseline_start") or None, "baseline_start")
+        be = _parse_date(data.get("baseline_end") or None, "baseline_end")
+        if (bs and not be) or (be and not bs):
+            raise HTTPException(status_code=400, detail="Provide both baseline dates, or neither.")
+        if bs and be and bs > be:
+            raise HTTPException(status_code=400, detail="baseline_start must be on or before baseline_end.")
+        cfg.baseline_start, cfg.baseline_end = bs, be
+    if "recipients" in data:
+        recips = data["recipients"]
+        if isinstance(recips, list):
+            recips = ",".join(recips)
+        cfg.recipients = (recips or "").strip() or None
+
+    db.commit()
+    db.refresh(cfg)
+    return {**_config_dict(cfg, project_id), "created": created}
+
+
+def _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics):
+    """Validate inputs and resolve the baseline source.
+
+    Explicit baseline dates in the query override (captured mode with those
+    dates). Otherwise the project's saved config supplies the baseline (its
+    mode + dates) and the default metric set — so the manual view/run match
+    what the scheduled report does.
+    Returns (night_date, baseline_mode, baseline_start, baseline_end, metric_keys).
+    """
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    nd = _parse_date(night_date, "night_date") or _default_night_date()
+    bs = _parse_date(baseline_start, "baseline_start")
+    be = _parse_date(baseline_end, "baseline_end")
+    if (bs and not be) or (be and not bs):
+        raise HTTPException(status_code=400, detail="Provide both baseline_start and baseline_end, or neither.")
+    if bs and be and bs > be:
+        raise HTTPException(status_code=400, detail="baseline_start must be on or before baseline_end.")
+
+    cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+    if bs and be:
+        baseline_mode = "captured"                       # explicit dates win
+    elif cfg:
+        baseline_mode = cfg.baseline_mode                # fall back to saved config
+        bs, be = cfg.baseline_start, cfg.baseline_end
+    else:
+        baseline_mode = "captured"
+
+    if metrics:
+        metric_keys = _parse_metrics(metrics)
+    elif cfg and cfg.metric_keys:
+        metric_keys = _parse_metrics(cfg.metric_keys)
+    else:
+        metric_keys = list(DEFAULT_METRICS)
+
+    return nd, baseline_mode, bs, be, metric_keys
+
+
+@router.get("/nightly/view", response_class=HTMLResponse)
+async def view_nightly_report(
+    project_id: str,
+    night_date: Optional[str] = Query(None, description="Evening date of the night (YYYY-MM-DD). Default: last night."),
+    baseline_start: Optional[str] = Query(None, description="Baseline range start (YYYY-MM-DD)."),
+    baseline_end: Optional[str] = Query(None, description="Baseline range end (YYYY-MM-DD)."),
+    metrics: Optional[str] = Query(None, description="Comma list, e.g. lmax,l01,l10,l90. Default: house set."),
+    db: Session = Depends(get_db),
+):
+    """Render the night report and return the HTML inline (preview — no write, no email)."""
+    nd, bmode, bs, be, metric_keys = _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics)
+    try:
+        result = run_nightly_report(
+            db, project_id, nd,
+            metric_keys=metric_keys, baseline_mode=bmode, baseline_start=bs, baseline_end=be,
+            send=False,           # preview: no email
+        )
+    except HTTPException:
+        raise
+    except Exception as e:  # noqa: BLE001
+        logger.error("nightly/view failed for %s (%s): %s", project_id, nd, e, exc_info=True)
+        raise HTTPException(status_code=500, detail=f"Report generation failed: {e}")
+    return HTMLResponse(result["html"])
+
+
+@router.post("/nightly/run")
+async def run_nightly_report_endpoint(
+    project_id: str,
+    night_date: Optional[str] = Query(None, description="Evening date of the night (YYYY-MM-DD). Default: last night."),
+    baseline_start: Optional[str] = Query(None, description="Baseline range start (YYYY-MM-DD)."),
+    baseline_end: Optional[str] = Query(None, description="Baseline range end (YYYY-MM-DD)."),
+    metrics: Optional[str] = Query(None, description="Comma list, e.g. lmax,l01,l10,l90. Default: house set."),
+    send: bool = Query(True, description="Attempt email (dry-run until SMTP is configured)."),
+    db: Session = Depends(get_db),
+):
+    """Run the night report: build → write report.html/report.json to disk → email (best-effort).
+
+    This is the same path the scheduled morning tick will call.  The `html` field
+    is omitted from the JSON response (it's large and on disk); use /view to see it.
+    """
+    nd, bmode, bs, be, metric_keys = _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics)
+    try:
+        result = run_nightly_report(
+            db, project_id, nd,
+            metric_keys=metric_keys, baseline_mode=bmode, baseline_start=bs, baseline_end=be,
+            send=send,
+        )
+    except HTTPException:
+        raise
+    except Exception as e:  # noqa: BLE001
+        logger.error("nightly/run failed for %s (%s): %s", project_id, nd, e, exc_info=True)
+        raise HTTPException(status_code=500, detail=f"Report generation failed: {e}")
+    result.pop("html", None)  # keep the JSON response lean — view it via /view or the file
+    result["view_url"] = (
+        f"/api/projects/{project_id}/reports/nightly/view"
+        f"?night_date={nd:%Y-%m-%d}"
+        + (f"&baseline_start={bs:%Y-%m-%d}&baseline_end={be:%Y-%m-%d}" if bs and be else "")
+        + (f"&metrics={','.join(metric_keys)}")
+    )
+    return result
+
+
+# ============================================================================
+# Test email + generated-report archive
+# ============================================================================
+
+_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
+
+
+@router.post("/test-email")
+async def send_test_email(project_id: str, request: Request, db: Session = Depends(get_db)):
+    """Send a small test email to verify the SMTP relay (dry-run if unconfigured).
+
+    Recipients: JSON body {"recipients": "..."} overrides; else the project's
+    configured recipients; else the REPORT_SMTP_RECIPIENTS env default.
+    """
+    project = db.query(Project).filter_by(id=project_id).first()
+    if not project:
+        raise HTTPException(status_code=404, detail="Project not found")
+    try:
+        data = await request.json()
+    except Exception:
+        data = {}
+
+    raw = (data or {}).get("recipients")
+    if not raw:
+        cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+        raw = cfg.recipients if cfg else None
+    recipients = None
+    if raw:
+        if isinstance(raw, list):
+            raw = ",".join(raw)
+        recipients = [r.strip() for r in raw.split(",") if r.strip()]
+
+    from backend.services.report_email import send_report_email
+    body = (
+        "<div style=\"font:14px Arial,sans-serif\">"
+        f"Terra-View test email for <b>{escape(project.name)}</b>.<br>"
+        "If you got this, the nightly sound-report email path is working.</div>"
+    )
+    return send_report_email("Terra-View — nightly report test email", body, recipients=recipients)
+
+
+@router.get("/list")
+async def list_reports(project_id: str, db: Session = Depends(get_db)):
+    """List the generated report artifacts on disk for this project (newest first)."""
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    base = Path("data/reports") / project_id
+    out = []
+    if base.exists():
+        for d in sorted((p for p in base.iterdir() if p.is_dir()), key=lambda p: p.name, reverse=True):
+            html_file = d / "report.html"
+            if html_file.exists():
+                st = html_file.stat()
+                out.append({
+                    "night_date": d.name,
+                    "view_url": f"/api/projects/{project_id}/reports/archive/{d.name}",
+                    "xlsx_url": (f"/api/projects/{project_id}/reports/archive/{d.name}/xlsx"
+                                 if (d / "report.xlsx").exists() else None),
+                    "size_bytes": st.st_size,
+                    "generated_at": datetime.utcfromtimestamp(st.st_mtime).isoformat(),
+                })
+    return {"reports": out, "count": len(out)}
+
+
+@router.get("/archive/{night_date}", response_class=HTMLResponse)
+async def view_archived_report(project_id: str, night_date: str, db: Session = Depends(get_db)):
+    """Serve a previously generated report.html from disk (the actual artifact)."""
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    if not _DATE_RE.match(night_date):
+        raise HTTPException(status_code=400, detail="Invalid date (YYYY-MM-DD)")
+    safe = _parse_date(night_date, "night_date")            # also guards path traversal
+    path = Path("data/reports") / project_id / f"{safe:%Y-%m-%d}" / "report.html"
+    if not path.exists():
+        raise HTTPException(status_code=404, detail="No saved report for that date")
+    return HTMLResponse(path.read_text(encoding="utf-8"))
+
+
+@router.get("/archive/{night_date}/xlsx")
+async def download_archived_xlsx(project_id: str, night_date: str, db: Session = Depends(get_db)):
+    """Download a previously generated report.xlsx from disk."""
+    from fastapi.responses import Response
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    if not _DATE_RE.match(night_date):
+        raise HTTPException(status_code=400, detail="Invalid date (YYYY-MM-DD)")
+    safe = _parse_date(night_date, "night_date")
+    path = Path("data/reports") / project_id / f"{safe:%Y-%m-%d}" / "report.xlsx"
+    if not path.exists():
+        raise HTTPException(status_code=404, detail="No saved spreadsheet for that date")
+    return Response(
+        content=path.read_bytes(),
+        media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+        headers={"Content-Disposition": f'attachment; filename="night_report_{safe:%Y-%m-%d}.xlsx"'},
+    )
+
+
+# ============================================================================
+# Reference baseline (fixed values typed per location — limits / prior averages)
+# ============================================================================
+
+@router.get("/baseline")
+async def get_baseline(project_id: str, db: Session = Depends(get_db)):
+    """Return the baseline mode + per-location reference values + the metric/window
+    grid to render the editor."""
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+    mode = cfg.baseline_mode if cfg else "captured"
+    metric_keys = _parse_metrics(cfg.metric_keys) if cfg and cfg.metric_keys else list(DEFAULT_METRICS)
+
+    locations = db.query(MonitoringLocation).filter_by(
+        project_id=project_id, location_type="sound",
+    ).order_by(MonitoringLocation.sort_order, MonitoringLocation.name).all()
+    locations = [l for l in locations if getattr(l, "removed_at", None) is None]
+
+    return {
+        "mode": mode,
+        "windows": [{"key": w.key, "label": w.label} for w in DEFAULT_WINDOWS],
+        "metrics": [{"key": k, "label": METRIC_REGISTRY[k].label} for k in metric_keys],
+        "locations": [
+            {"id": loc.id, "name": loc.name, "values": _location_reference_baseline(loc)}
+            for loc in locations
+        ],
+    }
+
+
+@router.put("/baseline")
+async def put_baseline(project_id: str, request: Request, db: Session = Depends(get_db)):
+    """Save the baseline mode (on config) and per-location reference values
+    (on each location's metadata).  Body:
+      {"mode": "reference",
+       "locations": {"<loc_id>": {"nighttime": {"l10": 85}, "evening": {...}}}}
+    """
+    if not db.query(Project).filter_by(id=project_id).first():
+        raise HTTPException(status_code=404, detail="Project not found")
+    data = await request.json()
+
+    if "mode" in data:
+        bm = str(data["mode"]).lower()
+        if bm not in ("captured", "reference"):
+            raise HTTPException(status_code=400, detail="mode must be 'captured' or 'reference'")
+        cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
+        if cfg is None:
+            cfg = SoundReportConfig(id=str(uuid.uuid4()), project_id=project_id)
+            db.add(cfg)
+        cfg.baseline_mode = bm
+
+    loc_values = data.get("locations") or {}
+    updated = 0
+    for loc_id, windows in loc_values.items():
+        loc = db.query(MonitoringLocation).filter_by(id=loc_id, project_id=project_id).first()
+        if not loc or not isinstance(windows, dict):
+            continue
+        try:
+            meta = json.loads(loc.location_metadata or "{}")
+        except (json.JSONDecodeError, TypeError):
+            meta = {}
+        clean: dict = {}
+        for wkey, mvals in windows.items():
+            if not isinstance(mvals, dict):
+                continue
+            cm = {}
+            for mkey, val in mvals.items():
+                if val in (None, ""):
+                    continue
+                try:
+                    cm[mkey] = round(float(val), 1)
+                except (ValueError, TypeError):
+                    continue
+            if cm:
+                clean[wkey] = cm
+        if clean:
+            meta["report_baseline"] = clean
+        else:
+            meta.pop("report_baseline", None)
+        loc.location_metadata = json.dumps(meta)
+        updated += 1
+
+    db.commit()
+    return {"ok": True, "locations_updated": updated}

backend/services/report_email.py

@@ -0,0 +1,172 @@
+"""
+Report email sender — config-driven SMTP via the Python standard library.
+
+Connection settings come from environment variables so the mail backend
+(internal relay / Microsoft 365 / Gmail / SendGrid) can be swapped without code
+changes — see the build plan: terra-mechanics.com is on M365 and has a smarthost
+relay that already sends the seismograph alerts as remote@terra-mechanics.com;
+reuse that relay's settings here.
+
+DRY-RUN: if SMTP isn't configured (no host/from), the message is built and
+logged but NOT sent, and the call still succeeds.  This keeps report generation
+working before the relay is wired up, and means a missing/incomplete mail config
+can never crash the nightly pipeline.
+
+Env vars
+--------
+  REPORT_SMTP_HOST        e.g. smtp.office365.com   (unset → dry-run)
+  REPORT_SMTP_PORT        default 587
+  REPORT_SMTP_SECURITY    starttls (default) | ssl | none
+  REPORT_SMTP_USER        optional — omit for IP-authenticated relays
+  REPORT_SMTP_PASSWORD    optional
+  REPORT_SMTP_FROM        e.g. "TMI Monitoring <monitoring@terra-mechanics.com>"
+  REPORT_SMTP_RECIPIENTS  comma-separated default recipient list
+  REPORT_SMTP_TIMEOUT     seconds, default 30
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import smtplib
+import ssl
+from dataclasses import dataclass, field
+from email.message import EmailMessage
+from typing import Optional
+
+logger = logging.getLogger(__name__)
+
+# Convenient MIME type for the Excel attachment.
+XLSX_MIME = ("application", "vnd.openxmlformats-officedocument.spreadsheetml.sheet")
+
+
+@dataclass
+class Attachment:
+    filename: str
+    content: bytes
+    maintype: str = "application"
+    subtype: str = "octet-stream"
+
+
+@dataclass
+class SMTPConfig:
+    host: str = ""
+    port: int = 587
+    security: str = "starttls"   # "starttls" | "ssl" | "none"
+    user: str = ""
+    password: str = ""
+    sender: str = ""
+    recipients: list[str] = field(default_factory=list)
+    timeout: float = 30.0
+
+    @classmethod
+    def from_env(cls) -> "SMTPConfig":
+        rec = os.getenv("REPORT_SMTP_RECIPIENTS", "")
+        return cls(
+            host=os.getenv("REPORT_SMTP_HOST", "").strip(),
+            port=int(os.getenv("REPORT_SMTP_PORT", "587") or 587),
+            security=os.getenv("REPORT_SMTP_SECURITY", "starttls").strip().lower(),
+            user=os.getenv("REPORT_SMTP_USER", "").strip(),
+            password=os.getenv("REPORT_SMTP_PASSWORD", ""),
+            sender=os.getenv("REPORT_SMTP_FROM", "").strip(),
+            recipients=[r.strip() for r in rec.split(",") if r.strip()],
+            timeout=float(os.getenv("REPORT_SMTP_TIMEOUT", "30") or 30),
+        )
+
+    @property
+    def configured(self) -> bool:
+        """True only when we have enough to actually send (host + from)."""
+        return bool(self.host and self.sender)
+
+
+def build_message(
+    cfg: SMTPConfig,
+    subject: str,
+    html_body: str,
+    recipients: list[str],
+    attachments: Optional[list[Attachment]] = None,
+    text_body: Optional[str] = None,
+) -> EmailMessage:
+    """Assemble a multipart message: plain-text fallback + HTML + attachments."""
+    msg = EmailMessage()
+    msg["From"] = cfg.sender or "terra-view@localhost"
+    msg["To"] = ", ".join(recipients)
+    msg["Subject"] = subject
+    # Plain-text part first, then the HTML alternative (clients prefer the HTML).
+    msg.set_content(text_body or "This report is best viewed in an HTML email client.")
+    msg.add_alternative(html_body, subtype="html")
+    for att in (attachments or []):
+        msg.add_attachment(
+            att.content, maintype=att.maintype, subtype=att.subtype, filename=att.filename,
+        )
+    return msg
+
+
+def send_report_email(
+    subject: str,
+    html_body: str,
+    *,
+    attachments: Optional[list[Attachment]] = None,
+    recipients: Optional[list[str]] = None,
+    text_body: Optional[str] = None,
+    cfg: Optional[SMTPConfig] = None,
+) -> dict:
+    """Send (or dry-run) the report email.
+
+    Returns a result dict: {sent, dry_run, recipients, error}.  Never raises on
+    a send failure — it logs and returns error, so the orchestrator can record
+    the failure without aborting the rest of the pipeline.
+    """
+    cfg = cfg or SMTPConfig.from_env()
+    recipients = recipients if recipients is not None else cfg.recipients
+    result = {"sent": False, "dry_run": False, "recipients": recipients, "error": None}
+
+    if not recipients:
+        result["error"] = "No recipients configured"
+        logger.warning("Report email: no recipients set; skipping send of %r", subject)
+        return result
+
+    msg = build_message(cfg, subject, html_body, recipients, attachments, text_body)
+
+    if not cfg.configured:
+        result["dry_run"] = True
+        logger.info(
+            "Report email DRY-RUN (SMTP not configured): would send %r to %s with %d attachment(s)",
+            subject, recipients, len(attachments or []),
+        )
+        return result
+
+    # Validate the security mode: an unrecognized value (typo) must NOT silently
+    # fall through to a plaintext connection while still sending credentials.
+    sec = cfg.security if cfg.security in ("ssl", "starttls", "none") else "starttls"
+    if sec != cfg.security:
+        logger.warning("Unknown REPORT_SMTP_SECURITY=%r — falling back to 'starttls'", cfg.security)
+
+    try:
+        if sec == "ssl":
+            ctx = ssl.create_default_context()
+            with smtplib.SMTP_SSL(cfg.host, cfg.port, timeout=cfg.timeout, context=ctx) as s:
+                if cfg.user:
+                    s.login(cfg.user, cfg.password)
+                s.send_message(msg)
+        else:
+            with smtplib.SMTP(cfg.host, cfg.port, timeout=cfg.timeout) as s:
+                s.ehlo()
+                if sec == "starttls":
+                    s.starttls(context=ssl.create_default_context())
+                    s.ehlo()
+                if cfg.user:
+                    if sec == "none":
+                        logger.warning(
+                            "Sending SMTP credentials over an UNENCRYPTED connection "
+                            "(REPORT_SMTP_SECURITY=none) — set starttls/ssl if the relay supports it."
+                        )
+                    s.login(cfg.user, cfg.password)
+                s.send_message(msg)
+        result["sent"] = True
+        logger.info("Report email sent: %r to %s", subject, recipients)
+    except Exception as e:  # noqa: BLE001 — surface as result, never abort the pipeline
+        result["error"] = str(e)
+        logger.error("Report email send failed: %s", e, exc_info=True)
+
+    return result

backend/services/report_orchestrator.py

@@ -0,0 +1,150 @@
+"""
+Nightly Report Orchestrator.
+
+Ties the pieces together: compute → render → write-to-disk → email.
+
+This is what the daily cycle (or a manual trigger) calls.  It ALWAYS writes the
+rendered report to disk — `data/reports/{project_id}/{night_date}/report.html`
+(+ `report.json` with the raw numbers) — so there's a viewable artifact even
+when email is in dry-run (SMTP not configured yet).  The email step is
+best-effort and never aborts the run.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from datetime import date
+from pathlib import Path
+from typing import Optional
+
+from sqlalchemy.orm import Session
+
+from backend.services.report_pipeline import (
+    ProjectNightReport, build_project_night_report, Window,
+)
+from backend.services.report_renderers import render_html_summary, render_excel
+from backend.services.report_email import send_report_email, Attachment, XLSX_MIME
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_OUTPUT_ROOT = "data/reports"
+
+
+def _report_to_dict(report: ProjectNightReport) -> dict:
+    """Serialise the report data model to plain JSON (for the on-disk record)."""
+    return {
+        "project_id": report.project_id,
+        "project_name": report.project_name,
+        "night_date": report.night_date.isoformat(),
+        "metrics": [m.key for m in report.metrics],
+        "locations": [
+            {
+                "name": loc.location_name,
+                "night_interval_count": loc.night_interval_count,
+                "baseline_nights_used": loc.baseline_nights_used,
+                "notes": loc.notes,
+                "windows": {
+                    w.key: {
+                        "label": w.label,
+                        "metrics": {
+                            m.key: {
+                                "label": m.label,
+                                "last_night": loc.table[w.key][m.key].last_night,
+                                "baseline": loc.table[w.key][m.key].baseline,
+                                "delta": loc.table[w.key][m.key].delta,
+                            }
+                            for m in loc.metrics
+                        },
+                    }
+                    for w in loc.windows
+                },
+            }
+            for loc in report.locations
+        ],
+    }
+
+
+def run_nightly_report(
+    db: Session,
+    project_id: str,
+    night_date: date,
+    *,
+    metric_keys: Optional[list[str]] = None,
+    windows: Optional[list[Window]] = None,
+    baseline_mode: str = "captured",
+    baseline_start: Optional[date] = None,
+    baseline_end: Optional[date] = None,
+    recipients: Optional[list[str]] = None,
+    output_root: str = DEFAULT_OUTPUT_ROOT,
+    send: bool = True,
+) -> dict:
+    """Build, persist, and (dry-run) email the night report for a project.
+
+    Returns a result dict with the on-disk artifact paths and the email result.
+    Designed to be called from the daily cycle or a manual trigger.
+    """
+    report = build_project_night_report(
+        db, project_id, night_date,
+        metric_keys=metric_keys, windows=windows,
+        baseline_mode=baseline_mode,
+        baseline_start=baseline_start, baseline_end=baseline_end,
+    )
+
+    html = render_html_summary(report)
+    subject = f"{report.project_name} — night report {night_date:%m/%d/%y}"
+
+    # --- Always persist a viewable copy ---
+    out_dir = Path(output_root) / project_id / f"{night_date:%Y-%m-%d}"
+    out_dir.mkdir(parents=True, exist_ok=True)
+    html_path = out_dir / "report.html"
+    html_path.write_text(html, encoding="utf-8")
+    json_path = out_dir / "report.json"
+    json_path.write_text(json.dumps(_report_to_dict(report), indent=2), encoding="utf-8")
+
+    # --- Excel (the email attachment; also written to disk for the archive) ---
+    attachments: list[Attachment] = []
+    xlsx_path = None
+    try:
+        xlsx_bytes = render_excel(report)
+        xlsx_path = out_dir / "report.xlsx"
+        xlsx_path.write_bytes(xlsx_bytes)
+        safe_name = "".join(c for c in report.project_name if c.isalnum() or c in " -_").strip().replace(" ", "_")
+        attachments.append(Attachment(
+            f"{safe_name or 'report'}_{night_date:%Y-%m-%d}_night_report.xlsx",
+            xlsx_bytes, *XLSX_MIME,
+        ))
+    except Exception as e:  # noqa: BLE001 — never let the spreadsheet sink the report
+        logger.error("Excel render failed for %s (%s): %s", project_id, night_date, e, exc_info=True)
+
+    # --- Email (best-effort; dry-run until SMTP is configured) ---
+    email_result = {"sent": False, "dry_run": False, "skipped": True, "error": None}
+    if send:
+        try:
+            email_result = send_report_email(
+                subject, html, attachments=attachments, recipients=recipients,
+            )
+        except Exception as e:  # noqa: BLE001 — artifacts are already written; never abort on email
+            logger.error("send_report_email raised for %s (%s): %s", project_id, night_date, e, exc_info=True)
+            email_result = {"sent": False, "dry_run": False, "skipped": False, "error": str(e)}
+
+    result = {
+        "project_id": project_id,
+        "project_name": report.project_name,
+        "night_date": night_date.isoformat(),
+        "subject": subject,
+        "location_count": len(report.locations),
+        "html_path": str(html_path),
+        "json_path": str(json_path),
+        "xlsx_path": str(xlsx_path) if xlsx_path else None,
+        "html": html,            # for callers that want to display it inline
+        "email": email_result,
+    }
+    logger.info(
+        "Nightly report for %s (%s): %d location(s) → %s; email=%s",
+        report.project_name, night_date, len(report.locations), html_path,
+        "sent" if email_result.get("sent") else
+        ("dry-run" if email_result.get("dry_run") else
+         ("skipped" if email_result.get("skipped") else f"error: {email_result.get('error')}")),
+    )
+    return result

backend/services/report_pipeline.py

@@ -0,0 +1,432 @@
+"""
+Nightly Report Pipeline — computation core.
+
+Builds the data model for the John-Myler-style "last night vs. baseline" sound
+report.  Source-agnostic: it reads the same on-disk Leq `.rnd` files the manual
+upload + FTP-pull ingest produce (see `project_locations.ingest_nrl_zip`).
+
+Design notes
+------------
+* **Ingest everything, report selectively.**  Ingest preserves every column of
+  the Leq file; this layer chooses which *metrics* to surface via `metric_keys`
+  (a future report wizard is just a UI over that list).
+* **House format match.**  Defaults reproduce the existing Excel report:
+  LAmax (max of interval maxima), LA01 / LA10 (arithmetic average), split into
+  Evening (7–10PM) and Nighttime (10PM–7AM) windows.  L90 (background) is added
+  for the baseline comparison.
+* **Metric labelling from the device.**  The LN→percentile assignment is
+  reconfigurable per job; we resolve which `LNx(Main)` column is L90/L10/etc.
+  from the percentile map captured in the session metadata at ingest, falling
+  back to the NL-43 default order.
+* **Correct averaging.**  Leq is energy-averaged (logarithmic); percentiles and
+  Lmax are arithmetic.  Baseline references combine the per-night values into a
+  "typical night" (arithmetic mean of per-night values — so baseline Lmax is the
+  typical nightly peak, not the worst-of-week).
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import math
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta, date
+from typing import Optional
+
+from sqlalchemy.orm import Session
+
+from backend.models import MonitoringSession, DataFile, MonitoringLocation, Project
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Metric registry
+# ---------------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class Metric:
+    """A reportable metric.
+
+    `agg` is the *within-night* aggregation used to collapse a window's 15-min
+    intervals into one value:
+      - "max"   → loudest interval (LAmax)
+      - "arith" → arithmetic mean (percentiles: L01/L10/L90…)
+      - "log"   → energy/logarithmic mean (Leq only)
+    `column` pins a fixed .rnd column; `percentile` instead resolves the LNx
+    column from the session's captured percentile map.
+    """
+    key: str
+    label: str
+    agg: str
+    column: Optional[str] = None
+    percentile: Optional[float] = None
+
+
+METRIC_REGISTRY: dict[str, Metric] = {
+    "lmax": Metric("lmax", "LAmax", "max",   column="Lmax(Main)"),
+    "leq":  Metric("leq",  "LAeq",  "log",   column="Leq(Main)"),
+    "lmin": Metric("lmin", "LAmin", "arith", column="Lmin(Main)"),
+    "l01":  Metric("l01",  "LA01",  "arith", percentile=1.0),
+    "l10":  Metric("l10",  "LA10",  "arith", percentile=10.0),
+    "l50":  Metric("l50",  "LA50",  "arith", percentile=50.0),
+    "l90":  Metric("l90",  "LA90",  "arith", percentile=90.0),
+    "l95":  Metric("l95",  "LA95",  "arith", percentile=95.0),
+}
+
+# House report metrics + L90 (background) for the baseline comparison.
+DEFAULT_METRICS: list[str] = ["lmax", "l01", "l10", "l90"]
+
+# NL-43 default percentile→slot assignment, used when a session has no captured map.
+_DEFAULT_SLOT_FOR_PCT: dict[float, int] = {1.0: 1, 10.0: 2, 50.0: 3, 90.0: 4, 95.0: 5}
+
+
+def _resolve_column(metric: Metric, pct_map: dict) -> Optional[str]:
+    """Resolve the .rnd column for a metric, using the session's percentile map."""
+    if metric.column:
+        return metric.column
+    if metric.percentile is None:
+        return None
+    # pct_map: {"1": "1.0", "2": "10.0", "4": "90.0", ...} → slot : percentile
+    if pct_map:
+        for slot, pval in pct_map.items():
+            try:
+                if float(pval) == metric.percentile:
+                    return f"LN{int(slot)}(Main)"
+            except (ValueError, TypeError):
+                continue
+    slot = _DEFAULT_SLOT_FOR_PCT.get(metric.percentile)
+    return f"LN{slot}(Main)" if slot else None
+
+
+# ---------------------------------------------------------------------------
+# Time windows
+# ---------------------------------------------------------------------------
+
+@dataclass(frozen=True)
+class Window:
+    key: str
+    label: str
+    start_hour: int
+    end_hour: int
+
+    def contains(self, hour: int) -> bool:
+        if self.start_hour < self.end_hour:
+            return self.start_hour <= hour < self.end_hour
+        return hour >= self.start_hour or hour < self.end_hour
+
+
+# Matches the existing Excel report's stats table.
+DEFAULT_WINDOWS: list[Window] = [
+    Window("evening",   "Evening (7PM–10PM)",   19, 22),
+    Window("nighttime", "Nighttime (10PM–7AM)", 22, 7),
+]
+
+# The full night used to select which intervals belong to "last night".
+NIGHT_START_HOUR = 19
+NIGHT_LENGTH_HOURS = 12
+
+
+# ---------------------------------------------------------------------------
+# Aggregation
+# ---------------------------------------------------------------------------
+
+def _aggregate(values: list, method: str) -> Optional[float]:
+    """Collapse a window's interval values into one number per `method`."""
+    vals = [v for v in values if isinstance(v, (int, float))]
+    if not vals:
+        return None
+    if method == "max":
+        return round(max(vals), 1)
+    if method == "log":
+        return round(10 * math.log10(sum(10 ** (v / 10.0) for v in vals) / len(vals)), 1)
+    return round(sum(vals) / len(vals), 1)  # arithmetic
+
+
+def _combine_across_nights(per_night: list, method: str) -> Optional[float]:
+    """Combine per-night window values into a baseline 'typical night' value.
+
+    Arithmetic mean for max/arith metrics (so baseline Lmax = typical nightly
+    peak, the agreed default), logarithmic mean for Leq.
+    """
+    vals = [v for v in per_night if v is not None]
+    if not vals:
+        return None
+    if method == "log":
+        return round(10 * math.log10(sum(10 ** (v / 10.0) for v in vals) / len(vals)), 1)
+    return round(sum(vals) / len(vals), 1)
+
+
+# ---------------------------------------------------------------------------
+# Row gathering
+# ---------------------------------------------------------------------------
+
+def _parse_dt(s: str) -> Optional[datetime]:
+    try:
+        return datetime.strptime(s, "%Y/%m/%d %H:%M:%S")
+    except (ValueError, TypeError):
+        return None
+
+
+def _location_leq_rows(db: Session, location_id: str) -> list[tuple[datetime, dict, dict]]:
+    """All Leq intervals at a location as (interval_dt, row, percentile_map).
+
+    Reuses the same .rnd readers as the report endpoints so parsing stays
+    identical.  Times are the meter's local clock (as written in the file).
+    """
+    # Lazy import avoids a service→router import cycle at module load.
+    from backend.routers.projects import (
+        _read_rnd_file_rows, _normalize_rnd_rows, _is_leq_file, _peek_rnd_headers,
+    )
+    from pathlib import Path
+
+    out: list[tuple[datetime, dict, dict]] = []
+    sessions = db.query(MonitoringSession).filter_by(
+        location_id=location_id, session_type="sound",
+    ).all()
+    for s in sessions:
+        try:
+            meta = json.loads(s.session_metadata or "{}")
+        except (json.JSONDecodeError, TypeError):
+            meta = {}
+        pct_map = meta.get("percentiles", {}) or {}
+        for f in db.query(DataFile).filter_by(session_id=s.id).all():
+            if not f.file_path or not f.file_path.lower().endswith(".rnd"):
+                continue
+            peek = _peek_rnd_headers(Path("data") / f.file_path)
+            if not _is_leq_file(f.file_path, peek):
+                continue
+            rows = _read_rnd_file_rows(f.file_path)
+            rows, _ = _normalize_rnd_rows(rows)
+            for r in rows:
+                dt = _parse_dt(r.get("Start Time", ""))
+                if dt:
+                    out.append((dt, r, pct_map))
+    out.sort(key=lambda t: t[0])
+    return out
+
+
+def _rows_in_night(rows: list, night_date: date) -> list:
+    """Rows falling in the night that *starts* on night_date (19:00 → +12h)."""
+    start = datetime(night_date.year, night_date.month, night_date.day, NIGHT_START_HOUR, 0)
+    end = start + timedelta(hours=NIGHT_LENGTH_HOURS)
+    return [(dt, r, p) for (dt, r, p) in rows if start <= dt < end]
+
+
+def _eligible_nights(rows: list, start_date: date, end_date: date) -> list[date]:
+    """Evening-dates in [start_date, end_date] that actually have night data."""
+    nights = []
+    cur = start_date
+    while cur <= end_date:
+        if _rows_in_night(rows, cur):
+            nights.append(cur)
+        cur += timedelta(days=1)
+    return nights
+
+
+def _window_value(rows: list, metric: Metric, window: Window) -> Optional[float]:
+    """Single aggregated value for one metric over one window of `rows`."""
+    vals = []
+    for dt, r, pct_map in rows:
+        if window.contains(dt.hour):
+            col = _resolve_column(metric, pct_map)
+            if col:
+                vals.append(r.get(col))
+    return _aggregate(vals, metric.agg)
+
+
+# ---------------------------------------------------------------------------
+# Report data model
+# ---------------------------------------------------------------------------
+
+@dataclass
+class CellPair:
+    last_night: Optional[float]
+    baseline: Optional[float]
+
+    @property
+    def delta(self) -> Optional[float]:
+        if self.last_night is None or self.baseline is None:
+            return None
+        return round(self.last_night - self.baseline, 1)
+
+
+@dataclass
+class LocationNightReport:
+    location_id: str
+    location_name: str
+    night_date: date
+    metrics: list[Metric]
+    windows: list[Window]
+    # table[window_key][metric_key] = CellPair
+    table: dict[str, dict[str, CellPair]]
+    interval_series: list[dict]
+    night_interval_count: int
+    baseline_nights_used: int
+    notes: list[str] = field(default_factory=list)
+
+
+def _location_reference_baseline(loc) -> dict:
+    """A location's manually-entered reference baseline, from its metadata.
+
+    Shape: {window_key: {metric_key: float}} e.g. {"nighttime": {"l10": 85.0}}.
+    Used when baseline_mode == "reference" — fixed targets/limits or prior-report
+    averages typed in, rather than computed from captured nights.
+    """
+    if not loc:
+        return {}
+    try:
+        meta = json.loads(loc.location_metadata or "{}")
+    except (json.JSONDecodeError, TypeError):
+        return {}
+    ref = meta.get("report_baseline") or {}
+    out: dict[str, dict[str, float]] = {}
+    if isinstance(ref, dict):
+        for wkey, mvals in ref.items():
+            if not isinstance(mvals, dict):
+                continue
+            clean = {}
+            for mkey, val in mvals.items():
+                try:
+                    clean[mkey] = float(val)
+                except (ValueError, TypeError):
+                    continue
+            if clean:
+                out[wkey] = clean
+    return out
+
+
+def build_location_night_report(
+    db: Session,
+    location_id: str,
+    night_date: date,
+    *,
+    metric_keys: Optional[list[str]] = None,
+    windows: Optional[list[Window]] = None,
+    baseline_mode: str = "captured",
+    baseline_start: Optional[date] = None,
+    baseline_end: Optional[date] = None,
+) -> LocationNightReport:
+    """Build the night-vs-baseline data model for one location.
+
+    `night_date` is the *evening* date of the night being reported (e.g. the
+    7/7 in "night of 7/7 → morning 7/8").  Baseline comes from one of:
+      - "captured": the typical-night value across eligible nights in
+        [baseline_start, baseline_end] (computed from recorded data);
+      - "reference": fixed values typed per location (a spec limit like
+        "L10 = 85", or a prior report's averages).
+    """
+    metric_keys = metric_keys or DEFAULT_METRICS
+    metrics = [METRIC_REGISTRY[k] for k in metric_keys]
+    windows = windows or DEFAULT_WINDOWS
+
+    loc = db.query(MonitoringLocation).filter_by(id=location_id).first()
+    loc_name = loc.name if loc else location_id
+
+    all_rows = _location_leq_rows(db, location_id)
+    night_rows = _rows_in_night(all_rows, night_date)
+
+    reference = _location_reference_baseline(loc) if baseline_mode == "reference" else {}
+
+    baseline_nights: list[date] = []
+    if baseline_mode != "reference" and baseline_start and baseline_end:
+        baseline_nights = _eligible_nights(all_rows, baseline_start, baseline_end)
+        # Don't let the reported night double as its own baseline.
+        baseline_nights = [n for n in baseline_nights if n != night_date]
+
+    table: dict[str, dict[str, CellPair]] = {}
+    for w in windows:
+        table[w.key] = {}
+        for m in metrics:
+            last_night_val = _window_value(night_rows, m, w)
+            if baseline_mode == "reference":
+                baseline_val = reference.get(w.key, {}).get(m.key)
+            elif baseline_nights:
+                per_night = [
+                    _window_value(_rows_in_night(all_rows, nd), m, w)
+                    for nd in baseline_nights
+                ]
+                baseline_val = _combine_across_nights(per_night, m.agg)
+            else:
+                baseline_val = None
+            table[w.key][m.key] = CellPair(last_night_val, baseline_val)
+
+    interval_series = []
+    for dt, r, pct_map in night_rows:
+        entry = {"dt": dt, "time": dt.strftime("%H:%M")}
+        for m in metrics:
+            col = _resolve_column(m, pct_map)
+            val = r.get(col) if col else None
+            entry[m.key] = val if isinstance(val, (int, float)) else None
+        interval_series.append(entry)
+
+    notes: list[str] = []
+    if not night_rows:
+        notes.append(f"No data found for the night of {night_date:%m/%d/%y}.")
+    if baseline_mode == "reference":
+        if not any(reference.values()):
+            notes.append("Reference-baseline mode is on but no reference values are set for this location.")
+    elif (baseline_start or baseline_end) and not baseline_nights:
+        notes.append("No baseline nights with data in the configured range.")
+
+    return LocationNightReport(
+        location_id=location_id,
+        location_name=loc_name,
+        night_date=night_date,
+        metrics=metrics,
+        windows=windows,
+        table=table,
+        interval_series=interval_series,
+        night_interval_count=len(night_rows),
+        baseline_nights_used=len(baseline_nights),
+        notes=notes,
+    )
+
+
+@dataclass
+class ProjectNightReport:
+    project_id: str
+    project_name: str
+    night_date: date
+    metrics: list[Metric]
+    locations: list[LocationNightReport]
+
+
+def build_project_night_report(
+    db: Session,
+    project_id: str,
+    night_date: date,
+    *,
+    metric_keys: Optional[list[str]] = None,
+    windows: Optional[list[Window]] = None,
+    baseline_mode: str = "captured",
+    baseline_start: Optional[date] = None,
+    baseline_end: Optional[date] = None,
+) -> ProjectNightReport:
+    """Build the night report for every active sound location in a project."""
+    metric_keys = metric_keys or DEFAULT_METRICS
+    project = db.query(Project).filter_by(id=project_id).first()
+    project_name = project.name if project else project_id
+
+    locations = db.query(MonitoringLocation).filter_by(
+        project_id=project_id, location_type="sound",
+    ).order_by(MonitoringLocation.sort_order, MonitoringLocation.name).all()
+    locations = [l for l in locations if getattr(l, "removed_at", None) is None]
+
+    reports = [
+        build_location_night_report(
+            db, loc.id, night_date,
+            metric_keys=metric_keys, windows=windows,
+            baseline_mode=baseline_mode,
+            baseline_start=baseline_start, baseline_end=baseline_end,
+        )
+        for loc in locations
+    ]
+
+    return ProjectNightReport(
+        project_id=project_id,
+        project_name=project_name,
+        night_date=night_date,
+        metrics=[METRIC_REGISTRY[k] for k in metric_keys],
+        locations=reports,
+    )

backend/services/report_renderers.py

@@ -0,0 +1,240 @@
+"""
+Nightly Report Renderers.
+
+Pluggable renderers over the `report_pipeline` data model.  v1 ships the HTML
+email body + the Excel attachment; PDF and an inline chart image are v1.1
+(each needs a new dependency).  Keeping renderers separate from the compute
+core means a future report wizard just toggles metrics/renderers — the data
+model is unchanged.
+
+Email-client constraints: the HTML uses a table layout with **inline styles
+only** (no <style> blocks, no external CSS, no fl/grid), which is the reliable
+common denominator across Outlook / Gmail / Apple Mail.
+"""
+
+from __future__ import annotations
+
+from html import escape
+
+from backend.services.report_pipeline import ProjectNightReport, LocationNightReport
+
+
+# Colours: louder-than-baseline reads as a concern (red), quieter as fine (green).
+_RED = "#b00020"
+_GREEN = "#1a7f37"
+_GREY = "#888888"
+
+
+def _fmt_value(v) -> str:
+    return f"{v:.1f}" if isinstance(v, (int, float)) else "—"
+
+
+def _fmt_delta(v) -> str:
+    """Signed delta with colour; positive (louder) = red, negative (quieter) = green."""
+    if not isinstance(v, (int, float)):
+        return f'<span style="color:{_GREY}">—</span>'
+    if v > 0:
+        return f'<span style="color:{_RED}">+{v:.1f}</span>'
+    if v < 0:
+        return f'<span style="color:{_GREEN}">{v:.1f}</span>'
+    return f'<span style="color:{_GREY}">0.0</span>'
+
+
+def _location_table(loc: LocationNightReport) -> str:
+    """One location block: heading + Metric × (window: Last / Base / Δ) table."""
+    th = ('padding:5px 9px;border:1px solid #ccc;background:#f2f2f2;'
+          'font:bold 12px Arial,sans-serif;text-align:center')
+    sub = ('padding:4px 8px;border:1px solid #ccc;background:#fafafa;'
+           'font:11px Arial,sans-serif;text-align:center;color:#555')
+    td = 'padding:4px 9px;border:1px solid #ccc;font:12px Arial,sans-serif;text-align:center'
+    td_l = 'padding:4px 9px;border:1px solid #ccc;font:bold 12px Arial,sans-serif;text-align:left'
+
+    # Top header: blank label cell + each window spanning Last/Base/Δ
+    top = f'<th rowspan="2" style="{th}">Metric (dBA)</th>'
+    for w in loc.windows:
+        top += f'<th colspan="3" style="{th}">{escape(w.label)}</th>'
+    sub_row = ''.join(
+        f'<th style="{sub}">Last</th><th style="{sub}">Base</th><th style="{sub}">&Delta;</th>'
+        for _ in loc.windows
+    )
+
+    body = ''
+    for m in loc.metrics:
+        cells = ''
+        for w in loc.windows:
+            cp = loc.table[w.key][m.key]
+            cells += (f'<td style="{td}">{_fmt_value(cp.last_night)}</td>'
+                      f'<td style="{td}">{_fmt_value(cp.baseline)}</td>'
+                      f'<td style="{td}">{_fmt_delta(cp.delta)}</td>')
+        body += f'<tr><td style="{td_l}">{escape(m.label)}</td>{cells}</tr>'
+
+    meta = (f'{loc.night_interval_count} intervals'
+            + (f' · baseline = {loc.baseline_nights_used} night(s)'
+               if loc.baseline_nights_used else ' · no baseline yet'))
+    notes = ''
+    if loc.notes:
+        notes = ('<div style="font:11px Arial,sans-serif;color:#b00020;margin:2px 0 0">'
+                 + '<br>'.join(escape(n) for n in loc.notes) + '</div>')
+
+    return (
+        f'<h3 style="font:bold 15px Arial,sans-serif;margin:18px 0 4px">{escape(loc.location_name)}</h3>'
+        f'<div style="font:11px Arial,sans-serif;color:#666;margin:0 0 6px">{escape(meta)}</div>'
+        f'<table style="border-collapse:collapse;border:1px solid #ccc">'
+        f'<thead><tr>{top}</tr><tr>{sub_row}</tr></thead>'
+        f'<tbody>{body}</tbody></table>{notes}'
+    )
+
+
+def render_html_summary(report: ProjectNightReport) -> str:
+    """Render the full email-body HTML for a project's night report."""
+    windows_desc = ", ".join(w.label for w in (report.locations[0].windows if report.locations else []))
+    header = (
+        f'<h2 style="font:bold 18px Arial,sans-serif;margin:0 0 2px">'
+        f'{escape(report.project_name)} — Night Report</h2>'
+        f'<div style="font:13px Arial,sans-serif;color:#444;margin:0 0 4px">'
+        f'Night of {report.night_date:%a %m/%d/%y} &nbsp;·&nbsp; last night vs. baseline</div>'
+        f'<div style="font:11px Arial,sans-serif;color:#888;margin:0 0 10px">'
+        f'Windows: {escape(windows_desc)}. '
+        f'&Delta; = last night minus baseline (<span style="color:{_RED}">+ louder</span>, '
+        f'<span style="color:{_GREEN}">− quieter</span>). '
+        f'LAmax = loudest interval; L-values are arithmetic averages; '
+        f'baseline = typical night.</div>'
+    )
+
+    if not report.locations:
+        body = ('<div style="font:13px Arial,sans-serif;color:#b00020">'
+                'No sound locations found for this project.</div>')
+    else:
+        body = ''.join(_location_table(loc) for loc in report.locations)
+
+    footer = ('<div style="font:10px Arial,sans-serif;color:#aaa;margin-top:18px">'
+              'Automated report — Terra-View. Full interval data in the attached spreadsheet.</div>')
+
+    return (f'<!DOCTYPE html><html><body style="margin:0;padding:16px;background:#fff">'
+            f'{header}{body}{footer}</body></html>')
+
+
+# ---------------------------------------------------------------------------
+# Excel renderer (the email attachment) — one sheet per location:
+# interval table + line chart + a Last/Baseline/Δ summary per window.
+# Metric-driven, so it adapts to whatever metric set is configured.
+# ---------------------------------------------------------------------------
+
+def _safe_sheet_name(name: str) -> str:
+    bad = set('[]:*?/\\')
+    cleaned = "".join(c for c in (name or "Location") if c not in bad).strip()
+    return (cleaned or "Location")[:31]
+
+
+def render_excel(report: ProjectNightReport) -> bytes:
+    """Render the night report as an .xlsx (bytes). One worksheet per location."""
+    import io as _io
+    import openpyxl
+    from openpyxl.chart import LineChart, Reference
+    from openpyxl.styles import Font, Alignment, PatternFill, Border, Side
+    from openpyxl.utils import get_column_letter
+
+    wb = openpyxl.Workbook()
+    wb.remove(wb.active)
+
+    f_title = Font(name="Arial", bold=True, size=13)
+    f_h = Font(name="Arial", bold=True, size=10)
+    f_d = Font(name="Arial", size=10)
+    f_note = Font(name="Arial", size=9, italic=True, color="888888")
+    center = Alignment(horizontal="center", vertical="center")
+    hdr_fill = PatternFill("solid", fgColor="F2F2F2")
+    thin = Side(style="thin")
+    box = Border(left=thin, right=thin, top=thin, bottom=thin)
+
+    if not report.locations:
+        ws = wb.create_sheet("No data")
+        ws["A1"] = f"{report.project_name} — no sound locations"
+        ws["A1"].font = f_title
+
+    used_names: set = set()
+    for loc in report.locations:
+        sheet_name = _safe_sheet_name(loc.location_name)
+        n, base = sheet_name, sheet_name
+        i = 2
+        while n in used_names:
+            n = (base[:28] + f"_{i}"); i += 1
+        used_names.add(n)
+        ws = wb.create_sheet(n)
+        metrics = loc.metrics
+
+        ws["A1"] = f"{report.project_name} — Night Report"; ws["A1"].font = f_title
+        ws["A2"] = loc.location_name; ws["A2"].font = f_h
+        ws["A3"] = f"Night of {loc.night_date:%m/%d/%y}  ·  7PM–7AM"; ws["A3"].font = f_d
+
+        # --- interval table ---
+        hr = 5
+        cols = ["Interval #", "Date", "Time"] + [m.label for m in metrics] + ["Comments"]
+        for ci, label in enumerate(cols, 1):
+            c = ws.cell(row=hr, column=ci, value=label)
+            c.font = f_h; c.alignment = center; c.fill = hdr_fill; c.border = box
+        r = hr + 1
+        for idx, entry in enumerate(loc.interval_series, 1):
+            ws.cell(row=r, column=1, value=idx).border = box
+            dt = entry.get("dt")
+            ws.cell(row=r, column=2, value=(dt.strftime("%m/%d/%y") if dt else "")).border = box
+            ws.cell(row=r, column=3, value=entry.get("time", "")).border = box
+            for mi, m in enumerate(metrics):
+                v = entry.get(m.key)
+                cc = ws.cell(row=r, column=4 + mi, value=(v if isinstance(v, (int, float)) else None))
+                cc.border = box; cc.alignment = center
+            ws.cell(row=r, column=4 + len(metrics), value="").border = box
+            r += 1
+        data_end = max(r - 1, hr + 1)
+
+        ws.column_dimensions["A"].width = 9
+        ws.column_dimensions["B"].width = 10
+        ws.column_dimensions["C"].width = 8
+        for mi in range(len(metrics)):
+            ws.column_dimensions[get_column_letter(4 + mi)].width = 11
+        ws.column_dimensions[get_column_letter(4 + len(metrics))].width = 22
+
+        # --- chart ---
+        if loc.interval_series and metrics:
+            chart = LineChart()
+            chart.title = f"{loc.location_name} — {loc.night_date:%m/%d/%y}"
+            chart.y_axis.title = "dBA"; chart.x_axis.title = "Time"
+            chart.height = 9; chart.width = 18
+            data_ref = Reference(ws, min_col=4, max_col=3 + len(metrics), min_row=hr, max_row=data_end)
+            cats = Reference(ws, min_col=3, min_row=hr + 1, max_row=data_end)
+            chart.add_data(data_ref, titles_from_data=True)
+            chart.set_categories(cats)
+            ws.add_chart(chart, f"{get_column_letter(6 + len(metrics))}5")
+
+        # --- summary: Metric × window (Last / Base / Δ) ---
+        sr = data_end + 3
+        ws.cell(row=sr, column=1, value="Summary — last night vs baseline").font = f_h
+        sr += 1
+        ws.cell(row=sr, column=1, value="Metric").font = f_h
+        win_col = {}
+        col = 2
+        for w in loc.windows:
+            c = ws.cell(row=sr, column=col, value=w.label); c.font = f_h; c.alignment = center
+            ws.merge_cells(start_row=sr, start_column=col, end_row=sr, end_column=col + 2)
+            win_col[w.key] = col
+            col += 3
+        sr += 1
+        for w in loc.windows:
+            b = win_col[w.key]
+            for j, lbl in enumerate(["Last", "Base", "Δ"]):
+                cc = ws.cell(row=sr, column=b + j, value=lbl); cc.font = f_h; cc.alignment = center
+        sr += 1
+        for m in metrics:
+            ws.cell(row=sr, column=1, value=m.label).font = f_d
+            for w in loc.windows:
+                cp = loc.table[w.key][m.key]
+                b = win_col[w.key]
+                ws.cell(row=sr, column=b + 0, value=cp.last_night).alignment = center
+                ws.cell(row=sr, column=b + 1, value=cp.baseline).alignment = center
+                ws.cell(row=sr, column=b + 2, value=cp.delta).alignment = center
+            sr += 1
+        if loc.notes:
+            ws.cell(row=sr + 1, column=1, value="; ".join(loc.notes)).font = f_note
+
+    out = _io.BytesIO()
+    wb.save(out)
+    return out.getvalue()

backend/services/scheduler.py

@@ -78,6 +78,9 @@ class SchedulerService:
                 # Execute pending actions
                 await self.execute_pending_actions()
 
+                # Run any due nightly sound reports (FTP report pipeline)
+                await self.run_due_reports()
+
                 # Generate actions from recurring schedules (every hour)
                 now = datetime.utcnow()
                 if (now - last_generation_check).total_seconds() >= 3600:
@@ -306,18 +309,11 @@ class SchedulerService:
         2. Enable FTP
         3. Download measurement folder to SLMM local storage
 
-        After stop_cycle, if download succeeded, this method fetches the ZIP
-        from SLMM and extracts it into Terra-View's project directory, creating
-        DataFile records for each file.
+        After stop_cycle, if download succeeded, this method ingests the folder
+        into Terra-View through the shared NRL ingest (same path as cycle and the
+        manual SD-card upload) so the resulting session is Leq-only, has its
+        `.rnh` parsed (percentile slot map + weightings), and is deduped.
         """
-        import hashlib
-        import io
-        import os
-        import zipfile
-        import httpx
-        from pathlib import Path
-        from backend.models import DataFile
-
         # Parse notes for download preference
         include_download = True
         try:
@@ -362,79 +358,39 @@ class SchedulerService:
 
         db.commit()
 
-        # If SLMM downloaded the folder successfully, fetch the ZIP from SLMM
-        # and extract it into Terra-View's project directory, creating DataFile records
-        files_created = 0
-        if include_download and cycle_response.get("download_success") and active_session:
+        # If SLMM downloaded the folder successfully, ingest it into Terra-View
+        # through the shared NRL ingest (the same path cycle and the manual SD
+        # upload use): keeps only the .rnh + Leq .rnd, parses the header
+        # (percentile slot map + weightings), dedups, and links the unit.  The
+        # transient "recording" marker session is dropped in favour of the clean
+        # ingested row.  (Replaces the old inline unzip that stored every file —
+        # incl. the 1-second _Lp_ data — without parsing the .rnh.)
+        ingest_result = None
+        ingested_session_id = None
+        if (include_download and cycle_response.get("download_success")
+                and active_session and action.device_type == "slm"):
             folder_name = cycle_response.get("downloaded_folder")  # e.g. "Auto_0058"
-            remote_path = f"/NL-43/{folder_name}"
-
+            if folder_name:
                 try:
-                SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
-                async with httpx.AsyncClient(timeout=600.0) as client:
-                    zip_response = await client.post(
-                        f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download-folder",
-                        json={"remote_path": remote_path}
+                    ingest_result = await self._ingest_and_link(
+                        db,
+                        location_id=action.location_id,
+                        unit_id=unit_id,
+                        folder_name=folder_name,
+                        placeholder_session=active_session,
                     )
-
-                if zip_response.is_success and len(zip_response.content) > 22:
-                    base_dir = Path(f"data/Projects/{action.project_id}/{active_session.id}/{folder_name}")
-                    base_dir.mkdir(parents=True, exist_ok=True)
-
-                    file_type_map = {
-                        '.wav': 'audio', '.mp3': 'audio',
-                        '.csv': 'data', '.txt': 'data', '.json': 'data', '.dat': 'data',
-                        '.rnd': 'data', '.rnh': 'data',
-                        '.log': 'log',
-                        '.zip': 'archive',
-                        '.jpg': 'image', '.jpeg': 'image', '.png': 'image',
-                        '.pdf': 'document',
-                    }
-
-                    with zipfile.ZipFile(io.BytesIO(zip_response.content)) as zf:
-                        for zip_info in zf.filelist:
-                            if zip_info.is_dir():
-                                continue
-                            file_data = zf.read(zip_info.filename)
-                            file_path = base_dir / zip_info.filename
-                            file_path.parent.mkdir(parents=True, exist_ok=True)
-                            with open(file_path, 'wb') as f:
-                                f.write(file_data)
-                            checksum = hashlib.sha256(file_data).hexdigest()
-                            ext = os.path.splitext(zip_info.filename)[1].lower()
-                            data_file = DataFile(
-                                id=str(uuid.uuid4()),
-                                session_id=active_session.id,
-                                file_path=str(file_path.relative_to("data")),
-                                file_type=file_type_map.get(ext, 'data'),
-                                file_size_bytes=len(file_data),
-                                downloaded_at=datetime.utcnow(),
-                                checksum=checksum,
-                                file_metadata=json.dumps({
-                                    "source": "stop_cycle",
-                                    "remote_path": remote_path,
-                                    "unit_id": unit_id,
-                                    "folder_name": folder_name,
-                                    "relative_path": zip_info.filename,
-                                }),
-                            )
-                            db.add(data_file)
-                            files_created += 1
-
-                    db.commit()
-                    logger.info(f"Created {files_created} DataFile records for session {active_session.id} from {folder_name}")
-                else:
-                    logger.warning(f"ZIP from SLMM for {folder_name} was empty or failed, skipping DataFile creation")
-
+                    ingested_session_id = ingest_result.get("session_id")
+                    logger.info(f"[STOP] Ingested {folder_name}: {ingest_result}")
                 except Exception as e:
-                logger.error(f"Failed to extract ZIP and create DataFile records for {folder_name}: {e}")
+                    logger.error(f"Failed to ingest {folder_name} on stop: {e}", exc_info=True)
                     # Don't fail the stop action — the device was stopped successfully
+                    ingest_result = {"success": False, "error": str(e)}
 
         return {
             "status": "stopped",
-            "session_id": active_session.id if active_session else None,
+            "session_id": ingested_session_id or (active_session.id if active_session else None),
             "cycle_response": cycle_response,
-            "files_created": files_created,
+            "ingest": ingest_result,
         }
 
     async def _execute_download(
@@ -487,12 +443,38 @@ class SchedulerService:
             files=None,  # Download all files in current measurement folder
         )
 
-        # TODO: Create DataFile records for downloaded files
+        # Ingest the downloaded folder into Terra-View via the shared NRL ingest
+        # (same path as stop/cycle): clean Leq-only session, .rnh parsed
+        # (percentiles + weightings), deduped, unit linked.  No placeholder
+        # session here — a standalone download isn't tied to a "recording" marker.
+        ingest_result = None
+        if action.device_type == "slm":
+            folder_name = (response or {}).get("folder_name")
+            if not folder_name:
+                try:
+                    folder_name = f"Auto_{int((response or {}).get('index_number')):04d}"
+                except (ValueError, TypeError):
+                    folder_name = None
+            if not folder_name:
+                ingest_result = {"success": False, "error": "no folder_name/index_number from download"}
+            else:
+                try:
+                    ingest_result = await self._ingest_and_link(
+                        db,
+                        location_id=action.location_id,
+                        unit_id=unit_id,
+                        folder_name=folder_name,
+                    )
+                    logger.info(f"[DOWNLOAD] Ingested {folder_name}: {ingest_result}")
+                except Exception as e:
+                    logger.error(f"Failed to ingest {folder_name} on download: {e}", exc_info=True)
+                    ingest_result = {"success": False, "error": str(e)}
 
         return {
             "status": "downloaded",
             "destination_path": destination_path,
             "device_response": response,
+            "ingest": ingest_result,
         }
 
     async def _execute_cycle(
@@ -633,6 +615,36 @@ class SchedulerService:
                 )
                 result["old_session_id"] = active_session.id
 
+            # Step 4b: Ingest the just-finished Auto_#### folder into Terra-View
+            # (clean session + DataFiles via ingest_nrl_zip — filters Lp, parses the
+            # .rnh, dedups).  This is what gives the nightly report its data.
+            if action.device_type == "slm" and result["steps"].get("download", {}).get("success"):
+                idx = None
+                try:
+                    idx = int((result["steps"]["download"].get("response") or {}).get("index_number"))
+                except (ValueError, TypeError):
+                    idx = None
+                if idx is None:
+                    result["steps"]["ingest"] = {"success": False, "error": "no index_number from download"}
+                else:
+                    folder_name = f"Auto_{idx:04d}"
+                    try:
+                        ing = await self._ingest_and_link(
+                            db,
+                            location_id=action.location_id,
+                            unit_id=unit_id,
+                            folder_name=folder_name,
+                            placeholder_session=active_session,
+                        )
+                        result["steps"]["ingest"] = ing
+                        # The marker session was dropped; repoint old_session_id at the real row.
+                        if ing.get("placeholder_dropped") and ing.get("session_id"):
+                            result["old_session_id"] = ing["session_id"]
+                        logger.info(f"[CYCLE] Ingested {folder_name}: {ing}")
+                    except Exception as e:
+                        logger.error(f"[CYCLE] Ingest failed for {folder_name}: {e}", exc_info=True)
+                        result["steps"]["ingest"] = {"success": False, "error": str(e)}
+
             # Step 5: Wait for device to settle before starting new measurement
             logger.info(f"[CYCLE] Step 5/7: Waiting 30s for device to settle...")
             await asyncio.sleep(30)
@@ -667,6 +679,54 @@ class SchedulerService:
 
                 logger.info(f"[CYCLE] New measurement started, session {new_session.id}")
 
+                # Step 6b: Verify the meter actually resumed measuring (fresh DOD).
+                # Polling is still paused here, so query directly.  If it didn't
+                # resume, retry ONCE with a plain start (start_recording — does NOT
+                # re-index, unlike start_cycle) before alerting: a meter left
+                # stopped overnight is the costly failure, and a transient restart
+                # hiccup is common on the NL-43.  We retry only on a *confident*
+                # not-measuring reading — never on a failed/inconclusive DOD read —
+                # so a flaky read can't disrupt an already-running measurement.
+                if action.device_type == "slm":
+                    async def _check_measuring():
+                        """Return (measuring, state); measuring is None if the DOD read failed."""
+                        try:
+                            await asyncio.sleep(2)
+                            live = await self.device_controller.get_live_data(unit_id, action.device_type)
+                            state = ((live or {}).get("measurement_state")
+                                     or ((live or {}).get("data") or {}).get("measurement_state") or "")
+                            ok = str(state).strip().lower() in ("start", "measure", "measuring", "run", "running")
+                            return ok, state
+                        except Exception as e:
+                            logger.warning(f"[CYCLE] Restart-verify DOD read failed: {e}")
+                            return None, None
+
+                    measuring, state = await _check_measuring()
+                    if measuring is False:
+                        logger.warning(f"[CYCLE] {unit_id} not measuring after restart (state={state!r}) — retrying start once.")
+                        result["steps"]["restart_retry"] = True
+                        try:
+                            await self.device_controller.start_recording(unit_id, action.device_type)
+                            measuring, state = await _check_measuring()
+                        except Exception as e:
+                            logger.error(f"[CYCLE] Restart retry (start_recording) failed for {unit_id}: {e}")
+
+                    result["steps"]["restart_verified"] = measuring
+                    if measuring:
+                        logger.info(f"[CYCLE] Restart verified — {unit_id} is measuring (state={state}).")
+                    elif measuring is False:
+                        logger.error(f"[CYCLE] Restart NOT verified for {unit_id} after retry — state={state!r}")
+                        try:
+                            get_alert_service(db).create_schedule_failed_alert(
+                                schedule_id=action.id, action_type="cycle", unit_id=unit_id,
+                                error_message=f"Meter did not resume measuring after the cycle + one retry (state={state!r}).",
+                                project_id=action.project_id, location_id=action.location_id,
+                            )
+                        except Exception as ae:
+                            logger.warning(f"[CYCLE] restart-verify alert failed: {ae}")
+                    else:
+                        logger.warning(f"[CYCLE] Restart verification inconclusive for {unit_id} (DOD read failed); keepalive poll will re-confirm.")
+
             except Exception as e:
                 logger.error(f"[CYCLE] Start failed: {e}")
                 result["steps"]["start"] = {"success": False, "error": str(e)}
@@ -689,6 +749,85 @@ class SchedulerService:
         logger.info(f"[CYCLE] === Cycle complete for {unit_id} ===")
         return result
 
+    async def _ingest_cycle_folder(self, db, location_id: str, unit_id: str, folder_name: str) -> dict:
+        """Fetch a just-finished Auto_#### folder from SLMM (FTP proxy) and ingest
+        it into Terra-View (clean MonitoringSession + DataFiles via ingest_nrl_zip).
+
+        Returns the ingest result dict, or {"success": False, "error": ...}.
+        Used by _execute_cycle Step 4b.
+        """
+        import os
+        import httpx
+        from backend.routers.project_locations import ingest_nrl_zip, IngestError
+
+        slmm_base = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
+        remote_path = f"/NL-43/{folder_name}"
+        try:
+            async with httpx.AsyncClient(timeout=600.0) as client:
+                resp = await client.post(
+                    f"{slmm_base}/api/nl43/{unit_id}/ftp/download-folder",
+                    json={"remote_path": remote_path},
+                )
+        except Exception as e:
+            return {"success": False, "error": f"download-folder request failed: {e}"}
+
+        if not resp.is_success or len(resp.content) <= 22:   # 22 bytes = empty-zip
+            return {"success": False, "error": f"empty/failed ZIP from SLMM (status {resp.status_code})"}
+
+        try:
+            res = ingest_nrl_zip(location_id, resp.content, db, source="ftp_cycle", dedupe=True)
+            return {"success": True, **res}
+        except IngestError as e:
+            return {"success": False, "error": str(e)}
+
+    async def _ingest_and_link(
+        self,
+        db,
+        *,
+        location_id: str,
+        unit_id: str,
+        folder_name: str,
+        placeholder_session=None,
+    ) -> dict:
+        """Ingest a just-finished Auto_#### folder and tie it to the unit.
+
+        This is the ONE ingest path that stop / cycle / download all funnel
+        through, so every route produces the same clean session: Leq-only,
+        `.rnh` parsed (percentile slot map + weightings captured), deduped.
+
+        Steps:
+          1. Fetch + ingest the folder via the shared NRL ingest
+             (`_ingest_cycle_folder` → `ingest_nrl_zip`).
+          2. `ingest_nrl_zip` leaves `unit_id` None — link it to the unit that
+             recorded the data so the session stays attributed.
+          3. If a `placeholder_session` (the transient "recording" marker) was
+             passed and it never accumulated DataFiles of its own, drop it — its
+             data now lives in the ingested, unit-linked session.
+
+        Returns the ingest result dict (with `success`); adds `placeholder_dropped`
+        when step 3 removed the marker.  On ingest failure the placeholder is
+        left untouched.
+        """
+        ing = await self._ingest_cycle_folder(db, location_id, unit_id, folder_name)
+        if not ing.get("success"):
+            return ing
+
+        sid = ing.get("session_id")
+        if sid:
+            s = db.query(MonitoringSession).filter_by(id=sid).first()
+            if s and not s.unit_id:
+                s.unit_id = unit_id
+                db.commit()
+
+        if placeholder_session is not None and sid:
+            from backend.models import DataFile
+            if db.query(DataFile).filter_by(session_id=placeholder_session.id).count() == 0:
+                db.delete(placeholder_session)
+                db.commit()
+                ing["placeholder_dropped"] = True
+
+        return ing
+
     # ========================================================================
     # Recurring Schedule Generation
     # ========================================================================
@@ -782,6 +921,92 @@ class SchedulerService:
 
         return cleaned
 
+    # ========================================================================
+    # Nightly Sound Report (FTP report pipeline)
+    # ========================================================================
+
+    async def run_due_reports(self):
+        """Run any project nightly sound reports that are due.
+
+        For each enabled SoundReportConfig: if local time is past report_time
+        and we haven't already reported last night, build the report (writes a
+        file always; emails if SMTP is configured, else dry-run) and stamp
+        last_run_date.  Idempotent across restarts via last_run_date.
+        """
+        from backend.models import SoundReportConfig
+        from backend.utils.timezone import utc_to_local
+
+        # Decide what's due (cheap, on the loop); run each OFF the event loop.
+        due_jobs = []
+        db = SessionLocal()
+        try:
+            configs = db.query(SoundReportConfig).filter_by(enabled=True).all()
+            if not configs:
+                return
+            local_now = utc_to_local(datetime.utcnow())
+            night_date = local_now.date() - timedelta(days=1)   # last night's evening date
+            for cfg in configs:
+                try:
+                    hh, mm = (int(x) for x in cfg.report_time.split(":"))
+                except (ValueError, AttributeError):
+                    hh, mm = 8, 0
+                if (local_now.hour, local_now.minute) < (hh, mm):
+                    continue
+                if cfg.last_run_date == night_date:
+                    continue
+                due_jobs.append({
+                    "project_id": cfg.project_id,
+                    "metric_keys": [m.strip() for m in (cfg.metric_keys or "").split(",") if m.strip()] or None,
+                    "recipients": [r.strip() for r in (cfg.recipients or "").split(",") if r.strip()] or None,
+                    "baseline_mode": cfg.baseline_mode,
+                    "baseline_start": cfg.baseline_start,
+                    "baseline_end": cfg.baseline_end,
+                })
+        finally:
+            db.close()
+
+        # run_nightly_report is synchronous (blocking file I/O + smtplib up to the
+        # SMTP timeout). Run it in a worker thread so it never stalls the scheduler
+        # loop (which also drives time-sensitive device cycles).
+        for job in due_jobs:
+            try:
+                logger.info(f"[REPORT] Running nightly report for project {job['project_id']} (night {night_date})")
+                result = await asyncio.to_thread(self._run_one_report, night_date, job)
+                email = (result or {}).get("email", {})
+                logger.info(
+                    f"[REPORT] project {job['project_id']}: {(result or {}).get('location_count')} location(s); "
+                    f"email={'sent' if email.get('sent') else ('dry-run' if email.get('dry_run') else (email.get('error') or 'skipped'))}"
+                )
+            except Exception as e:
+                logger.error(f"[REPORT] Failed nightly report for project {job['project_id']}: {e}", exc_info=True)
+
+    def _run_one_report(self, night_date, job) -> Dict[str, Any]:
+        """Sync worker: build/send one project's report and stamp last_run_date.
+        Uses its own DB session (runs in a thread, off the event loop)."""
+        from backend.models import SoundReportConfig
+        from backend.services.report_orchestrator import run_nightly_report
+
+        db = SessionLocal()
+        try:
+            result = run_nightly_report(
+                db, job["project_id"], night_date,
+                metric_keys=job["metric_keys"],
+                baseline_mode=job["baseline_mode"],
+                baseline_start=job["baseline_start"],
+                baseline_end=job["baseline_end"],
+                recipients=job["recipients"],
+            )
+            cfg = db.query(SoundReportConfig).filter_by(project_id=job["project_id"]).first()
+            if cfg:
+                cfg.last_run_date = night_date
+            db.commit()
+            return result
+        except Exception:
+            db.rollback()
+            raise
+        finally:
+            db.close()
+
     # ========================================================================
     # Manual Execution (for testing/debugging)
     # ========================================================================

docs/CLIENT_PORTAL.md

@@ -2,12 +2,6 @@
 
 **Status:** in development (`feat/client-portal`) · **Targets:** 0.14.x
 
-> **Update (Phase-1 auth landed):** the interim magic-link gate described below is
-> **retired** — client access is now a per-project secure link + shared password
-> (argon2). See the design at `docs/superpowers/specs/2026-06-15-portal-auth-design.md`
-> and the build plan at `docs/superpowers/plans/2026-06-15-portal-auth.md`. The
-> operator manages access from each project's **Portal access** panel.
-
 A client-facing, **read-only**, **scoped** view into a client's own monitoring
 data. The first internet-facing-with-real-clients surface in the system. Built
 *inside* the Terra-View app (new `/portal/*` namespace), reusing the cached SLMM

docs/adr/0001-device-data-ownership.md

@@ -0,0 +1,57 @@
+# ADR 0001 — Device data ownership: modules own raw data, Terra-View owns fleet context
+
+- **Status:** Accepted (SLMM grandfathered as a known exception — see Consequences)
+- **Date:** 2026-06-16
+- **Deciders:** Brian
+- **Applies to:** Terra-View core and all device modules (SFM, SLMM, and future modules)
+
+## Context
+
+Terra-View is a fleet-management / UI layer that talks to specialized **device modules**, each of which speaks one device's protocol (see the architecture note in `CLAUDE.md`). Two modules exist today, and they store their data **differently**:
+
+- **SFM (seismograph / seismo-relay).** Owns its own database **and** waveform store. Terra-View holds **no** seismic event or waveform data — it reads through live, e.g. `GET {SFM_BASE_URL}/db/events` (`backend/routers/activity.py`, `backend/routers/admin_modules.py`). Terra-View renders; SFM persists.
+- **SLMM (sound level meters).** A thin device-control shim. The sound **measurement data is stored in Terra-View** — `MonitoringSession` + `DataFile` rows in `data/seismo_fleet.db`, and the `.rnh` + Leq `.rnd` files under `data/Projects/{project_id}/{session_id}/` (`backend/routers/project_locations.py:_ingest_file_entries`). SLMM only keeps device config + a live-status cache (`slmm.db`) and a transient download staging area (`data/downloads/{unit_id}/`).
+
+This inconsistency is real, not cosmetic. It raises an obvious question every time we add a feature or a module: *where does this device's data live?* Without a stated rule, the answer drifts per-module, which is exactly how conceptual integrity erodes.
+
+### Why the asymmetry exists (history, not sloppiness)
+
+1. **Path dependence.** seismo-relay pre-existed as a complete data system; Terra-View integrated *with* it. SLMM was built fresh as a control shim, so persistence drifted up into Terra-View.
+2. **Coupling.** A seismic event is largely self-contained — Terra-View just tags it to a unit. A Leq interval is only meaningful against an NRL location + baseline + report config, which are **Terra-View concepts**. Sound data has stronger natural gravity toward Terra-View ownership than seismic events do.
+
+## Decision
+
+Adopt one explicit ownership rule for all device data:
+
+> **The device module owns the raw device data (waveforms, events, Leq files, raw telemetry). Terra-View owns the fleet/project/location/session/report context that gives that data meaning.**
+
+Note this is **not** "Terra-View stores nothing" — Terra-View remains the system of record for roster, projects, locations, deployments, history, schedules, and the associations between fleet entities and module-owned data. What it should **not** own is a second copy of raw device telemetry.
+
+**Litmus test for any "where does this live?" call:** *whose question does this data answer?*
+- "What did the sensor record?" (raw waveform / Leq rows) → **the module**.
+- "Which NRL, which night, versus which baseline?" (context) → **Terra-View**.
+
+### Application
+
+- **New device modules MUST follow the SFM pattern**: the module owns its data and exposes a read API (`/db/*` or equivalent); Terra-View references it and reads through, rather than ingesting a copy.
+- **SFM** already conforms. No change.
+- **SLMM does not conform** and is explicitly **grandfathered** (see Consequences).
+
+## Consequences
+
+**Positive**
+- Consistent module boundaries → lower cognitive load, fewer "which copy is authoritative?" bugs.
+- Terra-View stays thin; "add a device type = add a module" stays true (the CLAUDE.md north star).
+- Single source of truth for raw data; no silent duplication.
+
+**Negative / costs**
+- Realigning **SLMM** to this rule is a non-trivial refactor: move ingest + file storage into SLMM, build a SLMM read API, repoint the report engine and the Data Files UI to read through it, handle the session↔location association across the module boundary, and migrate existing `MonitoringSession`/`DataFile` data. The FTP night-report pipeline currently **assumes Terra-View ownership**.
+
+**SLMM grandfather clause**
+- SLMM stays as-is for now. Realignment is a **deliberate future project**, not a background cleanup, and should be triggered by a real signal — e.g. a 3rd device type arriving, or the duplication/coupling actually causing pain. Until then, Terra-View remains the system of record for sound data, and that is an accepted, documented exception rather than an aspiration.
+- The current sound data flow (for reference): `NL-43 SD card → (FTP) → SLMM data/downloads/ → (proxy ZIP) → Terra-View ingest → data/Projects/ + seismo_fleet.db`. The 1-second `_Lp_` files are dropped at ingest and never land in Terra-View.
+
+## Related
+
+- `CLAUDE.md` — module architecture ("Terra-View does NOT communicate directly with physical devices").
+- FTP night-report pipeline (`feat/ftp-report-pipeline`) — built on the current SLMM/Terra-View-ownership model; a future SLMM realignment would need to repoint it.

docs/superpowers/specs/2026-06-15-portal-auth-design.md

@@ -1,237 +0,0 @@
-# Portal Authentication — Design & Build Plan
-
-**Status:** in development (`feat/portal-auth`) · **Targets:** 0.14.x · **Date:** 2026-06-15
-
-Supersedes the interim shareable magic-link described in
-[CLIENT_PORTAL.md](../../CLIENT_PORTAL.md) with a real password gate.
-
-## Goal
-
-Give a client a **secure link + password** that opens a **read-only dashboard** —
-live data plus access to historical data — for the machines commissioned on
-**their project**. Nothing else: no device control, no editing, no internal pages.
-
-This is the first real, internet-facing, client-credentialed surface in the
-system.
-
-## Scope
-
-**Phase 1 (this spec — build now):** per-project, password-gated, read-only portal.
-
-**Deferred (designed, not built — captured below so nothing is lost):**
-- **Operator auth** — logins + roles for the *internal* app (you / parents).
-  Full design in [Deferred A](#deferred-a--operator-auth-designed-not-built).
-- **Full multi-tenancy** — per-client rollups, per-project separation within a
-  client, individual client user accounts, and extending the portal to all
-  client-relevant data. [Deferred B](#deferred-b--full-multi-tenancy).
-
-## Principles (the portal's standing charter)
-
-1. **Read-only.** A client can look, never touch.
-2. **Strictly scoped, server-side.** Never trust a project / location / unit id
-   from the request — always re-resolve ownership.
-3. **Cache-first.** Portal live data comes from SLMM's cache (the same cached
-   reads the internal dashboard uses). A client can never make us hit the device.
-4. **The gate is a swappable seam.** Everything routes through the scoping layer
-   the portal already has; auth is the thin thing in front of it.
-
-## The model
-
-- **Tenant unit = the project.** Each project is its own portal: one link, one
-  password, showing that project's commissioned machines.
-- **Shared credential — "company / project-manager wide."** No individual client
-  accounts. Because access is read-only, one shared password per project is an
-  acceptable trade. (Per-person accounts are a Deferred-B item.)
-- **The link identifies the project; the password authorizes.** A password alone
-  can't say *which* project — so the link carries an unguessable, revocable
-  per-project token, and the password is the shared secret gating it.
-
-## Architecture
-
-Two layers, two subdomains (hosting target: office Synology NAS behind a UniFi
-UXG Max; own domain `terra-mechanics.com`).
-
-```
-Internet
-   │
-UniFi UXG Max   ── Layer 1 (IT pro): firewall, IPS/IDS, GeoIP allow-list,
-   │                kill-switch rule, 443 only
-Synology NAS    ── DSM reverse proxy + Let's Encrypt wildcard TLS
-   │
-   ├─ terra-view.terra-mechanics.com → internal app  (operator auth = Deferred A)
-   └─ portal.terra-mechanics.com     → LOCKED to /portal/* only, password gate
-```
-
-The portal subdomain is **restricted to `/portal/*` at the reverse proxy** — a
-client on `portal.` physically cannot reach `/roster`, `/admin/*`, etc., even by
-guessing URLs. This path-lock is a load-bearing control for as long as the
-internal app remains unauthenticated (until Deferred A lands).
-
-## Data model
-
-Add three columns to **`Project`**:
-
-| Column | Type | Purpose |
-|---|---|---|
-| `portal_enabled` | bool, default `false` | Is the portal open for this project. |
-| `portal_password_hash` | text, nullable | argon2id hash of the shared password. Never plaintext. |
-| `portal_link_token` | text, unique, nullable | Unguessable token in the secure link; identifies the project without exposing its raw id, and is revocable (regenerate → old link dies). |
-
-**Reused unchanged:** the `Client → Project → MonitoringLocation →
-UnitAssignment → unit` scoping chain and the existing read-only scoped data
-routes (`resolve_client_location` + live / history / events).
-
-**Migration:** `migrate_add_project_portal_auth.py` — an `ALTER TABLE` adding the
-three columns to the existing (non-empty) `projects` table. Same pattern as
-`migrate_add_client_portal.py`; `create_all` won't add columns to an existing
-table.
-
-## Auth flow
-
-1. **Operator enables + shares.** On the project page, the operator turns the
-   portal on; the system generates a strong password + a `portal_link_token`; the
-   operator copies **link + password** to send the client.
-2. **Client opens the link** `portal.terra-mechanics.com/portal/p/{link_token}` →
-   the project is resolved from the token → a **password prompt** renders.
-3. **Client submits the password** → argon2-verified against
-   `portal_password_hash`. On success, a **signed session cookie scoped to that
-   project** is set (HMAC via the existing `SECRET_KEY` cookie machinery), and
-   they are redirected to the project dashboard.
-4. **Subsequent requests** re-validate the cookie (signature + project still
-   `portal_enabled` + within cookie max-age) and serve the existing read-only
-   scoped data.
-5. **Logout** clears the cookie. **Revoke** = disable the portal or regenerate the
-   token / password, which kills outstanding links and any session minted from
-   them on the next request.
-
-**Lockout:** track failed attempts (per token + IP); after 5 failures refuse for
-a 15-minute cooldown. Combined with the UniFi GeoIP/IPS edge, that's solid for a
-read-only surface.
-
-**Shared cookie machinery:** lift the portal's cookie sign/verify out of
-`portal_auth.py` into a small shared `backend/auth_cookies.py` — one signer, so
-the future operator auth (Deferred A) reuses it instead of copy-pasting crypto.
-
-### Relationship to the existing portal code
-
-The portal today is *client-scoped* (a `ClientAccessToken` magic-link → a cookie
-covering all of a client's projects, with a `/portal` overview). Phase 1 makes the
-entry point *project-scoped*:
-
-- The **`/portal/p/{link_token}` + password** flow becomes the way in; the
-  interim client magic-link (`/portal/enter/{token}`, `/portal/open/*`,
-  `PORTAL_OPEN_LINKS`) is **retired** in its favor.
-- The existing read-only views (`/portal/location/{id}`, live / history / events)
-  and the scoping helper are **reused as-is**, just resolved against the project in
-  the session cookie instead of the client.
-- `Client` / `ClientAccessToken` rows are **left in place** (no destructive
-  migration) — they become the substrate for the Deferred-B per-client rollup.
-
-## Operator "Portal access" panel
-
-On the project detail page (internal app), a panel that:
-- Toggles `portal_enabled`.
-- **Regenerate password** → shows a freshly generated strong password **once** for
-  the operator to copy.
-- **Copy link** → the `/portal/p/{token}` URL.
-- **Revoke** → regenerate the token (old link dies) and/or disable the portal.
-
-This is an operator action. Until operator auth lands (Deferred A), it sits behind
-the same posture as the rest of the internal app — see Security notes.
-
-## Error handling
-
-- **Bad password** → generic "incorrect password" + increment fail count.
-- **Unknown / disabled / revoked token** → generic "this portal link is no longer
-  active" page (no project-existence leak).
-- **Locked out** → "too many attempts, try again in 15 minutes."
-- **Expired / invalid cookie** → back to the password prompt.
-- **Portal disabled after a session started** → next request bounced to the prompt.
-
-## Rollout
-
-1. Implement on `feat/portal-auth` → review → merge to `dev`.
-2. **Migration** `migrate_add_project_portal_auth.py` on each DB (dev + prod), same
-   drill as the client-portal migration.
-3. **`SECRET_KEY`** must be a real value in prod (already required for the existing
-   portal cookie; the password gate reuses it).
-4. **Hosting:** DSM reverse proxy routes `portal.` → app, locked to `/portal/*`;
-   Let's Encrypt wildcard TLS; cookies `Secure` once on TLS. UXG Max GeoIP + IPS +
-   kill-switch handled by the IT pro.
-5. Enable a real project's portal, set a password, and test the full
-   link → password → dashboard flow over HTTPS before sending a client.
-
-## Testing
-
-- **Unit:** argon2 hash/verify; token resolution (valid / unknown / disabled);
-  lockout counter; cookie sign/verify + scope check; "disabled mid-session" bounce.
-- **Scoping:** a session for project A cannot read project B's locations / history
-  / events (404, no existence leak).
-- **Manual smoke:** enable → copy link + password → open in a fresh browser →
-  wrong password (lockout) → right password → see live + history → logout.
-
----
-
-## Deferred A — Operator auth (designed, not built)
-
-Logins + roles for the **internal** app (`terra-view.` subdomain). Closes the
-"internal app is wide open" hole. Full design, ready to lift into its own spec:
-
-- **Two layers:** UniFi UXG Max edge (IT-pro owned — firewall, IPS, GeoIP,
-  kill-switch, 443-only) + in-app auth (built by us). Internet-exposed with login
-  (no VPN — deliberately, to spare non-technical family members).
-- **`OperatorUser` model:** `id, email (unique, lowercased), display_name,
-  password_hash (argon2id), role, active, created_at, last_login_at,
-  sessions_valid_from, failed_login_count, locked_until` (+ later `totp_secret`,
-  `totp_enabled`).
-- **Role ladder:** `superadmin > admin > operator`.
-  - `superadmin` = you — everything + account management (create/disable users,
-    reset passwords, assign roles).
-  - `admin` = your parents (company owners) + you — full run of the app, no
-    operational restrictions.
-  - `operator` = **future** restricted tier for hires; the ladder accepts it with
-    no route changes.
-  - The only thing gated above plain `admin` in v1 is account management
-    (`superadmin`).
-- **Sessions:** stateless signed cookie reusing `auth_cookies.py` + `SECRET_KEY`
-  (distinct cookie name from the portal). `sessions_valid_from` gives "log out
-  everywhere" / revoke-on-password-change with no session table.
-- **Authorization:** one **deny-by-default middleware** gates the whole internal
-  app (exempt: `/login`, `/logout`, `/health`, `/static/*`, `/portal/*`);
-  `require_role("admin"|"superadmin")` guards specific routes. New routes are
-  protected automatically.
-- **Lockout:** 5 fails → 15-min cooldown (doubling).
-- **2FA:** deferred; TOTP later, admin/superadmin account first.
-- **Safe rollout (no self-lockout):** ship behind a feature flag
-  `OPERATOR_AUTH_ENABLED` (default **off** = app behaves as today) → seed the first
-  `superadmin` via a small CLI (`backend/operator_admin.py`, modeled on
-  `portal_admin.py`) → log in while still open → flip the flag on → create
-  parents' accounts. Flag back off = instant escape hatch; break-glass =
-  re-run seed / `reset-password` CLI in the container.
-- **`OperatorUser` is a brand-new table** → `create_all` builds it on startup; only
-  the seed step is required.
-
-## Deferred B — Full multi-tenancy
-
-- Per-client **rollup**: one login spanning all of a client's projects.
-- Per-project **separation within a client** (true tenant isolation).
-- **Individual client user accounts** (per-person, optional roles) replacing the
-  shared per-project password.
-- Extend the portal to **all client-relevant data types** (beyond sound:
-  vibration, reports, etc.) — the long-term goal of "everything we can show a
-  client."
-- All additive on the existing scoping seam — no teardown.
-
-## Security notes
-
-- Auth-gated from day one (even the shared password) — never wide-open like the
-  internal app currently is.
-- Scoping enforced server-side; client-supplied ids always re-checked.
-- Passwords argon2-hashed; link tokens unguessable + revocable; raw password shown
-  once.
-- `SECRET_KEY` a real secret in prod; cookies `HttpOnly` + `SameSite=Lax` +
-  `Secure` (once on TLS).
-- **Known risk:** the operator "Portal access" panel — and the whole internal app —
-  is unauthenticated until Deferred A. Mitigated for now by the `/portal/*`
-  path-lock on the public subdomain plus keeping the internal app off the public
-  internet. Tracked in the hardening backlog (CLIENT_PORTAL.md).

report-changelog.md

@@ -0,0 +1,54 @@
+# FTP Night-Report Pipeline — changelog entry
+
+> **How to use:** paste the block below into Terra-View's `CHANGELOG.md`.
+> The current `[Unreleased]` section targets **0.14.0** (SLM live monitoring); this
+> is a separate, larger feature, so it's drafted here as **0.15.0** — fold it into
+> `[Unreleased]` or bump the version as you prefer. Set the release date when you ship.
+
+---
+
+## [0.15.0] - 2026-XX-XX
+
+FTP night-report pipeline.  Automated **daily morning report** of last night's noise (7PM–7AM) versus a baseline, per location, for 24/7 remote sound jobs.  The meter records 24/7 to its SD card regardless of TCP state, so the report pulls the meter's own stored 15-minute `_Leq_` intervals over FTP (through the existing `/api/slmm/.../ftp/...` proxy) — accurate Leq/Lmax/Ln straight from the device, and resilient to a TCP-control wedge.  The report engine is source-agnostic and metric-driven; delivery is an HTML email body plus an Excel attachment.  Built around the existing `MonitoringSession`/`DataFile` store and the existing scheduled `cycle` action — the meter is cycled each morning (stop → download → ingest → increment store index → restart), and the report runs off the just-finished, finalized folder.
+
+### Added
+
+- **Callable ingest — `ingest_nrl_zip(location_id, zip_bytes, db)`** (`backend/routers/project_locations.py`).  The manual SD-card upload (`upload_nrl_data`) was refactored into a shared core so the same path runs programmatically from the scheduler.  Keeps `.rnh` + the averaged `_Leq_ .rnd`, drops the 1-second `_Lp_` files, parses the header (now also capturing the device's **percentile→slot map** and weightings into session metadata), and **dedups** repeated pulls of the same folder by store-name + start time.  Metric-agnostic: every column of the Leq file is preserved on disk; metric selection happens in the report layer.
+- **Report compute engine** (`backend/services/report_pipeline.py`).  Per-location night model: **LAmax / LA01 / LA10 / LA90 / LAeq** over **Evening (7–10PM)** and **Nighttime (10PM–7AM)** windows, with correct aggregation — Lmax = loudest interval, percentiles = arithmetic mean, **Leq = logarithmic (energy) mean**.  The LN→percentile mapping is read from the device's own `.rnh` config, not hardcoded.
+- **Two baseline sources.**  *Captured* — computed from recorded nights in a configurable date range (the "typical night" = mean of per-night values).  *Reference* — fixed values typed per location, for a spec limit (e.g. *"L10 = 85"*) or a prior report's averages when the raw data isn't in the system.  Blank reference cells aren't compared.
+- **Renderers** (`backend/services/report_renderers.py`).  HTML email body (per-location Last / Baseline / Δ table, colored louder/quieter) **+ an Excel attachment** — one worksheet per NRL with the 15-minute interval table, a line chart, and a Last/Base/Δ summary per window.  Metric-driven, so it tracks whatever metric set is configured.
+- **Config-driven SMTP sender** (`backend/services/report_email.py`).  Reads host/port/security/user/password/from/recipients from env (`REPORT_SMTP_*`); **dry-run** when unconfigured, so reports still generate and persist without credentials.
+- **Per-project config + automatic morning run.**  New `SoundReportConfig` table (enabled, report time, metrics, baseline mode + range, recipients) and a scheduler tick (`SchedulerService.run_due_reports`) that builds + emails each enabled project's report once per morning, off the event loop.  The orchestrator (`report_orchestrator.py`) always writes `report.html` / `report.json` / `report.xlsx` to `data/reports/{project}/{date}/`, then emails.
+- **Capture hook in the daily cycle.**  `_execute_cycle` now ingests the just-finished `Auto_####` folder into Terra-View after the download, and verifies the meter resumed measuring via a fresh DOD (`measurement_state`) — alerting if not.
+- **UI on the sound project header.**  A **Night Report** button (modal: view a night, *Run & Email* on demand, and a *Recent reports* list with HTML + Excel links) and a **gear → Settings** modal (enable/time, **baseline source toggle** with a per-NRL value editor, metrics, recipients, a **Send test email** button, and a schedule/last-run status line).
+- **Endpoints** (`backend/routers/reports.py`): `GET/PUT …/reports/config`, `GET/PUT …/reports/baseline`, `GET …/reports/nightly/view`, `POST …/reports/nightly/run`, `POST …/reports/test-email`, `GET …/reports/list`, `GET …/reports/archive/{date}` (+ `/xlsx`).
+
+### Changed
+
+- **Manual SD upload now shares the new ingest core.**  `POST …/nrl/{location_id}/upload-data` behaves as before (zip or loose files) but routes through `_ingest_file_entries`, so manually-uploaded sessions also get the captured percentile map.
+
+### Security / hardening
+
+- HTML modal fields built from user-controlled data (location names, baseline values) are HTML-escaped before insertion (stored-XSS fix).
+- The SMTP sender refuses to silently downgrade to a plaintext connection on an unrecognized `REPORT_SMTP_SECURITY` value (falls back to STARTTLS), and warns when credentials would go over an unencrypted link.
+
+### Upgrade Notes
+
+- **No database migration.**  `sound_report_configs` is a brand-new table created automatically by `create_all` on startup (the `baseline_mode` column lives on it).  Templates and Python are baked into the image, so **rebuild** (don't just restart):
+
+  ```bash
+  cd /home/serversdown/terra-view && docker compose build terra-view && docker compose up -d terra-view
+  ```
+
+- **To actually send email**, set the relay env vars (e.g. on the `terra-view` service in `docker-compose.yml`).  Until then, reports still build and write to `data/reports/…` in dry-run:
+
+  ```
+  REPORT_SMTP_HOST, REPORT_SMTP_PORT, REPORT_SMTP_SECURITY=starttls|ssl|none,
+  REPORT_SMTP_USER, REPORT_SMTP_PASSWORD, REPORT_SMTP_FROM, REPORT_SMTP_RECIPIENTS
+  ```
+
+  Use **Settings → Send test email** to verify the relay once set.
+
+- **To turn on automation for a job:** configure a daily `cycle` recurring schedule per NRL (~7:15 AM, after the night ends) so the meter is stopped/downloaded/ingested/restarted, then **enable** the report in the gear (report time ~8 AM) and set the baseline (range or fixed values).
+
+- **Not yet field-tested on a physical meter** — the live device-control portion of the cycle hook (download + restart-verify) was validated against a mocked SLMM only.

requirements-dev.txt

@@ -1,2 +0,0 @@
--r requirements.txt
-pytest==8.3.3

requirements.txt

@@ -10,4 +10,3 @@ httpx==0.25.2
 openpyxl==3.1.2
 rapidfuzz==3.10.1
 schedule==1.2.2
-argon2-cffi==23.1.0

templates/nrl_detail.html

@@ -357,6 +357,16 @@
 
     <!-- Data Files Tab -->
     <div id="data-tab" class="tab-panel hidden">
+        <!-- Download Files from SLMs (FTP browser, scoped to this NRL's assigned unit) -->
+        <div id="ftp-browser" class="mb-6"
+             hx-get="/api/projects/{{ project_id }}/ftp-browser?location_id={{ location_id }}"
+             hx-trigger="load"
+             hx-swap="innerHTML">
+            <div class="bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6">
+                <div class="text-center py-8 text-gray-500">Loading FTP browser...</div>
+            </div>
+        </div>
+
         <div class="bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6">
             <div class="flex items-center justify-between mb-6">
                 <h2 class="text-xl font-semibold text-gray-900 dark:text-white">Data Files</h2>
@@ -369,6 +379,13 @@
                         </svg>
                         Upload Data
                     </button>
+                    <button onclick="htmx.trigger('#unified-files', 'refresh')"
+                            class="px-3 py-1.5 text-sm bg-gray-100 dark:bg-gray-700 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-200 dark:hover:bg-gray-600 transition-colors flex items-center gap-1.5">
+                        <svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                            <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
+                        </svg>
+                        Refresh
+                    </button>
                 </div>
             </div>
 
@@ -408,11 +425,11 @@
                 </div>
             </div>
 
-            <div id="data-files-list"
-                 hx-get="/api/projects/{{ project_id }}/nrl/{{ location_id }}/files"
-                 hx-trigger="load"
+            <div id="unified-files"
+                 hx-get="/api/projects/{{ project_id }}/files-unified?location_id={{ location_id }}"
+                 hx-trigger="load, refresh from:#unified-files"
                  hx-swap="innerHTML">
-                <div class="text-center py-8 text-gray-500">Loading data files...</div>
+                <div class="text-center py-12 text-gray-500">Loading files...</div>
             </div>
         </div>
     </div>
@@ -715,7 +732,7 @@ function submitUpload() {
                 status.textContent = parts.join(' ');
                 status.className = 'text-sm text-green-600 dark:text-green-400';
                 input.value = '';
-                htmx.trigger(document.getElementById('data-files-list'), 'load');
+                htmx.trigger(document.getElementById('unified-files'), 'refresh');
             } else {
                 status.textContent = `Error: ${data.detail || 'Upload failed'}`;
                 status.className = 'text-sm text-red-600 dark:text-red-400';

templates/partials/projects/ftp_browser.html

@@ -32,19 +32,19 @@
                             </svg>
                             Settings
                         </button>
-                        <button onclick="enableFTP('{{ unit_item.unit.id }}')"
+                        <button onclick="FtpBrowser.enableFTP('{{ unit_item.unit.id }}')"
                                 id="enable-ftp-{{ unit_item.unit.id }}"
                                 class="px-3 py-1 text-xs bg-green-600 text-white rounded-lg hover:bg-green-700 transition-colors"
                                 disabled>
                             Enable FTP
                         </button>
-                        <button onclick="disableFTP('{{ unit_item.unit.id }}')"
+                        <button onclick="FtpBrowser.disableFTP('{{ unit_item.unit.id }}')"
                                 id="disable-ftp-{{ unit_item.unit.id }}"
                                 class="px-3 py-1 text-xs bg-red-600 text-white rounded-lg hover:bg-red-700 transition-colors"
                                 disabled>
                             Disable FTP
                         </button>
-                        <button onclick="loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
+                        <button onclick="FtpBrowser.loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
                                 id="browse-ftp-{{ unit_item.unit.id }}"
                                 class="px-3 py-1 text-xs bg-seismo-orange text-white rounded-lg hover:bg-seismo-navy transition-colors"
                                 disabled>
@@ -61,7 +61,7 @@
                                 <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M3 7v10a2 2 0 002 2h14a2 2 0 002-2V9a2 2 0 00-2-2h-6l-2-2H5a2 2 0 00-2 2z"></path>
                             </svg>
                             <span id="current-path-{{ unit_item.unit.id }}" class="text-sm font-mono text-gray-600 dark:text-gray-400">/NL-43</span>
-                            <button onclick="loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
+                            <button onclick="FtpBrowser.loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
                                     class="ml-auto text-xs px-2 py-1 bg-gray-100 dark:bg-gray-700 rounded hover:bg-gray-200 dark:hover:bg-gray-600">
                                 <svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
                                     <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
@@ -87,6 +87,11 @@
 </div>
 
 <script>
+// Self-contained namespace: this partial is reusable (project-wide Data Files
+// tab AND per-NRL Data Files tab), and may be co-loaded with other FTP-browsing
+// partials (e.g. slm_live_view). Wrapping in an IIFE keeps its helpers off the
+// global scope; only window.FtpBrowser is exposed (see the export at the end).
+(function () {
 async function checkFTPStatus(unitId) {
     const statusSpan = document.getElementById(`ftp-status-${unitId}`);
     const enableBtn = document.getElementById(`enable-ftp-${unitId}`);
@@ -228,7 +233,7 @@ async function loadFTPFiles(unitId, path) {
                 html += `
                     <div class="border border-gray-200 dark:border-gray-600 rounded mb-1">
                         <div class="flex items-center justify-between p-3 hover:bg-gray-50 dark:hover:bg-gray-700 rounded transition-colors cursor-pointer"
-                             onclick="toggleFTPFolderProject('${unitId}', '${escapeForAttribute(file.path)}', '${folderId}', this)">
+                             onclick="FtpBrowser.toggleFTPFolderProject('${unitId}', '${escapeForAttribute(file.path)}', '${folderId}', this)">
                             <div class="flex items-center flex-1">
                                 <svg class="w-4 h-4 mr-2 text-gray-400 transition-transform folder-chevron" fill="none" stroke="currentColor" viewBox="0 0 24 24">
                                     <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"></path>
@@ -239,7 +244,7 @@ async function loadFTPFiles(unitId, path) {
                             </div>
                             <div class="flex items-center gap-2 flex-shrink-0 ml-4">
                                 <span class="text-xs text-gray-500 hidden sm:inline">${file.modified || ''}</span>
-                                <button onclick="event.stopPropagation(); downloadFolderToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
+                                <button onclick="event.stopPropagation(); FtpBrowser.downloadFolderToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
                                         class="px-3 py-1 text-xs bg-seismo-orange text-white rounded hover:bg-seismo-navy transition-colors flex items-center"
                                         title="Download folder from device to server and add to project database">
                                     <svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -264,7 +269,7 @@ async function loadFTPFiles(unitId, path) {
                         <div class="flex items-center gap-3 flex-shrink-0 ml-4">
                             <span class="text-xs text-gray-500 hidden sm:inline">${sizeStr}</span>
                             <span class="text-xs text-gray-500 hidden md:inline">${file.modified || ''}</span>
-                            <button onclick="downloadToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
+                            <button onclick="FtpBrowser.downloadToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
                                     class="px-3 py-1 text-xs bg-seismo-orange text-white rounded hover:bg-seismo-navy transition-colors flex items-center"
                                     title="Download file from device to server and add to project database">
                                 <svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -379,7 +384,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
                     html += `
                         <div class="border border-gray-200 dark:border-gray-600 rounded">
                             <div class="flex items-center justify-between p-2 hover:bg-gray-50 dark:hover:bg-gray-700 rounded transition-colors cursor-pointer text-sm"
-                                 onclick="toggleFTPFolderProject('${unitId}', '${escapeForAttribute(fullPath)}', '${subFolderId}', this)">
+                                 onclick="FtpBrowser.toggleFTPFolderProject('${unitId}', '${escapeForAttribute(fullPath)}', '${subFolderId}', this)">
                                 <div class="flex items-center flex-1">
                                     <svg class="w-3 h-3 mr-2 text-gray-400 transition-transform folder-chevron" fill="none" stroke="currentColor" viewBox="0 0 24 24">
                                         <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"></path>
@@ -389,7 +394,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
                                     <span class="ml-2 text-xs text-gray-400 folder-status"></span>
                                 </div>
                                 <div class="flex items-center gap-2 flex-shrink-0 ml-2">
-                                    <button onclick="event.stopPropagation(); downloadFolderToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
+                                    <button onclick="event.stopPropagation(); FtpBrowser.downloadFolderToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
                                             class="px-2 py-1 bg-seismo-orange hover:bg-seismo-navy text-white text-xs rounded transition-colors flex items-center"
                                             title="Download folder to server">
                                         <svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -412,7 +417,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
                             </div>
                             <div class="flex items-center gap-2 flex-shrink-0 ml-2">
                                 <span class="text-xs text-gray-500 hidden sm:inline">${sizeText}</span>
-                                <button onclick="downloadToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
+                                <button onclick="FtpBrowser.downloadToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
                                         class="px-2 py-1 bg-seismo-orange hover:bg-seismo-navy text-white text-xs rounded transition-colors"
                                         title="Download to server">
                                     <svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -542,8 +547,11 @@ async function downloadFolderToServer(unitId, remotePath, folderName) {
         const data = await response.json();
 
         if (response.ok) {
-            // Show success message
-            alert(`✓ Folder "${folderName}" downloaded successfully!\n\n${data.file_count} files extracted\nTotal size: ${formatFileSize(data.total_size)}\n\nFiles are now available in the Project Files section below.`);
+            // Show success message — surface how long the measurement ran
+            alert(`✓ Folder "${folderName}" saved!\n\n` +
+                  (data.message || `${data.file_count} file(s) imported`) +
+                  formatRunLength(data) +
+                  `\n\nNow saved as a session in the Project Files section below.`);
 
             // Refresh the unified files list
             htmx.trigger('#unified-files', 'refresh');
@@ -585,7 +593,11 @@ async function downloadToServer(unitId, remotePath, fileName) {
 
         if (response.ok) {
             // Show success message
-            alert(`✓ ${fileName} downloaded to server successfully!\n\nFile ID: ${data.file_id}\nSize: ${formatFileSize(data.file_size)}`);
+            const sizeLine = `\nSize: ${formatFileSize(data.file_size)}`;
+            const msg = data.ingested
+                ? `✓ ${fileName} imported as measurement data!` + formatRunLength(data) + sizeLine
+                : `✓ ${fileName} downloaded to server successfully!\n\nFile ID: ${data.file_id}` + sizeLine;
+            alert(msg);
 
             // Refresh the unified files list
             htmx.trigger('#unified-files', 'refresh');
@@ -607,6 +619,17 @@ function formatFileSize(bytes) {
     return (bytes / 1073741824).toFixed(2) + ' GB';
 }
 
+// Build a "how long did it run" line from an ingest response.  Duration is
+// timezone-independent (stop − start), so it's the reliable number to show.
+function formatRunLength(data) {
+    if (data.duration_seconds == null) return '';
+    const s = data.duration_seconds;
+    const h = Math.floor(s / 3600);
+    const m = Math.floor((s % 3600) / 60);
+    let txt = h > 0 ? `${h}h ${m}m` : `${m}m`;
+    return `\n\nRecorded for: ${txt}`;
+}
+
 // Check FTP status for all units on load
 // Use setTimeout to ensure DOM elements exist when HTMX loads this partial
 setTimeout(function() {
@@ -614,6 +637,17 @@ setTimeout(function() {
     checkFTPStatus('{{ unit_item.unit.id }}');
     {% endfor %}
 }, 100);
+
+// The only global surface — the handlers referenced by inline onclick attributes.
+window.FtpBrowser = {
+    loadFTPFiles,
+    enableFTP,
+    disableFTP,
+    toggleFTPFolderProject,
+    downloadFolderToServer,
+    downloadToServer,
+};
+})();
 </script>
 
 <!-- Include the unified SLM Settings Modal -->

templates/partials/projects/project_header.html

@@ -74,6 +74,22 @@
                 </svg>
                 Generate Combined Report
             </a>
+            <button onclick="openNightReportModal()"
+                    title="Last night's noise vs baseline, per location (FTP report pipeline)"
+                    class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors flex items-center gap-2 text-sm">
+                <svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M20.354 15.354A9 9 0 018.646 3.646 9.003 9.003 0 0012 21a9.003 9.003 0 008.354-5.646z"></path>
+                </svg>
+                Night Report
+            </button>
+            <button onclick="openReportSettings('{{ project.id }}')"
+                    title="Nightly report settings — schedule, baseline range, recipients"
+                    class="px-3 py-2 border border-gray-300 dark:border-gray-600 text-gray-600 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors flex items-center text-sm">
+                <svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
+                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10.325 4.317c.426-1.756 2.924-1.756 3.35 0a1.724 1.724 0 002.573 1.066c1.543-.94 3.31.826 2.37 2.37a1.724 1.724 0 001.065 2.572c1.756.426 1.756 2.924 0 3.35a1.724 1.724 0 00-1.066 2.573c.94 1.543-.826 3.31-2.37 2.37a1.724 1.724 0 00-2.572 1.065c-.426 1.756-2.924 1.756-3.35 0a1.724 1.724 0 00-2.573-1.066c-1.543.94-3.31-.826-2.37-2.37a1.724 1.724 0 00-1.065-2.572c-1.756-.426-1.756-2.924 0-3.35a1.724 1.724 0 001.066-2.573c-.94-1.543.826-3.31 2.37-2.37.996.608 2.296.07 2.572-1.065z"></path>
+                    <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 12a3 3 0 11-6 0 3 3 0 016 0z"></path>
+                </svg>
+            </button>
             {% endif %}
             <button onclick="openMergeModal()"
                     title="Merge this project into another (consolidates duplicates)"
@@ -87,6 +103,338 @@
     </div>
 </div>
 
+<!-- Night Report Modal -->
+<div id="night-report-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/60 backdrop-blur-sm">
+    <div class="bg-white dark:bg-slate-800 rounded-xl shadow-2xl w-full max-w-md mx-4">
+        <div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700 flex items-center justify-between">
+            <h3 class="text-lg font-semibold text-gray-900 dark:text-white">Night Report</h3>
+            <button onclick="closeNightReportModal()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
+                <svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path></svg>
+            </button>
+        </div>
+        <div class="px-6 py-5 space-y-4">
+            <p class="text-xs text-gray-500 dark:text-gray-400">Last night's noise (7&nbsp;PM–7&nbsp;AM) vs a baseline range, per location. Opens in a new tab.</p>
+            <div>
+                <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Night (evening date)</label>
+                <input type="date" id="nr-night-date" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+            </div>
+            <div class="grid grid-cols-2 gap-3">
+                <div>
+                    <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline start <span class="text-gray-400 font-normal">(optional)</span></label>
+                    <input type="date" id="nr-baseline-start" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                </div>
+                <div>
+                    <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline end</label>
+                    <input type="date" id="nr-baseline-end" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                </div>
+            </div>
+            <div>
+                <div class="flex items-center justify-between mb-1">
+                    <label class="block text-sm font-medium text-gray-700 dark:text-gray-300">Recent reports</label>
+                    <span id="nr-recent-count" class="text-xs text-gray-400"></span>
+                </div>
+                <div id="nr-recent" class="max-h-40 overflow-y-auto rounded-lg border border-gray-200 dark:border-gray-700 divide-y divide-gray-100 dark:divide-gray-700">
+                    <div class="px-3 py-2 text-xs text-gray-400">Loading…</div>
+                </div>
+            </div>
+            <p id="nr-status" class="text-xs"></p>
+        </div>
+        <div class="px-6 py-4 border-t border-gray-200 dark:border-gray-700 flex justify-end gap-2">
+            <button onclick="closeNightReportModal()" class="px-4 py-2 border border-gray-300 dark:border-gray-600 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors text-sm">Cancel</button>
+            <button onclick="runNightReport('{{ project.id }}')" class="px-4 py-2 bg-emerald-600 text-white rounded-lg hover:bg-emerald-700 transition-colors text-sm">Run &amp; Email</button>
+            <button onclick="viewNightReport('{{ project.id }}')" class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors text-sm">View Report</button>
+        </div>
+    </div>
+</div>
+<script>
+var NR_PROJECT_ID = '{{ project.id }}';
+function openNightReportModal() {
+    var el = document.getElementById('nr-night-date');
+    if (el && !el.value) {                       // default to last night
+        var d = new Date(); d.setDate(d.getDate() - 1);
+        el.value = d.getFullYear() + '-'
+                 + String(d.getMonth() + 1).padStart(2, '0') + '-'
+                 + String(d.getDate()).padStart(2, '0');
+    }
+    document.getElementById('nr-status').textContent = '';
+    document.getElementById('night-report-modal').classList.remove('hidden');
+    loadRecentReports(NR_PROJECT_ID);
+}
+function closeNightReportModal() {
+    document.getElementById('night-report-modal').classList.add('hidden');
+}
+function _nrParams() {
+    var night = document.getElementById('nr-night-date').value;
+    var bs = document.getElementById('nr-baseline-start').value;
+    var be = document.getElementById('nr-baseline-end').value;
+    if (!night) { alert('Pick a night (evening date).'); return null; }
+    if ((bs && !be) || (be && !bs)) { alert('Provide both baseline dates, or leave both empty.'); return null; }
+    var qs = 'night_date=' + night;
+    if (bs && be) qs += '&baseline_start=' + bs + '&baseline_end=' + be;
+    return qs;
+}
+function viewNightReport(projectId) {
+    var qs = _nrParams(); if (!qs) return;
+    window.open('/api/projects/' + projectId + '/reports/nightly/view?' + qs, '_blank');
+}
+function runNightReport(projectId) {
+    var qs = _nrParams(); if (!qs) return;
+    var st = document.getElementById('nr-status');
+    st.style.color = ''; st.textContent = 'Running…';
+    fetch('/api/projects/' + projectId + '/reports/nightly/run?' + qs + '&send=true', { method: 'POST' })
+        .then(function (r) { return r.json().then(function (j) { return { ok: r.ok, j: j }; }); })
+        .then(function (res) {
+            if (!res.ok) { st.style.color = '#b00020'; st.textContent = 'Error: ' + (res.j.detail || 'run failed'); return; }
+            var em = res.j.email || {};
+            var emailMsg = em.sent ? 'emailed' : (em.dry_run ? 'email dry-run (SMTP not set)' : (em.error || 'email skipped'));
+            st.style.color = '#1a7f37';
+            st.innerHTML = 'Done — saved &amp; ' + _mergeEsc(emailMsg) + '. <a href="' + _mergeEsc(res.j.view_url) + '" target="_blank" class="underline">view</a>';
+            loadRecentReports(projectId);
+        })
+        .catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
+}
+function loadRecentReports(projectId) {
+    var box = document.getElementById('nr-recent');
+    var cnt = document.getElementById('nr-recent-count');
+    fetch('/api/projects/' + projectId + '/reports/list')
+        .then(function (r) { return r.json(); })
+        .then(function (j) {
+            cnt.textContent = (j.count || 0) + ' generated';
+            if (!j.reports || !j.reports.length) {
+                box.innerHTML = '<div class="px-3 py-2 text-xs text-gray-400">None yet. Run one above.</div>';
+                return;
+            }
+            box.innerHTML = j.reports.map(function (rp) {
+                var when = (rp.generated_at || '').replace('T', ' ').slice(0, 16);
+                var xlsx = rp.xlsx_url ? ' · <a href="' + _mergeEsc(rp.xlsx_url) + '" class="text-indigo-600 dark:text-indigo-400 hover:underline">Excel</a>' : '';
+                return '<div class="flex items-center justify-between px-3 py-2 text-sm">'
+                     + '<a href="' + _mergeEsc(rp.view_url) + '" target="_blank" class="font-medium text-gray-800 dark:text-gray-200 hover:underline">Night of ' + _mergeEsc(rp.night_date) + '</a>'
+                     + '<span class="text-xs text-gray-400">' + _mergeEsc(when) + ' UTC' + xlsx + '</span></div>';
+            }).join('');
+        })
+        .catch(function () { box.innerHTML = '<div class="px-3 py-2 text-xs text-red-500">Failed to load.</div>'; });
+}
+</script>
+
+<!-- Nightly Report Settings Modal -->
+<div id="report-settings-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/60 backdrop-blur-sm">
+    <div class="bg-white dark:bg-slate-800 rounded-xl shadow-2xl w-full max-w-md mx-4 max-h-[90vh] overflow-y-auto">
+        <div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700 flex items-center justify-between">
+            <h3 class="text-lg font-semibold text-gray-900 dark:text-white">Nightly Report Settings</h3>
+            <button onclick="closeReportSettings()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
+                <svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path></svg>
+            </button>
+        </div>
+        <div class="px-6 py-5 space-y-4">
+            <div id="rs-schedule-status" class="text-xs text-gray-500 dark:text-gray-400 bg-gray-50 dark:bg-gray-900/40 rounded-lg px-3 py-2"></div>
+            <label class="flex items-center gap-2 text-sm font-medium text-gray-800 dark:text-gray-200">
+                <input type="checkbox" id="rs-enabled" class="rounded border-gray-300 text-indigo-600 focus:ring-indigo-500">
+                Email the report automatically each morning
+            </label>
+            <div>
+                <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Report time (local)</label>
+                <input type="time" id="rs-report-time" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                <p class="text-xs text-gray-400 mt-1">Runs after this time for the night that just ended.</p>
+            </div>
+            <div>
+                <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline source</label>
+                <div class="flex gap-4 text-sm mb-2">
+                    <label class="flex items-center gap-1.5"><input type="radio" name="rs-baseline-mode" value="captured" onchange="rsToggleBaselineMode()" class="text-indigo-600"> Captured nights</label>
+                    <label class="flex items-center gap-1.5"><input type="radio" name="rs-baseline-mode" value="reference" onchange="rsToggleBaselineMode()" class="text-indigo-600"> Fixed values</label>
+                </div>
+                <div id="rs-baseline-captured" class="grid grid-cols-2 gap-3">
+                    <div>
+                        <label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Range start</label>
+                        <input type="date" id="rs-baseline-start" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                    </div>
+                    <div>
+                        <label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Range end</label>
+                        <input type="date" id="rs-baseline-end" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                    </div>
+                </div>
+                <div id="rs-baseline-reference" class="hidden">
+                    <p class="text-xs text-gray-400 mb-2">Values to compare against (a spec limit like L10 = 85, or a prior report's averages). Blank cells aren't compared.</p>
+                    <div class="flex justify-end mb-1"><button type="button" onclick="rsCopyFirstNrl()" class="text-xs text-indigo-600 dark:text-indigo-400 hover:underline">Copy first NRL → all</button></div>
+                    <div id="rs-ref-grid" class="space-y-3"></div>
+                </div>
+            </div>
+            <div>
+                <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Metrics</label>
+                <input type="text" id="rs-metrics" placeholder="lmax,l01,l10,l90" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                <p class="text-xs text-gray-400 mt-1">Comma list. Options: lmax, l01, l10, l50, l90, l95, leq.</p>
+            </div>
+            <div>
+                <label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Recipients</label>
+                <input type="text" id="rs-recipients" placeholder="brian@…, dad@…" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
+                <p class="text-xs text-gray-400 mt-1">Comma list. Blank → the default SMTP recipients.</p>
+            </div>
+            <div>
+                <button type="button" onclick="sendTestEmail('{{ project.id }}')" class="text-sm text-indigo-600 dark:text-indigo-400 hover:underline">Send test email</button>
+                <span id="rs-test-status" class="text-xs ml-2"></span>
+            </div>
+            <p id="rs-status" class="text-xs"></p>
+        </div>
+        <div class="px-6 py-4 border-t border-gray-200 dark:border-gray-700 flex justify-end gap-2">
+            <button onclick="closeReportSettings()" class="px-4 py-2 border border-gray-300 dark:border-gray-600 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors text-sm">Cancel</button>
+            <button onclick="saveReportSettings('{{ project.id }}')" class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors text-sm">Save</button>
+        </div>
+    </div>
+</div>
+<script>
+function openReportSettings(projectId) {
+    var show = function () { document.getElementById('report-settings-modal').classList.remove('hidden'); };
+    document.getElementById('rs-status').textContent = '';
+    fetch('/api/projects/' + projectId + '/reports/config')
+        .then(function (r) { return r.json(); })
+        .then(function (c) {
+            document.getElementById('rs-enabled').checked = !!c.enabled;
+            document.getElementById('rs-report-time').value = c.report_time || '08:00';
+            document.getElementById('rs-baseline-start').value = c.baseline_start || '';
+            document.getElementById('rs-baseline-end').value = c.baseline_end || '';
+            document.getElementById('rs-metrics').value = c.metric_keys || 'lmax,l01,l10,l90';
+            document.getElementById('rs-recipients').value = c.recipients || '';
+            var ss = document.getElementById('rs-schedule-status');
+            var last = c.last_run_date || '—';
+            if (c.enabled) {
+                ss.innerHTML = '<span style="color:#1a7f37">●</span> Automatic — runs daily at ' + (c.report_time || '08:00') + '. Last reported night: ' + last + '.';
+            } else {
+                ss.innerHTML = '<span style="color:#9ca3af">●</span> Automatic sending is off. Last reported night: ' + last + '.';
+            }
+            document.getElementById('rs-test-status').textContent = '';
+            rsSetMode(c.baseline_mode || 'captured');
+            loadBaselineEditor(projectId);
+            show();
+        })
+        .catch(show);
+}
+function closeReportSettings() {
+    document.getElementById('report-settings-modal').classList.add('hidden');
+}
+function saveReportSettings(projectId) {
+    var st = document.getElementById('rs-status');
+    var mode = rsGetMode();
+    var bs = document.getElementById('rs-baseline-start').value;
+    var be = document.getElementById('rs-baseline-end').value;
+    if (mode === 'captured' && ((bs && !be) || (be && !bs))) {
+        st.style.color = '#b00020'; st.textContent = 'Provide both baseline dates, or neither.'; return;
+    }
+    var body = {
+        enabled: document.getElementById('rs-enabled').checked,
+        report_time: document.getElementById('rs-report-time').value || '08:00',
+        metric_keys: document.getElementById('rs-metrics').value || 'lmax,l01,l10,l90',
+        baseline_mode: mode,
+        baseline_start: bs || null,
+        baseline_end: be || null,
+        recipients: document.getElementById('rs-recipients').value || ''
+    };
+    st.style.color = ''; st.textContent = 'Saving…';
+    fetch('/api/projects/' + projectId + '/reports/config', {
+        method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body)
+    }).then(function (r) { return r.json().then(function (j) { return { ok: r.ok, j: j }; }); })
+      .then(function (res) {
+          if (!res.ok) { st.style.color = '#b00020'; st.textContent = 'Error: ' + (res.j.detail || 'save failed'); return; }
+          if (mode === 'reference') {
+              return fetch('/api/projects/' + projectId + '/reports/baseline', {
+                  method: 'PUT', headers: { 'Content-Type': 'application/json' },
+                  body: JSON.stringify({ locations: gatherRefValues() })
+              }).then(function (r2) {
+                  if (!r2.ok) throw new Error('baseline values failed to save');
+                  st.style.color = '#1a7f37'; st.textContent = 'Saved.'; setTimeout(closeReportSettings, 700);
+              });
+          }
+          st.style.color = '#1a7f37'; st.textContent = 'Saved.'; setTimeout(closeReportSettings, 700);
+      })
+      .catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
+}
+var RS_BASELINE = { metrics: [], windows: [], locations: [] };
+function rsGetMode() {
+    var r = document.querySelector('input[name="rs-baseline-mode"]:checked');
+    return r ? r.value : 'captured';
+}
+function rsSetMode(mode) {
+    document.querySelectorAll('input[name="rs-baseline-mode"]').forEach(function (el) { el.checked = (el.value === mode); });
+    rsToggleBaselineMode();
+}
+function rsToggleBaselineMode() {
+    var ref = rsGetMode() === 'reference';
+    document.getElementById('rs-baseline-captured').classList.toggle('hidden', ref);
+    document.getElementById('rs-baseline-reference').classList.toggle('hidden', !ref);
+}
+function loadBaselineEditor(projectId) {
+    fetch('/api/projects/' + projectId + '/reports/baseline')
+        .then(function (r) { return r.json(); })
+        .then(function (d) { RS_BASELINE = d; renderRefGrid(); })
+        .catch(function () {});
+}
+function _refId(loc, w, m) { return 'ref__' + loc + '__' + w + '__' + m; }
+function renderRefGrid() {
+    var box = document.getElementById('rs-ref-grid');
+    if (!RS_BASELINE.locations || !RS_BASELINE.locations.length) {
+        box.innerHTML = '<div class="text-xs text-gray-400">No NRLs in this project yet.</div>'; return;
+    }
+    var W = RS_BASELINE.windows, M = RS_BASELINE.metrics;
+    box.innerHTML = RS_BASELINE.locations.map(function (loc) {
+        var head = '<tr><th></th>' + W.map(function (w) {
+            return '<th class="text-xs text-gray-400 font-normal pb-1 px-1">' + w.label.replace(/\s*\(.*\)/, '') + '</th>';
+        }).join('') + '</tr>';
+        var rows = M.map(function (m) {
+            var cells = W.map(function (w) {
+                var v = (loc.values[w.key] && loc.values[w.key][m.key] != null) ? loc.values[w.key][m.key] : '';
+                return '<td class="px-1"><input type="number" step="0.1" id="' + _refId(loc.id, w.key, m.key) + '" value="' + _mergeEsc(v) + '" class="w-16 px-1.5 py-1 border border-gray-300 dark:border-gray-600 rounded bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm text-center"></td>';
+            }).join('');
+            return '<tr><td class="text-sm text-gray-700 dark:text-gray-300 pr-2">' + m.label + '</td>' + cells + '</tr>';
+        }).join('');
+        return '<div class="border border-gray-200 dark:border-gray-700 rounded-lg p-2">'
+             + '<div class="text-sm font-medium text-gray-800 dark:text-gray-200 mb-1">' + _mergeEsc(loc.name) + '</div>'
+             + '<table class="w-full">' + head + rows + '</table></div>';
+    }).join('');
+}
+function gatherRefValues() {
+    var out = {};
+    (RS_BASELINE.locations || []).forEach(function (loc) {
+        var wins = {};
+        RS_BASELINE.windows.forEach(function (w) {
+            var mv = {};
+            RS_BASELINE.metrics.forEach(function (m) {
+                var el = document.getElementById(_refId(loc.id, w.key, m.key));
+                if (el && el.value !== '') mv[m.key] = el.value;
+            });
+            if (Object.keys(mv).length) wins[w.key] = mv;
+        });
+        out[loc.id] = wins;
+    });
+    return out;
+}
+function rsCopyFirstNrl() {
+    if (!RS_BASELINE.locations || RS_BASELINE.locations.length < 2) return;
+    var first = RS_BASELINE.locations[0].id;
+    RS_BASELINE.locations.slice(1).forEach(function (loc) {
+        RS_BASELINE.windows.forEach(function (w) {
+            RS_BASELINE.metrics.forEach(function (m) {
+                var src = document.getElementById(_refId(first, w.key, m.key));
+                var dst = document.getElementById(_refId(loc.id, w.key, m.key));
+                if (src && dst) dst.value = src.value;
+            });
+        });
+    });
+}
+function sendTestEmail(projectId) {
+    var st = document.getElementById('rs-test-status');
+    st.style.color = ''; st.textContent = 'Sending…';
+    var recips = document.getElementById('rs-recipients').value;
+    fetch('/api/projects/' + projectId + '/reports/test-email', {
+        method: 'POST', headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify(recips ? { recipients: recips } : {})
+    }).then(function (r) { return r.json(); })
+      .then(function (j) {
+          if (j.sent) { st.style.color = '#1a7f37'; st.textContent = 'Sent to ' + (j.recipients || []).join(', '); }
+          else if (j.dry_run) { st.style.color = '#b8860b'; st.textContent = 'Dry-run (SMTP not set) — would send to ' + (j.recipients || []).join(', '); }
+          else { st.style.color = '#b00020'; st.textContent = 'Error: ' + (j.error || 'failed'); }
+      })
+      .catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
+}
+</script>
+
 <!-- Merge Modal —
      min-h on the body ensures the typeahead dropdown has room to render
      below the input without forcing the operator to scroll inside the

templates/portal/password.html

@@ -1,26 +0,0 @@
-{% extends "portal/base.html" %}
-{% block title %}{{ project_name }}{% endblock %}
-{% block content %}
-<div class="max-w-md mx-auto mt-20 text-center reveal">
-    <div class="panel inline-flex items-center justify-center w-16 h-16 rounded-2xl mb-6">
-        <svg class="w-7 h-7 text-[var(--text-dim)]" fill="none" stroke="currentColor" viewBox="0 0 24 24">
-            <path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"
-                  d="M12 15v2m-6 4h12a2 2 0 002-2v-6a2 2 0 00-2-2H6a2 2 0 00-2 2v6a2 2 0 002 2zm10-10V7a4 4 0 00-8 0v4h8z"></path>
-        </svg>
-    </div>
-    <h1 class="text-2xl font-bold tracking-tight mb-1">{{ project_name }}</h1>
-    <p class="text-[var(--text-dim)] text-sm mb-6">Enter the password to view this monitoring portal.</p>
-    {% if error %}
-    <p class="text-[var(--lvl-bad)] text-sm mb-4">{{ error }}</p>
-    {% endif %}
-    <form method="post" action="/portal/p/{{ link_token }}" class="panel p-5 text-left">
-        <label class="block text-xs text-[var(--text-dim)] mb-1" for="password">Password</label>
-        <input id="password" name="password" type="password" autofocus required
-               class="w-full px-3 py-2 rounded-lg border border-[var(--border)] bg-[var(--panel-b)] text-[var(--text)] mb-4">
-        <button type="submit"
-                class="w-full px-4 py-2 rounded-lg bg-seismo-orange text-white font-medium hover:opacity-90">
-            View portal
-        </button>
-    </form>
-</div>
-{% endblock %}

templates/projects/detail.html

@@ -18,16 +18,16 @@
         <span class="text-gray-900 dark:text-white font-medium" id="project-name-breadcrumb">Project</span>
     </nav>
 
-    <!-- Client portal access for this project -->
+    <!-- Client portal actions for this project -->
     <div class="shrink-0 flex items-center gap-2">
-        <button type="button" onclick="openPortalAccess()"
+        <button type="button" onclick="openShareModal()"
                 class="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm rounded-lg border border-slate-600 bg-slate-700/40 text-gray-200 hover:bg-slate-700 transition-colors"
-                title="Manage this project's client portal access">
+                title="Get a shareable link to this project's client portal">
             <svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
-                <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2"
-                      d="M12 15v2m-6 4h12a2 2 0 002-2v-6a2 2 0 00-2-2H6a2 2 0 00-2 2v6a2 2 0 002 2zm10-10V7a4 4 0 00-8 0v4h8z"></path>
+                <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13.828 10.172a4 4 0 010 5.656l-3 3a4 4 0 11-5.656-5.656l1.5-1.5"></path>
+                <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10.172 13.828a4 4 0 010-5.656l3-3a4 4 0 115.656 5.656l-1.5 1.5"></path>
             </svg>
-            Portal access
+            Copy client link
         </button>
         <a href="/projects/{{ project_id }}/portal-preview" target="_blank" rel="noopener"
            class="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm rounded-lg border border-seismo-orange/40 bg-seismo-orange/10 text-seismo-orange hover:bg-seismo-orange/20 transition-colors"
@@ -36,7 +36,7 @@
                 <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 12a3 3 0 11-6 0 3 3 0 016 0z"></path>
                 <path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M2.458 12C3.732 7.943 7.523 5 12 5c4.478 0 8.268 2.943 9.542 7-1.274 4.057-5.064 7-9.542 7-4.477 0-8.268-2.943-9.542-7z"></path>
             </svg>
-            Preview
+            View client portal
         </a>
     </div>
 </div>
@@ -2098,95 +2098,123 @@ document.addEventListener('DOMContentLoaded', function() {
 
 </script>
 
-<!-- Portal access modal -->
-<div id="portal-access-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/50 p-4"
-     onclick="if(event.target===this)closePortalAccess()">
+<!-- Share client portal link modal -->
+<div id="share-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/50 p-4"
+     onclick="if(event.target===this)closeShareModal()">
     <div class="bg-white dark:bg-slate-800 rounded-xl shadow-xl w-full max-w-lg p-6">
         <div class="flex items-center justify-between mb-1">
-            <h3 class="text-lg font-semibold text-gray-900 dark:text-white">Client portal access</h3>
-            <button onclick="closePortalAccess()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
+            <h3 class="text-lg font-semibold text-gray-900 dark:text-white">Client portal link</h3>
+            <button onclick="closeShareModal()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
                 <svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"/></svg>
             </button>
         </div>
         <p class="text-sm text-gray-500 dark:text-gray-400 mb-4">
-            Send the client the link <em>and</em> the password. Read-only. Disabling rotates the link.
+            Anyone with a link can view this project's client portal (read-only). Links are revocable.
         </p>
 
-        <div class="flex items-center justify-between mb-4">
-            <span class="text-sm font-medium text-gray-700 dark:text-gray-200">Portal enabled</span>
-            <button id="pa-toggle" onclick="togglePortalEnabled()"
-                    class="px-3 py-1.5 text-sm rounded-lg border border-slate-300 dark:border-slate-600">…</button>
+        {% if portal_open_links %}
+        <!-- Dev quick link: plain, no-token URL anyone can open (PORTAL_OPEN_LINKS on) -->
+        <div class="mb-4 p-3 rounded-lg bg-amber-50 dark:bg-amber-900/20 border border-amber-200 dark:border-amber-800/50">
+            <label class="block text-xs font-medium text-amber-700 dark:text-amber-300 mb-1">Quick share link (dev — anyone can open, no login)</label>
+            <div class="flex gap-2">
+                <input id="open-url" readonly
+                       class="flex-1 px-3 py-2 text-sm rounded-lg border border-amber-300 dark:border-amber-700 bg-white dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
+                <button onclick="copyOpenUrl(this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
+            </div>
+            <p class="text-xs text-amber-600 dark:text-amber-400 mt-1">For feedback during development. Disable <code>PORTAL_OPEN_LINKS</code> before real clients.</p>
+        </div>
+        {% endif %}
+
+        <div id="share-new" class="hidden mb-4">
+            <label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">New link &mdash; copy it now</label>
+            <div class="flex gap-2">
+                <input id="share-new-url" readonly
+                       class="flex-1 px-3 py-2 text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-gray-50 dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
+                <button onclick="copyShareUrl(this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
+            </div>
         </div>
 
-        <div id="pa-details" class="hidden space-y-4">
-            <div>
-                <label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Portal link</label>
-                <div class="flex gap-2">
-                    <input id="pa-link" readonly class="flex-1 px-3 py-2 text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-gray-50 dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
-                    <button onclick="copyField('pa-link', this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
-                </div>
-            </div>
-            <div>
-                <label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Password</label>
-                <div class="flex gap-2">
-                    <input id="pa-pass" readonly placeholder="•••••••• (set one below)"
-                           class="flex-1 px-3 py-2 text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-gray-50 dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
-                    <button onclick="copyField('pa-pass', this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
-                </div>
-                <button onclick="regeneratePassword()" class="mt-2 text-sm text-seismo-orange hover:text-seismo-navy font-medium">↻ Generate new password</button>
-                <p class="text-xs text-gray-400 mt-1">Shown once — copy it now. Regenerating invalidates the old one.</p>
-            </div>
+        <div class="flex items-center justify-between mb-2">
+            <span class="text-xs font-medium text-gray-500 dark:text-gray-400 uppercase tracking-wide">Active links</span>
+            <button onclick="generateShareLink()" class="text-sm text-seismo-orange hover:text-seismo-navy font-medium">+ Generate new link</button>
         </div>
+        <div id="share-list" class="space-y-2 max-h-56 overflow-y-auto"></div>
     </div>
 </div>
 
 <script>
-const PA_PROJECT_ID = "{{ project_id }}";
-let paEnabled = false;
-function paToast(msg) { if (window.showToast) showToast(msg, 'error'); else alert(msg); }
-function openPortalAccess() { document.getElementById('portal-access-modal').classList.remove('hidden'); loadPortalAccess(); }
-function closePortalAccess() { document.getElementById('portal-access-modal').classList.add('hidden'); }
+const SHARE_PROJECT_ID = "{{ project_id }}";
+function openShareModal() {
+    document.getElementById('share-modal').classList.remove('hidden');
+    document.getElementById('share-new').classList.add('hidden');
+    const ou = document.getElementById('open-url');   // only present when PORTAL_OPEN_LINKS on
+    if (ou) ou.value = `${location.origin}/portal/open/${SHARE_PROJECT_ID}`;
+    loadShareLinks();
+}
+function closeShareModal() { document.getElementById('share-modal').classList.add('hidden'); }
 
-function copyField(id, btn) {
-    const inp = document.getElementById(id); inp.select();
+function copyOpenUrl(btn) {
+    const inp = document.getElementById('open-url');
+    inp.select();
     const done = () => { btn.textContent = 'Copied!'; setTimeout(() => btn.textContent = 'Copy', 1500); };
     if (navigator.clipboard) navigator.clipboard.writeText(inp.value).then(done).catch(() => { document.execCommand('copy'); done(); });
     else { document.execCommand('copy'); done(); }
 }
 
-async function loadPortalAccess() {
+async function loadShareLinks() {
+    const list = document.getElementById('share-list');
+    list.innerHTML = '<div class="text-sm text-gray-400">Loading…</div>';
     try {
-        const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access`);
-        if (!r.ok) throw new Error('load failed');
-        renderPortalAccess(await r.json());
-    } catch (e) { paToast('Could not load portal access.'); }
+        const j = await (await fetch(`/projects/${SHARE_PROJECT_ID}/portal-links`)).json();
+        if (!j.links || !j.links.length) {
+            list.innerHTML = '<div class="text-sm text-gray-400">No links yet — generate one above.</div>';
+            return;
         }
-function renderPortalAccess(j) {
-    paEnabled = !!j.enabled;
-    const toggle = document.getElementById('pa-toggle');
-    const details = document.getElementById('pa-details');
-    toggle.textContent = paEnabled ? 'On — click to disable' : 'Off — click to enable';
-    toggle.className = 'px-3 py-1.5 text-sm rounded-lg border ' +
-        (paEnabled ? 'border-green-500 text-green-600 dark:text-green-400' : 'border-slate-300 dark:border-slate-600');
-    details.classList.toggle('hidden', !paEnabled);
-    document.getElementById('pa-link').value = (paEnabled && j.link_url) ? j.link_url : '';
+        list.innerHTML = '';
+        for (const l of j.links) {
+            const last = l.last_used_at ? ('last used ' + new Date(l.last_used_at + 'Z').toLocaleString()) : 'never used';
+            const row = document.createElement('div');
+            row.className = 'flex items-center justify-between gap-2 px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-700';
+            row.innerHTML = `<div class="text-sm min-w-0">
+                <div class="text-gray-800 dark:text-gray-200 truncate">${l.label || 'Link'}</div>
+                <div class="text-xs text-gray-400">${last}</div></div>`;
+            const btn = document.createElement('button');
+            btn.className = 'shrink-0 text-xs text-red-600 hover:text-red-700';
+            btn.textContent = 'Revoke';
+            btn.onclick = () => revokeShareLink(l.id);
+            row.appendChild(btn);
+            list.appendChild(row);
         }
-async function togglePortalEnabled() {
-    const action = paEnabled ? 'disable' : 'enable';
+    } catch (e) {
+        list.innerHTML = '<div class="text-sm text-red-500">Failed to load links.</div>';
+    }
+}
+
+async function generateShareLink() {
     try {
-        const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access/${action}`, { method: 'POST' });
-        if (!r.ok) throw new Error('toggle failed');
-        const j = await r.json();
-        renderPortalAccess(action === 'disable' ? { enabled: false, link_url: null } : j);
-    } catch (e) { paToast(`Could not ${action} the portal.`); }
+        const j = await (await fetch(`/projects/${SHARE_PROJECT_ID}/portal-link`, { method: 'POST' })).json();
+        if (j.url) {
+            document.getElementById('share-new').classList.remove('hidden');
+            document.getElementById('share-new-url').value = j.url;
+            loadShareLinks();
         }
-async function regeneratePassword() {
-    try {
-        const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access/password`, { method: 'POST' });
-        if (!r.ok) throw new Error('password failed');
-        const j = await r.json();
-        if (j.password) { const f = document.getElementById('pa-pass'); f.value = j.password; f.placeholder = ''; }
-    } catch (e) { paToast('Could not generate a password.'); }
+    } catch (e) {
+        if (window.showToast) showToast('Failed to generate link', 'error');
+    }
+}
+
+function copyShareUrl(btn) {
+    const inp = document.getElementById('share-new-url');
+    inp.select();
+    const done = () => { btn.textContent = 'Copied!'; setTimeout(() => btn.textContent = 'Copy', 1500); };
+    if (navigator.clipboard) navigator.clipboard.writeText(inp.value).then(done).catch(() => { document.execCommand('copy'); done(); });
+    else { document.execCommand('copy'); done(); }
+}
+
+async function revokeShareLink(id) {
+    if (!confirm('Revoke this link? Anyone using it will be signed out on their next action.')) return;
+    try { await fetch(`/projects/${SHARE_PROJECT_ID}/portal-link/${id}/revoke`, { method: 'POST' }); loadShareLinks(); }
+    catch (e) { if (window.showToast) showToast('Failed to revoke', 'error'); }
 }
 </script>
 {% endblock %}

tests/conftest.py

@@ -1,64 +0,0 @@
-"""Test harness: a throwaway SQLite DB per test, get_db overridden, a TestClient
-that does NOT run lifespan startup (so schedulers/SLMM polling stay off)."""
-import uuid
-import pytest
-from datetime import datetime
-from sqlalchemy import create_engine
-from sqlalchemy.orm import sessionmaker
-from starlette.testclient import TestClient
-
-from backend.database import Base, get_db
-import backend.models as models  # noqa: F401  (ensure all tables are registered on Base)
-
-
-@pytest.fixture()
-def db_session(tmp_path):
-    db_file = tmp_path / "test.db"
-    engine = create_engine(f"sqlite:///{db_file}", connect_args={"check_same_thread": False})
-    Base.metadata.create_all(bind=engine)
-    TestingSession = sessionmaker(autocommit=False, autoflush=False, bind=engine)
-    sess = TestingSession()
-    try:
-        yield sess
-    finally:
-        sess.close()
-        engine.dispose()
-
-
-@pytest.fixture()
-def client(db_session):
-    from backend.main import app  # imported lazily so module side effects are contained
-    def _override():
-        yield db_session
-    app.dependency_overrides[get_db] = _override
-    # No `with` → lifespan/startup events do not run (no scheduler/SLMM threads).
-    c = TestClient(app)
-    yield c
-    app.dependency_overrides.pop(get_db, None)
-
-
-@pytest.fixture(autouse=True)
-def _reset_portal_lockout():
-    """Portal lockout state is a module-global dict; clear it between tests so
-    one test's failed attempts can't lock out another."""
-    try:
-        import backend.portal_auth as _pa
-        if hasattr(_pa, "_failures"):
-            _pa._failures.clear()
-    except Exception:
-        pass
-    yield
-
-
-def make_project(db_session, name=None, **kwargs):
-    """Insert and return a Project with a unique name."""
-    p = models.Project(
-        id=str(uuid.uuid4()),
-        name=name or f"Proj {uuid.uuid4().hex[:8]}",
-        status="active",
-        created_at=datetime.utcnow(),
-        **kwargs,
-    )
-    db_session.add(p)
-    db_session.commit()
-    return p

tests/test_auth_passwords.py

@@ -1,23 +0,0 @@
-from backend.auth_passwords import hash_password, verify_password, generate_password
-
-
-def test_hash_is_not_plaintext_and_verifies():
-    h = hash_password("hunter2")
-    assert h != "hunter2"
-    assert h.startswith("$argon2")
-    assert verify_password("hunter2", h) is True
-
-
-def test_verify_rejects_wrong_password():
-    h = hash_password("hunter2")
-    assert verify_password("nope", h) is False
-
-
-def test_verify_is_safe_on_garbage_hash():
-    assert verify_password("anything", "not-a-real-hash") is False
-
-
-def test_generated_password_is_strong_and_unique():
-    a, b = generate_password(), generate_password()
-    assert a != b
-    assert len(a) >= 12

tests/test_cookie_secure.py

@@ -1,16 +0,0 @@
-import importlib
-from tests.conftest import make_project
-from backend.auth_passwords import hash_password
-
-
-def test_cookie_secure_flag_is_applied(monkeypatch, client, db_session):
-    import backend.portal_auth as pa
-    monkeypatch.setattr(pa, "COOKIE_SECURE", True, raising=False)
-    # also patch the name imported into the router module
-    import backend.routers.portal as pr
-    monkeypatch.setattr(pr, "COOKIE_SECURE", True, raising=False)
-
-    make_project(db_session, portal_enabled=True, portal_link_token="ts",
-                 portal_password_hash=hash_password("pw"))
-    r = client.post("/portal/p/ts", data={"password": "pw"}, follow_redirects=False)
-    assert "secure" in r.headers.get("set-cookie", "").lower()

tests/test_portal_access_admin.py

@@ -1,40 +0,0 @@
-from tests.conftest import make_project
-from backend.models import Project
-
-
-def test_enable_creates_link_token_and_reports_state(client, db_session):
-    p = make_project(db_session)
-    r = client.post(f"/projects/{p.id}/portal-access/enable")
-    assert r.status_code == 200
-    body = r.json()
-    assert body["enabled"] is True
-    assert body["link_url"].endswith(f"/portal/p/{db_session.get(Project, p.id).portal_link_token}")
-
-
-def test_set_password_returns_raw_once_and_stores_hash(client, db_session):
-    p = make_project(db_session)
-    client.post(f"/projects/{p.id}/portal-access/enable")
-    r = client.post(f"/projects/{p.id}/portal-access/password")
-    assert r.status_code == 200
-    raw = r.json()["password"]
-    assert len(raw) >= 12
-    fresh = db_session.get(Project, p.id)
-    assert fresh.portal_password_hash and fresh.portal_password_hash != raw
-
-
-def test_disable_turns_off_and_rotates_token(client, db_session):
-    p = make_project(db_session)
-    client.post(f"/projects/{p.id}/portal-access/enable")
-    old = db_session.get(Project, p.id).portal_link_token
-    r = client.post(f"/projects/{p.id}/portal-access/disable")
-    assert r.status_code == 200
-    fresh = db_session.get(Project, p.id)
-    assert fresh.portal_enabled is False
-    assert fresh.portal_link_token != old
-
-
-def test_get_state(client, db_session):
-    p = make_project(db_session)
-    r = client.get(f"/projects/{p.id}/portal-access")
-    assert r.status_code == 200
-    assert r.json() == {"enabled": False, "has_password": False, "link_url": None}

tests/test_portal_auth_helpers.py

@@ -1,46 +0,0 @@
-import time
-from tests.conftest import make_project
-from backend import portal_auth as pa
-from backend.models import Client, ClientAccessToken
-
-
-def test_portal_client_for_project_is_1to1_and_idempotent(db_session):
-    p = make_project(db_session)
-    c1 = pa.portal_client_for_project(p, db_session)
-    c2 = pa.portal_client_for_project(p, db_session)
-    assert isinstance(c1, Client) and c1.id == c2.id
-    assert c1.slug == f"portal-{p.id}"
-    assert db_session.query(Client).filter_by(slug=f"portal-{p.id}").count() == 1
-    # the project must be linked to its portal client, or client-scoped routes find nothing
-    assert p.client_id == c1.id
-
-
-def test_mint_portal_session_returns_usable_token_id(db_session):
-    p = make_project(db_session)
-    tid = pa.mint_portal_session(p, db_session)
-    tok = db_session.query(ClientAccessToken).filter_by(id=tid, revoked_at=None).first()
-    assert tok is not None
-    cookie = pa.make_session_cookie(tid)
-    client = pa.client_from_cookie(cookie, db_session)
-    assert client is not None and client.slug == f"portal-{p.id}"
-
-
-def test_resolve_project_by_link_token(db_session):
-    p = make_project(db_session, portal_enabled=True, portal_link_token="tok-abc")
-    assert pa.resolve_project_by_link_token("tok-abc", db_session).id == p.id
-    assert pa.resolve_project_by_link_token("nope", db_session) is None
-
-
-def test_resolve_project_ignores_disabled_portal(db_session):
-    make_project(db_session, portal_enabled=False, portal_link_token="tok-off")
-    assert pa.resolve_project_by_link_token("tok-off", db_session) is None
-
-
-def test_lockout_after_max_attempts():
-    pa.clear_failures("k1")
-    assert pa.is_locked("k1") is False
-    for _ in range(pa.MAX_ATTEMPTS):
-        pa.register_failure("k1")
-    assert pa.is_locked("k1") is True
-    pa.clear_failures("k1")
-    assert pa.is_locked("k1") is False

tests/test_portal_gate.py

@@ -1,60 +0,0 @@
-from tests.conftest import make_project
-from backend import portal_auth as pa
-from backend.auth_passwords import hash_password
-
-
-def _enabled_project(db_session, token="tok-1", password="secretpw"):
-    return make_project(db_session, portal_enabled=True, portal_link_token=token,
-                        portal_password_hash=hash_password(password))
-
-
-def test_get_prompt_renders_for_valid_token(client, db_session):
-    _enabled_project(db_session)
-    r = client.get("/portal/p/tok-1")
-    assert r.status_code == 200
-    assert "password" in r.text.lower()
-
-
-def test_get_unknown_token_shows_generic_page(client, db_session):
-    r = client.get("/portal/p/does-not-exist")
-    assert r.status_code in (403, 404)
-    assert "password" not in r.text.lower() or "isn't valid" in r.text.lower()
-
-
-def test_wrong_password_is_rejected(client, db_session):
-    _enabled_project(db_session, password="rightpw")
-    r = client.post("/portal/p/tok-1", data={"password": "wrongpw"}, follow_redirects=False)
-    assert r.status_code == 200            # re-renders the form, no cookie
-    assert "portal_session" not in r.headers.get("set-cookie", "")
-
-
-def test_correct_password_sets_cookie_and_redirects(client, db_session):
-    _enabled_project(db_session, password="rightpw")
-    r = client.post("/portal/p/tok-1", data={"password": "rightpw"}, follow_redirects=False)
-    assert r.status_code == 303
-    assert r.headers["location"] == "/portal"
-    assert "portal_session=" in r.headers.get("set-cookie", "")
-
-
-def test_lockout_after_five_wrong(client, db_session):
-    _enabled_project(db_session, token="tok-lock", password="rightpw")
-    for _ in range(5):
-        client.post("/portal/p/tok-lock", data={"password": "x"}, follow_redirects=False)
-    # 6th attempt — even the CORRECT password is refused while locked
-    r = client.post("/portal/p/tok-lock", data={"password": "rightpw"}, follow_redirects=False)
-    assert r.status_code == 200
-    assert "portal_session=" not in r.headers.get("set-cookie", "")
-    assert "too many" in r.text.lower()
-
-
-def test_enabled_without_password_is_not_accessible(client, db_session):
-    # enabled portal but no password set yet (operator enabled before generating one)
-    # must NOT show a usable form — looks like an invalid link, no self-lockout.
-    make_project(db_session, portal_enabled=True, portal_link_token="tok-nopw")
-    r = client.get("/portal/p/tok-nopw")
-    assert r.status_code == 404
-    assert "isn't valid" in r.text.lower()
-    # and a POST can't succeed or set a cookie either
-    r2 = client.post("/portal/p/tok-nopw", data={"password": "anything"}, follow_redirects=False)
-    assert r2.status_code == 404
-    assert "portal_session=" not in r2.headers.get("set-cookie", "")

tests/test_portal_migration.py

@@ -1,29 +0,0 @@
-import sqlite3
-import importlib
-
-
-def _columns(db_file):
-    conn = sqlite3.connect(db_file)
-    cols = {r[1] for r in conn.execute("PRAGMA table_info(projects)")}
-    conn.close()
-    return cols
-
-
-def test_migration_adds_columns_and_is_idempotent(tmp_path, monkeypatch):
-    db_file = tmp_path / "seismo_fleet.db"
-    conn = sqlite3.connect(db_file)
-    conn.execute("CREATE TABLE projects (id TEXT PRIMARY KEY, name TEXT)")
-    conn.commit()
-    conn.close()
-
-    monkeypatch.chdir(tmp_path)            # migration resolves data/ relative to cwd
-    (tmp_path / "data").mkdir()
-    (tmp_path / "data" / "seismo_fleet.db").write_bytes(db_file.read_bytes())
-
-    mod = importlib.import_module("backend.migrate_add_project_portal_auth")
-    mod.migrate()
-    cols = _columns(tmp_path / "data" / "seismo_fleet.db")
-    assert {"portal_enabled", "portal_password_hash", "portal_link_token"} <= cols
-
-    mod.migrate()  # second run must not raise
-    assert {"portal_enabled", "portal_password_hash", "portal_link_token"} <= _columns(tmp_path / "data" / "seismo_fleet.db")

tests/test_portal_scope.py

@@ -1,81 +0,0 @@
-import uuid
-from datetime import datetime
-import pytest
-from sqlalchemy.orm import sessionmaker
-from starlette.testclient import WebSocketDisconnect
-from tests.conftest import make_project
-from backend import portal_auth as pa
-from backend.auth_passwords import hash_password
-from backend.models import MonitoringLocation
-
-
-def _sound_location(db_session, project):
-    loc = MonitoringLocation(
-        id=str(uuid.uuid4()), project_id=project.id, name="Site",
-        location_type="sound", created_at=datetime.utcnow(),
-        sort_order=0)
-    db_session.add(loc)
-    db_session.commit()
-    return loc
-
-
-def test_session_for_A_cannot_open_B_location(client, db_session):
-    a = make_project(db_session, portal_enabled=True, portal_link_token="ta",
-                     portal_password_hash=hash_password("pw"))
-    b = make_project(db_session)
-    b_loc = _sound_location(db_session, b)
-
-    # Establish an A session
-    r = client.post("/portal/p/ta", data={"password": "pw"}, follow_redirects=False)
-    assert r.status_code == 303
-
-    # Try to open B's location page → 404 (not 403), no leak
-    r2 = client.get(f"/portal/location/{b_loc.id}")
-    assert r2.status_code == 404
-
-
-def test_session_can_open_its_own_location(client, db_session):
-    # Positive case: proves the negative test's 404 is real scoping, not a blanket
-    # "client owns nothing" failure — an A session CAN open A's own location.
-    a = make_project(db_session, portal_enabled=True, portal_link_token="ta2",
-                     portal_password_hash=hash_password("pw"))
-    a_loc = _sound_location(db_session, a)
-    r = client.post("/portal/p/ta2", data={"password": "pw"}, follow_redirects=False)
-    assert r.status_code == 303
-    r2 = client.get(f"/portal/location/{a_loc.id}")
-    assert r2.status_code == 200
-
-
-def test_ws_stream_rejects_unauthenticated(client, db_session):
-    # The live-feed WebSocket must refuse a connection with no session cookie (1008).
-    a = make_project(db_session, portal_enabled=True, portal_link_token="tw1",
-                     portal_password_hash=hash_password("pw"))
-    a_loc = _sound_location(db_session, a)
-    with pytest.raises(WebSocketDisconnect) as exc:
-        with client.websocket_connect(f"/portal/api/location/{a_loc.id}/stream") as ws:
-            ws.receive_text()
-    assert exc.value.code == 1008
-
-
-def test_ws_stream_rejects_cross_project(client, db_session, monkeypatch):
-    # The WebSocket enforces the SAME per-project ownership as the HTTP routes: a
-    # B-session opening A's stream is closed 1008 (ownership) before any device feed.
-    # The handler uses SessionLocal() directly (not the get_db override), so point it
-    # at the test DB engine so this genuinely exercises the ownership check (not a
-    # vacuous "client not found").
-    import backend.routers.portal as portal_router
-    monkeypatch.setattr(portal_router, "SessionLocal",
-                        sessionmaker(bind=db_session.get_bind()))
-
-    a = make_project(db_session, portal_enabled=True, portal_link_token="tw2",
-                     portal_password_hash=hash_password("pw"))
-    a_loc = _sound_location(db_session, a)
-    make_project(db_session, portal_enabled=True, portal_link_token="tw3",
-                 portal_password_hash=hash_password("pw"))
-    # Log in as project B, then aim the stream at project A's location.
-    assert client.post("/portal/p/tw3", data={"password": "pw"},
-                       follow_redirects=False).status_code == 303
-    with pytest.raises(WebSocketDisconnect) as exc:
-        with client.websocket_connect(f"/portal/api/location/{a_loc.id}/stream") as ws:
-            ws.receive_text()
-    assert exc.value.code == 1008

tests/test_retired_routes.py

@@ -1,20 +0,0 @@
-from tests.conftest import make_project
-
-
-def test_enter_and_open_are_gone(client, db_session):
-    assert client.get("/portal/enter/anything", follow_redirects=False).status_code == 404
-    assert client.get("/portal/open/anything", follow_redirects=False).status_code == 404
-
-
-def test_portal_link_endpoints_are_gone(client, db_session):
-    p = make_project(db_session)
-    assert client.post(f"/projects/{p.id}/portal-link").status_code == 404
-    assert client.get(f"/projects/{p.id}/portal-links").status_code == 404
-    assert client.post(f"/projects/{p.id}/portal-link/sometoken/revoke").status_code == 404
-
-
-def test_preview_still_mints_a_session(client, db_session):
-    p = make_project(db_session)
-    r = client.get(f"/projects/{p.id}/portal-preview", follow_redirects=False)
-    assert r.status_code == 303
-    assert "portal_session=" in r.headers.get("set-cookie", "")