# FTP Report Pipeline — session brief

**Branch:** `feat/ftp-report-pipeline` (off `dev`), worktree `/home/serversdown/terra-view-reports`.
**Scope:** Terra-View only. Do NOT touch SLMM — the SLMM alert/monitor work is live in a
parallel session on `slmm` branch `feat/drd-fix`. Pull device data through the **existing**
SLMM FTP proxy endpoints; add no SLMM code (for v1).

See memory note `client_sound_monitoring_job_2026-07` for the client requirements + timeline.

## Goal
Automated **daily morning report** for the John Myler 3-location sound job: each AM, last
night's noise levels vs the **baseline week**, per location. Data pulled from the meters via
FTP (the meter records 24/7 to SD regardless of TCP wedges). Alerts are a *separate* workstream
(SLMM, real-time DOD) — not in scope here.

## The big realization (why this is small)
The hard parts already exist:
- **SLMM (use as-is, via the `/api/slmm/...` proxy):**
  - `GET  /api/slmm/{unit}/ftp/files?path=/NL-43` → list files/folders
  - `POST /api/slmm/{unit}/ftp/download-folder` → returns the `Auto_####` folder as a **ZIP**
- **Terra-View ingest (reuse):** `backend/routers/project_locations.py:1743` `upload_nrl_data`
  already accepts a **ZIP**, extracts, keeps `.rnh` + `_Leq_ .rnd` (drops `_Lp_`/junk via
  `_is_wanted`), runs `_parse_rnh` (line 1687) → creates `MonitoringSession` + `DataFile`.
- **Report generator (reuse, source-agnostic):** `backend/routers/projects.py`. The `.rnd`
  file reads funnel through 3 helpers — `_peek_rnd_headers` (~135), `_is_leq_file` (~147),
  `_read_rnd_file_rows` (~256). `.rnd` files live on disk under `data/{file_path}` (DataFile
  holds the path, not a BLOB). The stats/Excel/formatting logic doesn't care where bytes come from.

## Build (Terra-View)
1. **Refactor** `upload_nrl_data`'s core into a callable `ingest_nrl_zip(location_id, zip_bytes, db)`
   so it can be invoked programmatically (not only via HTTP UploadFile).
2. **Scheduled pull job** (reuse the existing scheduler): per project location/unit →
   `GET /ftp/files` to find new `Auto_####` folders → `POST /ftp/download-folder` (zip) →
   `ingest_nrl_zip(...)`. **Dedup** so repeated pulls don't duplicate sessions/files
   (track ingested folder names per location).
3. **Baseline aggregation:** aggregate the baseline-week `_Leq_` intervals per location →
   reference values (nighttime Leq, L90 floor, typical Lmax).
4. **Nightly report + email:** compute last night's metrics per location, compare to baseline
   (deltas), render (reuse the Excel/report machinery), email each morning.

## Data-location decision (light version, agreed)
Keep `MonitoringSession`/`DataFile` **metadata in TV** for now; reuse the existing on-disk file
store. Optional refinement (later): have SLMM keep the pulled files and TV read them through a
SLMM file-serve endpoint (avoids the copy-into-TV step). Don't do that refinement under the
deadline unless trivial — the report logic is identical either way.

## Open questions to resolve early
1. **What's actually in a `_Leq_ .rnd`** — Leq only, or Leq + Lmax + Ln per 15-min interval?
   Decides whether the night-vs-baseline report can show L90/Lmax or just Leq. Inspect a real file.
2. **Session rollover / dedup** — does a 2-week run write one growing `Auto_####` folder or new
   folders? Drives the "what's new" logic.
3. **`download-folder` over a multi-day run** — confirm it zips cleanly (size/time).

## Client params (confirm with Dave before locking)
Threshold/metric + their "night" window; report recipients + format (email body vs PDF/Excel).

## Timeline
Setup ~7/1–7/2 (baseline week), shutdown week through ~7/17. Reports needed by ~7/8 (before
shutdown). Today is ~3 weeks out — reliability > features.