- #1 XSS: escape user-controlled values (location name, baseline values, recent-
report fields, SMTP status message) in the modals via the existing _mergeEsc
helper — they were concatenated raw into innerHTML (stored XSS via location name).
- #2 SMTP: an unrecognized REPORT_SMTP_SECURITY no longer silently downgrades to a
plaintext connection while still calling login() — it falls back to starttls and
warns; warn on intentional security=none + auth.
- #3 scheduler: run the (blocking smtplib + Excel) nightly report in a worker thread
(asyncio.to_thread + its own DB session) so it can't stall the loop that drives
time-sensitive device cycles. New _run_one_report helper.
- #4 cycle ingest: set unit_id on the ingested data session (ingest_nrl_zip leaves
it None) before dropping the empty placeholder, preserving the unit<->session link;
repoint old_session_id at the real row.
- #7 robustness: wrap send_report_email in the orchestrator and run_nightly_report in
/view + /run so a render/SMTP error returns a clean error instead of a raw 500
after artifacts are written.
Verified: SMTP paths (typo->starttls, none, starttls, ssl), off-thread tick stamps
last_run_date + writes the file, /view 200, escaping wired, app imports.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- PORTAL_OPEN_LINKS now defaults OFF — /portal/open/* is an unauthenticated,
proxy-reachable session-minting path (and a linked project's open link grants
the whole client's scope), so it must be explicitly enabled in dev.
- Session cookie: enforce server-side expiry (check iat vs COOKIE_MAX_AGE — was
browser-only) and guard a non-dict signed body (was an uncaught AttributeError →
500, reachable if SECRET_KEY is the insecure default).
- Escape operator-set strings (location/rule/event names) before innerHTML +
Leaflet tooltips — they're client-facing, so a name with markup was stored XSS
in the client's browser. Global esc() helper applied at every injection point.
- WS _scrub_frame drops a non-JSON frame instead of forwarding it raw; /history
rows now whitelisted like the other scoped endpoints.
- Preview-client slug uses the full project id (an 8-char prefix could collide
two projects onto one client).
Verified: cookie reader (fresh/expired/non-dict/missing-iat) + open-links default
off; templates parse; scoped scrubbing intact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New scoped GET /portal/api/location/{id}/thresholds returns the enabled alert
rules (scrubbed: name/metric/comparison/threshold/duration/schedule — no cooldown
or hysteresis internals). Location page renders an "Alert limits" panel above the
history, e.g. "Night noise · Leq above 65 dB for 60s · 22:00–07:00", hidden when
no limits are set. Gives the breach history context.
Verified: portal.py compiles; location script balances; template parses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
render_excel(report): one worksheet per location — interval table, a line chart,
and a Last/Base/Δ summary per window. Metric-driven, so it tracks whatever metric
set is configured.
- orchestrator: render report.xlsx alongside report.html, attach it to the email
(dry-run until SMTP set), expose xlsx_path. Never lets a spreadsheet error sink
the report.
- reports router: /list includes xlsx_url when present; new
GET /archive/{date}/xlsx serves the saved spreadsheet.
- UI: Recent-reports rows get an "Excel" download link.
Verified: real Feb data -> valid .xlsx (sheet per NRL, interval table + chart +
summary with real values), attachment path runs, both archive routes registered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Baseline can now come from fixed values typed per location, not just captured
data — for a spec limit ("L10 = 85") or a prior report's averages when the raw
data isn't available.
- SoundReportConfig.baseline_mode ("captured" | "reference").
- report_pipeline: _location_reference_baseline() reads per-location values from
location_metadata; build_*_night_report honor baseline_mode (reference cells
use the typed value; unset metrics compare against nothing).
- reports router: GET/PUT /reports/baseline (mode on config + per-location values
in location_metadata); config carries baseline_mode; manual view/run fall back
to the saved config's baseline when no explicit dates are given.
- orchestrator + scheduler tick thread baseline_mode through.
Verified end-to-end: PUT/GET /baseline, reference deltas (L10 66.6 vs 85 -> -18.4),
unset metrics compare against nothing, captured-mode regression intact.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Internal (SLM detail page): live alarm-state badge in the Alerts header
(● N active / ✓ all clear), a History list of fired events (onset → clear, peak
dB, ack status) with an Ack button, refreshed every 20s. Reads the existing SLMM
/alerts/events + /ack via the proxy.
Portal (client, read-only, scoped): new GET /portal/api/location/{id}/events —
ownership-gated, returns a scrubbed projection (rule_name/metric/threshold/onset/
peak/clear/status only; no internal ids or ack-by) plus an `active` count. The
location page shows a red "Currently above threshold" banner when active and a
read-only breach history, polled every 20s. No ack on the client side.
Verified: portal.py compiles; both scripts balance; both templates parse.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a frictionless shareable link so anyone can open a project's client portal
during dev without minting/copying a magic token. GET /portal/open/{project_id}
(gated by PORTAL_OPEN_LINKS) provisions the client session and lands on /portal;
lives under /portal so it works through a proxy exposing only /portal/*.
The project page's "Copy client link" modal now leads with this Quick share link
(amber, host taken from window.location.origin so it always matches the host you
copied it from — no more LAN-vs-public foot-gun). The token-based generate/list/
revoke stays below for the eventual secure path.
PORTAL_OPEN_LINKS defaults ON for the prototype (whole app is open anyway) and logs
a warning; set =false before real clients. The get_current_client seam is
untouched, so M4 auth still layers in front of the same routes regardless.
Verified: compiles, share script balances, detail.html parses, flag default
on / =false off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backend (reports router):
- POST /reports/test-email — send a test email (body/config recipients; dry-run
if SMTP unset) to verify the relay.
- GET /reports/list — list generated report artifacts on disk (newest first).
- GET /reports/archive/{date} — serve a saved report.html (traversal-guarded).
Frontend (sound project header modals):
- Night Report modal: "Run & Email" button (POST /run) + a "Recent reports" list
(GET /list → opens the archived report.html in a new tab).
- Settings modal: schedule + last-run status line, and a "Send test email" button.
Verified: endpoints (run→list→archive, traversal blocked, test-email recipient
fallback) and the template renders with all four wired + gated to sound projects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add SoundReportConfig (one row per project) + the scheduler tick that runs the
nightly report on its own:
- model SoundReportConfig (enabled, report_time, metric_keys, baseline range,
recipients, last_run_date) — new table, auto-created by create_all (no migration).
- GET/PUT /api/projects/{id}/reports/config with validation.
- SchedulerService.run_due_reports(): each loop, for every enabled config past
its report_time, run last night's report once (dedup via last_run_date),
writing the file + emailing (dry-run until SMTP is set).
- UI: gear button beside "Night Report" opens a settings modal (enable, time,
baseline range, metrics, recipients) that GET/PUTs the config.
Verified: table registers + auto-creates, config CRUD + validation, tick
runs/dedups, templates render and gate to sound projects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The portal location view is now genuinely live, not a 15s poll. Scoped WS endpoint
/portal/api/location/{id}/stream: authenticates via the session cookie, enforces
ownership (resolve_client_location), then bridges the unit's shared SLMM /monitor
fan-out feed to the browser — a viewer is just one more subscriber, no extra
device connection. Frames are scrubbed to the portal whitelist (drops unit_id,
raw_payload, counter, lmin) before reaching the client.
location.html: cache prefill for instant first paint, then upgrades to the live
socket (cards tick ~1Hz, chart scrolls). Auto-close so an abandoned tab can't pin
the device at 1Hz polling (~8x cellular data):
- closes when the tab is hidden, reopens when visible (Page Visibility) — the main
guard;
- hard 15-min cap -> "Live paused — click to resume" overlay.
Refactor: client_from_cookie() extracted from get_current_client so the WS handler
(no Request-based Depends) can auth the same way.
Verified: scrub drops internal fields / keeps metrics + heartbeat (7/7), auth
refactor (3/3), portal compiles, location.html JS balances + parses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add backend/routers/reports.py (registered in main.py):
- GET /api/projects/{id}/reports/nightly/view — render the night report
HTML inline (preview; no write, no email)
- POST /api/projects/{id}/reports/nightly/run — build -> write
report.html/report.json to disk -> dry-run email -> JSON result + view_url
Same entry point the scheduled morning tick will reuse. Query params:
night_date (default last night, local tz), baseline_start/end, metrics, send.
Orchestrator now also returns the rendered html for inline display.
Verified via FastAPI TestClient on real meter data (200 HTML with the computed
numbers, files written to disk, 400/404 validation paths).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/portal overview: client's active sound locations as live tiles (current Lp +
Live/Stopped badge + "updated Xm ago", polled from the scoped cache every 15s)
plus a Leaflet map of locations with coordinates. /portal/location/{id}: 404-gated
read-only live panel — Lp/Leq/Lmax/L1/L10 cards + a 4-line Chart.js trace
(backfilled from /history) + measuring/freshness badge. Cache-only, 15s poll, no
device controls, no refresh-from-device. _client_locations() feeds the overview.
Verified: portal.py compiles; both inline scripts balance; all four portal
templates parse in Jinja2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resolve_client_location() enforces ownership (sound location in one of the
client's active projects) and 404s everything else — same response for missing
and not-yours, so location existence never leaks. active_unit_for_location()
resolves the currently-assigned SLM.
Scoped GET /portal/api/location/{id}/live and /history: gate -> resolve unit ->
read SLMM cache (never the device). /live returns a SCRUBBED projection (sound
metrics + run state only; no battery/SD/raw_payload). Both degrade gracefully
when there's no device or SLMM is down.
Verified: ownership gate (owns / other-client / vibration / deleted-project /
removed / nonexistent) + active-vs-completed unit resolution — 8/8 on a temp DB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
backend/portal_auth.py: stdlib HMAC-signed session cookie carrying the access-
token id (re-validated against the DB each request, so revoke kills live
sessions), hash_token, resolve_token, and the get_current_client dependency
(raises PortalAuthError). SECRET_KEY env (insecure dev default + warning).
routers/portal.py: /portal/enter/{token} mints the cookie -> /portal; /logout;
/access; /portal home stub. main.py registers the router + a PortalAuthError
handler (HTML access page for pages, 401 JSON for /portal/api/*).
Portal shell templates (base, access_required, overview stub), branded dark.
Verified: cookie round-trip + tamper/garbage rejection, token resolution
(valid/bad), get_current_client (valid/no-cookie/revoked) — 8/8 against a temp DB.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Terra-View side of the daily night-vs-baseline sound report for the John Myler
24/7 job. Engine is built and verified end-to-end against real meter data;
SMTP send + scheduler/capture wiring still pending.
- ingest: refactor upload_nrl_data into a callable ingest_nrl_zip(location_id,
zip_bytes, db) sharing one core with the HTTP endpoint. Capture the .rnh
percentile map + weightings into session metadata; dedup on store-name +
start time. Ingest stays metric-agnostic (every Leq column preserved).
- report_pipeline.py: metric registry, Evening/Nighttime windows, correct
aggregation (Lmax=max, Ln=arithmetic, Leq=logarithmic), baseline = typical
night, per-location + per-project builders.
- report_renderers.py: HTML email-body renderer (Last/Base/delta layout).
- report_email.py: config-driven SMTP via stdlib (env vars) with a dry-run
fallback so the pipeline runs without credentials.
- report_orchestrator.py: compute -> render -> always write report.html +
report.json to disk -> best-effort email.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. "No recent check-in" was always shown because the row's last-check text read
unit.slm_last_check (a Terra-View roster field the monitor never updates),
while the live freshness lives in SLMM's cached NL43Status.last_seen. Carry
that last_seen onto the unit (unit.cache_last_seen) and display it (falling
back to slm_last_check). Also treat "Measure" as Measuring in the badge, to
match the panel and the cache's MEASURING_STATES.
2. The dashboard card chart only had Lp + Leq datasets, so L1/L10 never drew even
though the cards showed them. Add L1 (purple) and L10 (orange) datasets and
feed ln1/ln2 in both the /history backfill and the live /monitor frames.
Percentiles parse via numOrNull so a missing "-.-" leaves a gap (spanGaps)
instead of dropping the line to 0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
get_live_view fired two device calls on every command-center load:
/measurement-state (sends Measure?) and /live (fresh DOD read) — competing
with the monitor's DOD polling. Both are now redundant: the keepalive monitor
keeps NL43Status fresh (~1.3s) and the live-stream WS handles ongoing updates.
Read the cached /status once instead (no device call); derive is_measuring
from measurement_state. Command center opens instantly without poking the
device. (Relies on monitor_start_time now being in /status.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
"No recent check-in" read a roster field (slm_last_check) that nothing
stamps, and the live-status fetch hit /measurement-state — which sends
Measure? to the DEVICE every refresh, competing with DOD polling.
Now read SLMM's /roster once: it carries each unit's cached NL43Status
(last_seen, measurement_state) — a cache read, no device call. is_recent is
derived from last_seen (advances only on a successful monitor poll, so
staleness == not being reached) within 5 min, for all non-retired units
(benched units can still be monitored). Net: fewer device calls AND the
dashboard reflects the live monitor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /monitor WS proxy cancelled its sibling task on disconnect but then
`except Exception` failed to swallow the resulting CancelledError (a
BaseException), so stopping the stream raised "Exception in ASGI
application". It also only awaited the pending task, leaving the done
task's WebSocketDisconnect unretrieved ("Task exception was never
retrieved"). Await all tasks and catch (CancelledError, Exception).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SLM live view now consumes SLMM's shared DOD /monitor feed instead of
the per-client DRD /stream. This fixes the single-connection contention
(many viewers share one device feed) and finally puts L1/L10 in the live
chart (DRD couldn't carry percentiles).
- New WS proxy handler /api/slmm/{unit}/monitor -> SLMM /api/nl43/{unit}/monitor.
Uses asyncio.wait(FIRST_COMPLETED) + cancel-sibling instead of gather(), so
it doesn't leave a task sending into a closed socket ("Unexpected ASGI
message after close").
- Live view JS points at /monitor; onmessage reflects feed_status and ignores
heartbeat / unreachable frames so they don't blank the cards or zero-spike
the chart. Adds a small Live/Device-offline badge.
Still on the old /live (DRD): the dashboard live tile (sound_level_meters.html)
— next slice.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v0.13.0 shipped the mic_unit_pref default as "dBL", which made the
website chart's mic axis inconsistent with the PDF report (which
renders psi). Original brief was always "psi on charts, dBL on
peaks" — I implemented the default backwards. Operator caught it
within an hour of rollout.
Same-day patch:
- backend/models.py: default "dBL" → "psi"
- migrate_add_mic_unit_pref.py: idempotent across both fresh DB
("add column with psi default") and v0.13.0 upgrade ("flip dBL
rows to psi"). One-row table, freshness assumed.
- backend/routers/settings.py: GET/PUT fallback "dBL" → "psi"
- templates/settings.html: dropdown's `selected` flag moves to psi
+ reorders options + relabels with "(matches PDF report)" hint
- backend/static/event-modal.js: module-level fallback + branch
conditions flip to make psi the unset/error default
Includes the "Captured at" → "Time received" relabel from earlier
in the day (already-shipped commit 43c804d) rolled into the
release notes.
Migration is idempotent + safe to re-run; rolled out on the dev
container during this commit's smoke test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New UserPreferences field controls the mic channel's unit on the
SFM event-detail modal's waveform chart only. "dBL" default,
"psi" alternate. Peaks everywhere else (tables, KPI tiles, modal
summary) stay in dBL regardless — this is strictly a chart-axis
preference.
Surfaced as a single dropdown on Settings → General, below the
auto-refresh interval.
Setting up the storage half ahead of the chart port in the next
commit, so the chart can read the value from /api/settings/preferences
on first render instead of needing a follow-up wiring pass.
Includes idempotent backend/migrate_add_mic_unit_pref.py for fleets
already on an older schema.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deployment timestamps were stored correctly as UTC but rendered raw —
a 1:30 PM EDT swap displayed as "5:30" because the frontend sliced the
naive UTC ISO string straight to the screen.
Display side: deployment_timeline.py now converts every emitted
timestamp (starts_at, ends_at, event_overlay.peak_pvs_at and last_event)
through `utc_to_local()` using the user's configured timezone from
UserPreferences before serializing. Frontend slice keeps working — it
just slices a local-time string now.
Write side (so the new edit / add-historical-assignment modals stay
consistent):
- PATCH /api/projects/{pid}/assignments/{aid}
- POST /api/projects/{pid}/locations/{loc}/assign
both now interpret a *naive* assigned_at / assigned_until ISO string as
the user's local time and convert to UTC for storage via
`local_to_utc()`. Explicit tz-aware strings ("...Z" or "...+00:00")
skip the conversion so programmatic callers that already speak UTC
keep working.
Verified live: BE13121's stored 2026-01-28 18:06:29 UTC now serializes
as 2026-01-28 13:06:29 in the timeline endpoint; PATCHing
"2026-01-28T13:06:29" round-trips back to the same UTC value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The legacy RosterUnit.deployed flag drives heartbeat polling and
benched-vs-deployed roster filters. Three workflows ended an
assignment without flipping it, so the outgoing unit kept being
polled and showed up as "deployed" forever:
- swap endpoint (POST /locations/{loc}/swap)
- unassign endpoint (POST /assignments/{aid}/unassign)
- promote-pending endpoint (POST /deployments/pending/{id}/promote)
All three now: close the previous active assignment, break the
outgoing unit's modem pairing (both directions), and set
`deployed = False` on the outgoing unit. Unassign and swap also
clear the modem's back-reference.
The promote-pending path additionally handles the case where the
target location already has an active assignment — that previously
silently created two active assignments at the same location. Now
the old one is closed (assigned_until = pending capture time, status
= completed), the old unit is benched and unpaired, and an
"assignment_swapped" history row is written. Incoming unit gets
`deployed = True` if it was on the bench.
Verified live: triggered a swap via the existing endpoint and saw
the outgoing unit flip True → False while the incoming flipped
False → True. Test mutations rolled back.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each assignment row in the timeline now gets an inline edit (pencil)
that opens a modal with `assigned_at`, `assigned_until`, and notes.
Save calls the existing `PATCH /api/projects/{pid}/assignments/{aid}`;
delete (for misclicks) calls the existing `DELETE`. Open-ended
checkbox clears `assigned_until` and the endpoint flips status back
to "active".
Adds an "+ Add deployment record" button at the top of the timeline
for backfilling historical windows when orphan events sit outside any
assignment. Modal: project → location → assigned_at → assigned_until
(optional open-ended) → notes.
Backend: the `/locations/{loc}/assign` endpoint now accepts an
`assigned_at` form field and a closed-window assignment. The previous
blanket "location already has an active assignment" check is replaced
with same-location overlap detection — closed historical windows that
don't overlap an existing assignment are accepted (which is exactly
the backfill case).
After any save/delete the timeline reloads and the SFM-events list
re-fetches so previously-orphaned events flip to "attributed" when
their timestamp now falls inside an assignment window.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`available-units` and `available-modems` now accept `include_benched=true`
to also return units/modems with `deployed=False`. Default is False so
the existing location-detail swap modal is unchanged. Each row carries
a `deployed` boolean for badge rendering. The Unit Swap wizard fetches
with the flag enabled — exactly the candidates a field tech pulls off
the shelf.
The /swap endpoint now flips the incoming unit (and modem) back to
`deployed=True` when they came in benched, keeping the legacy roster
flag consistent with the active-assignment signal.
Adds the symmetric half of the orphan-pairing fix: when a newly-paired
modem still claims a different seismograph (whose
`deployed_with_modem_id` was never cleared in a past swap), break that
stale back-reference before re-pairing.
`locations-with-assignments` includes `modem.deployed` so the wizard
can badge the current modem in the location card, the "Keep current
modem" choice, the picker rows, and the review screen.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
UI for the pending-deployment workflow (commits 2 + 3 from the plan,
landed together since commit 1 already shipped the full backend).
New surfaces
- /deploy — mobile-first 3-step wizard. Pick unit → take photo (uses
<input capture="environment"> so it opens the phone camera) → add
optional note + submit. EXIF GPS auto-extracted on the server.
Success page shows the captured coords + links to either "Deploy
another" or "View pending hopper." Whole flow is meant to take
under 90 seconds on site.
- /tools/pending-deployments — the hopper. Filter pills: Awaiting /
Assigned / Cancelled. Each card shows photo thumbnail, unit serial
link, captured-at timestamp, coordinates, operator note, and
status-appropriate actions.
- Classify modal on the hopper: two modes — "Assign to existing
location" (project + location pickers, scoped to vibration_monitoring)
or "Create new location" (with new-or-existing project, plus a
"use captured coords" checkbox that writes the pending row's coords
onto the new location). Calls /pending/{id}/promote on submit.
- Cancel button uses prompt() for the optional reason → POSTs to
/pending/{id}/cancel.
Backend additions
- GET /api/deployments/seismograph-picker — JSON list of non-retired
seismograph units for the /deploy unit picker. Annotates each unit
with has_pending so the picker can flag units that already have a
pending capture waiting.
Discovery
- New "Field Deploy" + "Pending Deployments" cards on /tools.
- Dashboard banner: auto-shows when there are awaiting captures,
polled every 30s. Hides when count drops to 0. Click → /tools/
pending-deployments.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Field-install workflow needs to be fast: arrive on site, snap a photo
of the seismograph in place, leave. Project / location classification
happens later at a desk. This adds the data model + capture endpoint
for that workflow.
Data model
- New PendingDeployment table. Lifecycle: awaiting → assigned (when
promoted to a real UnitAssignment) or → cancelled (operator's
mistake). Photos are filesystem files under data/photos/{unit_id}/
with the filename stored on the row.
- Migration: backend/migrate_add_pending_deployments.py (idempotent).
Endpoints
- POST /api/deployments/capture — multipart upload (unit_id, photo,
optional note). Refuses non-seismographs. Extracts EXIF GPS
(cribbing extract_exif_data from routers/photos.py) and stores
the captured "lat,lon" on the row. Saves the photo under
data/photos/{unit_id}/install_YYYYMMDD_HHMMSS_<uuid8>.<ext>.
Returns the new pending_deployment_id + extracted coords + photo
URL for the client to render confirmation.
- GET /api/deployments/pending — list by status (default awaiting)
- GET /api/deployments/pending/{id} — single row detail
- POST /api/deployments/pending/{id}/promote — classify → create
UnitAssignment. Body accepts two shapes: assign-to-existing-location
OR create-new-location (with new-or-existing project). Sets
status=assigned, resulting_assignment_id, promoted_at.
- POST /api/deployments/pending/{id}/cancel — abandon with optional reason.
All four routes write UnitHistory audit rows
(pending_deployment_captured / _promoted / _cancelled).
Events from a unit with an unclassified pending deployment land in the
unit's "Unattributed" events bucket as usual. Once promoted, the new
UnitAssignment's window retroactively attributes them — same mechanism
the metadata-backfill tool uses.
Seismograph-only for v1. SLM deployments don't follow this pattern
and are tracked elsewhere. Capture refuses non-seismograph unit_ids
with HTTP 400.
UI (commits 2 + 3) lands next.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The per-unit Gantt chart on /unit/{id} (Phase 1, v0.11.0) was scoped
to one unit's deployment timeline. This adds the fleet-wide view as
a new entry under /tools.
What it shows
- 12-month calendar grid styled like the Job Planner (4 months per
row, responsive down to single column on mobile).
- Each day cell shows up to 4 colored mini-bars — one per project
that had ≥1 active UnitAssignment that day, color deterministically
hashed from project_id. Days with >4 active projects show "+N".
- KPI strip at the top: project count, distinct unit count, total
assignment count in the window.
- Collapsible project legend: ordered by first-active date (which
matches the deployment-history reading order), each row links to
the project page, shows the assignment count.
Click-a-day side panel
- Click any populated day cell → slide-over panel from the right
- Groups by project, lists every (unit, location) active that day
- Per-deployment: unit link, location link, window dates, active /
closed badge, "auto-backfilled" tag for metadata_backfill source
- Sources from a new GET /api/admin/deployment-history/day endpoint
Navigation
- Prev / Next month buttons shift the 12-month window by one month
- "Recent" button jumps back to default (12 months ending now)
- Default window is 11 months back from current month — operator
sees the recent past on first load, not future emptiness
Files
- backend/services/deployment_history.py — data builder + day-detail
helper. Walks UnitAssignment windows, intersects with the 12-month
range, computes per-project active-day sets.
- backend/routers/deployment_history.py — page route + day-detail JSON
endpoint. Wired into main.py.
- templates/admin/deployment_history.html — page + side-panel
- templates/tools.html — new card linking to the page
Phase 3 (deferred): drag-to-resize bars to retroactively adjust
assignment windows from inside the calendar; per-unit row view
(complement to the project-row view) for "where has unit X been across
all jobs"; horizontal scroll for >12-month windows.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
For vibration projects, "Sessions: 0" on every location card was
misleading — monitoring sessions don't exist under the watcher-forward
pipeline. The relevant number is how many SFM events have been
attributed to the location.
get_project_locations now fans out events_for_location() concurrently
across all vibration locations in the project (via asyncio.gather) and
injects event_count into each item's payload. Sound locations are
unchanged — they still show session_count.
The template already had the conditional rendering ready from the
previous commit:
{% if item.event_count is defined and item.location.location_type == 'vibration' %}
<span><strong>{{ event_count }}</strong> events</span>
{% else %}
<span>Sessions: {{ session_count }}</span>
{% endif %}
so this commit is purely the data-layer change that activates it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Project location cards now reorderable via drag-and-drop, and the
four inline action buttons (Unassign/Edit/Remove/Delete) collapse into
a single three-dot kebab menu — much cleaner card layout, especially
for projects with many locations.
Data
- MonitoringLocation.sort_order: nullable Integer, default 0.
Migration `migrate_add_location_sort_order.py` adds the column and
seeds existing rows with sort_order = alphabetical index per project
(so the post-migration display order matches what operators see
today — no surprise reordering).
- get_project_locations + locations-json: ORDER BY sort_order, name.
- Location-create: assigns max(sort_order) + 1 so new locations land
at the END of the list rather than being interleaved alphabetically.
Reorder endpoint
- POST /api/projects/{p}/locations/reorder
Body: { location_ids: [uuid, uuid, ...] }
Validates: all ids belong to this project; raises 404 on missing.
Applies 0-indexed sort_order matching the provided order.
UI changes (templates/partials/projects/location_list.html)
- Active cards get a draggable="true" attribute + native HTML5
drag/drop handlers. Drop reorders the DOM immediately, then posts
the new order to the reorder endpoint. Drop-zone visual feedback
(orange ring on hover, opacity on source during drag).
- Six-dot drag handle icon on the left of each active card; whole
card body is the drag source but the handle is the visual cue.
- Right side: small Assign pill (only shown when unassigned) +
three-dot kebab menu containing Unassign/Edit/Remove/Delete.
Click ⋮ to toggle; click outside or Escape to close. Only one
menu open at a time.
- Removed locations are NOT draggable (their order is historical) and
keep their existing Restore button visible.
The card also shows "{N} events" instead of "Sessions: N" when the
location_type is vibration AND the backend passes event_count in
the payload — which lands in commit 2 of this redesign.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
rapidfuzz.fuzz.WRatio inflates scores when two strings share substring
tokens, even when the shared tokens are common boilerplate. For
project names this is desirable (catches typos like '1-80' vs 'I-80')
but for location names it produces obvious false positives:
'Area 2 - Brookville Dam - Loc 2 East'
vs
'Area 1 - Loc 1 - 87 Jenks' → WRatio 85.5 (above 0.80 fuzzy threshold)
These share only 'area' + 'loc' + a digit but score 85%+ because WRatio
weights partial-substring overlap heavily. Operator reported the
backfill tool suggesting completely unrelated locations as 86% matches.
Fix: introduce `location_similarity()` — token_set_ratio + multi-digit
mismatch penalty. Used for location matching everywhere; WRatio stays
as the scorer for project names where its leniency is correct.
The multi-digit penalty (-0.30) triggers when both strings contain 2+-
digit numbers and none overlap. Catches the harder "same project,
different address identifier" case:
'Area 1 - Loc 2 - 68 Jenks' vs 'Area 1 - Loc 1 - 87 Jenks'
token_set_ratio = 0.91 (would still match without penalty)
multi-digit tokens {68} and {87} disjoint → -0.30 → 0.61 (rejected)
Single-digit tokens ('Loc 1', 'Area 2') are excluded from the penalty
because they're often coincidentally shared.
Updated:
- backend/services/metadata_backfill.py: new location_similarity()
function; _find_best_match() gains a `kind` parameter that selects
scorer; cluster-match call site passes kind='location'
- backend/routers/metadata_backfill.py: locations_search endpoint
(the typeahead dropdown's data source) uses location_similarity
instead of similarity for the same reason
Verified all six test cases land correctly:
- user-reported false positive: 0.85 → 0.59 (rejected)
- '87 Jenks' vs '68 Jenks': 0.90 → 0.61 (rejected)
- NRL-01 vs NRL-02: 0.83 → 0.53 (rejected)
- 'Loc 2 - 735 Bunola' vs 'Loc 2 735 Bunola Rd': 1.00 (still matches)
- punctuation-only difference: 1.00 (still matches)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The safety check that refuses to delete assignments with real recording
history referenced MonitoringSession.start_time, but the actual column
is MonitoringSession.started_at. Every DELETE call to /assignments/{id}
crashed with AttributeError before doing anything.
Now uses started_at correctly. Verified end-to-end on dev.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a unit had its assignment closed-then-reopened (e.g. via the
recent location remove/restore flow) or had metadata-backfill auto-
create a retroactive window adjacent to a manual one, the deployment
timeline showed N stacked rows that represented one continuous
deployment. Visual noise that didn't match reality.
Merge feature
- New endpoint POST /api/projects/{p}/assignments/merge
- Body: { assignment_ids: [uuid, ...] }
- Keeps earliest record, extends its window to span all inputs,
deletes the others, logs `assignment_merged` to UnitHistory
- Validates: all assignments share same unit + location, all
belong to the same project
- deployment_timeline_for_unit() now auto-detects mergeable groups
(consecutive same-location assignments within 7-day gap tolerance)
and returns them in `merge_groups` as a list of id-lists
- Unit detail page shows a blue banner above the timeline list when
groups exist, with one "Merge into one" button per group. Each
mergeable row gets a small "mergeable" badge to make the
relationship obvious.
Per-unit Gantt chart (Phase 1 of the deployment-history calendar)
- Plain-SVG horizontal timeline rendered above the existing Deployment
Timeline list, ~140px tall
- One colored bar per assignment, color-keyed by location (auto-
assigned palette + legend)
- Reduced opacity for closed bars; small white dot at the right edge
of active bars; today marker as a dashed orange vertical line
- Month gridlines (or every-3-month gridlines when domain > 24 months)
- Metadata-backfilled assignments get a blue outline so you spot
which were auto-attributed
- Mergeable groups get a dashed blue underline tying their bars
together visually
- Click any bar → smooth-scrolls the matching list row into view
and flashes a ring around it
- Hover any bar → tooltip with location + window + event count
- Auto-hides on units with no deployment history
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When an operator accidentally clicks Assign multiple times on the same
location (or assigns the wrong unit), the resulting bogus assignment
rows cluttered the location's deployment history with no way to clean
them up — Unassign just sets assigned_until to now, which preserves
the row.
New DELETE /api/projects/{p}/assignments/{a} endpoint hard-deletes the
row entirely, intended for mis-clicks that never represented a real
deployment.
Safety:
- Refuses if any MonitoringSession exists in the assignment's window
for the same (unit, location). If there's a recording session
backing it, this isn't a mis-click — operator should Edit or
Unassign instead.
- Records UnitHistory `assignment_deleted` so the unit's deployment
timeline still shows the deletion happened, even though the row
itself is gone.
UI: trash icon added next to the existing pencil (Edit) icon on each
row of the vibration location's "Deployment History" panel. Confirms
intent with a descriptive prompt that explains the consequence
(attribution becomes unattributed for that window) and points to
Edit/Unassign as alternatives.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
When a client drops a location from scope mid-project (e.g. the office
half of a museum+office monitoring job), operators couldn't previously
mark it as no-longer-active without either deleting it (which would
orphan historical events) or leaving it in the active list looking
deployable. Now there's a proper middle ground.
Data model
- MonitoringLocation gets two new nullable columns:
- removed_at — NULL means active; set means soft-removed
- removal_reason — optional operator note
Migration: backend/migrate_add_location_removed.py (idempotent)
Endpoints
- POST /api/projects/{p}/locations/{l}/remove
Body: { effective_date?: ISO-datetime, reason?: str }
Side effects (cascade):
1. Closes active UnitAssignment rows at this location
(assigned_until = effective_date, status = "completed")
2. Cancels pending ScheduledActions at this location
3. Marks location.removed_at = effective_date
Returns counts of assignments closed + actions cancelled.
- POST /api/projects/{p}/locations/{l}/restore
Clears removed_at + removal_reason. Does NOT auto-reopen
assignments — operator creates new ones if resuming monitoring.
Active-surface filters
- locations-json defaults to active-only; pass include_removed=true
for historical / reporting views. Schedule modal dropdowns now
exclude removed locations automatically.
- Metadata-backfill fuzzy matcher excludes removed locations from
proposed targets (don't want backfill creating new assignments at
decommissioned locations).
- Vibration-summary per_location rollup includes removed locations
(so historical event totals stay accurate) but tags each with
removed_at so the UI can show a badge.
UI
- Project detail page's Monitoring Locations section now splits into:
Active locations (full card with Assign / Edit / Remove / Delete)
Removed locations (collapsed <details>, greyed cards, Restore button,
shows removal date + reason)
- New per-card "Remove" button → opens confirmation modal explaining
the cascade, with optional effective-date (defaults to now,
backdateable) and reason fields.
- Unit detail's SFM Events attribution cell shows a small "removed"
badge next to historical attributions whose location is no longer
active. Same pattern in vibration_summary's top-locations list.
- Soft-removal indicator surfaced through the events_for_unit
attribution payload as location_removed_at.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New /admin/sfm page (linked from Settings → Developer):
- Health banner — green/red with version + last-checked timestamp
- Connection panel — shows SFM_BASE_URL terra-view is configured with
- 4 KPI tiles — known units, total events, stale monitor_log rows,
stale ach_sessions rows (the deprecated tables from the paused
Python-ACH experiment, useful for confirming nothing's growing them)
- Per-unit roll-up table — serial, last_seen, event count, stale
per-unit counts, sourced from SFM's /db/units
- Recent events with forwarding latency — color-coded gap between
the event's recorded timestamp and SFM ingest time, so operators
can spot watchers that are forwarding stale files (e.g. after a
jobsite outage)
- Raw API tester — text input + GET button against any /api/sfm/*
path, response rendered as prettified JSON
New /admin/slmm page — same layout, stripped down to health + connection
+ raw API tester. For per-device SLM control the existing
/sound-level-meters dashboard remains the right entry point.
Backend (backend/routers/admin_modules.py):
- GET /admin/sfm, GET /admin/slmm — HTML pages
- GET /api/admin/sfm/overview — single aggregated probe that returns
health, units, last 25 events with computed latency, stale-table
counts, cache stats. Tolerant of partial failures: any sub-fetch
error is captured into errors{} so a flaky SFM endpoint doesn't
break the whole page
- GET /api/admin/slmm/overview — health + connection info only for now
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Top row left→right: Recent Alerts | Recent Call-Ins (2 cols) | Fleet Summary
- Today's Schedule becomes a horizontal collapsible card below Fleet Map.
Collapsed by default; auto-expands when pending actions are detected in
the rendered partial; manual toggle sticks via localStorage.
- New /api/recent-event-callins proxies SFM /db/events and bulk-joins each
serial against RosterUnit for in-roster annotation. Phases the
heartbeat-derived /api/recent-callins out of the UI while keeping it as
a backup endpoint for now.
- Call-ins card renders a dense 2-column grid (last 10 events) showing
PVS, sensor_location, false-trigger badge, event timestamp, and
links to the unit page when rostered.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 5b first slice. Surfaces near-duplicate projects (typo variants,
abbreviation differences, spacing variations like "SR81" vs "SR 81")
as side-by-side pairs the operator can merge with one click.
Backend (backend/services/project_tidy.py):
- find_duplicate_pairs(db, threshold=0.85) walks all active projects and
computes rapidfuzz.WRatio similarity for every pair. Pre-filters
too-short normalised names (< 4 chars) to avoid noise. Skips
soft-deleted projects. Returns pairs sorted by score desc, then by
total content (more assignments → review first).
- Each pair carries a suggested merge target with a human-readable
reason. Priorities (in order): manual source over parser source,
populated project_number, more locations, more assignments, shorter
name. Operator can override the suggestion by clicking the OTHER
direction button.
- O(N^2) over project count. Fine up to ~500 projects. Token-prefix
blocking is the obvious next optimisation if it becomes slow.
Backend (backend/routers/projects.py):
- GET /api/projects/admin/duplicate_pairs?threshold=&max_pairs= returns
pairs as JSON for the Tidy page.
Frontend (templates/admin/project_tidy.html):
- New admin page at /settings/developer/project-tidy. Threshold selector
(95% / 90% / 85% / 80%) at the top; rescan button next to it; auto-
scans on load.
- Each pair card shows side-by-side project summaries (name, project_
number, client, source-badge, location/assignment counts) with the
suggested target visually highlighted (orange border). Three buttons:
"Merge A → B", "Merge B → A", "Not a dup" (hide locally).
- Click-to-merge opens a native confirm with the preview totals
(assignments/sessions/data files moving, consolidations) — same data
the project_header.html merge modal shows. On confirm, hits the
existing /merge_into endpoint and re-scans automatically.
- Source badges distinguish parser-created (`metadata_backfill`) from
manual projects — at a glance the operator can see "this duplicate is
parser-generated; safe to merge into the manual one".
Frontend (templates/admin/metadata_backfill.html):
- Apply-result handling now surfaces failed[] cluster reasons in a
dedicated failure panel (bottom-left, dismissable). Previously a 200
OK with all-failures showed a misleading "1 cluster applied" success
toast because the count and the failure list weren't being reconciled.
This bit us during the DB-revert recovery earlier — the
project_modules table was missing, every apply silently rolled back,
user saw success toasts. Fixed.
Smoke-verified against current state (10K events, 9 projects, post-
merge): tool correctly finds 0 pairs at threshold 0.85 (data is clean),
1 false-positive at 0.70 (two unrelated projects sharing the token "81"
— example of why the 0.85 default is correct).
Settings link added under Developer → Project Tidy.
Phase 5c (swap-detection daily background job + notification inbox)
remains deferred to the next session.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operator-facing tool for cleaning up duplicate projects. Common after
the metadata-backfill parser auto-creates near-duplicates from operator
name variations ("SR81" vs "SR 81", "Swank-Karns Crossing" vs
"Swank-Karns Crossings", "Trumbull-Bryman Mont.Dam" vs
"Trumbull-Brayman-Mont Dam", etc.).
Workflow: visit the duplicate project's detail page, click "Merge into…"
in the header, search for the canonical target project from a typeahead,
review the preview (what assignments / locations / sessions will move,
any conflicts), confirm. Source is soft-deleted; everything else
re-points to the target. Smart consolidation: same-named locations in
both projects merge into one (source's assignments move to target's
existing location with the same name; source's empty location is then
deleted). Different-named locations move as-is.
Backend:
- backend/services/project_merge.py (new): preview() and execute()
functions. Transaction-safe. Per-assignment UnitHistory audit row
with change_type='assignment_merged' so the deployment timeline shows
the merge. Source modules disabled; missing modules added to target.
Handles edge cases: same project_id rejected, deleted projects rejected,
orphan project-direct assignments (no location) re-pointed defensively.
- backend/routers/projects.py: new endpoints
GET /api/projects/{source_id}/merge_preview?target_id=...
POST /api/projects/{source_id}/merge_into?target_id=...
Frontend (templates/partials/projects/project_header.html):
- "Merge into…" button in Project Actions area.
- Modal with typeahead (reuses /api/admin/metadata_backfill/projects_search)
scoped to existing projects only (no create-new option). Filters out
the source project from candidates so operator can't accidentally pick
it as target.
- Preview pane shows totals + per-location plan (consolidate vs move) +
warnings (mismatched client names, location consolidation note).
- Red "Merge (permanent)" confirm button only enables after a target is
picked and preview loads.
- On success, browser redirects to target project page.
Smoke verified: "Swank-Karns Crossing" (1 assignment) merged into
"Swank-Karns Crossings"; target now has 2 locations + 2 assignments,
source has 0 dangling rows, 1 project_merge audit entry written.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operator no longer has to accept the parser's suggested project /
location verbatim. Each cluster card now has editable typeahead inputs
that search existing projects (and existing locations within the chosen
project), with a "Create new: <typed>" fallback always available.
Solves the I-80-North-Fork case: of the 20+ cluster variants
("I-80-North Fork Bridges-I80 E. Abutment", "I-80- North Fork
Bridges-543 Plank Rd", etc.), operator types "I-80" in the Project
input, picks the existing project from the dropdown, and the cluster
attaches to it. Repeat for the other variants. No need to pre-create
the canonical project — though pre-creation still works fine if you'd
rather.
Backend (backend/routers/metadata_backfill.py):
- GET /api/admin/metadata_backfill/projects_search?q=&limit=
Returns existing projects matching by case-insensitive substring OR
rapidfuzz WRatio score >= 0.50. Substring matches sort to the top
(treated as exact for ordering). Includes location_count and
project_number/client_name in each result for disambiguation. Always
emits a "Create new: <q>" suggestion alongside the matches.
- GET /api/admin/metadata_backfill/locations_search?project_id=&q=&limit=
Same shape, scoped to a single project's vibration locations.
- POST /api/admin/metadata_backfill/apply now accepts four override
keys per cluster (was previously two):
project_id → attach to existing Project (operator picked from
typeahead)
project_name → create new with this name (operator typed a
custom name; existing project_name behaviour)
location_id → attach to existing MonitoringLocation; validated
against the chosen project_id so a stale location
FK can't sneak in
location_name → create new location with this name
Frontend (templates/admin/metadata_backfill.html):
- Each non-blank-meta cluster card now has two editable typeahead inputs
(Project + Location) pre-populated with the parser's suggested
values. Old static "Project: + Create new: X" / "≈ Fuzzy match" pills
replaced with compact hint lines under the inputs showing what the
current value will do.
- Typeahead dropdown opens on focus, debounced 150ms on type. Shows
matched existing entities with score badges (exact / NN%) plus a
"Create new: <typed>" option at the bottom. Click-to-pick fills the
text input and writes the entity id into a hidden field.
- Picking a new project clears the location id (forces re-pick under
the new project, avoids cross-project location FKs).
- _gatherOverrides re-wired to emit the new project_id / location_id
keys when the operator picked from the dropdown, falling back to
*_name when they typed free-form.
Backward-compatible: blank-meta clusters keep their existing "project_name
/ location_name" plain inputs and the override path still honours them.
Verified end-to-end:
- /projects_search?q=I-80 returns the existing "I-80 - North Fork
Bridge" project (score 1.0, has 4 locations) plus a "Create new"
option.
- /locations_search requires project_id (400 without it).
- Wizard page renders with typeahead wiring confirmed in HTML.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operators sometimes bake location identifiers into the project string
for email-readability — "Fay - Locks & Dam No3 - Loc 2 - 735 Bunola"
where "Fay - Locks & Dam No3" is the actual project and "- Loc 2 -
735 Bunola" is location info that already lives in sensor_location.
Without stripping, every "- Loc N" variant became a separate project,
fragmenting what should be one project with several locations.
Backend:
- New _extract_project_root() helper. Regex matches " - Loc N" / "-Loc3" /
" - Location #5" / etc. with case-insensitive multi-dash support; strips
from that marker forward and cleans up dangling separators. Strings
without a Loc-marker pass through unchanged.
- Cluster dataclass adds project_root field alongside project_raw.
project_raw stays the operator-typed string for display ("hover to see
what was actually typed"). project_root is what gets normalised for
matching and used as the suggested project name.
- _ensure_project + _ensure_location now do normalisation-aware dedup
before creating: a cluster of "SR81" and a cluster of "SR 81" (which
normalise to the same string) collapse into one project on apply,
even when applied in the same bulk operation. Avoids UNIQUE
constraint collisions and duplicate-named-by-spacing projects.
Frontend:
- Wizard cluster cards show "↳ stripped trailing 'Loc N' suffix; operator
typed: <raw>" when project_root differs from project_raw, so the
operator can see at a glance what the parser did to the string.
Real-data results: against the same 10,055 SFM events, confidence
distribution improved from 37/14/8 (high/med/low) to 43/9/7. "Fay -
Locks & Dam No3" now appears as ONE project across 6 cluster instances
spanning 3 serials and 6 different locations — exactly the
"one project, many locations" model the user described.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operator clicks one button. Parser reads SFM's events table (operator-typed
project / client / sensor_location strings), clusters by serial + time +
metadata, fuzzy-matches against existing projects, and proposes
Project / MonitoringLocation / UnitAssignment chains to create.
Auto-applies high-confidence non-conflicting clusters in bulk; queues
medium/low confidence for individual review.
Verified against real data: 10,052 events → 59 clusters → 37 high-
confidence + 14 medium + 8 low. Test-applied one cluster end-to-end;
Project + Module + Location + Assignment + UnitHistory + Decision rows
all created correctly, and Phase 2's attribution walk picked up the
events automatically on the new location's detail page.
Pipeline (backend/services/metadata_backfill.py, ~700 lines):
1. Pull all SFM events via /db/events per serial.
2. Pre-filter: drop events already covered by an existing UnitAssignment
window (Phase 2 handles those automatically).
3. Time-cluster what's left: serial + 7-day gap is the cluster identity.
4. Metadata-split each time-cluster on persistent metadata transitions
(≥ 2 consecutive events) so a single typo doesn't fork the cluster.
5. Match against existing graph (rapidfuzz.WRatio multi-signal scoring,
normalisation that handles abbreviations / reorders / separator
variations). Thresholds: 0.95 exact, 0.80 fuzzy, min-shorter-input
5 chars to guardrail false positives on single common words.
6. Score confidence (high/medium/low) using event count, span,
blank-meta, conflict, ambiguity rules.
7. Detect conflicts: overlap with existing UnitAssignment at a different
location for the same serial → blocking. Operator must reconcile.
8. Apply: ensure auto_imported ProjectType exists, ensure
vibration_monitoring ProjectModule on the project, write
Project / MonitoringLocation / UnitAssignment / UnitHistory all in
one transaction.
Migration (backend/migrate_add_metadata_backfill.py): adds
unit_assignments.source column (default 'manual') and
metadata_backfill_decisions table. Idempotent, non-destructive.
API (backend/routers/metadata_backfill.py):
GET /api/admin/metadata_backfill/scan — clusters + suggestions
POST /api/admin/metadata_backfill/apply — bulk apply by cluster_ids
w/ optional per-cluster
project/location overrides
POST /api/admin/metadata_backfill/skip — mark skipped (persistent)
UI (templates/admin/metadata_backfill.html, accessible at
/settings/developer/metadata-backfill via the Developer tab of Settings):
- One-button "Run scan" entry.
- Summary KPI tiles (scanned / already attributed / pending / conflicts).
- "Apply all high-confidence" bulk button at the top — primary path.
- Per-cluster cards below with Apply / Skip / Preview event actions.
- Blank-meta clusters get inline input fields for operator-typed project +
location names before applying.
- Blocking-conflict clusters render with the conflicting assignment
information and a disabled Apply button.
- Live progress toast during apply.
- Reuses the Phase 1+2+4 event-detail modal for "Preview event" — operator
can sanity-check the BW report data against the cluster's sample event.
Dependencies: rapidfuzz==3.10.1 added to requirements.txt. Pre-built C
wheels for all platforms, ~5s docker build hit.
Phase 5b (deferred to next session): swap-detection daily background job,
notification inbox for auto-applied swaps, recently-applied audit view,
"Tidy" page for renaming/merging auto-created projects.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 4. Rebuilds the seismograph "Deployment History" + "Timeline"
sections on the unit detail page as a single derived view computed from
three sources: unit_assignments (authoritative project/location windows),
unit_history (calibration/retirement/deployed state changes), and SFM
events overlaid per assignment window (count + peak PVS + last event).
Fixes the wonky-timeline symptoms: missing entries, duplicate/contradictory
rows, and no visibility into what the unit was actually doing during each
deployment window.
Backend:
- backend/services/deployment_timeline.py: new deployment_timeline_for_unit()
helper. Merges UnitAssignment rows (with SFM event overlay fetched
concurrently via httpx), UnitHistory state-change rows (filtered to
meaningful change_types and de-noised by dropping rows where
old_value == new_value — there's noise in legacy audit log from
record_history() being called on every save), and synthetic "gap"
entries between assignments >= 1 day apart. Sorts newest first.
- backend/routers/units.py: new GET /api/units/{unit_id}/deployment_timeline
endpoint with optional include_events=false flag.
- backend/routers/project_locations.py: assign / unassign / swap /
update endpoints now write UnitHistory rows on every assignment
lifecycle event. New change_types: assignment_created,
assignment_ended, assignment_swapped, assignment_updated. These
surface in the unified timeline (where the assignment row itself
shows the structural data; the audit row is filtered out to avoid
double-rendering). Closes a real gap — assignment changes were
previously invisible to any audit consumer.
- backend/migrate_deprecate_deployment_records.py: non-destructive
migration. Adds deployment_records.deprecated_at column. For each
legacy row without a matching UnitAssignment, best-effort
synthesizes one (with the free-text location_name preserved in
notes). Marks every processed row. Idempotent. DROP TABLE
deferred to a follow-up release.
Frontend (templates/unit_detail.html):
- Removed legacy "Deployment History" card (with Log Deployment button)
and the separate "Timeline" card. Replaced with a single
"Deployment Timeline" section.
- Three entry visual styles: assignment rows (orange dot, location +
project link, event-overlay summary), gap rows (dashed outline, idle
day count), and state_change rows (navy dot, friendly label, old →
new value). Active assignments get a green dot + "active" badge.
- Existing loadUnitHistory() and loadDeploymentHistory() functions kept
as shims that delegate to loadDeploymentTimeline(), so modal-save
callbacks that referenced them still trigger a refresh of the visible
section. Legacy function bodies preserved under _legacy_*_unused
names for archeology; not called by anything.
Verified end-to-end:
- BE11529 timeline now shows 2 entries (active assignment with 24-event
overlay + the deployed→benched state change), compared to the previous
noisy mix that included 6 no-op state-change rows.
- Migration ran against real DB: 1 legacy row processed (had no
project_id, marked deprecated without backfill).
- Assign / unassign / swap / edit now leave a paper trail in
unit_history.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
serversdown