feat(sfm): unified deployment timeline (deprecate deployment_records)
Phase 4. Rebuilds the seismograph "Deployment History" + "Timeline"
sections on the unit detail page as a single derived view computed from
three sources: unit_assignments (authoritative project/location windows),
unit_history (calibration/retirement/deployed state changes), and SFM
events overlaid per assignment window (count + peak PVS + last event).
Fixes the wonky-timeline symptoms: missing entries, duplicate/contradictory
rows, and no visibility into what the unit was actually doing during each
deployment window.
Backend:
- backend/services/deployment_timeline.py: new deployment_timeline_for_unit()
helper. Merges UnitAssignment rows (with SFM event overlay fetched
concurrently via httpx), UnitHistory state-change rows (filtered to
meaningful change_types and de-noised by dropping rows where
old_value == new_value — there's noise in legacy audit log from
record_history() being called on every save), and synthetic "gap"
entries between assignments >= 1 day apart. Sorts newest first.
- backend/routers/units.py: new GET /api/units/{unit_id}/deployment_timeline
endpoint with optional include_events=false flag.
- backend/routers/project_locations.py: assign / unassign / swap /
update endpoints now write UnitHistory rows on every assignment
lifecycle event. New change_types: assignment_created,
assignment_ended, assignment_swapped, assignment_updated. These
surface in the unified timeline (where the assignment row itself
shows the structural data; the audit row is filtered out to avoid
double-rendering). Closes a real gap — assignment changes were
previously invisible to any audit consumer.
- backend/migrate_deprecate_deployment_records.py: non-destructive
migration. Adds deployment_records.deprecated_at column. For each
legacy row without a matching UnitAssignment, best-effort
synthesizes one (with the free-text location_name preserved in
notes). Marks every processed row. Idempotent. DROP TABLE
deferred to a follow-up release.
Frontend (templates/unit_detail.html):
- Removed legacy "Deployment History" card (with Log Deployment button)
and the separate "Timeline" card. Replaced with a single
"Deployment Timeline" section.
- Three entry visual styles: assignment rows (orange dot, location +
project link, event-overlay summary), gap rows (dashed outline, idle
day count), and state_change rows (navy dot, friendly label, old →
new value). Active assignments get a green dot + "active" badge.
- Existing loadUnitHistory() and loadDeploymentHistory() functions kept
as shims that delegate to loadDeploymentTimeline(), so modal-save
callbacks that referenced them still trigger a refresh of the visible
section. Legacy function bodies preserved under _legacy_*_unused
names for archeology; not called by anything.
Verified end-to-end:
- BE11529 timeline now shows 2 entries (active assignment with 24-event
overlay + the deployed→benched state change), compared to the previous
noisy mix that included 6 no-op state-change rows.
- Migration ran against real DB: 1 legacy row processed (had no
project_id, marked deprecated without backfill).
- Assign / unassign / swap / edit now leave a paper trail in
unit_history.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,256 @@
|
||||
"""
|
||||
Deployment timeline service — replaces the legacy `deployment_records`-driven
|
||||
timeline on the seismograph unit detail page.
|
||||
|
||||
Architecture:
|
||||
- `unit_assignments` is the authoritative source for "where was this unit"
|
||||
(one row per location/time-window). Auto-written by the project location
|
||||
swap/assign/unassign/update workflows.
|
||||
- `unit_history` is the audit log for non-location state changes
|
||||
(calibration toggles, retirement, allocation, etc.).
|
||||
- SFM events are overlaid per assignment window to show "what was the unit
|
||||
actually doing during this deployment" (count + peak PVS + last-event).
|
||||
|
||||
Gaps between assignments are emitted as synthetic "gap" entries so operators
|
||||
can see when the unit was idle vs out-of-service.
|
||||
|
||||
`deployment_records` is being deprecated; this module does not read it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.models import (
|
||||
UnitAssignment,
|
||||
UnitHistory,
|
||||
MonitoringLocation,
|
||||
Project,
|
||||
RosterUnit,
|
||||
)
|
||||
from backend.services.sfm_events import (
|
||||
SFM_BASE_URL,
|
||||
_fetch_events_for_serial,
|
||||
_iso_utc,
|
||||
)
|
||||
|
||||
log = logging.getLogger("backend.services.deployment_timeline")
|
||||
|
||||
# Don't emit synthetic gap entries shorter than this (seconds). Avoids visual
|
||||
# clutter from a sub-second handoff during a swap workflow.
|
||||
_MIN_GAP_SECONDS = 24 * 3600 # 1 day
|
||||
|
||||
# Per-call timeout when querying SFM for the event overlay.
|
||||
_SFM_TIMEOUT = 10.0
|
||||
_SFM_FETCH_CEILING = 5000
|
||||
|
||||
|
||||
# ── Public API ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def deployment_timeline_for_unit(
|
||||
db: Session,
|
||||
unit_id: str,
|
||||
*,
|
||||
include_event_overlay: bool = True,
|
||||
) -> dict:
|
||||
"""Build a chronological timeline for a unit.
|
||||
|
||||
Returns:
|
||||
{
|
||||
"unit_id": str,
|
||||
"device_type": str,
|
||||
"entries": [
|
||||
{
|
||||
"kind": "assignment" | "gap" | "state_change",
|
||||
"starts_at": ISO timestamp,
|
||||
"ends_at": ISO timestamp | None,
|
||||
"duration_days": float | None,
|
||||
# — assignment-only fields —
|
||||
"assignment_id": str,
|
||||
"location_id": str,
|
||||
"location_name": str,
|
||||
"project_id": str,
|
||||
"project_name": str,
|
||||
"is_active": bool,
|
||||
"event_overlay": {event_count, peak_pvs, peak_pvs_at, last_event}
|
||||
or None if include_event_overlay=False,
|
||||
"notes": str | None,
|
||||
# — gap-only fields —
|
||||
"context": "between assignments" | None,
|
||||
# — state_change-only fields —
|
||||
"change_type": str,
|
||||
"field_name": str | None,
|
||||
"old_value": str | None,
|
||||
"new_value": str | None,
|
||||
"source": str,
|
||||
"history_notes": str | None,
|
||||
},
|
||||
... # newest first
|
||||
],
|
||||
}
|
||||
"""
|
||||
unit = db.query(RosterUnit).filter_by(id=unit_id).first()
|
||||
if not unit:
|
||||
return {"unit_id": unit_id, "device_type": None, "entries": []}
|
||||
|
||||
# 1. Load assignments + their location/project lookups in bulk.
|
||||
assignments = (
|
||||
db.query(UnitAssignment)
|
||||
.filter(UnitAssignment.unit_id == unit_id)
|
||||
.order_by(UnitAssignment.assigned_at.asc())
|
||||
.all()
|
||||
)
|
||||
|
||||
loc_ids = {a.location_id for a in assignments}
|
||||
proj_ids = {a.project_id for a in assignments}
|
||||
loc_map = {
|
||||
l.id: l for l in db.query(MonitoringLocation).filter(
|
||||
MonitoringLocation.id.in_(loc_ids)
|
||||
).all()
|
||||
} if loc_ids else {}
|
||||
proj_map = {
|
||||
p.id: p for p in db.query(Project).filter(
|
||||
Project.id.in_(proj_ids)
|
||||
).all()
|
||||
} if proj_ids else {}
|
||||
|
||||
# 2. Load relevant unit_history rows. We surface state changes that
|
||||
# operators care about on a deployment timeline: calibration status,
|
||||
# retirement, deployed flag, allocation, calibration date, and the
|
||||
# assignment_* events we just added (those are redundant with the
|
||||
# assignment rows themselves, so we skip them to avoid double-rendering).
|
||||
interesting_change_types = (
|
||||
"calibration_status_change",
|
||||
"retired_change",
|
||||
"deployed_change",
|
||||
"allocation_change",
|
||||
"last_calibrated_change",
|
||||
"next_calibration_due_change",
|
||||
)
|
||||
history = (
|
||||
db.query(UnitHistory)
|
||||
.filter(UnitHistory.unit_id == unit_id)
|
||||
.filter(UnitHistory.change_type.in_(interesting_change_types))
|
||||
.order_by(UnitHistory.changed_at.asc())
|
||||
.all()
|
||||
)
|
||||
|
||||
now = datetime.utcnow()
|
||||
|
||||
# 3. Optionally fetch SFM event overlay for each assignment window.
|
||||
# Concurrent fan-out via httpx + asyncio.gather.
|
||||
overlays: dict[str, dict] = {}
|
||||
if include_event_overlay and assignments and unit.device_type == "seismograph":
|
||||
async with httpx.AsyncClient(timeout=_SFM_TIMEOUT) as client:
|
||||
results = await asyncio.gather(
|
||||
*(
|
||||
_fetch_events_for_serial(
|
||||
client,
|
||||
serial=unit_id,
|
||||
from_dt=a.assigned_at,
|
||||
to_dt=a.assigned_until or now,
|
||||
false_trigger=None,
|
||||
limit=_SFM_FETCH_CEILING,
|
||||
)
|
||||
for a in assignments
|
||||
),
|
||||
return_exceptions=False,
|
||||
)
|
||||
for a, events in zip(assignments, results):
|
||||
peak = None
|
||||
peak_at = None
|
||||
last_ev = None
|
||||
for ev in events:
|
||||
pvs = ev.get("peak_vector_sum")
|
||||
if pvs is not None and (peak is None or pvs > peak):
|
||||
peak = pvs
|
||||
peak_at = ev.get("timestamp")
|
||||
ts = ev.get("timestamp")
|
||||
if ts and (last_ev is None or ts > last_ev):
|
||||
last_ev = ts
|
||||
overlays[a.id] = {
|
||||
"event_count": len(events),
|
||||
"peak_pvs": peak,
|
||||
"peak_pvs_at": peak_at,
|
||||
"last_event": last_ev,
|
||||
}
|
||||
|
||||
# 4. Build entries. Start by emitting assignment rows + gap rows between
|
||||
# consecutive assignments, then add state-change rows from unit_history.
|
||||
entries: list[dict] = []
|
||||
|
||||
for idx, a in enumerate(assignments):
|
||||
loc = loc_map.get(a.location_id)
|
||||
proj = proj_map.get(a.project_id)
|
||||
is_active = a.assigned_until is None
|
||||
ends_at = a.assigned_until or now
|
||||
duration_days = (ends_at - a.assigned_at).total_seconds() / 86400 if a.assigned_at else None
|
||||
|
||||
entry = {
|
||||
"kind": "assignment",
|
||||
"starts_at": _iso_utc(a.assigned_at),
|
||||
"ends_at": _iso_utc(a.assigned_until),
|
||||
"duration_days": round(duration_days, 1) if duration_days is not None else None,
|
||||
"assignment_id": a.id,
|
||||
"location_id": a.location_id,
|
||||
"location_name": loc.name if loc else None,
|
||||
"project_id": a.project_id,
|
||||
"project_name": proj.name if proj else None,
|
||||
"is_active": is_active,
|
||||
"notes": a.notes,
|
||||
"event_overlay": overlays.get(a.id),
|
||||
}
|
||||
entries.append(entry)
|
||||
|
||||
# Gap detection: from the end of this assignment to the start of the
|
||||
# next one. Only emit gaps that are at least _MIN_GAP_SECONDS long
|
||||
# so trivial sub-second handoffs during swaps don't clutter the view.
|
||||
if idx + 1 < len(assignments):
|
||||
next_a = assignments[idx + 1]
|
||||
gap_start = a.assigned_until or now
|
||||
gap_end = next_a.assigned_at
|
||||
gap_seconds = (gap_end - gap_start).total_seconds() if gap_end and gap_start else 0
|
||||
if gap_seconds >= _MIN_GAP_SECONDS:
|
||||
entries.append({
|
||||
"kind": "gap",
|
||||
"starts_at": _iso_utc(gap_start),
|
||||
"ends_at": _iso_utc(gap_end),
|
||||
"duration_days": round(gap_seconds / 86400, 1),
|
||||
"context": "between assignments",
|
||||
})
|
||||
|
||||
# 5. State changes — interleaved by timestamp. Skip no-op rows where
|
||||
# old_value == new_value (an artifact of the legacy record_history()
|
||||
# being called on every save regardless of whether the field changed).
|
||||
for h in history:
|
||||
if h.old_value == h.new_value:
|
||||
continue
|
||||
entries.append({
|
||||
"kind": "state_change",
|
||||
"starts_at": _iso_utc(h.changed_at),
|
||||
"ends_at": None,
|
||||
"duration_days": None,
|
||||
"change_type": h.change_type,
|
||||
"field_name": h.field_name,
|
||||
"old_value": h.old_value,
|
||||
"new_value": h.new_value,
|
||||
"source": h.source,
|
||||
"history_notes": h.notes,
|
||||
})
|
||||
|
||||
# 6. Sort newest first. Active assignments (no end) sort by start time,
|
||||
# same as everything else.
|
||||
entries.sort(key=lambda e: e.get("starts_at") or "", reverse=True)
|
||||
|
||||
return {
|
||||
"unit_id": unit.id,
|
||||
"device_type": unit.device_type,
|
||||
"entries": entries,
|
||||
}
|
||||
Reference in New Issue
Block a user