fix(db): /db/units surfaces events-only serials too

Previous query_units() only joined on ach_sessions, which is created
exclusively by the live ACH server.  The BW-importer path
(/db/import/blastware_file → WaveformStore.save_imported_bw →
SeismoDb.insert_events) populates `events` but never creates an
ach_sessions row.  Consequence: every serial whose events flowed in
through the series3-watcher forwarder was invisible to
/db/units (and therefore to the SFM webapp's fleet overview / units
list), even though the events were correctly populated in the
events table with proper serial attribution.

Rewrite query_units() to aggregate from BOTH tables and union the
serials:
  - total_events / last_event_at  come from `events` (every ingest path)
  - last_session_at / total_monitor_entries / total_sessions
                                  come from `ach_sessions` (ACH-only),
                                  0 when no sessions exist for the serial
  - last_seen = max(last_event_at, last_session_at)

Verified on the user's actual prod DB after the
repair_unknown_serials run: /db/units now returns 24 serials instead
of 2.  All 3,257 watcher-forwarded events become visible in the
fleet overview without any further DB surgery.
This commit is contained in:
2026-05-11 05:15:09 +00:00
parent 082e5946bc
commit cc57a8e618
+72 -14
View File
@@ -564,21 +564,79 @@ class SeismoDb:
def query_units(self) -> list[dict]: def query_units(self) -> list[dict]:
""" """
Return one row per known serial with summary stats: Return one row per known serial with summary stats.
last_seen, total_events, total_monitor_entries.
Aggregates from BOTH source tables:
- `events` — populated by every ingest path
(live ACH, /db/import/blastware_file
from the series3-watcher forwarder, etc.)
- `ach_sessions` — only populated by the live ACH server;
empty for events that came in via the
BW-importer route.
Earlier this method only joined on `ach_sessions`, which made
watcher-forwarded units invisible to the SFM webapp's fleet
overview even though their events were correctly populated in
`events`. Now we union the two and surface every serial that
has activity in either table.
Fields:
serial — unit serial number (e.g. "BE11529")
last_seen — most recent of MAX(events.timestamp)
and MAX(ach_sessions.session_time)
total_events — COUNT(*) from `events` (the
authoritative count regardless of
ingest path)
total_monitor_entries — from `ach_sessions`, 0 when absent
total_sessions — COUNT(*) from `ach_sessions`, 0 when absent
""" """
with self._connect() as conn: with self._connect() as conn:
rows = conn.execute( event_stats = {
""" row["serial"]: row
SELECT for row in conn.execute(
s.serial,
MAX(s.session_time) AS last_seen,
SUM(s.events_downloaded) AS total_events,
SUM(s.monitor_entries) AS total_monitor_entries,
COUNT(*) AS total_sessions
FROM ach_sessions s
GROUP BY s.serial
ORDER BY last_seen DESC
""" """
SELECT serial,
MAX(timestamp) AS last_event_at,
COUNT(*) AS total_events
FROM events
GROUP BY serial
""",
).fetchall() ).fetchall()
return [dict(r) for r in rows] }
session_stats = {
row["serial"]: row
for row in conn.execute(
"""
SELECT serial,
MAX(session_time) AS last_session_at,
SUM(monitor_entries) AS total_monitor_entries,
COUNT(*) AS total_sessions
FROM ach_sessions
GROUP BY serial
""",
).fetchall()
}
all_serials = set(event_stats) | set(session_stats)
units = []
for serial in all_serials:
e = event_stats.get(serial)
s = session_stats.get(serial)
last_event_at = e["last_event_at"] if e else None
last_session_at = s["last_session_at"] if s else None
# Prefer whichever timestamp is more recent
last_seen = max(
(t for t in (last_event_at, last_session_at) if t),
default=None,
)
units.append({
"serial": serial,
"last_seen": last_seen,
"total_events": e["total_events"] if e else 0,
"total_monitor_entries": s["total_monitor_entries"] if s else 0,
"total_sessions": s["total_sessions"] if s else 0,
})
# Sort by last_seen desc; serials with no timestamp at all sink to the bottom.
units.sort(key=lambda u: u.get("last_seen") or "", reverse=True)
return units