sfm: stored-event browser at /events

New standalone HTML page (sfm/event_browser.html, ~470 lines, Chart.js)
that lets you browse persisted events from the SeismoDb + WaveformStore.
Companion to the existing live-device viewer at /waveform:

  /waveform  — connect to a unit and pull events in real time
  /events    — browse events already stored in the DB

Flow:
  1. Page loads → GET /db/units → populate serial dropdown
  2. Select serial → GET /db/events?serial=X&limit=500 → event list
  3. Click event → GET /db/events/{id}/waveform.json → render

Layout is Instantel-printout-ready: channels stacked vertically in
Tran / Vert / Long / MicL order, trigger line at t=0, peak labels,
clean dark theme.  Frames the future PDF-export feature without
needing extra layout work.

Smoke-tested against the dev prod-snapshot — 4 channels render with
correct peaks for K558 events (L=0.3 in/s = the offset-fault peak
we've been chasing all week).

CHANGELOG entry added under [Unreleased] per the v0.20.0 release plan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-23 06:53:48 +00:00
parent 8710b8f327
commit 460006e5cd
3 changed files with 603 additions and 1 deletions
+24
View File
@@ -4,6 +4,30 @@ All notable changes to seismo-relay are documented here.
---
## [Unreleased]
### Added
- **Stored-event browser** — new standalone HTML page at `GET /events` (`sfm/event_browser.html`). Pick a serial from the unit dropdown, scroll through that unit's events (newest-first), click any event to render its decoded waveform via the existing `/db/events/{id}/waveform.json` endpoint. Dark-themed Chart.js viewer, channels stacked vertically (Tran / Vert / Long / MicL — Instantel printout order, designed PDF-export-ready), trigger line at t=0, peak labels, search/filter, false-trigger flag honored. Companion to the existing live-device viewer at `/waveform`; the two routes are now clearly delineated in their docstrings.
- **Histogram body codec — uint8 peak count fix.** Per-channel peak fields at `block[6]/[10]/[14]/[18]` are `uint8`, not `uint16 LE` spanning `block[6:8]` etc. The original interpretation was byte-exact on the N844 fixture corpus only because every annotation byte (`block[7]/[11]/[15]/[19]`) in those fixtures was zero. On non-N844 events with non-zero annotation bytes (observed across BE9558 Tran-drift and BE18003 Histogram+Continuous units), the old interpretation produced peaks up to 268 in/s per channel and 35× inflated PVS sums when first deployed to prod (rolled back same day; properly fixed in this release). Cross-correlated against BW's per-interval ASCII export on K558 / T003 / N599 / N844 corpora — 100% byte-exact on T/V/L, 99%+ on M (sub-precision rounding). Annotation byte preserved on each record as `record["annotations"]` for future RE. Verified against ~3,500 blocks across 5 in-repo fixtures + a synthetic K558 interval-12 regression block.
- **`apply_bw_report_dict_to_event` helper** in `minimateplus.event_file_io`. Mirror of `apply_report_to_event` for the projected sidecar dict shape — used by the backfill path, which has the preserved `bw_report` block but not the original `.TXT` file. BW's reported peaks (and `sample_rate` / `record_time`) now win over codec output during `--force` backfill, matching ingest-path behavior.
- **`scripts/check_bw_report_preservation.py`** — two-step snapshot/diff tool to verify that `backfill_sidecars.py` doesn't wipe the `bw_report` block from existing sidecars. Classifies every sidecar as PRESERVED / CHANGED / WIPED / STILL_MISSING / NEW / ADDED / REMOVED. Exit code 1 if any WIPED or CHANGED entries are found, so it can gate a CI step or deploy script.
### Fixed
- **`scripts/backfill_sidecars.py` no longer wipes `bw_report`.** Before this fix, `event_to_sidecar_dict` silently dropped the preserved `bw_report` block during every backfill, since the function only emits a `bw_report` when called with a live `BwAsciiReport` dataclass (which the backfill doesn't have — only the projected sidecar dict). Now we read the existing sidecar's `bw_report` and overlay it onto the regenerated sidecar, alongside the existing `review` and `extensions` preservation.
- **`scripts/backfill_sidecars.py --force` no longer overwrites BW-overlaid DB peaks with codec output.** The backfill path now calls `apply_bw_report_dict_to_event` before the DB upsert, mirroring what the ingest path does (`/db/import/blastware_file` parses the `.TXT` into a `BwAsciiReport`, calls `apply_report_to_event`, then upserts). Without this, events where the codec doesn't fully decode (waveform walker edge cases on SP0/SS0/SV0-style events, histogram `byte[5]!=0` sub-format) ended up with PVS=0 in the DB after a `--force` backfill; bit on prod 2026-05-22, rolled back the same day.
- **Thor IDF files no longer attempted as BW events in backfill.** `scripts/backfill_sidecars.py` now filters out `.IDFW` / `.IDFH` files in `_looks_like_event_file()`; they share the `.X0W` / `.X0H` suffix shape but use a separate ingest path (`WaveformStore.save_imported_idf`) and aren't decodable by `event_file_io.read_blastware_file`.
### Docs
- **CLAUDE.md** — added a three-tier conceptual architecture model (SFM / SDM / shared codec library) near the top of the file, with a placement rule for where new code goes. Documents that what is conceptually SDM (database, waveform store, ingest, `/db/*` endpoints) still lives under `sfm/` for historical reasons; rename deferred until the codebase is quiet enough for a clean refactor.
- **README.md** — added a "Strategic direction" lead-in to the Roadmap that frames seismo-relay as a suite of cooperating components (not a single app), and an explicit "Terra-View ↔ SFM device control" roadmap section with a concrete implementation checklist (auth as hard prerequisite, embedded live-monitor view, action history, Series IV live-device support).
- **`docs/histogram_codec_re_status.md`** updated with the uint8 retraction and the annotation-byte status.
- Three known issues recorded in the Roadmap that were discovered during prod validation: (1) `bw_ascii_report` parser misses PPV / `vector_sum` on some `.TXT` formats (5 events on prod); (2) NULL-timestamp duplicate-row dedup needed (2 events on prod); (3) histogram body sub-format with `byte[5] != 0` not yet decoded (~3 events on prod with empty `.h5` plots).
---
## v0.19.0 — 2026-05-20
The "device-family separation" release. Tightens the boundary between Series III (MiniMate Plus / Blastware) and Series IV (Micromate / Thor) so the UI and storage layer dispatch deterministically by family instead of sniffing filename extensions or magnitude heuristics.
+564
View File
@@ -0,0 +1,564 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>SFM Event Browser</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/4.4.1/chart.umd.min.js"></script>
<style>
* { box-sizing: border-box; margin: 0; padding: 0; }
body {
background: #0d1117;
color: #c9d1d9;
font-family: 'Segoe UI', system-ui, sans-serif;
font-size: 13px;
height: 100vh;
display: flex;
flex-direction: column;
overflow: hidden;
}
header {
background: #161b22;
border-bottom: 1px solid #30363d;
padding: 12px 20px;
display: flex;
align-items: center;
gap: 16px;
flex-shrink: 0;
}
header h1 {
font-size: 15px;
font-weight: 600;
color: #f0f6fc;
white-space: nowrap;
}
label { color: #8b949e; font-size: 12px; }
select, input[type="text"], input[type="search"] {
background: #0d1117;
border: 1px solid #30363d;
border-radius: 6px;
color: #c9d1d9;
padding: 5px 8px;
font-size: 13px;
}
select { min-width: 140px; }
input[type="search"] { width: 200px; }
select:focus, input:focus { outline: none; border-color: #388bfd; }
button {
background: #1f6feb;
border: none;
border-radius: 6px;
color: #fff;
cursor: pointer;
font-size: 13px;
font-weight: 500;
padding: 5px 14px;
}
button:hover { background: #388bfd; }
button:disabled { background: #21262d; color: #484f58; cursor: not-allowed; }
#main {
flex: 1;
display: flex;
overflow: hidden;
}
/* ── Event list (left sidebar) ────────────────────────────────── */
#event-list-wrap {
width: 320px;
flex-shrink: 0;
background: #0d1117;
border-right: 1px solid #21262d;
display: flex;
flex-direction: column;
}
#event-list-header {
padding: 10px 14px;
border-bottom: 1px solid #21262d;
font-size: 11px;
color: #8b949e;
text-transform: uppercase;
letter-spacing: 0.06em;
display: flex;
justify-content: space-between;
}
#event-list {
flex: 1;
overflow-y: auto;
}
.event-row {
padding: 8px 14px;
border-bottom: 1px solid #161b22;
cursor: pointer;
transition: background 0.1s;
}
.event-row:hover { background: #161b22; }
.event-row.active { background: #1f3a5f; border-left: 3px solid #58a6ff; padding-left: 11px; }
.event-row .er-top {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 2px;
}
.event-row .er-ts { font-family: monospace; font-size: 12px; color: #c9d1d9; }
.event-row .er-pvs { font-family: monospace; font-size: 12px; color: #58a6ff; font-weight: 600; }
.event-row .er-meta { font-size: 11px; color: #8b949e; }
.event-row.false_trigger .er-pvs { color: #f85149; text-decoration: line-through; }
/* ── Main viewer (right side) ─────────────────────────────────── */
#viewer {
flex: 1;
display: flex;
flex-direction: column;
overflow: hidden;
}
#event-meta {
padding: 12px 20px;
background: #161b22;
border-bottom: 1px solid #21262d;
display: grid;
grid-template-columns: repeat(auto-fit, minmax(160px, 1fr));
gap: 8px 24px;
flex-shrink: 0;
}
.meta-field {
display: flex;
flex-direction: column;
gap: 1px;
}
.meta-field .mf-label {
font-size: 10px;
color: #484f58;
text-transform: uppercase;
letter-spacing: 0.05em;
}
.meta-field .mf-value {
font-family: monospace;
font-size: 13px;
color: #c9d1d9;
}
.meta-field .mf-value.highlight { color: #58a6ff; font-weight: 600; }
#charts {
flex: 1;
overflow-y: auto;
padding: 12px 16px;
display: flex;
flex-direction: column;
gap: 10px;
}
.chart-wrap {
background: #161b22;
border: 1px solid #21262d;
border-radius: 8px;
padding: 10px 12px 8px;
}
.chart-label {
font-size: 11px;
font-weight: 600;
letter-spacing: 0.06em;
text-transform: uppercase;
margin-bottom: 4px;
display: flex;
justify-content: space-between;
}
.chart-canvas-wrap { position: relative; height: 130px; }
.ch-tran { color: #58a6ff; }
.ch-vert { color: #3fb950; }
.ch-long { color: #d29922; }
.ch-micl { color: #bc8cff; }
#status-bar {
background: #161b22;
border-top: 1px solid #21262d;
padding: 5px 20px;
font-size: 12px;
color: #8b949e;
min-height: 26px;
flex-shrink: 0;
}
#status-bar.error { color: #f85149; }
#status-bar.ok { color: #3fb950; }
#empty-state {
flex: 1;
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
color: #484f58;
gap: 8px;
}
#empty-state svg { opacity: 0.3; }
.pill {
background: #21262d;
border-radius: 4px;
padding: 2px 8px;
color: #c9d1d9;
font-family: monospace;
font-size: 11px;
margin-left: 8px;
}
</style>
</head>
<body>
<header>
<h1>SFM Event Browser</h1>
<label>Serial</label>
<select id="serial-select">
<option value="">Loading…</option>
</select>
<input type="search" id="event-filter" placeholder="filter events…" />
<span class="pill" id="count-pill"></span>
<button id="reload-btn" onclick="loadSerials()" style="margin-left:auto">Reload</button>
</header>
<div id="main">
<div id="event-list-wrap">
<div id="event-list-header">
<span>Events</span>
<span id="event-list-count"></span>
</div>
<div id="event-list"></div>
</div>
<div id="viewer">
<div id="empty-state">
<svg width="48" height="48" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5">
<polyline points="22 12 18 12 15 21 9 3 6 12 2 12"/>
</svg>
<p>Select a unit and event to view its waveform.</p>
</div>
<div id="event-meta" style="display:none"></div>
<div id="charts" style="display:none"></div>
</div>
</div>
<div id="status-bar">Ready.</div>
<script>
const CHANNEL_COLORS = {
Tran: '#58a6ff',
Vert: '#3fb950',
Long: '#d29922',
MicL: '#bc8cff',
};
const CHANNEL_ORDER = ['Tran', 'Vert', 'Long', 'MicL'];
let allEvents = [];
let filteredEvents = [];
let currentEventId = null;
let charts = {};
const apiBase = window.location.origin;
function setStatus(msg, cls = '') {
const bar = document.getElementById('status-bar');
bar.textContent = msg;
bar.className = cls;
}
async function loadSerials() {
setStatus('Loading serials…');
try {
const r = await fetch(`${apiBase}/db/units`);
if (!r.ok) throw new Error(r.statusText);
// /db/units returns a bare list[dict], not {units:[...]}
const units = await r.json();
const sel = document.getElementById('serial-select');
sel.innerHTML = '';
if (!units || units.length === 0) {
sel.innerHTML = '<option value="">(no units found)</option>';
setStatus('No units in DB.', 'error');
return;
}
sel.innerHTML = '<option value="">— pick a unit —</option>' +
units.map(u => {
const n = u.total_events ?? 0;
return `<option value="${u.serial}">${u.serial} (${n} events)</option>`;
}).join('');
setStatus(`Loaded ${units.length} units.`, 'ok');
} catch (e) {
setStatus(`Failed to load units: ${e.message}`, 'error');
}
}
async function loadEventsForSerial(serial) {
if (!serial) {
allEvents = [];
renderEventList();
return;
}
setStatus(`Loading events for ${serial}…`);
try {
const r = await fetch(`${apiBase}/db/events?serial=${encodeURIComponent(serial)}&limit=500`);
if (!r.ok) throw new Error(r.statusText);
const d = await r.json();
allEvents = d.events || [];
document.getElementById('count-pill').textContent = `${allEvents.length} events`;
applyFilter();
setStatus(`Loaded ${allEvents.length} events for ${serial}.`, 'ok');
} catch (e) {
setStatus(`Failed to load events: ${e.message}`, 'error');
}
}
function applyFilter() {
const q = document.getElementById('event-filter').value.toLowerCase().trim();
if (!q) {
filteredEvents = allEvents;
} else {
filteredEvents = allEvents.filter(ev =>
(ev.blastware_filename || '').toLowerCase().includes(q) ||
(ev.timestamp || '').toLowerCase().includes(q) ||
(ev.record_type || '').toLowerCase().includes(q) ||
(ev.project || '').toLowerCase().includes(q)
);
}
document.getElementById('event-list-count').textContent = `${filteredEvents.length} / ${allEvents.length}`;
renderEventList();
}
function renderEventList() {
const list = document.getElementById('event-list');
list.innerHTML = '';
if (filteredEvents.length === 0) {
list.innerHTML = '<div style="padding:14px;color:#484f58;font-size:12px">No events.</div>';
return;
}
for (const ev of filteredEvents) {
const row = document.createElement('div');
row.className = 'event-row' + (ev.false_trigger ? ' false_trigger' : '');
if (ev.id === currentEventId) row.className += ' active';
const ts = (ev.timestamp || '').replace('T', ' ').replace('Z', '');
const pvs = ev.peak_vector_sum != null ? `${ev.peak_vector_sum.toFixed(3)} in/s` : '—';
row.innerHTML = `
<div class="er-top">
<span class="er-ts">${ts || '(no ts)'}</span>
<span class="er-pvs">${pvs}</span>
</div>
<div class="er-meta">${ev.record_type || '?'} · ${ev.blastware_filename || ev.id.slice(0,8)}</div>
`;
row.onclick = () => loadEvent(ev.id);
list.appendChild(row);
}
}
async function loadEvent(eventId) {
currentEventId = eventId;
renderEventList();
setStatus('Loading waveform…');
try {
const r = await fetch(`${apiBase}/db/events/${eventId}/waveform.json`);
if (!r.ok) {
if (r.status === 404) {
showEmpty('No waveform data for this event (codec returned no samples).');
return;
}
throw new Error(r.statusText);
}
const data = await r.json();
renderWaveform(data);
// Also fetch metadata from the events list for richer header
const ev = allEvents.find(e => e.id === eventId);
renderMeta(data, ev);
setStatus(`Event loaded.`, 'ok');
} catch (e) {
setStatus(`Failed to load event: ${e.message}`, 'error');
showEmpty(`Error: ${e.message}`);
}
}
function showEmpty(msg) {
document.getElementById('empty-state').style.display = 'flex';
document.getElementById('empty-state').querySelector('p').textContent = msg;
document.getElementById('event-meta').style.display = 'none';
document.getElementById('charts').style.display = 'none';
Object.values(charts).forEach(c => c.destroy());
charts = {};
}
function renderMeta(data, ev) {
const metaDiv = document.getElementById('event-meta');
const fields = [
['Serial', data.serial || ev?.serial || '—'],
['Timestamp', (data.timestamp || ev?.timestamp || '—').replace('T', ' ').replace('Z', '')],
['Record', data.record_type || ev?.record_type || '—'],
['Sample rate', data.sample_rate ? `${data.sample_rate} sps` : '—'],
['Geo range', data.geo_range ? `${data.geo_range} (${data.geo_full_scale_ips} in/s FS)` : '—'],
['Project', ev?.project || '—'],
['Location', ev?.sensor_location || '—'],
['PVS', ev?.peak_vector_sum != null ? `${ev.peak_vector_sum.toFixed(4)} in/s` : '—'],
];
metaDiv.innerHTML = fields.map(([l, v]) =>
`<div class="meta-field"><span class="mf-label">${l}</span><span class="mf-value${l === 'PVS' ? ' highlight' : ''}">${v}</span></div>`
).join('');
metaDiv.style.display = 'grid';
}
function renderWaveform(data) {
document.getElementById('empty-state').style.display = 'none';
const chartsDiv = document.getElementById('charts');
chartsDiv.style.display = 'flex';
chartsDiv.innerHTML = '';
Object.values(charts).forEach(c => c.destroy());
charts = {};
const channels = data.channels || {};
const timeAxis = data.time_axis || null; // ms relative to trigger
const triggerMs = data.trigger_ms ?? 0;
for (const ch of CHANNEL_ORDER) {
const chData = channels[ch];
if (!chData) continue;
const values = chData.values || [];
if (values.length === 0) {
// Render an empty card so user sees the channel exists but is missing
const wrap = document.createElement('div');
wrap.className = 'chart-wrap';
wrap.innerHTML = `
<div class="chart-label ch-${ch.toLowerCase()}">
<span>${ch}</span>
<span style="color:#484f58">no samples decoded</span>
</div>
<div class="chart-canvas-wrap" style="display:flex;align-items:center;justify-content:center;color:#484f58;font-size:12px">empty</div>
`;
chartsDiv.appendChild(wrap);
continue;
}
const unit = chData.unit || 'unit';
const peak = chData.peak;
const peakT = chData.peak_t_ms;
const peakLabel = peak != null
? `peak ${(typeof peak === 'number' ? peak.toExponential(3) : peak)} ${unit}`
+ (peakT != null ? ` @ ${peakT.toFixed(1)} ms` : '')
: '';
const wrap = document.createElement('div');
wrap.className = 'chart-wrap';
const lbl = document.createElement('div');
lbl.className = `chart-label ch-${ch.toLowerCase()}`;
lbl.innerHTML = `<span>${ch}</span><span style="color:#8b949e;font-weight:normal">${peakLabel}</span>`;
wrap.appendChild(lbl);
const canvasWrap = document.createElement('div');
canvasWrap.className = 'chart-canvas-wrap';
const canvas = document.createElement('canvas');
canvasWrap.appendChild(canvas);
wrap.appendChild(canvasWrap);
chartsDiv.appendChild(wrap);
// Build time labels — use server-provided time_axis if present, else derive from sample_rate
let times;
if (timeAxis && timeAxis.length === values.length) {
times = timeAxis;
} else {
const sr = data.sample_rate || 1024;
times = values.map((_, i) => (i / sr * 1000 - triggerMs));
}
// Downsample for rendering
const MAX_POINTS = 4000;
let rT = times, rV = values;
if (values.length > MAX_POINTS) {
const step = Math.ceil(values.length / MAX_POINTS);
rT = times.filter((_, i) => i % step === 0);
rV = values.filter((_, i) => i % step === 0);
}
const chart = new Chart(canvas, {
type: 'line',
data: {
labels: rT.map(t => (typeof t === 'number' ? t.toFixed(2) : t)),
datasets: [{
data: rV,
borderColor: CHANNEL_COLORS[ch],
borderWidth: 1,
pointRadius: 0,
tension: 0,
}],
},
options: {
animation: false,
responsive: true,
maintainAspectRatio: false,
plugins: {
legend: { display: false },
tooltip: {
mode: 'index',
intersect: false,
callbacks: {
title: items => `t = ${items[0].label} ms`,
label: item => `${ch}: ${item.raw} ${unit}`,
},
},
},
scales: {
x: {
type: 'category',
ticks: {
color: '#484f58',
maxTicksLimit: 10,
maxRotation: 0,
callback: (val, i) => rT[i] + ' ms',
},
grid: { color: '#21262d' },
},
y: {
ticks: { color: '#484f58', maxTicksLimit: 5 },
grid: { color: '#21262d' },
title: { display: true, text: unit, color: '#484f58', font: { size: 10 } },
},
},
},
plugins: [{
// Vertical trigger line at t=0
id: 'triggerLine',
afterDraw(chart) {
const ctx = chart.ctx;
const xAxis = chart.scales.x;
const yAxis = chart.scales.y;
const zeroIdx = rT.findIndex(t => parseFloat(t) >= 0);
if (zeroIdx < 0) return;
const x = xAxis.getPixelForValue(zeroIdx);
ctx.save();
ctx.beginPath();
ctx.moveTo(x, yAxis.top);
ctx.lineTo(x, yAxis.bottom);
ctx.strokeStyle = 'rgba(248, 81, 73, 0.7)';
ctx.lineWidth = 1.5;
ctx.setLineDash([4, 3]);
ctx.stroke();
ctx.restore();
},
}],
});
charts[ch] = chart;
}
}
// Wire up handlers
document.getElementById('serial-select').addEventListener('change', e => {
loadEventsForSerial(e.target.value);
});
document.getElementById('event-filter').addEventListener('input', applyFilter);
// Initial load
loadSerials();
</script>
</body>
</html>
+15 -1
View File
@@ -381,10 +381,24 @@ def webapp():
@app.get("/waveform", response_class=FileResponse)
def waveform_viewer():
"""Serve the standalone waveform viewer."""
"""Serve the standalone LIVE-device waveform viewer.
Talks to ``/device/*`` endpoints for plotting events pulled from
a connected unit in real time. For the stored-event browser that
reads from the SeismoDb + WaveformStore, see ``/events``.
"""
return str(Path(__file__).parent / "waveform_viewer.html")
@app.get("/events", response_class=FileResponse)
def event_browser():
"""Serve the stored-event browser — pick a serial, list its events,
render any one's waveform from the persisted ``.h5`` via the
``/db/events/{id}/waveform.json`` endpoint. Standalone HTML +
Chart.js, no auth, no build step."""
return str(Path(__file__).parent / "event_browser.html")
@app.get("/device/info")
def device_info(
port: Optional[str] = Query(None, description="Serial port (e.g. COM5, /dev/ttyUSB0)"),