Merge feat/event-timestamp-fix: exact waveform trigger time from the binary

read_blastware_file stamped waveforms with footer ts1 (the monitoring-session
start, hours off — vomit-list #3).  The event time is ts2 (recording stop) and
the trigger = ts2 - record time, a float32 in the recording-setup config block,
so the exact Blastware trigger is recovered from the binary alone (no .TXT).
Histograms keep ts1; a paired report's event_datetime stays authoritative.

Needs a re-decode backfill to correct existing stored events' timestamps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf

CHANGELOG.md

@@ -6,6 +6,91 @@ All notable changes to seismo-relay are documented here.
 
 ## Unreleased
 
+### Fixed
+
+- **Waveform event times were the monitoring-session start, not the trigger
+  (~hours off).**  `read_blastware_file` stamped events with footer `ts1`, which
+  for a waveform is the session start a unit shares across every event that day
+  (a unit arming at 06:00 stamped 06:00 on all of them — the modal and PDF both
+  showed it, since it's the stored value).  The event time is footer `ts2` (the
+  recording stop), and Blastware's trigger = `ts2 - record time`.  The record
+  time is a big-endian float32 in the recording-setup config block (30 bytes
+  before the `Standard Recording Setup` marker), so the **exact trigger is now
+  recovered from the binary alone** — all 7 BE12844 oracle events decode to
+  their exact Blastware time (e.g. N844LQHB 10:33:29), no paired `.TXT` needed.
+  Histograms keep `ts1` (the ~24 h window start).  A paired report's
+  `event_datetime` stays authoritative (unit-clock drift).
+  ⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
+
+### Added
+
+- **Diagnostics tab in the SFM standalone webapp.**  Surfaces the device
+  endpoints that previously existed only as `curl`: `events/storage_range` and
+  `events/index` alongside `monitor/status`, then stop monitoring, disable ACH
+  (`rescue?erase=false`, so stored events survive), and erase.  The wedged-unit
+  ladder — slow drip and blind stop — sits under its own heading pointing at
+  `docs/runbooks/wedged_unit_recovery.md`, with the reminder that `slow_drip`'s
+  success signal is `bytes_received > 0` and not a clean duration.  Erase is
+  guarded by typing the unit's serial: auth answers *who*, not *did you mean
+  it*, and Swagger's try-it-out button on `/device/events/erase` is live on
+  `:8200/docs`.
+
+- **`docs/sfm_tool_status.md`** — an honest per-capability maturity assessment:
+  what is production-grade (the codec library, the data side), what is
+  emergency-grade (the device side), what is a research artifact, the
+  known-issues table, and the gap to a real tool.  Also records the **5A
+  page-boundary bug** as known: `parse_strt_end_offset()` discards the key's
+  page byte, so once a unit has recorded more than 64 KB since its last erase,
+  an event spanning the boundary reads an `end_offset` *behind* its own start —
+  the chunk loop fetches nothing and TERM packs a negative `offset_word`, which
+  500s.  Reproduced on BE12599.  Production is unaffected: it ingests complete
+  files via the watcher path and never runs this walk.
+
+### Changed
+
+- **Connecting to a unit no longer walks its event chain.**  `/device/events`
+  reads every event header over the cellular link; on a unit with a large or
+  wrapped chain that takes minutes or fails outright, and it fired
+  automatically on every connect.  Connect now uses only ~2 s probes —
+  `/device/info` (which already carried the compliance config the walk was
+  re-reading) plus `events/storage_range` — and the Device tab gains an Event
+  Chain card.  The walk moved behind a **Load events** button in the Events
+  toolbar.  Knowing whether a unit's ACH is on no longer requires reading every
+  event it has stored.
+
+### Migration
+
+**None.**  Frontend and documentation only — no codec, waveform-store or DB
+change, no schema change, and no `TOOL_VERSION` bump.  The webapp is served
+from the image, so the change appears after the next `sfm` rebuild.
+
+---
+
+## v0.31.0 — 2026-09-18
+
+**Report parity, and a second way to rescue a runaway unit.**  Two threads.
+
+The first closes out Blastware Event/FFT-Report parity: the FFT, the USBM
+RI8507 compliance chart and the sensor self-check now render on the event
+report, reverse-engineered against BE12844 (MiniMate Plus) and UM (Thor)
+events.  The sensor check is decoded for **both** series and standardized into
+the `.h5` (schema **v2**, a new `/sensor_check` group), so SFM serves it
+device-agnostically rather than decoding at report time.  The Inspector — an
+annotated hex reader for series-3 binaries — is what made the trailing-block
+structure findable, and it earned its keep by *ruling out* a stored FFT block
+and proving Blastware computes it from the samples.
+
+The second came out of a field emergency.  BE12599's connector fault drove its
+Tran channel to its trigger level, so the unit recorded back-to-back and dialed
+the office ACH server every ~75 s, unreachable the whole time.
+`bridges/ach_server.py` gained `--stop-monitoring` / `--disable-ach` /
+`--rescue`, which **invert** the recovery: instead of racing a Stop into the
+gaps between dial-outs, point the modem's Destination at our own ACH server and
+answer the call.  Proven in production the same night — the stop landed on the
+first call-in and held.  See `docs/runbooks/wedged_unit_recovery.md`.
+
+⚠ **This release owes prod a backfill** — see Migration below.
+
 ### Added
 - **Rescue-on-connect for `bridges/ach_server.py`** — `--stop-monitoring`
   (SUB 0x97), `--disable-ach` (SUB 0x2C read → 0x7E write → 0x7F confirm) and
@@ -46,18 +131,25 @@ All notable changes to seismo-relay are documented here.
   reference PDF. A technical breakdown of the curve is in
   `docs/ri8507_compliance_curve.md`.
 
-- **Sensor self-check waveforms decoded and drawn (`minimateplus.sensor_check`).**
-  The "Sensor Check" traces Blastware shows to the right of the waveform panel
-  live in the series-3 binary's trailing block as four length-prefixed records
-  (`0x3c`–`0x3f`) using the same delta-block codec as the main waveform:
-  Tran/Vert/Long geophone ring-downs (the transducer's damped impulse response —
-  resonant frequency + overswing/damping) and a MicL pulse train (the mic's
-  known-signal gain check). `gather_report_data` decodes them from the retained
-  BW binary at report time; the report renders them as a strip flush against the
-  waveform panel plus the **Sensor Check → Frequency / Overswing Ratio** sub-rows
-  in the stats table. Verified against the reports on all 7 oracle events (mic
-  zero-crossing frequency = 20.1 Hz exact; geophone ring-downs consistent
-  ~7.5 Hz with overswing ~3.5). Tests in `tests/test_sensor_check.py`.
+- **Sensor self-check waveforms decoded and drawn — both series.** The little
+  "Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
+  response — plus a MicL pulse train, the mic's known-signal gain check) are the
+  unit's proof its sensors were healthy when it recorded the event.
+  - **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
+    the binary's trailing block, same delta-block codec as the main waveform.
+    Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
+    geophone ring-downs ~7.5 Hz, overswing ~3.5).
+  - **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
+    fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
+    raw int16 traces; three-channel (mic-disabled) units carry only the three
+    geophones. Validated by shape + cross-event consistency.
+  - **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
+    decoder attaches the traces to the event at decode, the writer persists
+    them, and `gather_report_data` reads them back — so SFM renders the strip
+    (flush against the waveform panel) plus the **Sensor Check → Frequency /
+    Overswing Ratio** sub-rows without knowing the source instrument.
+  - Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
+    `tests/test_event_hdf5_sensor_check.py`.
 
 - **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
   binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
@@ -77,11 +169,21 @@ All notable changes to seismo-relay are documented here.
 
 ### Migration
 
-**None.**  Every change here is additive and reads from data already on disk —
-the `.h5` samples and the retained raw BW binary.  No `.h5`/DB change, no
-schema change, no migration, no backfill, and **no `TOOL_VERSION` bump**: a
-report regenerated for an existing event simply gains the new panels, and the
-`ach_server` rescue flags don't touch the codec.
+⚠ **The sensor-check needs a backfill.**  Existing `.h5` files are schema v1
+and carry no `/sensor_check` group, so their reports show no sensor-check strip
+until regenerated.  `TOOL_VERSION` is bumped to **0.31.0**, so the standard
+backfill regenerates every event and picks up the traces with **no `--force`**:
+`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
+backfill — this rides along) and the series-3 sidecar/shape backfill for
+MiniMate events.  Purely additive — no decoded value changes, and v1 `.h5`
+files read fine until then (empty strip).  DB backup first, as always.
+
+⚠ Budget **~2 h on the NAS** — ~1.5 files/sec there versus ~85/sec on the dev
+box (gzip-4 in `sfm/event_hdf5.py` against a Synology CPU).
+
+Everything else in this release owes nothing: the FFT, the USBM compliance
+chart and the `ach_server` rescue flags are additive and read data already on
+disk — no schema change, no DB migration.
 
 ---
 

CLAUDE.md

@@ -2,7 +2,7 @@
 
 Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
 managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
-(Sierra Wireless RV50 / RV55). Current version: **v0.30.0**.
+(Sierra Wireless RV50 / RV55). Current version: **v0.31.0**.
 
 Stack-level context — which repo owns what, and how the three project versions
 pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
@@ -61,6 +61,24 @@ Read this first when picking the project back up.
   4th-decimal tick and are Thor's own rounding — no single linear LSB can
   reproduce every printed value (the constraints are infeasible by 7e-5
   relative), so do NOT retune `_GEO_LSB_IPS`.
+- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
+  `parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
+  discarding the key's page byte.  An event starting at `0x0111F2A2` and ending
+  at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
+  start.  The chunk loop then exits before fetching anything and TERM computes
+  a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
+  `/device/events` walk 500s.  Reproduced on BE12599 (2026-09-19), which had
+  78 KB stored and had rolled into page `0x12`.
+  **Why it hid so long:** every 5A capture the walk was verified against came
+  from a freshly-erased BE11529 — all three confirmed TERM examples in
+  `framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`.  Prod is
+  unaffected: it ingests complete files via BW ACH, never this walk.
+  **Fixing it has two layers** — the arithmetic (`if end < start: end +=
+  0x10000`) stops the crash and bounds the loop correctly; carrying the page
+  byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
+  over) needs a BW capture of a spanning event first.  Do not ship layer one
+  alone without a loud truncation warning — a silently short event is the
+  failure mode this codec has been bitten by repeatedly.
 - **Open, not blocking:** 14 sensitive-range files show an exact 8x
   (= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
   inserts DB rows for store files that have none (one-time per store) and the

README.md

@@ -1,4 +1,4 @@
-# seismo-relay  `v0.30.0`
+# seismo-relay  `v0.31.0`
 
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing seismographs.  Supports both the **MiniMate Plus
@@ -496,6 +496,11 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
 
 ## Roadmap (Future)
 
+> **Where it stands *today*** — an honest per-capability maturity assessment,
+> what to rely on, known issues, and the gap to a real tool:
+> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md).  This section covers
+> where it is *going*.
+
 ### Strategic direction — where this is going
 
 seismo-relay is being built as a **suite of cooperating components**

docs/sfm_tool_status.md

@@ -0,0 +1,150 @@
+# SFM — where it actually stands as a tool
+
+**Status as of 2026-09-20 (v0.31.0).**  This is the honest assessment, not the
+roadmap — `README.md § Roadmap` covers where it is *going*.  Expect this file to
+go stale; re-date it when you revise it.
+
+---
+
+## The framing
+
+SFM is **three different things wearing one name**, at three very different
+levels of maturity:
+
+| | what it is | maturity |
+|---|---|---|
+| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
+| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
+| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
+| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
+
+Brian's own description — *"right now it's an emergency tool and a research
+project"* — is accurate, and it applies specifically to the **device side**.
+The data side is not an emergency tool; it has been carrying production for
+months.
+
+Most confusion about "is SFM reliable?" comes from answering for the wrong
+tier.
+
+---
+
+## 1. What you can rely on
+
+### Production-grade — trust it
+
+- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
+  preserved Blastware ASCII exports, 45 units, files back to 2018.
+- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
+  Thor's own CSV exports; production IDFW 575/575 with zero truncations.
+- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
+  842,442 per-interval frequency comparisons with zero mismatches.
+- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
+  fed by the watchers — this is how prod actually gets its data, and it has
+  been running unattended for months.
+- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
+  listing, event detail and report.
+- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
+  binaries, with operator review state preserved across regeneration.
+- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
+  Proven in the field, including as a rescue tool (see the runbook).
+
+### Emergency-grade — works, but you are the error handling
+
+- **`/device/*` live endpoints.**  They do what they say.  But they are
+  synchronous, unauthenticated, and a single cellular download can exceed the
+  60 s timeouts that sit in front of them.
+- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
+  Each has worked in a real incident — but each has been used a handful of
+  times, by one person, with the runbook open.
+- **The standalone webapp.**  Perfectly usable, and as of v0.31.0 the cheap
+  probes and rescue actions are reachable without curl.  No auth of any kind.
+
+### Research artifacts — useful, not products
+
+- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
+  Inspector).  Desktop-only, single-user, no tests.
+- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
+  `verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
+  These produced the numbers the production claims rest on, so they matter —
+  but they are analysis scripts, not maintained code.
+- **`docs/offset_investigation.md`** — an open investigation, not a feature.
+
+---
+
+## 2. What to use when
+
+| you want to… | use | notes |
+|---|---|---|
+| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
+| Know whether ACH is on | `GET /device/call_home` | ~2 s.  **Not** `/device/events`. |
+| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
+| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
+| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
+| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
+| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
+
+The single most useful habit: **the cheap probes are cheap and the event walk
+is not.**  Reaching for `/device/events` to answer a yes/no question about a
+unit is the mistake that motivated the v0.31.0 webapp changes.
+
+---
+
+## 3. Known issues
+
+| issue | impact | status |
+|---|---|---|
+| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`.  Needs a BW capture of a spanning event to fix properly. |
+| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
+| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial.  `/docs` itself is unguarded. |
+| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug.  Surfaced in the UI as "unreliable". |
+| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint.  Must be POST-starts-job / GET-polls before any remote lab. |
+| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known.  Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
+| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
+| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
+
+---
+
+## 4. What stands between this and a real tool
+
+Roughly in dependency order — each unblocks the ones below it.
+
+**1. Authentication.**  Everything else is gated on this.  SFM has none, and
+the modem IP whitelist gives zero protection because SFM *is* the whitelisted
+origin.  The agreed design delegates rather than builds: Terra-View becomes the
+authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
+auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
+
+**2. Async long operations.**  POST starts a job, GET polls.  Retrofitting this
+after building a remote lab on top of synchronous endpoints would be far worse
+than designing for it now.
+
+**3. Confirm-guards on the remaining destructive endpoints.**  Auth answers
+*who*, not *did you mean it*.  The webapp's erase is guarded; the other seven
+destructive POSTs and `/docs` are not.
+
+**4. The 5A page-boundary fix.**  Until this lands, live event download is
+unreliable on exactly the units most likely to need attention — the ones that
+have been recording heavily.  Wants a Blastware capture of an event spanning a
+page boundary before the chunk-addressing half is trustworthy.
+
+**5. A live Thor / Micromate client.**  The device side is MiniMate-only.
+Series-4 units can only be read from forwarded files, so half the fleet has no
+live path at all.
+
+**6. Test coverage that runs from a clean checkout.**  15 of 16 current
+failures are missing fixture bundles.  A test suite that cannot go green on a
+fresh clone cannot gate anything.
+
+**7. The SDM rename.**  Cosmetic relative to the above, but the longer `sfm/`
+holds the data-side code the more the tiers blur.  ~30–50 files here, ~10–15 in
+Terra-View, plus a Docker volume migration.  Do it when the codebase is quiet.
+
+---
+
+## The short version
+
+The **data side is a real tool already**.  The **device side is a set of sharp
+instruments** that work in the hands of the person who wrote them, with the
+runbook open.  The gap between those two states is mostly **auth, async, and
+guardrails** — not protocol work.  The protocol is the part that is actually
+finished.

micromate/sensor_check.py

@@ -0,0 +1,89 @@
+r"""Decode the Thor / Micromate (series-4) sensor self-check waveforms from an
+IDFW event binary.
+
+Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events.  The IDFW
+binary carries the sensor self-check in its fixed-header region (before the
+waveform body), as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
+channel ids as the series-3 MiniMate Plus (Tran / Vert / Long / MicL), which is
+the physical self-test:
+
+  * 3c / 3d / 3e = Tran / Vert / Long geophone ring-downs (a damped impulse
+    response — resonant frequency + damping).
+  * 3f          = MicL pulse train (the mic's known-signal gain check).  Absent
+    on three-channel (mic-disabled) units.
+
+Record framing (per record)::
+
+    01 0e [id:1] [flags:3] [count:2 BE] [pad:10] [int16-BE samples × count]
+    \___ 18-byte header ___/
+
+Unlike series-3's delta-coded trailing block, series-4 stores each trace as a
+raw int16 big-endian array.  ``count`` (the 2-byte field at header offset +8)
+is the sample count; the record is padded to a fixed stride after that.
+"""
+from __future__ import annotations
+
+import struct
+from typing import Dict, List
+
+# Record id → channel.  Same ids/order as series-3 (minimateplus.sensor_check).
+_ID_TO_CHANNEL = {0x3C: "Tran", 0x3D: "Vert", 0x3E: "Long", 0x3F: "MicL"}
+_CHAIN_IDS = (0x3C, 0x3D, 0x3E, 0x3F)
+
+_MARKER = b"\x01\x0e"       # precedes the 1-byte channel id
+_HEADER_LEN = 18            # bytes from the marker start to the first sample
+_COUNT_OFF = 8              # 2-byte BE sample count, from the marker start
+_MAX_COUNT = 4000           # sanity cap (traces are ~70-200 samples)
+
+
+def _find_chain(raw: bytes):
+    """Locate the sensor-check record chain.  Returns a list of
+    ``(offset, id, count)`` for the first run of markers whose ids run
+    3c, 3d, 3e[, 3f] in order, or ``[]``.
+
+    Records are padded to a fixed stride, so the next marker is not at
+    ``header + count*2``; instead collect every ``01 0e [id]`` marker with a
+    sane count and take the first id-ordered run.  Validating the id sequence
+    (not a lone ``01 0e 3c``) keeps a stray marker in the waveform body from
+    matching — the real chain sits in the fixed header, ahead of the body.
+    """
+    n = len(raw)
+    markers = []
+    for p in range(n - _HEADER_LEN):
+        if raw[p:p + 2] == _MARKER and raw[p + 2] in _ID_TO_CHANNEL:
+            count = int.from_bytes(raw[p + _COUNT_OFF:p + _COUNT_OFF + 2], "big")
+            if 0 < count <= _MAX_COUNT:
+                markers.append((p, raw[p + 2], count))
+
+    for i, (off, rid, _c) in enumerate(markers):
+        if rid != 0x3C:
+            continue
+        run = [markers[i]]
+        for m in markers[i + 1:]:
+            if len(run) < len(_CHAIN_IDS) and m[1] == _CHAIN_IDS[len(run)]:
+                run.append(m)
+            else:
+                break
+        if len(run) >= 3:       # 3-channel (mic-disabled) units are valid
+            return run
+    return []
+
+
+def decode_idf_sensor_check(raw: bytes) -> Dict[str, List[int]]:
+    """Decode the sensor self-check traces from a Thor/Micromate IDFW binary.
+
+    Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}`` in
+    raw int16 ADC counts (MicL omitted on 3-channel units), or ``{}`` if the
+    binary carries no sensor-check chain (a non-IDF file, or an IDFH histogram).
+    """
+    chain = _find_chain(raw)
+    if not chain:
+        return {}
+    out: Dict[str, List[int]] = {}
+    for off, rid, count in chain:
+        start = off + _HEADER_LEN
+        blob = raw[start:start + count * 2]
+        if len(blob) < count * 2:
+            continue
+        out[_ID_TO_CHANNEL[rid]] = list(struct.unpack(">%dh" % count, blob))
+    return out

minimateplus/event_file_io.py

@@ -50,7 +50,7 @@ SIDECAR_KIND   = "sfm.event"
 # bumped without a `pip install` re-run — leading to confusing stale
 # version stamps in sidecars.  Bump this constant and CHANGELOG.md
 # together at release time.
-TOOL_VERSION = "0.30.0"
+TOOL_VERSION = "0.31.0"   # +/sensor_check group (schema v2); gates the backfill regen
 
 try:
     # Best-effort: prefer the installed metadata when it's NEWER than the
@@ -296,6 +296,16 @@ def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
         event.sample_rate = report.sample_rate_sps
     if report.record_time_s is not None:
         event.rectime_seconds = report.record_time_s
+    # The report's event_datetime is Blastware's exact trigger time (parsed
+    # from Event Time + Event Date).  Prefer it over the binary footer's stop
+    # time so a report-paired import matches BW to the second.
+    edt = report.event_datetime
+    if edt is not None:
+        event.timestamp = Timestamp(
+            raw=b"", flag=0x10,
+            year=edt.year, unknown_byte=0, month=edt.month, day=edt.day,
+            hour=edt.hour, minute=edt.minute, second=edt.second,
+        )
 
 
 def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
@@ -808,6 +818,30 @@ def derive_record_type_from_filename(filename, default: str = "Waveform") -> str
     return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
 
 
+# Marker for the recording-setup config block, and the offset of the record-time
+# float32 within it.  The configured post-trigger record time (seconds) is a
+# big-endian float32 exactly 30 bytes before the "Standard Recording Setup"
+# label.  Verified across the corpus reading 1.0 / 2.0 / 3.0 s on different
+# setups — and ts2 - record_time reproduces Blastware's trigger to the second
+# (N844LQHB: stop 10:33:32 - 3.0 = 10:33:29).
+_RECSETUP_MARKER = b"Standard Recording Setup"
+_RECTIME_OFFSET_BEFORE_MARKER = 30
+
+
+def _parse_record_time_seconds(raw: bytes) -> Optional[float]:
+    """The configured post-trigger record time in seconds, from the recording-
+    setup config block, or None when absent / implausible."""
+    a = raw.find(_RECSETUP_MARKER)
+    if a < _RECTIME_OFFSET_BEFORE_MARKER:
+        return None
+    off = a - _RECTIME_OFFSET_BEFORE_MARKER
+    try:
+        rt = struct.unpack(">f", raw[off:off + 4])[0]
+    except struct.error:
+        return None
+    return rt if 0.05 <= rt <= 600.0 else None
+
+
 def read_blastware_file(path: Union[str, Path]) -> Event:
     """
     Parse a Blastware waveform file into an Event.
@@ -917,6 +951,10 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
     # rest of the event (timestamp, waveform_key, project strings) is
     # still recoverable and useful.
     decoded = decode_waveform_v2(body)
+    # Discriminator for the timestamp logic below: a waveform (trigger) event
+    # vs a histogram window.  Keyed on the codec, not the filename — the
+    # save_imported_bw path passes a tmp ".bw" name whose extension lies.
+    is_waveform_body = decoded is not None
     if decoded is None:
         decoded = decode_histogram_body(body)
     if decoded is None:
@@ -948,7 +986,31 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
     ev.total_samples   = strt_fields.get("total_samples")
     ev.pretrig_samples = strt_fields.get("pretrig_samples")
 
-    if ts1 is not None:
+    # Event timestamp.  The footer's two timestamps mean different things by
+    # record type:
+    #   * Waveform: ts1 = the monitoring-SESSION start (shared across every
+    #     event that day — a unit arming at 06:00 stamps 06:00 on all of them),
+    #     ts2 = THIS event's recording STOP.  Blastware's Date/Time is the
+    #     TRIGGER = ts2 - record time, and the record time is a float32 in the
+    #     recording-setup config block (see _parse_record_time_seconds), so the
+    #     exact trigger is recoverable from the binary alone.  Falls back to ts2
+    #     (the stop, within the record duration) if the config block is absent.
+    #     (Stamping ts1 showed the session start, hours off.)
+    #   * Histogram / undecodable: ts1 = the window start, which IS the event
+    #     time — keep it.
+    # Discriminate by ``is_waveform_body`` (the codec), not the filename.
+    if is_waveform_body and ts2 is not None:
+        _stop = datetime.datetime(ts2.year, ts2.month, ts2.day,
+                                  ts2.hour, ts2.minute, ts2.second)
+        _rt = _parse_record_time_seconds(raw)
+        _trig = _stop - datetime.timedelta(seconds=_rt) if _rt is not None else _stop
+        ev.timestamp = Timestamp(
+            raw=footer[10:18],
+            flag=0x10,
+            year=_trig.year, unknown_byte=0, month=_trig.month, day=_trig.day,
+            hour=_trig.hour, minute=_trig.minute, second=_trig.second,
+        )
+    elif ts1 is not None:
         ev.timestamp = Timestamp(
             raw=footer[2:10],
             flag=0x10,
@@ -960,6 +1022,11 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
         project=project, client=client, operator=user, sensor_location=seisloc,
     )
     ev.raw_samples = samples
+    # Sensor self-check traces from the binary's trailing block (waveform
+    # events only; returns {} for histograms / when absent).  Carried on the
+    # Event so the .h5 writer persists them device-agnostically.
+    from minimateplus.sensor_check import decode_sensor_check
+    ev.sensor_check = decode_sensor_check(raw) or None
     # Only compute peaks from samples when we actually have samples.
     # For events the codec couldn't decode (histogram-mode bodies, until
     # the §7.6.2 histogram codec is wired in), samples is an empty dict

minimateplus/models.py

@@ -544,6 +544,15 @@ class Event:
     pretrig_samples: Optional[int] = None   # from STRT record: pre-trigger sample count
     rectime_seconds: Optional[int] = None   # from STRT record: record duration (seconds)
 
+    # Sensor self-check traces keyed by channel label — the short diagnostic
+    # waveforms the unit records when it pulses each sensor before monitoring
+    # (geophone ring-downs + a mic pulse train).  Decoded from the binary by
+    # the per-series decoder (minimateplus.sensor_check / micromate.sensor_check)
+    # and carried here so the .h5 writer can persist them device-agnostically.
+    # Raw ADC counts; the source series' scale differs but the trace is a
+    # shape diagnostic (rendered fit-to-box).  None when absent.
+    sensor_check: Optional[dict] = None     # {"Tran": [...], ..., "MicL": [...]}
+
     # ── Debug / introspection ─────────────────────────────────────────────────
     # Raw 210-byte waveform record bytes, set when debug mode is active.
     # Exposed by the SFM server via ?debug=true so field layouts can be verified.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "seismo-relay"
-version = "0.30.0"
+version = "0.31.0"
 description = "Python client and REST server for MiniMate Plus seismographs"
 requires-python = ">=3.10"
 dependencies = [

scripts/backfill_thor_events.py

@@ -305,6 +305,11 @@ def main(argv=None) -> int:
                                 default=0,
                             )
                             ev.total_samples = ev.total_samples or n_samp
+                            # Sensor self-check traces from the IDFW fixed
+                            # header, so regenerated .h5 files gain the v2
+                            # /sensor_check group (mirrors save_imported_idf).
+                            from micromate.sensor_check import decode_idf_sensor_check
+                            ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
 
                         event_hdf5.write_event_hdf5(
                             hdf5_path, ev,

sfm/event_hdf5.py

@@ -12,8 +12,11 @@ Layout written to `<filename>.h5`:
   ├─ samples_int16/ (optional)
   │   ├─ Tran  (int16, raw ADC counts) shape: (N,)
   │   └─ ... per channel (only when present in the source)
+  ├─ sensor_check/ (optional, schema v2+)
+  │   ├─ Tran  (int32, raw counts)     shape: (M,)   M ≪ N
+  │   └─ ... per channel present in the source (MicL absent on 3-channel units)
   └─ root attrs (event metadata):
-       schema_version       int    = 1
+       schema_version       int    = 2
        kind                 str    = "sfm.event.hdf5"
        serial               str
        waveform_key         str    (8-hex)
@@ -64,7 +67,7 @@ from minimateplus.models import Event
 
 log = logging.getLogger(__name__)
 
-SCHEMA_VERSION = 1
+SCHEMA_VERSION = 2   # v2 adds the optional /sensor_check group
 HDF5_KIND = "sfm.event.hdf5"
 
 # Geophone full-scale velocity per range (in/s).  Confirmed in CLAUDE.md
@@ -270,6 +273,22 @@ def write_event_hdf5(
                     )
             igrp.attrs["mic_psi_per_count"] = float(mic_factor)
 
+        # /sensor_check — optional short diagnostic self-check traces (schema
+        # v2+).  Raw ADC counts (a shape diagnostic; the per-series count scale
+        # differs, and the renderer fits each trace to its box).  Only channels
+        # the decoder found are written — 3-channel units carry no MicL.
+        sc = event.sensor_check or {}
+        if sc:
+            scgrp = f.create_group("sensor_check")
+            for ch in ("Tran", "Vert", "Long", "MicL"):
+                vals = sc.get(ch)
+                if vals:
+                    scgrp.create_dataset(
+                        ch, data=np.asarray(vals, dtype=np.int32),
+                        compression="gzip", compression_opts=4, shuffle=True,
+                    )
+            scgrp.attrs["units"] = "raw_counts"
+
     import os
     os.replace(tmp, path)
 
@@ -334,6 +353,16 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
             if mic_attr is not None:
                 mic_psi = float(mic_attr)
 
+        # /sensor_check — optional (schema v2+); absent on older files.
+        sensor_check = None
+        scgrp = f.get("sensor_check")
+        if scgrp is not None:
+            sensor_check = {}
+            for ch in ("Tran", "Vert", "Long", "MicL"):
+                ds = scgrp.get(ch)
+                if ds is not None:
+                    sensor_check[ch] = np.asarray(ds[()])
+
     return {
         "schema_version":     sv,
         "kind":               attrs.get("kind"),
@@ -341,6 +370,7 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
         "samples":            samples,
         "samples_int16":      samples_int16,
         "mic_psi_per_count":  mic_psi,
+        "sensor_check":       sensor_check,
     }
 
 
@@ -431,11 +461,16 @@ def plot_json_from_hdf5(
     event_id: Optional[str] = None,
     index: Optional[int] = None,
 ) -> dict:
-    """Build a `sfm.plot.v1` JSON dict from a stored .h5 file."""
+    """Build a `sfm.plot.v1` JSON dict from a stored .h5 file.
+
+    The dict also carries a top-level ``sensor_check`` key (the raw self-check
+    traces as ``{ch: [int]}``, or None) beyond the plot schema, so report
+    generation can read the traces from the same single .h5 load.
+    """
     data = read_event_hdf5(path)
     a = data["attrs"]
     s = data["samples"]
-    return _build_plot_dict(
+    out = _build_plot_dict(
         n_samples=len(s["Tran"]) if "Tran" in s else 0,
         sample_rate=int(a.get("sample_rate", 1024) or 1024),
         pretrig_samples=int(a.get("pretrig_samples", 0) or 0),
@@ -463,6 +498,11 @@ def plot_json_from_hdf5(
         event_id=event_id,
         index=index,
     )
+    scd = data.get("sensor_check")
+    out["sensor_check"] = (
+        {ch: v.tolist() for ch, v in scd.items()} if scd else None
+    )
+    return out
 
 
 def _build_plot_dict(

sfm/report_pdf.py

@@ -121,9 +121,11 @@ class ReportData:
     t0_ms:                  Optional[float] = None
     dt_ms:                  Optional[float] = None
 
-    # Sensor self-check traces — {ch: [samples]} in raw decode units, decoded
-    # from the binary's trailing block (see minimateplus.sensor_check). The
-    # little waveforms BW draws in its "Sensor Check" strip. Empty when absent.
+    # Sensor self-check traces — {ch: [samples]} in raw counts, read from the
+    # standardized .h5 (/sensor_check group, schema v2+) where the per-series
+    # decoder stored them at ingest.  The little diagnostic waveforms BW draws
+    # in its "Sensor Check" strip.  Empty when absent (pre-v2 .h5, histogram,
+    # or 3-channel unit's MicL).
     sensor_check_waveforms: dict = field(default_factory=dict)
 
     # Record-type discriminator
@@ -294,22 +296,15 @@ def gather_report_data(
             rd.pretrig_samples  = ta.get("pretrig_samples")
             rd.t0_ms            = ta.get("t0_ms")
             rd.dt_ms            = ta.get("dt_ms")
+            # Sensor self-check traces — read from the standardized .h5 (schema
+            # v2+).  Device-agnostic: whichever decoder produced the event
+            # stored them at ingest, so SFM reads them here without knowing or
+            # caring about the source instrument series.  Empty on pre-v2 files
+            # (until backfilled) and on 3-channel / histogram events.
+            rd.sensor_check_waveforms = wf.get("sensor_check") or {}
         except Exception as exc:
             log.warning("gather_report_data: hdf5 read failed: %s", exc)
 
-    # ── Sensor self-check traces — decoded from the retained raw binary ──
-    # The .h5 holds only the main waveform; the sensor-check traces live in the
-    # binary's trailing block, so decode them straight from the kept BW file.
-    # Waveform events only (histograms have no sensor-check strip).
-    if not rd.is_histogram:
-        try:
-            from minimateplus.sensor_check import decode_sensor_check
-            bw_path, _a5 = store.paths_for(serial, filename)
-            if bw_path.exists():
-                rd.sensor_check_waveforms = decode_sensor_check(bw_path.read_bytes())
-        except Exception as exc:
-            log.warning("gather_report_data: sensor-check decode failed: %s", exc)
-
     # ── Histogram aggregation ──
     # Codec emits ~N per-block samples (typically 1/sec); BW reports
     # one bar per configured interval (1 min / 5 min / etc.).  When

sfm/sfm_webapp.html

@@ -108,6 +108,12 @@
       color: var(--text);
     }
     .btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
+    .btn-danger   { background: var(--red);    color: #fff; }
+    .btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
+    .diag-result  { display:block; margin-top:6px; font-size:12px; opacity:.85;
+                    white-space:pre-wrap; word-break:break-word; }
+    .diag-result.ok    { color: var(--green); }
+    .diag-result.error { color: var(--red); }
     .btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
 
     /* #connect-btn styles moved to #live-connect-bar block */
@@ -910,6 +916,7 @@
   <button class="tab-btn"        data-tab="events"    onclick="switchTab('events')">Events</button>
   <button class="tab-btn"        data-tab="config"    onclick="switchTab('config')">Config</button>
   <button class="tab-btn"        data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
+  <button class="tab-btn"        data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
 </div>
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -938,6 +945,10 @@
 <div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
 
   <div class="event-toolbar">
+    <button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
+            title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
+      ⟳ Load events
+    </button>
     <button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
     <button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
             title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
@@ -1205,6 +1216,77 @@
 
 </div><!-- end #tab-call-home -->
 
+<!-- ════════════════════════════════════════════════════════════════
+     TAB: Diagnostics
+═══════════════════════════════════════════════════════════════════ -->
+<div id="tab-diagnostics" class="tab-pane">
+
+  <div class="cfg-grid">
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Device State</div>
+      <div class="hint" style="margin-bottom:10px">
+        Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
+      </div>
+      <div class="dev-table" id="diag-table"></div>
+      <div class="cfg-actions" style="margin-top:12px">
+        <button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
+        <span id="diag-status"></span>
+      </div>
+    </div>
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Actions</div>
+
+      <div class="cfg-field">
+        <label>Stop Monitoring</label>
+        <button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
+        <div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
+        <span class="diag-result" id="diag-stop-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Disable Auto Call Home</label>
+        <button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
+        <div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
+        <span class="diag-result" id="diag-ach-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Erase All Events</label>
+        <input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
+               oninput="diagCheckEraseConfirm()" autocomplete="off" />
+        <button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
+        <div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
+        <span class="diag-result" id="diag-erase-result"></span>
+      </div>
+    </div>
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Unresponsive Unit</div>
+      <div class="hint" style="margin-bottom:10px">
+        The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
+        to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
+        <code>ach_server</code> and answer its call — before racing it with these.
+      </div>
+
+      <div class="cfg-field">
+        <label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
+        <button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
+        <div class="hint">Success is <code>bytes_received &gt; 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
+        <span class="diag-result" id="diag-drip-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
+        <button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
+        <span class="diag-result" id="diag-blind-result"></span>
+      </div>
+    </div>
+
+  </div>
+</div><!-- end #tab-diagnostics -->
+
 </div><!-- end #section-live -->
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -1361,6 +1443,8 @@
 // ── State ──────────────────────────────────────────────────────────────────────
 let unitInfo      = null;
 let eventList     = [];
+let storageInfo   = null;   // /device/events/storage_range — cheap, read on connect
+let eventsLoaded  = false;  // the event chain walk is opt-in; see loadEventList()
 let currentEvent  = 0;
 let charts        = {};
 let geoAdcScale      = 6.206;
@@ -1458,6 +1542,7 @@ function switchTab(name) {
   if (name === 'units')    { if (!unitsLoaded)  loadUnits(); }
   if (name === 'monlog')   { if (!monlogLoaded) loadMonitorLog(); }
   if (name === 'sessions') { if (!sessLoaded)   loadSessions(); }
+  if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
 }
 
 // ── Connect ────────────────────────────────────────────────────────────────────
@@ -1478,18 +1563,13 @@ async function connectUnit() {
     btn.disabled = false; btn.textContent = 'Connect'; return;
   }
 
-  setStatus('Fetching event list…', 'loading');
-  try {
-    const r = await fetch(`${api()}/device/events?${deviceParams()}`);
-    if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
-    const evData = await r.json();
-    eventList = evData.events || [];
-    // Merge compliance from /device/events response (it re-reads it)
-    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
-  } catch (e) {
-    setStatus(`Event fetch failed: ${e.message}`, 'error');
-    btn.disabled = false; btn.textContent = 'Reconnect'; return;
-  }
+  // Connecting deliberately does NOT walk the event chain.  That walk reads
+  // every event header over the cellular link and can take minutes — or fail
+  // outright on a unit whose buffer has wrapped past 0xFFFF.  Use the ~2 s
+  // probes instead; the event list is opt-in via loadEventList().
+  eventList = []; eventsLoaded = false;
+  setStatus('Reading device state…', 'loading');
+  storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
 
   populateDeviceBar();
   populateDeviceTab();
@@ -1498,11 +1578,9 @@ async function connectUnit() {
 
   document.getElementById('device-bar').style.display    = 'flex';
   document.getElementById('monitor-panel').style.display = 'flex';
-  document.getElementById('load-btn').disabled     = eventList.length === 0;
-  document.getElementById('save-btn').disabled     = eventList.length === 0;
-  document.getElementById('download-btn').disabled = eventList.length === 0;
-  document.getElementById('prev-btn').disabled     = true;
-  document.getElementById('next-btn').disabled     = eventList.length <= 1;
+  setEventButtonsEnabled();
+  document.getElementById('load-events-btn').disabled = false;
+  setDiagButtonsEnabled(true);
   document.getElementById('cfg-read-btn').disabled  = false;
   document.getElementById('cfg-write-btn').disabled = false;
   document.getElementById('ch-read-btn').disabled   = false;
@@ -1510,7 +1588,9 @@ async function connectUnit() {
 
   btn.disabled = false; btn.textContent = 'Reconnect';
 
-  setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
+  setStatus(storageInfo && storageInfo.is_empty
+    ? 'Connected — no events stored.'
+    : 'Connected. Event list not loaded (Events → Load events).', 'ok');
 
   // Fetch monitor status in background (non-blocking)
   refreshMonitorStatus().catch(() => {});
@@ -1522,6 +1602,48 @@ async function connectUnit() {
   }
 }
 
+// ── Shared fetch helper ────────────────────────────────────────────────────────
+async function fetchJson(path, opts) {
+  const sep = path.includes('?') ? '&' : '?';
+  const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
+  const body = await r.json().catch(() => ({}));
+  if (!r.ok) throw new Error(body.detail || r.statusText);
+  return body;
+}
+
+function setEventButtonsEnabled() {
+  const n = eventList.length;
+  document.getElementById('load-btn').disabled     = n === 0;
+  document.getElementById('save-btn').disabled     = n === 0;
+  document.getElementById('download-btn').disabled = n === 0;
+  document.getElementById('prev-btn').disabled     = true;
+  document.getElementById('next-btn').disabled     = n <= 1;
+}
+
+// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
+async function loadEventList() {
+  if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
+  const btn = document.getElementById('load-events-btn');
+  btn.disabled = true;
+  setStatus('Walking the event chain — this can take a while…', 'loading');
+  try {
+    const evData = await fetchJson('/device/events');
+    eventList = evData.events || [];
+    eventsLoaded = true;
+    // /device/events re-reads compliance; fold it in.
+    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
+  } catch (e) {
+    setStatus(`Event fetch failed: ${e.message}`, 'error');
+    btn.disabled = false; return;
+  }
+  populateDeviceBar();
+  populateDeviceTab();
+  populateEventChips();
+  setEventButtonsEnabled();
+  btn.disabled = false;
+  setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
+}
+
 // ── Device bar ─────────────────────────────────────────────────────────────────
 function populateDeviceBar() {
   qs('di-serial').textContent   = unitInfo.serial || '—';
@@ -1530,7 +1652,7 @@ function populateDeviceBar() {
   qs('di-sr').textContent       = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
   qs('di-rt').textContent       = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
   qs('di-trig').textContent     = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
-  qs('di-count').textContent    = eventList.length;
+  qs('di-count').textContent    = eventsLoaded ? eventList.length : '—';
   qs('di-project').textContent  = cc.project || '—';
   qs('di-client').textContent   = cc.client  || '—';
   qs('di-operator').textContent = cc.operator || '—';
@@ -1660,7 +1782,8 @@ function populateDeviceTab() {
     { label:'DSP',            value: unitInfo.dsp_version || '—' },
     { label:'Model',          value: unitInfo.model || '—' },
     { label:'Manufacturer',   value: unitInfo.manufacturer || '—' },
-    { label:'Stored Events',  value: eventList.length },
+    { label:'Stored Events',  value: eventsLoaded ? eventList.length : 'not loaded' },
+    { label:'Storage Used',   value: storageUsedLabel() },
   ];
   for (const {label, value} of cardData) {
     const c = document.createElement('div');
@@ -1707,6 +1830,158 @@ function renderTable(id, rows) {
   }
 }
 
+// ── Diagnostics ────────────────────────────────────────────────────────────────
+// Everything here is a cheap probe (POLL + one read) or a single write.  None of
+// it walks the event chain.  See docs/runbooks/wedged_unit_recovery.md.
+
+function storageUsedLabel() {
+  if (!storageInfo) return '—';
+  if (storageInfo.is_empty) return 'empty';
+  const f = storageInfo.first_key, l = storageInfo.last_key;
+  return (f && l) ? `${f} → ${l}` : '—';
+}
+
+function setDiagButtonsEnabled(on) {
+  for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
+                    'diag-drip-btn','diag-blind-btn']) {
+    const el = document.getElementById(id);
+    if (el) el.disabled = !on;
+  }
+  diagCheckEraseConfirm();
+}
+
+// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
+function diagCheckEraseConfirm() {
+  const box = document.getElementById('diag-erase-confirm');
+  const btn = document.getElementById('diag-erase-btn');
+  if (!box || !btn) return;
+  const serial = (unitInfo && unitInfo.serial) || '';
+  btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
+}
+
+function diagResult(id, text, cls) {
+  const el = document.getElementById(id);
+  if (!el) return;
+  el.textContent = text;
+  el.className = 'diag-result' + (cls ? ' ' + cls : '');
+}
+
+async function refreshDiagnostics() {
+  if (!devHost()) return;
+  const st = document.getElementById('diag-status');
+  if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
+
+  const [mon, store, idx] = await Promise.all([
+    fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
+    fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
+    fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
+  ]);
+
+  if (!store._err) storageInfo = store;
+
+  const err = v => `<span style="color:var(--red)">${v}</span>`;
+  const rows = [];
+
+  rows.push(['Monitoring', mon._err ? err(mon._err)
+    : (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
+  if (!mon._err) {
+    rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
+    if (mon.memory_total_bytes) {
+      const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
+      const pct  = (used / mon.memory_total_bytes * 100).toFixed(1);
+      rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
+    }
+  }
+
+  rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
+  if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
+
+  // SUB 0x08.  Known to report 0 on units with years of history — suspected
+  // field-offset bug in the decode, so show it but do not trust it.
+  rows.push(['Lifetime events', idx._err ? err(idx._err)
+    : `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
+
+  renderTable('diag-table', rows);
+  populateDeviceTab();
+  if (st) { st.textContent = ''; st.className = ''; }
+}
+
+async function diagStopMonitoring() {
+  const btn = document.getElementById('diag-stop-btn');
+  btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
+  try {
+    await fetchJson('/device/monitor/stop', { method: 'POST' });
+    diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
+    refreshDiagnostics();
+  } catch (e) {
+    diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagDisableAch() {
+  const btn = document.getElementById('diag-ach-btn');
+  btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
+  try {
+    const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
+    const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
+    diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
+  } catch (e) {
+    diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagEraseEvents() {
+  const serial = (unitInfo && unitInfo.serial) || 'this unit';
+  if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
+  const btn = document.getElementById('diag-erase-btn');
+  btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
+  try {
+    await fetchJson('/device/events/erase', { method: 'POST' });
+    diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
+    document.getElementById('diag-erase-confirm').value = '';
+    eventList = []; eventsLoaded = false;
+    setEventButtonsEnabled(); populateEventChips();
+    refreshDiagnostics();
+  } catch (e) {
+    diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
+  }
+  diagCheckEraseConfirm();
+}
+
+async function diagSlowDrip() {
+  const btn = document.getElementById('diag-drip-btn');
+  btn.disabled = true;
+  diagResult('diag-drip-result', 'Holding a session for 120 s…');
+  try {
+    const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
+                              { method: 'POST' });
+    const good = (r.bytes_received || 0) > 0;
+    diagResult('diag-drip-result',
+      `drips ${r.drips_sent}  ·  held ${r.duration_s}s  ·  bytes back ${r.bytes_received}` +
+      (r.send_error ? `  ·  ${r.send_error}` : '') +
+      (good ? '  → device responded' : '  → no response; the modem may not be bridging'),
+      good ? 'ok' : 'error');
+  } catch (e) {
+    diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagBlindStop() {
+  const btn = document.getElementById('diag-blind-btn');
+  btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
+  try {
+    const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
+    diagResult('diag-blind-result',
+      `Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
+  } catch (e) {
+    diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
 // ── Config form ────────────────────────────────────────────────────────────────
 function populateConfigFromDeviceInfo() {
   if (!unitInfo) return;

sfm/waveform_store.py

@@ -662,6 +662,11 @@ class WaveformStore:
             ev.raw_samples = idf_samples
             n_samples = max((len(idf_samples.get(ch, [])) for ch in ("Tran", "Vert", "Long", "MicL")), default=0)
             ev.total_samples = ev.total_samples or n_samples
+            # Sensor self-check traces from the IDFW fixed header (waveform
+            # events only; {} on histograms / when absent).  Carried on the
+            # bridged Event so the .h5 writer persists them like series-3.
+            from micromate.sensor_check import decode_idf_sensor_check
+            ev.sensor_check = decode_idf_sensor_check(idf_bytes) or None
 
         # For IDFH histograms there are no per-sample waveform arrays — the
         # device stores one peak ADC count per interval per channel.  Synthesise

tests/test_event_hdf5_sensor_check.py

@@ -0,0 +1,71 @@
+"""The event .h5 carries the sensor self-check traces (schema v2).
+
+The sensor check is decoded by the per-series decoder and attached to the
+standardized Event, so the .h5 writer persists it device-agnostically and SFM
+reads it back without knowing which instrument produced it.  Old v1 files (no
+sensor_check group) must still read cleanly.
+"""
+import tempfile
+from pathlib import Path
+
+import numpy as np
+
+from minimateplus.models import Event
+from minimateplus.event_file_io import read_blastware_file
+from sfm import event_hdf5
+
+S3_FIX = Path(__file__).parent / "fixtures" / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"
+
+
+def _write(ev, **kw):
+    d = Path(tempfile.mkdtemp())
+    p = d / "e.h5"
+    event_hdf5.write_event_hdf5(p, ev, serial="BE12844", **kw)
+    return p
+
+
+def test_sensor_check_roundtrips_through_hdf5():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, -3], "Vert": [0, 1], "Long": [2], "MicL": [5, -5]}
+    ev.sample_rate = 1024
+    sc = {"Tran": [0, -990, -500, -100], "Vert": [0, -980, -480],
+          "Long": [0, -986, -470], "MicL": [0, -1800, 1800, -1800]}
+    ev.sensor_check = sc
+
+    r = event_hdf5.read_event_hdf5(_write(ev))
+    assert r["schema_version"] == 2
+    assert set(r["sensor_check"]) == {"Tran", "Vert", "Long", "MicL"}
+    for ch, vals in sc.items():
+        assert r["sensor_check"][ch].tolist() == vals
+
+
+def test_plot_json_carries_sensor_check():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, 3]}
+    ev.sample_rate = 1024
+    ev.sensor_check = {"Tran": [0, -990, -500], "Vert": [0, -980],
+                       "Long": [0, -986]}   # 3-channel: no MicL
+    pj = event_hdf5.plot_json_from_hdf5(_write(ev))
+    assert pj["sensor_check"] is not None
+    assert "MicL" not in pj["sensor_check"]
+    assert pj["sensor_check"]["Tran"] == [0, -990, -500]
+
+
+def test_event_without_sensor_check_still_reads_as_v2():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, 3]}
+    ev.sample_rate = 1024
+    r = event_hdf5.read_event_hdf5(_write(ev))
+    assert r["schema_version"] == 2
+    assert r["sensor_check"] is None
+    assert event_hdf5.plot_json_from_hdf5(_write(ev))["sensor_check"] is None
+
+
+def test_series3_decode_populates_event_sensor_check():
+    # The real series-3 decoder attaches the traces to the Event, so the
+    # ingest/backfill .h5 write picks them up with no extra plumbing.
+    ev = read_blastware_file(S3_FIX)
+    assert ev.sensor_check is not None
+    assert set(ev.sensor_check) == {"Tran", "Vert", "Long", "MicL"}
+    tran = np.asarray(ev.sensor_check["Tran"], dtype=float)
+    assert tran.min() < -800   # the geophone ring-down deflection

tests/test_event_timestamp.py

@@ -0,0 +1,54 @@
+"""Event timestamp decode — waveform trigger/stop vs histogram window start.
+
+The Blastware footer holds two timestamps: ts1 = footer[2:10], ts2 = footer[10:18].
+Their meaning depends on record type:
+
+  * Waveform: ts1 is the monitoring-SESSION start (e.g. 06:00 for a unit that
+    arms at 06:00 daily — shared across every event that day), and ts2 is THIS
+    event's recording STOP.  read_blastware_file used to stamp events with ts1 →
+    every waveform showed the session start (~4.5 h off).  Binary-only, the best
+    estimate is ts2 (the stop); the exact trigger BW displays (= ts2 - record
+    duration) comes from the paired report's event_datetime, since the binary
+    STRT record-time byte is a misparsed record-type marker.
+  * Histogram: ts1/ts2 are the ~24 h window [start, stop]; the event time is the
+    window start = ts1 (unchanged).
+"""
+import datetime
+from pathlib import Path
+
+from minimateplus.event_file_io import read_blastware_file, apply_report_to_event
+from minimateplus.bw_ascii_report import BwAsciiReport
+from minimateplus.models import Event
+
+FIX = Path(__file__).parent / "fixtures"
+WAVEFORM = FIX / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"   # footer ts2 = 2026-08-25 10:33:32
+HISTOGRAM = FIX / "ts-fix" / "K441LKZU.C30H"                 # window start 2026-05-10 19:04:50
+
+
+def _tuple(ts):
+    return (ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second)
+
+
+def test_waveform_timestamp_is_exact_trigger_from_binary():
+    ev = read_blastware_file(WAVEFORM)
+    # The EXACT Blastware trigger, from the binary alone: ts2 (stop 10:33:32)
+    # minus the config record time (3.0 s) = 10:33:29 — NOT the 06:00:13
+    # monitoring-session start the old decode used.
+    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29), _tuple(ev.timestamp)
+
+
+def test_histogram_timestamp_is_window_start_unchanged():
+    ev = read_blastware_file(HISTOGRAM)
+    # Histogram event time = the window start (ts1); must NOT get the waveform
+    # ts2 treatment (that would land ~24 h off).
+    assert _tuple(ev.timestamp) == (2026, 5, 10, 19, 4, 50), _tuple(ev.timestamp)
+
+
+def test_report_event_datetime_is_authoritative_over_binary():
+    # The binary already yields the exact trigger, but a paired report stays
+    # authoritative (e.g. if the unit clock had drifted) — applying it wins.
+    ev = read_blastware_file(WAVEFORM)
+    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29)   # exact, from binary
+    apply_report_to_event(ev, BwAsciiReport(
+        event_datetime=datetime.datetime(2026, 8, 25, 10, 35, 0)))
+    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 35, 0)    # report wins

tests/test_sensor_check_idf.py

@@ -0,0 +1,67 @@
+"""Series-4 (Thor / Micromate IDFW) sensor self-check waveform decode.
+
+Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events.  The IDFW
+binary carries the sensor self-check in its fixed-header region (before the
+waveform body) as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
+channel ids as series-3 (Tran/Vert/Long/MicL).  Unlike series-3's delta-coded
+trailing block, series-4 stores each trace as a raw int16-BE array after an
+18-byte record header whose sample count is a 2-byte field at offset +8.
+
+Three-channel (mic-disabled) Thor units carry only 3c/3d/3e — no MicL record.
+
+Validated by shape (geophone ring-down / mic pulse train) and cross-event
+consistency, since there's no Thor Event-Report strip to exact-match against.
+"""
+from pathlib import Path
+
+import numpy as np
+
+from micromate.sensor_check import decode_idf_sensor_check
+
+FIXDIR = Path(__file__).parent / "fixtures" / "thor-idf-sc"
+EVENTS = sorted(p.name for p in FIXDIR.glob("*.IDFW"))
+
+
+def _decode(name):
+    return decode_idf_sensor_check((FIXDIR / name).read_bytes())
+
+
+def test_geo_channels_present_and_ringdown_shaped():
+    # Every IDFW event has the three geophone self-checks; each is a large
+    # one-sided deflection (~15000 raw counts) that rings back — the geophone's
+    # damped impulse response.
+    for name in EVENTS:
+        sc = _decode(name)
+        for ch in ("Tran", "Vert", "Long"):
+            assert ch in sc, f"{name} missing {ch}"
+            tr = np.asarray(sc[ch], dtype=float)
+            tr = tr - tr[:4].mean()          # reference to the pre-trigger baseline
+            assert 40 <= len(tr) <= 300, f"{name}:{ch} n={len(tr)}"
+            assert tr.min() < -8000, f"{name}:{ch} min {tr.min()}"
+            # deflects one way and rings back toward / past the baseline
+            assert tr.max() < abs(tr.min()), f"{name}:{ch} not one-sided"
+
+
+def test_mic_present_only_on_four_channel_units():
+    # UM11719 / UM12947 record a mic; UM13981 / UM20147 are 3-channel
+    # (mic-disabled) units and carry no MicL self-check.
+    got = {name: ("MicL" in _decode(name)) for name in EVENTS}
+    assert any(got.values()), "expected at least one 4-channel unit"
+    assert not all(got.values()), "expected at least one 3-channel unit"
+    for name, has_mic in got.items():
+        if has_mic:
+            tr = np.asarray(_decode(name)["MicL"], dtype=float)
+            tr = tr - tr[:4].mean()
+            # mic self-check is a bipolar pulse train — swings both ways, wide range
+            assert tr.max() > 5000 and tr.min() < -5000, f"{name} mic not bipolar"
+
+
+def test_channel_ids_and_order():
+    # ids decode to the canonical channel names, geo always in Tran/Vert/Long order
+    sc = _decode(EVENTS[0])
+    assert [c for c in ("Tran", "Vert", "Long") if c in sc] == ["Tran", "Vert", "Long"]
+
+
+def test_returns_empty_on_non_idf_input():
+    assert decode_idf_sensor_check(b"not an IDF file") == {}
+    assert decode_idf_sensor_check(b"") == {}