docs: record the 5A page-boundary bug, and assess SFM as a tool

Two things Brian asked for after the BE12599 work.

The known bug: the 5A walk discards the key's page byte, so once a unit has
recorded more than 64 KB since its last erase, an event spanning the boundary
reads an end_offset behind its own start. The chunk loop then fetches nothing
and TERM packs a negative offset_word, which is the 500. Reproduced on BE12599.
It hid this long because every capture the walk was verified against came from
a freshly-erased BE11529 — all three confirmed TERM examples sit inside page
0x11. Prod is unaffected; it ingests complete files and never runs this walk.

The status doc exists because "is SFM reliable?" has three different answers
depending on which tier is meant. The codec library and the data side are
production — verified per-sample at scale, carrying Terra-View daily. The
device side is emergency-grade: it works, but it is synchronous,
unauthenticated, and thinly tested. The lab is research artifacts. Most
confusion comes from answering for the wrong tier.

It covers all three of what Brian asked for: maturity per capability, an
operator-facing "what to use when" (the cheap probes are cheap and the event
walk is not), the known-issues table, and the gap analysis. That gap is mostly
auth, async and guardrails — not protocol work. The protocol is the finished
part.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN

CHANGELOG.md

@@ -4,7 +4,30 @@ All notable changes to seismo-relay are documented here.
 
 ---
 
-## Unreleased
+## v0.31.0 — 2026-09-18
+
+**Report parity, and a second way to rescue a runaway unit.**  Two threads.
+
+The first closes out Blastware Event/FFT-Report parity: the FFT, the USBM
+RI8507 compliance chart and the sensor self-check now render on the event
+report, reverse-engineered against BE12844 (MiniMate Plus) and UM (Thor)
+events.  The sensor check is decoded for **both** series and standardized into
+the `.h5` (schema **v2**, a new `/sensor_check` group), so SFM serves it
+device-agnostically rather than decoding at report time.  The Inspector — an
+annotated hex reader for series-3 binaries — is what made the trailing-block
+structure findable, and it earned its keep by *ruling out* a stored FFT block
+and proving Blastware computes it from the samples.
+
+The second came out of a field emergency.  BE12599's connector fault drove its
+Tran channel to its trigger level, so the unit recorded back-to-back and dialed
+the office ACH server every ~75 s, unreachable the whole time.
+`bridges/ach_server.py` gained `--stop-monitoring` / `--disable-ach` /
+`--rescue`, which **invert** the recovery: instead of racing a Stop into the
+gaps between dial-outs, point the modem's Destination at our own ACH server and
+answer the call.  Proven in production the same night — the stop landed on the
+first call-in and held.  See `docs/runbooks/wedged_unit_recovery.md`.
+
+⚠ **This release owes prod a backfill** — see Migration below.
 
 ### Added
 - **Rescue-on-connect for `bridges/ach_server.py`** — `--stop-monitoring`
@@ -46,18 +69,25 @@ All notable changes to seismo-relay are documented here.
   reference PDF. A technical breakdown of the curve is in
   `docs/ri8507_compliance_curve.md`.
 
-- **Sensor self-check waveforms decoded and drawn (`minimateplus.sensor_check`).**
-  The "Sensor Check" traces Blastware shows to the right of the waveform panel
-  live in the series-3 binary's trailing block as four length-prefixed records
-  (`0x3c`–`0x3f`) using the same delta-block codec as the main waveform:
-  Tran/Vert/Long geophone ring-downs (the transducer's damped impulse response —
-  resonant frequency + overswing/damping) and a MicL pulse train (the mic's
-  known-signal gain check). `gather_report_data` decodes them from the retained
-  BW binary at report time; the report renders them as a strip flush against the
-  waveform panel plus the **Sensor Check → Frequency / Overswing Ratio** sub-rows
-  in the stats table. Verified against the reports on all 7 oracle events (mic
-  zero-crossing frequency = 20.1 Hz exact; geophone ring-downs consistent
-  ~7.5 Hz with overswing ~3.5). Tests in `tests/test_sensor_check.py`.
+- **Sensor self-check waveforms decoded and drawn — both series.** The little
+  "Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
+  response — plus a MicL pulse train, the mic's known-signal gain check) are the
+  unit's proof its sensors were healthy when it recorded the event.
+  - **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
+    the binary's trailing block, same delta-block codec as the main waveform.
+    Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
+    geophone ring-downs ~7.5 Hz, overswing ~3.5).
+  - **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
+    fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
+    raw int16 traces; three-channel (mic-disabled) units carry only the three
+    geophones. Validated by shape + cross-event consistency.
+  - **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
+    decoder attaches the traces to the event at decode, the writer persists
+    them, and `gather_report_data` reads them back — so SFM renders the strip
+    (flush against the waveform panel) plus the **Sensor Check → Frequency /
+    Overswing Ratio** sub-rows without knowing the source instrument.
+  - Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
+    `tests/test_event_hdf5_sensor_check.py`.
 
 - **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
   binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
@@ -77,11 +107,21 @@ All notable changes to seismo-relay are documented here.
 
 ### Migration
 
-**None.**  Every change here is additive and reads from data already on disk —
-the `.h5` samples and the retained raw BW binary.  No `.h5`/DB change, no
-schema change, no migration, no backfill, and **no `TOOL_VERSION` bump**: a
-report regenerated for an existing event simply gains the new panels, and the
-`ach_server` rescue flags don't touch the codec.
+⚠ **The sensor-check needs a backfill.**  Existing `.h5` files are schema v1
+and carry no `/sensor_check` group, so their reports show no sensor-check strip
+until regenerated.  `TOOL_VERSION` is bumped to **0.31.0**, so the standard
+backfill regenerates every event and picks up the traces with **no `--force`**:
+`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
+backfill — this rides along) and the series-3 sidecar/shape backfill for
+MiniMate events.  Purely additive — no decoded value changes, and v1 `.h5`
+files read fine until then (empty strip).  DB backup first, as always.
+
+⚠ Budget **~2 h on the NAS** — ~1.5 files/sec there versus ~85/sec on the dev
+box (gzip-4 in `sfm/event_hdf5.py` against a Synology CPU).
+
+Everything else in this release owes nothing: the FFT, the USBM compliance
+chart and the `ach_server` rescue flags are additive and read data already on
+disk — no schema change, no DB migration.
 
 ---
 

CLAUDE.md

@@ -2,7 +2,7 @@
 
 Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
 managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
-(Sierra Wireless RV50 / RV55). Current version: **v0.30.0**.
+(Sierra Wireless RV50 / RV55). Current version: **v0.31.0**.
 
 Stack-level context — which repo owns what, and how the three project versions
 pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
@@ -61,6 +61,24 @@ Read this first when picking the project back up.
   4th-decimal tick and are Thor's own rounding — no single linear LSB can
   reproduce every printed value (the constraints are infeasible by 7e-5
   relative), so do NOT retune `_GEO_LSB_IPS`.
+- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
+  `parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
+  discarding the key's page byte.  An event starting at `0x0111F2A2` and ending
+  at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
+  start.  The chunk loop then exits before fetching anything and TERM computes
+  a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
+  `/device/events` walk 500s.  Reproduced on BE12599 (2026-09-19), which had
+  78 KB stored and had rolled into page `0x12`.
+  **Why it hid so long:** every 5A capture the walk was verified against came
+  from a freshly-erased BE11529 — all three confirmed TERM examples in
+  `framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`.  Prod is
+  unaffected: it ingests complete files via BW ACH, never this walk.
+  **Fixing it has two layers** — the arithmetic (`if end < start: end +=
+  0x10000`) stops the crash and bounds the loop correctly; carrying the page
+  byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
+  over) needs a BW capture of a spanning event first.  Do not ship layer one
+  alone without a loud truncation warning — a silently short event is the
+  failure mode this codec has been bitten by repeatedly.
 - **Open, not blocking:** 14 sensitive-range files show an exact 8x
   (= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
   inserts DB rows for store files that have none (one-time per store) and the

README.md

@@ -1,4 +1,4 @@
-# seismo-relay  `v0.30.0`
+# seismo-relay  `v0.31.0`
 
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing seismographs.  Supports both the **MiniMate Plus
@@ -496,6 +496,11 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
 
 ## Roadmap (Future)
 
+> **Where it stands *today*** — an honest per-capability maturity assessment,
+> what to rely on, known issues, and the gap to a real tool:
+> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md).  This section covers
+> where it is *going*.
+
 ### Strategic direction — where this is going
 
 seismo-relay is being built as a **suite of cooperating components**

docs/sfm_tool_status.md

@@ -0,0 +1,150 @@
+# SFM — where it actually stands as a tool
+
+**Status as of 2026-09-20 (v0.31.0).**  This is the honest assessment, not the
+roadmap — `README.md § Roadmap` covers where it is *going*.  Expect this file to
+go stale; re-date it when you revise it.
+
+---
+
+## The framing
+
+SFM is **three different things wearing one name**, at three very different
+levels of maturity:
+
+| | what it is | maturity |
+|---|---|---|
+| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
+| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
+| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
+| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
+
+Brian's own description — *"right now it's an emergency tool and a research
+project"* — is accurate, and it applies specifically to the **device side**.
+The data side is not an emergency tool; it has been carrying production for
+months.
+
+Most confusion about "is SFM reliable?" comes from answering for the wrong
+tier.
+
+---
+
+## 1. What you can rely on
+
+### Production-grade — trust it
+
+- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
+  preserved Blastware ASCII exports, 45 units, files back to 2018.
+- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
+  Thor's own CSV exports; production IDFW 575/575 with zero truncations.
+- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
+  842,442 per-interval frequency comparisons with zero mismatches.
+- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
+  fed by the watchers — this is how prod actually gets its data, and it has
+  been running unattended for months.
+- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
+  listing, event detail and report.
+- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
+  binaries, with operator review state preserved across regeneration.
+- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
+  Proven in the field, including as a rescue tool (see the runbook).
+
+### Emergency-grade — works, but you are the error handling
+
+- **`/device/*` live endpoints.**  They do what they say.  But they are
+  synchronous, unauthenticated, and a single cellular download can exceed the
+  60 s timeouts that sit in front of them.
+- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
+  Each has worked in a real incident — but each has been used a handful of
+  times, by one person, with the runbook open.
+- **The standalone webapp.**  Perfectly usable, and as of v0.31.0 the cheap
+  probes and rescue actions are reachable without curl.  No auth of any kind.
+
+### Research artifacts — useful, not products
+
+- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
+  Inspector).  Desktop-only, single-user, no tests.
+- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
+  `verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
+  These produced the numbers the production claims rest on, so they matter —
+  but they are analysis scripts, not maintained code.
+- **`docs/offset_investigation.md`** — an open investigation, not a feature.
+
+---
+
+## 2. What to use when
+
+| you want to… | use | notes |
+|---|---|---|
+| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
+| Know whether ACH is on | `GET /device/call_home` | ~2 s.  **Not** `/device/events`. |
+| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
+| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
+| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
+| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
+| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
+
+The single most useful habit: **the cheap probes are cheap and the event walk
+is not.**  Reaching for `/device/events` to answer a yes/no question about a
+unit is the mistake that motivated the v0.31.0 webapp changes.
+
+---
+
+## 3. Known issues
+
+| issue | impact | status |
+|---|---|---|
+| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`.  Needs a BW capture of a spanning event to fix properly. |
+| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
+| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial.  `/docs` itself is unguarded. |
+| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug.  Surfaced in the UI as "unreliable". |
+| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint.  Must be POST-starts-job / GET-polls before any remote lab. |
+| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known.  Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
+| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
+| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
+
+---
+
+## 4. What stands between this and a real tool
+
+Roughly in dependency order — each unblocks the ones below it.
+
+**1. Authentication.**  Everything else is gated on this.  SFM has none, and
+the modem IP whitelist gives zero protection because SFM *is* the whitelisted
+origin.  The agreed design delegates rather than builds: Terra-View becomes the
+authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
+auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
+
+**2. Async long operations.**  POST starts a job, GET polls.  Retrofitting this
+after building a remote lab on top of synchronous endpoints would be far worse
+than designing for it now.
+
+**3. Confirm-guards on the remaining destructive endpoints.**  Auth answers
+*who*, not *did you mean it*.  The webapp's erase is guarded; the other seven
+destructive POSTs and `/docs` are not.
+
+**4. The 5A page-boundary fix.**  Until this lands, live event download is
+unreliable on exactly the units most likely to need attention — the ones that
+have been recording heavily.  Wants a Blastware capture of an event spanning a
+page boundary before the chunk-addressing half is trustworthy.
+
+**5. A live Thor / Micromate client.**  The device side is MiniMate-only.
+Series-4 units can only be read from forwarded files, so half the fleet has no
+live path at all.
+
+**6. Test coverage that runs from a clean checkout.**  15 of 16 current
+failures are missing fixture bundles.  A test suite that cannot go green on a
+fresh clone cannot gate anything.
+
+**7. The SDM rename.**  Cosmetic relative to the above, but the longer `sfm/`
+holds the data-side code the more the tiers blur.  ~30–50 files here, ~10–15 in
+Terra-View, plus a Docker volume migration.  Do it when the codebase is quiet.
+
+---
+
+## The short version
+
+The **data side is a real tool already**.  The **device side is a set of sharp
+instruments** that work in the hands of the person who wrote them, with the
+runbook open.  The gap between those two states is mostly **auth, async, and
+guardrails** — not protocol work.  The protocol is the part that is actually
+finished.

micromate/sensor_check.py

@@ -0,0 +1,89 @@
+r"""Decode the Thor / Micromate (series-4) sensor self-check waveforms from an
+IDFW event binary.
+
+Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events.  The IDFW
+binary carries the sensor self-check in its fixed-header region (before the
+waveform body), as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
+channel ids as the series-3 MiniMate Plus (Tran / Vert / Long / MicL), which is
+the physical self-test:
+
+  * 3c / 3d / 3e = Tran / Vert / Long geophone ring-downs (a damped impulse
+    response — resonant frequency + damping).
+  * 3f          = MicL pulse train (the mic's known-signal gain check).  Absent
+    on three-channel (mic-disabled) units.
+
+Record framing (per record)::
+
+    01 0e [id:1] [flags:3] [count:2 BE] [pad:10] [int16-BE samples × count]
+    \___ 18-byte header ___/
+
+Unlike series-3's delta-coded trailing block, series-4 stores each trace as a
+raw int16 big-endian array.  ``count`` (the 2-byte field at header offset +8)
+is the sample count; the record is padded to a fixed stride after that.
+"""
+from __future__ import annotations
+
+import struct
+from typing import Dict, List
+
+# Record id → channel.  Same ids/order as series-3 (minimateplus.sensor_check).
+_ID_TO_CHANNEL = {0x3C: "Tran", 0x3D: "Vert", 0x3E: "Long", 0x3F: "MicL"}
+_CHAIN_IDS = (0x3C, 0x3D, 0x3E, 0x3F)
+
+_MARKER = b"\x01\x0e"       # precedes the 1-byte channel id
+_HEADER_LEN = 18            # bytes from the marker start to the first sample
+_COUNT_OFF = 8              # 2-byte BE sample count, from the marker start
+_MAX_COUNT = 4000           # sanity cap (traces are ~70-200 samples)
+
+
+def _find_chain(raw: bytes):
+    """Locate the sensor-check record chain.  Returns a list of
+    ``(offset, id, count)`` for the first run of markers whose ids run
+    3c, 3d, 3e[, 3f] in order, or ``[]``.
+
+    Records are padded to a fixed stride, so the next marker is not at
+    ``header + count*2``; instead collect every ``01 0e [id]`` marker with a
+    sane count and take the first id-ordered run.  Validating the id sequence
+    (not a lone ``01 0e 3c``) keeps a stray marker in the waveform body from
+    matching — the real chain sits in the fixed header, ahead of the body.
+    """
+    n = len(raw)
+    markers = []
+    for p in range(n - _HEADER_LEN):
+        if raw[p:p + 2] == _MARKER and raw[p + 2] in _ID_TO_CHANNEL:
+            count = int.from_bytes(raw[p + _COUNT_OFF:p + _COUNT_OFF + 2], "big")
+            if 0 < count <= _MAX_COUNT:
+                markers.append((p, raw[p + 2], count))
+
+    for i, (off, rid, _c) in enumerate(markers):
+        if rid != 0x3C:
+            continue
+        run = [markers[i]]
+        for m in markers[i + 1:]:
+            if len(run) < len(_CHAIN_IDS) and m[1] == _CHAIN_IDS[len(run)]:
+                run.append(m)
+            else:
+                break
+        if len(run) >= 3:       # 3-channel (mic-disabled) units are valid
+            return run
+    return []
+
+
+def decode_idf_sensor_check(raw: bytes) -> Dict[str, List[int]]:
+    """Decode the sensor self-check traces from a Thor/Micromate IDFW binary.
+
+    Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}`` in
+    raw int16 ADC counts (MicL omitted on 3-channel units), or ``{}`` if the
+    binary carries no sensor-check chain (a non-IDF file, or an IDFH histogram).
+    """
+    chain = _find_chain(raw)
+    if not chain:
+        return {}
+    out: Dict[str, List[int]] = {}
+    for off, rid, count in chain:
+        start = off + _HEADER_LEN
+        blob = raw[start:start + count * 2]
+        if len(blob) < count * 2:
+            continue
+        out[_ID_TO_CHANNEL[rid]] = list(struct.unpack(">%dh" % count, blob))
+    return out

minimateplus/event_file_io.py

@@ -50,7 +50,7 @@ SIDECAR_KIND   = "sfm.event"
 # bumped without a `pip install` re-run — leading to confusing stale
 # version stamps in sidecars.  Bump this constant and CHANGELOG.md
 # together at release time.
-TOOL_VERSION = "0.30.0"
+TOOL_VERSION = "0.31.0"   # +/sensor_check group (schema v2); gates the backfill regen
 
 try:
     # Best-effort: prefer the installed metadata when it's NEWER than the
@@ -960,6 +960,11 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
         project=project, client=client, operator=user, sensor_location=seisloc,
     )
     ev.raw_samples = samples
+    # Sensor self-check traces from the binary's trailing block (waveform
+    # events only; returns {} for histograms / when absent).  Carried on the
+    # Event so the .h5 writer persists them device-agnostically.
+    from minimateplus.sensor_check import decode_sensor_check
+    ev.sensor_check = decode_sensor_check(raw) or None
     # Only compute peaks from samples when we actually have samples.
     # For events the codec couldn't decode (histogram-mode bodies, until
     # the §7.6.2 histogram codec is wired in), samples is an empty dict

minimateplus/models.py

@@ -544,6 +544,15 @@ class Event:
     pretrig_samples: Optional[int] = None   # from STRT record: pre-trigger sample count
     rectime_seconds: Optional[int] = None   # from STRT record: record duration (seconds)
 
+    # Sensor self-check traces keyed by channel label — the short diagnostic
+    # waveforms the unit records when it pulses each sensor before monitoring
+    # (geophone ring-downs + a mic pulse train).  Decoded from the binary by
+    # the per-series decoder (minimateplus.sensor_check / micromate.sensor_check)
+    # and carried here so the .h5 writer can persist them device-agnostically.
+    # Raw ADC counts; the source series' scale differs but the trace is a
+    # shape diagnostic (rendered fit-to-box).  None when absent.
+    sensor_check: Optional[dict] = None     # {"Tran": [...], ..., "MicL": [...]}
+
     # ── Debug / introspection ─────────────────────────────────────────────────
     # Raw 210-byte waveform record bytes, set when debug mode is active.
     # Exposed by the SFM server via ?debug=true so field layouts can be verified.

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "seismo-relay"
-version = "0.30.0"
+version = "0.31.0"
 description = "Python client and REST server for MiniMate Plus seismographs"
 requires-python = ">=3.10"
 dependencies = [

scripts/backfill_thor_events.py

@@ -305,6 +305,11 @@ def main(argv=None) -> int:
                                 default=0,
                             )
                             ev.total_samples = ev.total_samples or n_samp
+                            # Sensor self-check traces from the IDFW fixed
+                            # header, so regenerated .h5 files gain the v2
+                            # /sensor_check group (mirrors save_imported_idf).
+                            from micromate.sensor_check import decode_idf_sensor_check
+                            ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
 
                         event_hdf5.write_event_hdf5(
                             hdf5_path, ev,

sfm/event_hdf5.py

@@ -12,8 +12,11 @@ Layout written to `<filename>.h5`:
   ├─ samples_int16/ (optional)
   │   ├─ Tran  (int16, raw ADC counts) shape: (N,)
   │   └─ ... per channel (only when present in the source)
+  ├─ sensor_check/ (optional, schema v2+)
+  │   ├─ Tran  (int32, raw counts)     shape: (M,)   M ≪ N
+  │   └─ ... per channel present in the source (MicL absent on 3-channel units)
   └─ root attrs (event metadata):
-       schema_version       int    = 1
+       schema_version       int    = 2
        kind                 str    = "sfm.event.hdf5"
        serial               str
        waveform_key         str    (8-hex)
@@ -64,7 +67,7 @@ from minimateplus.models import Event
 
 log = logging.getLogger(__name__)
 
-SCHEMA_VERSION = 1
+SCHEMA_VERSION = 2   # v2 adds the optional /sensor_check group
 HDF5_KIND = "sfm.event.hdf5"
 
 # Geophone full-scale velocity per range (in/s).  Confirmed in CLAUDE.md
@@ -270,6 +273,22 @@ def write_event_hdf5(
                     )
             igrp.attrs["mic_psi_per_count"] = float(mic_factor)
 
+        # /sensor_check — optional short diagnostic self-check traces (schema
+        # v2+).  Raw ADC counts (a shape diagnostic; the per-series count scale
+        # differs, and the renderer fits each trace to its box).  Only channels
+        # the decoder found are written — 3-channel units carry no MicL.
+        sc = event.sensor_check or {}
+        if sc:
+            scgrp = f.create_group("sensor_check")
+            for ch in ("Tran", "Vert", "Long", "MicL"):
+                vals = sc.get(ch)
+                if vals:
+                    scgrp.create_dataset(
+                        ch, data=np.asarray(vals, dtype=np.int32),
+                        compression="gzip", compression_opts=4, shuffle=True,
+                    )
+            scgrp.attrs["units"] = "raw_counts"
+
     import os
     os.replace(tmp, path)
 
@@ -334,6 +353,16 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
             if mic_attr is not None:
                 mic_psi = float(mic_attr)
 
+        # /sensor_check — optional (schema v2+); absent on older files.
+        sensor_check = None
+        scgrp = f.get("sensor_check")
+        if scgrp is not None:
+            sensor_check = {}
+            for ch in ("Tran", "Vert", "Long", "MicL"):
+                ds = scgrp.get(ch)
+                if ds is not None:
+                    sensor_check[ch] = np.asarray(ds[()])
+
     return {
         "schema_version":     sv,
         "kind":               attrs.get("kind"),
@@ -341,6 +370,7 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
         "samples":            samples,
         "samples_int16":      samples_int16,
         "mic_psi_per_count":  mic_psi,
+        "sensor_check":       sensor_check,
     }
 
 
@@ -431,11 +461,16 @@ def plot_json_from_hdf5(
     event_id: Optional[str] = None,
     index: Optional[int] = None,
 ) -> dict:
-    """Build a `sfm.plot.v1` JSON dict from a stored .h5 file."""
+    """Build a `sfm.plot.v1` JSON dict from a stored .h5 file.
+
+    The dict also carries a top-level ``sensor_check`` key (the raw self-check
+    traces as ``{ch: [int]}``, or None) beyond the plot schema, so report
+    generation can read the traces from the same single .h5 load.
+    """
     data = read_event_hdf5(path)
     a = data["attrs"]
     s = data["samples"]
-    return _build_plot_dict(
+    out = _build_plot_dict(
         n_samples=len(s["Tran"]) if "Tran" in s else 0,
         sample_rate=int(a.get("sample_rate", 1024) or 1024),
         pretrig_samples=int(a.get("pretrig_samples", 0) or 0),
@@ -463,6 +498,11 @@ def plot_json_from_hdf5(
         event_id=event_id,
         index=index,
     )
+    scd = data.get("sensor_check")
+    out["sensor_check"] = (
+        {ch: v.tolist() for ch, v in scd.items()} if scd else None
+    )
+    return out
 
 
 def _build_plot_dict(

sfm/report_pdf.py

@@ -121,9 +121,11 @@ class ReportData:
     t0_ms:                  Optional[float] = None
     dt_ms:                  Optional[float] = None
 
-    # Sensor self-check traces — {ch: [samples]} in raw decode units, decoded
-    # from the binary's trailing block (see minimateplus.sensor_check). The
-    # little waveforms BW draws in its "Sensor Check" strip. Empty when absent.
+    # Sensor self-check traces — {ch: [samples]} in raw counts, read from the
+    # standardized .h5 (/sensor_check group, schema v2+) where the per-series
+    # decoder stored them at ingest.  The little diagnostic waveforms BW draws
+    # in its "Sensor Check" strip.  Empty when absent (pre-v2 .h5, histogram,
+    # or 3-channel unit's MicL).
     sensor_check_waveforms: dict = field(default_factory=dict)
 
     # Record-type discriminator
@@ -294,22 +296,15 @@ def gather_report_data(
             rd.pretrig_samples  = ta.get("pretrig_samples")
             rd.t0_ms            = ta.get("t0_ms")
             rd.dt_ms            = ta.get("dt_ms")
+            # Sensor self-check traces — read from the standardized .h5 (schema
+            # v2+).  Device-agnostic: whichever decoder produced the event
+            # stored them at ingest, so SFM reads them here without knowing or
+            # caring about the source instrument series.  Empty on pre-v2 files
+            # (until backfilled) and on 3-channel / histogram events.
+            rd.sensor_check_waveforms = wf.get("sensor_check") or {}
         except Exception as exc:
             log.warning("gather_report_data: hdf5 read failed: %s", exc)
 
-    # ── Sensor self-check traces — decoded from the retained raw binary ──
-    # The .h5 holds only the main waveform; the sensor-check traces live in the
-    # binary's trailing block, so decode them straight from the kept BW file.
-    # Waveform events only (histograms have no sensor-check strip).
-    if not rd.is_histogram:
-        try:
-            from minimateplus.sensor_check import decode_sensor_check
-            bw_path, _a5 = store.paths_for(serial, filename)
-            if bw_path.exists():
-                rd.sensor_check_waveforms = decode_sensor_check(bw_path.read_bytes())
-        except Exception as exc:
-            log.warning("gather_report_data: sensor-check decode failed: %s", exc)
-
     # ── Histogram aggregation ──
     # Codec emits ~N per-block samples (typically 1/sec); BW reports
     # one bar per configured interval (1 min / 5 min / etc.).  When

sfm/sfm_webapp.html

@@ -108,6 +108,12 @@
       color: var(--text);
     }
     .btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
+    .btn-danger   { background: var(--red);    color: #fff; }
+    .btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
+    .diag-result  { display:block; margin-top:6px; font-size:12px; opacity:.85;
+                    white-space:pre-wrap; word-break:break-word; }
+    .diag-result.ok    { color: var(--green); }
+    .diag-result.error { color: var(--red); }
     .btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
 
     /* #connect-btn styles moved to #live-connect-bar block */
@@ -910,6 +916,7 @@
   <button class="tab-btn"        data-tab="events"    onclick="switchTab('events')">Events</button>
   <button class="tab-btn"        data-tab="config"    onclick="switchTab('config')">Config</button>
   <button class="tab-btn"        data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
+  <button class="tab-btn"        data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
 </div>
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -938,6 +945,10 @@
 <div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
 
   <div class="event-toolbar">
+    <button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
+            title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
+      ⟳ Load events
+    </button>
     <button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
     <button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
             title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
@@ -1205,6 +1216,77 @@
 
 </div><!-- end #tab-call-home -->
 
+<!-- ════════════════════════════════════════════════════════════════
+     TAB: Diagnostics
+═══════════════════════════════════════════════════════════════════ -->
+<div id="tab-diagnostics" class="tab-pane">
+
+  <div class="cfg-grid">
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Device State</div>
+      <div class="hint" style="margin-bottom:10px">
+        Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
+      </div>
+      <div class="dev-table" id="diag-table"></div>
+      <div class="cfg-actions" style="margin-top:12px">
+        <button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
+        <span id="diag-status"></span>
+      </div>
+    </div>
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Actions</div>
+
+      <div class="cfg-field">
+        <label>Stop Monitoring</label>
+        <button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
+        <div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
+        <span class="diag-result" id="diag-stop-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Disable Auto Call Home</label>
+        <button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
+        <div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
+        <span class="diag-result" id="diag-ach-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Erase All Events</label>
+        <input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
+               oninput="diagCheckEraseConfirm()" autocomplete="off" />
+        <button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
+        <div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
+        <span class="diag-result" id="diag-erase-result"></span>
+      </div>
+    </div>
+
+    <div class="cfg-section">
+      <div class="cfg-section-title">Unresponsive Unit</div>
+      <div class="hint" style="margin-bottom:10px">
+        The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
+        to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
+        <code>ach_server</code> and answer its call — before racing it with these.
+      </div>
+
+      <div class="cfg-field">
+        <label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
+        <button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
+        <div class="hint">Success is <code>bytes_received &gt; 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
+        <span class="diag-result" id="diag-drip-result"></span>
+      </div>
+
+      <div class="cfg-field">
+        <label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
+        <button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
+        <span class="diag-result" id="diag-blind-result"></span>
+      </div>
+    </div>
+
+  </div>
+</div><!-- end #tab-diagnostics -->
+
 </div><!-- end #section-live -->
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -1361,6 +1443,8 @@
 // ── State ──────────────────────────────────────────────────────────────────────
 let unitInfo      = null;
 let eventList     = [];
+let storageInfo   = null;   // /device/events/storage_range — cheap, read on connect
+let eventsLoaded  = false;  // the event chain walk is opt-in; see loadEventList()
 let currentEvent  = 0;
 let charts        = {};
 let geoAdcScale      = 6.206;
@@ -1458,6 +1542,7 @@ function switchTab(name) {
   if (name === 'units')    { if (!unitsLoaded)  loadUnits(); }
   if (name === 'monlog')   { if (!monlogLoaded) loadMonitorLog(); }
   if (name === 'sessions') { if (!sessLoaded)   loadSessions(); }
+  if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
 }
 
 // ── Connect ────────────────────────────────────────────────────────────────────
@@ -1478,18 +1563,13 @@ async function connectUnit() {
     btn.disabled = false; btn.textContent = 'Connect'; return;
   }
 
-  setStatus('Fetching event list…', 'loading');
-  try {
-    const r = await fetch(`${api()}/device/events?${deviceParams()}`);
-    if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
-    const evData = await r.json();
-    eventList = evData.events || [];
-    // Merge compliance from /device/events response (it re-reads it)
-    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
-  } catch (e) {
-    setStatus(`Event fetch failed: ${e.message}`, 'error');
-    btn.disabled = false; btn.textContent = 'Reconnect'; return;
-  }
+  // Connecting deliberately does NOT walk the event chain.  That walk reads
+  // every event header over the cellular link and can take minutes — or fail
+  // outright on a unit whose buffer has wrapped past 0xFFFF.  Use the ~2 s
+  // probes instead; the event list is opt-in via loadEventList().
+  eventList = []; eventsLoaded = false;
+  setStatus('Reading device state…', 'loading');
+  storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
 
   populateDeviceBar();
   populateDeviceTab();
@@ -1498,11 +1578,9 @@ async function connectUnit() {
 
   document.getElementById('device-bar').style.display    = 'flex';
   document.getElementById('monitor-panel').style.display = 'flex';
-  document.getElementById('load-btn').disabled     = eventList.length === 0;
-  document.getElementById('save-btn').disabled     = eventList.length === 0;
-  document.getElementById('download-btn').disabled = eventList.length === 0;
-  document.getElementById('prev-btn').disabled     = true;
-  document.getElementById('next-btn').disabled     = eventList.length <= 1;
+  setEventButtonsEnabled();
+  document.getElementById('load-events-btn').disabled = false;
+  setDiagButtonsEnabled(true);
   document.getElementById('cfg-read-btn').disabled  = false;
   document.getElementById('cfg-write-btn').disabled = false;
   document.getElementById('ch-read-btn').disabled   = false;
@@ -1510,7 +1588,9 @@ async function connectUnit() {
 
   btn.disabled = false; btn.textContent = 'Reconnect';
 
-  setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
+  setStatus(storageInfo && storageInfo.is_empty
+    ? 'Connected — no events stored.'
+    : 'Connected. Event list not loaded (Events → Load events).', 'ok');
 
   // Fetch monitor status in background (non-blocking)
   refreshMonitorStatus().catch(() => {});
@@ -1522,6 +1602,48 @@ async function connectUnit() {
   }
 }
 
+// ── Shared fetch helper ────────────────────────────────────────────────────────
+async function fetchJson(path, opts) {
+  const sep = path.includes('?') ? '&' : '?';
+  const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
+  const body = await r.json().catch(() => ({}));
+  if (!r.ok) throw new Error(body.detail || r.statusText);
+  return body;
+}
+
+function setEventButtonsEnabled() {
+  const n = eventList.length;
+  document.getElementById('load-btn').disabled     = n === 0;
+  document.getElementById('save-btn').disabled     = n === 0;
+  document.getElementById('download-btn').disabled = n === 0;
+  document.getElementById('prev-btn').disabled     = true;
+  document.getElementById('next-btn').disabled     = n <= 1;
+}
+
+// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
+async function loadEventList() {
+  if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
+  const btn = document.getElementById('load-events-btn');
+  btn.disabled = true;
+  setStatus('Walking the event chain — this can take a while…', 'loading');
+  try {
+    const evData = await fetchJson('/device/events');
+    eventList = evData.events || [];
+    eventsLoaded = true;
+    // /device/events re-reads compliance; fold it in.
+    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
+  } catch (e) {
+    setStatus(`Event fetch failed: ${e.message}`, 'error');
+    btn.disabled = false; return;
+  }
+  populateDeviceBar();
+  populateDeviceTab();
+  populateEventChips();
+  setEventButtonsEnabled();
+  btn.disabled = false;
+  setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
+}
+
 // ── Device bar ─────────────────────────────────────────────────────────────────
 function populateDeviceBar() {
   qs('di-serial').textContent   = unitInfo.serial || '—';
@@ -1530,7 +1652,7 @@ function populateDeviceBar() {
   qs('di-sr').textContent       = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
   qs('di-rt').textContent       = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
   qs('di-trig').textContent     = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
-  qs('di-count').textContent    = eventList.length;
+  qs('di-count').textContent    = eventsLoaded ? eventList.length : '—';
   qs('di-project').textContent  = cc.project || '—';
   qs('di-client').textContent   = cc.client  || '—';
   qs('di-operator').textContent = cc.operator || '—';
@@ -1660,7 +1782,8 @@ function populateDeviceTab() {
     { label:'DSP',            value: unitInfo.dsp_version || '—' },
     { label:'Model',          value: unitInfo.model || '—' },
     { label:'Manufacturer',   value: unitInfo.manufacturer || '—' },
-    { label:'Stored Events',  value: eventList.length },
+    { label:'Stored Events',  value: eventsLoaded ? eventList.length : 'not loaded' },
+    { label:'Storage Used',   value: storageUsedLabel() },
   ];
   for (const {label, value} of cardData) {
     const c = document.createElement('div');
@@ -1707,6 +1830,158 @@ function renderTable(id, rows) {
   }
 }
 
+// ── Diagnostics ────────────────────────────────────────────────────────────────
+// Everything here is a cheap probe (POLL + one read) or a single write.  None of
+// it walks the event chain.  See docs/runbooks/wedged_unit_recovery.md.
+
+function storageUsedLabel() {
+  if (!storageInfo) return '—';
+  if (storageInfo.is_empty) return 'empty';
+  const f = storageInfo.first_key, l = storageInfo.last_key;
+  return (f && l) ? `${f} → ${l}` : '—';
+}
+
+function setDiagButtonsEnabled(on) {
+  for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
+                    'diag-drip-btn','diag-blind-btn']) {
+    const el = document.getElementById(id);
+    if (el) el.disabled = !on;
+  }
+  diagCheckEraseConfirm();
+}
+
+// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
+function diagCheckEraseConfirm() {
+  const box = document.getElementById('diag-erase-confirm');
+  const btn = document.getElementById('diag-erase-btn');
+  if (!box || !btn) return;
+  const serial = (unitInfo && unitInfo.serial) || '';
+  btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
+}
+
+function diagResult(id, text, cls) {
+  const el = document.getElementById(id);
+  if (!el) return;
+  el.textContent = text;
+  el.className = 'diag-result' + (cls ? ' ' + cls : '');
+}
+
+async function refreshDiagnostics() {
+  if (!devHost()) return;
+  const st = document.getElementById('diag-status');
+  if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
+
+  const [mon, store, idx] = await Promise.all([
+    fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
+    fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
+    fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
+  ]);
+
+  if (!store._err) storageInfo = store;
+
+  const err = v => `<span style="color:var(--red)">${v}</span>`;
+  const rows = [];
+
+  rows.push(['Monitoring', mon._err ? err(mon._err)
+    : (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
+  if (!mon._err) {
+    rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
+    if (mon.memory_total_bytes) {
+      const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
+      const pct  = (used / mon.memory_total_bytes * 100).toFixed(1);
+      rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
+    }
+  }
+
+  rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
+  if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
+
+  // SUB 0x08.  Known to report 0 on units with years of history — suspected
+  // field-offset bug in the decode, so show it but do not trust it.
+  rows.push(['Lifetime events', idx._err ? err(idx._err)
+    : `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
+
+  renderTable('diag-table', rows);
+  populateDeviceTab();
+  if (st) { st.textContent = ''; st.className = ''; }
+}
+
+async function diagStopMonitoring() {
+  const btn = document.getElementById('diag-stop-btn');
+  btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
+  try {
+    await fetchJson('/device/monitor/stop', { method: 'POST' });
+    diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
+    refreshDiagnostics();
+  } catch (e) {
+    diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagDisableAch() {
+  const btn = document.getElementById('diag-ach-btn');
+  btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
+  try {
+    const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
+    const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
+    diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
+  } catch (e) {
+    diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagEraseEvents() {
+  const serial = (unitInfo && unitInfo.serial) || 'this unit';
+  if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
+  const btn = document.getElementById('diag-erase-btn');
+  btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
+  try {
+    await fetchJson('/device/events/erase', { method: 'POST' });
+    diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
+    document.getElementById('diag-erase-confirm').value = '';
+    eventList = []; eventsLoaded = false;
+    setEventButtonsEnabled(); populateEventChips();
+    refreshDiagnostics();
+  } catch (e) {
+    diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
+  }
+  diagCheckEraseConfirm();
+}
+
+async function diagSlowDrip() {
+  const btn = document.getElementById('diag-drip-btn');
+  btn.disabled = true;
+  diagResult('diag-drip-result', 'Holding a session for 120 s…');
+  try {
+    const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
+                              { method: 'POST' });
+    const good = (r.bytes_received || 0) > 0;
+    diagResult('diag-drip-result',
+      `drips ${r.drips_sent}  ·  held ${r.duration_s}s  ·  bytes back ${r.bytes_received}` +
+      (r.send_error ? `  ·  ${r.send_error}` : '') +
+      (good ? '  → device responded' : '  → no response; the modem may not be bridging'),
+      good ? 'ok' : 'error');
+  } catch (e) {
+    diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
+async function diagBlindStop() {
+  const btn = document.getElementById('diag-blind-btn');
+  btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
+  try {
+    const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
+    diagResult('diag-blind-result',
+      `Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
+  } catch (e) {
+    diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
+  }
+  btn.disabled = false;
+}
+
 // ── Config form ────────────────────────────────────────────────────────────────
 function populateConfigFromDeviceInfo() {
   if (!unitInfo) return;

sfm/waveform_store.py

@@ -662,6 +662,11 @@ class WaveformStore:
             ev.raw_samples = idf_samples
             n_samples = max((len(idf_samples.get(ch, [])) for ch in ("Tran", "Vert", "Long", "MicL")), default=0)
             ev.total_samples = ev.total_samples or n_samples
+            # Sensor self-check traces from the IDFW fixed header (waveform
+            # events only; {} on histograms / when absent).  Carried on the
+            # bridged Event so the .h5 writer persists them like series-3.
+            from micromate.sensor_check import decode_idf_sensor_check
+            ev.sensor_check = decode_idf_sensor_check(idf_bytes) or None
 
         # For IDFH histograms there are no per-sample waveform arrays — the
         # device stores one peak ADC count per interval per channel.  Synthesise

tests/test_event_hdf5_sensor_check.py

@@ -0,0 +1,71 @@
+"""The event .h5 carries the sensor self-check traces (schema v2).
+
+The sensor check is decoded by the per-series decoder and attached to the
+standardized Event, so the .h5 writer persists it device-agnostically and SFM
+reads it back without knowing which instrument produced it.  Old v1 files (no
+sensor_check group) must still read cleanly.
+"""
+import tempfile
+from pathlib import Path
+
+import numpy as np
+
+from minimateplus.models import Event
+from minimateplus.event_file_io import read_blastware_file
+from sfm import event_hdf5
+
+S3_FIX = Path(__file__).parent / "fixtures" / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"
+
+
+def _write(ev, **kw):
+    d = Path(tempfile.mkdtemp())
+    p = d / "e.h5"
+    event_hdf5.write_event_hdf5(p, ev, serial="BE12844", **kw)
+    return p
+
+
+def test_sensor_check_roundtrips_through_hdf5():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, -3], "Vert": [0, 1], "Long": [2], "MicL": [5, -5]}
+    ev.sample_rate = 1024
+    sc = {"Tran": [0, -990, -500, -100], "Vert": [0, -980, -480],
+          "Long": [0, -986, -470], "MicL": [0, -1800, 1800, -1800]}
+    ev.sensor_check = sc
+
+    r = event_hdf5.read_event_hdf5(_write(ev))
+    assert r["schema_version"] == 2
+    assert set(r["sensor_check"]) == {"Tran", "Vert", "Long", "MicL"}
+    for ch, vals in sc.items():
+        assert r["sensor_check"][ch].tolist() == vals
+
+
+def test_plot_json_carries_sensor_check():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, 3]}
+    ev.sample_rate = 1024
+    ev.sensor_check = {"Tran": [0, -990, -500], "Vert": [0, -980],
+                       "Long": [0, -986]}   # 3-channel: no MicL
+    pj = event_hdf5.plot_json_from_hdf5(_write(ev))
+    assert pj["sensor_check"] is not None
+    assert "MicL" not in pj["sensor_check"]
+    assert pj["sensor_check"]["Tran"] == [0, -990, -500]
+
+
+def test_event_without_sensor_check_still_reads_as_v2():
+    ev = Event(index=0)
+    ev.raw_samples = {"Tran": [1, 2, 3]}
+    ev.sample_rate = 1024
+    r = event_hdf5.read_event_hdf5(_write(ev))
+    assert r["schema_version"] == 2
+    assert r["sensor_check"] is None
+    assert event_hdf5.plot_json_from_hdf5(_write(ev))["sensor_check"] is None
+
+
+def test_series3_decode_populates_event_sensor_check():
+    # The real series-3 decoder attaches the traces to the Event, so the
+    # ingest/backfill .h5 write picks them up with no extra plumbing.
+    ev = read_blastware_file(S3_FIX)
+    assert ev.sensor_check is not None
+    assert set(ev.sensor_check) == {"Tran", "Vert", "Long", "MicL"}
+    tran = np.asarray(ev.sensor_check["Tran"], dtype=float)
+    assert tran.min() < -800   # the geophone ring-down deflection

tests/test_sensor_check_idf.py

@@ -0,0 +1,67 @@
+"""Series-4 (Thor / Micromate IDFW) sensor self-check waveform decode.
+
+Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events.  The IDFW
+binary carries the sensor self-check in its fixed-header region (before the
+waveform body) as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
+channel ids as series-3 (Tran/Vert/Long/MicL).  Unlike series-3's delta-coded
+trailing block, series-4 stores each trace as a raw int16-BE array after an
+18-byte record header whose sample count is a 2-byte field at offset +8.
+
+Three-channel (mic-disabled) Thor units carry only 3c/3d/3e — no MicL record.
+
+Validated by shape (geophone ring-down / mic pulse train) and cross-event
+consistency, since there's no Thor Event-Report strip to exact-match against.
+"""
+from pathlib import Path
+
+import numpy as np
+
+from micromate.sensor_check import decode_idf_sensor_check
+
+FIXDIR = Path(__file__).parent / "fixtures" / "thor-idf-sc"
+EVENTS = sorted(p.name for p in FIXDIR.glob("*.IDFW"))
+
+
+def _decode(name):
+    return decode_idf_sensor_check((FIXDIR / name).read_bytes())
+
+
+def test_geo_channels_present_and_ringdown_shaped():
+    # Every IDFW event has the three geophone self-checks; each is a large
+    # one-sided deflection (~15000 raw counts) that rings back — the geophone's
+    # damped impulse response.
+    for name in EVENTS:
+        sc = _decode(name)
+        for ch in ("Tran", "Vert", "Long"):
+            assert ch in sc, f"{name} missing {ch}"
+            tr = np.asarray(sc[ch], dtype=float)
+            tr = tr - tr[:4].mean()          # reference to the pre-trigger baseline
+            assert 40 <= len(tr) <= 300, f"{name}:{ch} n={len(tr)}"
+            assert tr.min() < -8000, f"{name}:{ch} min {tr.min()}"
+            # deflects one way and rings back toward / past the baseline
+            assert tr.max() < abs(tr.min()), f"{name}:{ch} not one-sided"
+
+
+def test_mic_present_only_on_four_channel_units():
+    # UM11719 / UM12947 record a mic; UM13981 / UM20147 are 3-channel
+    # (mic-disabled) units and carry no MicL self-check.
+    got = {name: ("MicL" in _decode(name)) for name in EVENTS}
+    assert any(got.values()), "expected at least one 4-channel unit"
+    assert not all(got.values()), "expected at least one 3-channel unit"
+    for name, has_mic in got.items():
+        if has_mic:
+            tr = np.asarray(_decode(name)["MicL"], dtype=float)
+            tr = tr - tr[:4].mean()
+            # mic self-check is a bipolar pulse train — swings both ways, wide range
+            assert tr.max() > 5000 and tr.min() < -5000, f"{name} mic not bipolar"
+
+
+def test_channel_ids_and_order():
+    # ids decode to the canonical channel names, geo always in Tran/Vert/Long order
+    sc = _decode(EVENTS[0])
+    assert [c for c in ("Tran", "Vert", "Long") if c in sc] == ["Tran", "Vert", "Long"]
+
+
+def test_returns_empty_on_non_idf_input():
+    assert decode_idf_sensor_check(b"not an IDF file") == {}
+    assert decode_idf_sensor_check(b"") == {}