Merge pull request 'Release v0.31.0 — report parity + the inverted rescue (0.29.0 → 0.31.0)' (#40) from dev into main

Reviewed-on: #40

CHANGELOG.md

@@ -4,68 +4,6 @@ All notable changes to seismo-relay are documented here.
 
 ---
 
-## Unreleased
-
-### Fixed
-
-- **Waveform event times were the monitoring-session start, not the trigger
-  (~hours off).**  `read_blastware_file` stamped events with footer `ts1`, which
-  for a waveform is the session start a unit shares across every event that day
-  (a unit arming at 06:00 stamped 06:00 on all of them — the modal and PDF both
-  showed it, since it's the stored value).  The event time is footer `ts2` (the
-  recording stop), and Blastware's trigger = `ts2 - record time`.  The record
-  time is a big-endian float32 in the recording-setup config block (30 bytes
-  before the `Standard Recording Setup` marker), so the **exact trigger is now
-  recovered from the binary alone** — all 7 BE12844 oracle events decode to
-  their exact Blastware time (e.g. N844LQHB 10:33:29), no paired `.TXT` needed.
-  Histograms keep `ts1` (the ~24 h window start).  A paired report's
-  `event_datetime` stays authoritative (unit-clock drift).
-  ⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
-
-### Added
-
-- **Diagnostics tab in the SFM standalone webapp.**  Surfaces the device
-  endpoints that previously existed only as `curl`: `events/storage_range` and
-  `events/index` alongside `monitor/status`, then stop monitoring, disable ACH
-  (`rescue?erase=false`, so stored events survive), and erase.  The wedged-unit
-  ladder — slow drip and blind stop — sits under its own heading pointing at
-  `docs/runbooks/wedged_unit_recovery.md`, with the reminder that `slow_drip`'s
-  success signal is `bytes_received > 0` and not a clean duration.  Erase is
-  guarded by typing the unit's serial: auth answers *who*, not *did you mean
-  it*, and Swagger's try-it-out button on `/device/events/erase` is live on
-  `:8200/docs`.
-
-- **`docs/sfm_tool_status.md`** — an honest per-capability maturity assessment:
-  what is production-grade (the codec library, the data side), what is
-  emergency-grade (the device side), what is a research artifact, the
-  known-issues table, and the gap to a real tool.  Also records the **5A
-  page-boundary bug** as known: `parse_strt_end_offset()` discards the key's
-  page byte, so once a unit has recorded more than 64 KB since its last erase,
-  an event spanning the boundary reads an `end_offset` *behind* its own start —
-  the chunk loop fetches nothing and TERM packs a negative `offset_word`, which
-  500s.  Reproduced on BE12599.  Production is unaffected: it ingests complete
-  files via the watcher path and never runs this walk.
-
-### Changed
-
-- **Connecting to a unit no longer walks its event chain.**  `/device/events`
-  reads every event header over the cellular link; on a unit with a large or
-  wrapped chain that takes minutes or fails outright, and it fired
-  automatically on every connect.  Connect now uses only ~2 s probes —
-  `/device/info` (which already carried the compliance config the walk was
-  re-reading) plus `events/storage_range` — and the Device tab gains an Event
-  Chain card.  The walk moved behind a **Load events** button in the Events
-  toolbar.  Knowing whether a unit's ACH is on no longer requires reading every
-  event it has stored.
-
-### Migration
-
-**None.**  Frontend and documentation only — no codec, waveform-store or DB
-change, no schema change, and no `TOOL_VERSION` bump.  The webapp is served
-from the image, so the change appears after the next `sfm` rebuild.
-
----
-
 ## v0.31.0 — 2026-09-18
 
 **Report parity, and a second way to rescue a runaway unit.**  Two threads.

CLAUDE.md

@@ -61,24 +61,6 @@ Read this first when picking the project back up.
   4th-decimal tick and are Thor's own rounding — no single linear LSB can
   reproduce every printed value (the constraints are infeasible by 7e-5
   relative), so do NOT retune `_GEO_LSB_IPS`.
-- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
-  `parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
-  discarding the key's page byte.  An event starting at `0x0111F2A2` and ending
-  at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
-  start.  The chunk loop then exits before fetching anything and TERM computes
-  a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
-  `/device/events` walk 500s.  Reproduced on BE12599 (2026-09-19), which had
-  78 KB stored and had rolled into page `0x12`.
-  **Why it hid so long:** every 5A capture the walk was verified against came
-  from a freshly-erased BE11529 — all three confirmed TERM examples in
-  `framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`.  Prod is
-  unaffected: it ingests complete files via BW ACH, never this walk.
-  **Fixing it has two layers** — the arithmetic (`if end < start: end +=
-  0x10000`) stops the crash and bounds the loop correctly; carrying the page
-  byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
-  over) needs a BW capture of a spanning event first.  Do not ship layer one
-  alone without a loud truncation warning — a silently short event is the
-  failure mode this codec has been bitten by repeatedly.
 - **Open, not blocking:** 14 sensitive-range files show an exact 8x
   (= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
   inserts DB rows for store files that have none (one-time per store) and the

README.md

@@ -496,11 +496,6 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
 
 ## Roadmap (Future)
 
-> **Where it stands *today*** — an honest per-capability maturity assessment,
-> what to rely on, known issues, and the gap to a real tool:
-> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md).  This section covers
-> where it is *going*.
-
 ### Strategic direction — where this is going
 
 seismo-relay is being built as a **suite of cooperating components**

docs/sfm_tool_status.md

@@ -1,150 +0,0 @@
-# SFM — where it actually stands as a tool
-
-**Status as of 2026-09-20 (v0.31.0).**  This is the honest assessment, not the
-roadmap — `README.md § Roadmap` covers where it is *going*.  Expect this file to
-go stale; re-date it when you revise it.
-
----
-
-## The framing
-
-SFM is **three different things wearing one name**, at three very different
-levels of maturity:
-
-| | what it is | maturity |
-|---|---|---|
-| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
-| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
-| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
-| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
-
-Brian's own description — *"right now it's an emergency tool and a research
-project"* — is accurate, and it applies specifically to the **device side**.
-The data side is not an emergency tool; it has been carrying production for
-months.
-
-Most confusion about "is SFM reliable?" comes from answering for the wrong
-tier.
-
----
-
-## 1. What you can rely on
-
-### Production-grade — trust it
-
-- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
-  preserved Blastware ASCII exports, 45 units, files back to 2018.
-- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
-  Thor's own CSV exports; production IDFW 575/575 with zero truncations.
-- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
-  842,442 per-interval frequency comparisons with zero mismatches.
-- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
-  fed by the watchers — this is how prod actually gets its data, and it has
-  been running unattended for months.
-- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
-  listing, event detail and report.
-- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
-  binaries, with operator review state preserved across regeneration.
-- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
-  Proven in the field, including as a rescue tool (see the runbook).
-
-### Emergency-grade — works, but you are the error handling
-
-- **`/device/*` live endpoints.**  They do what they say.  But they are
-  synchronous, unauthenticated, and a single cellular download can exceed the
-  60 s timeouts that sit in front of them.
-- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
-  Each has worked in a real incident — but each has been used a handful of
-  times, by one person, with the runbook open.
-- **The standalone webapp.**  Perfectly usable, and as of v0.31.0 the cheap
-  probes and rescue actions are reachable without curl.  No auth of any kind.
-
-### Research artifacts — useful, not products
-
-- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
-  Inspector).  Desktop-only, single-user, no tests.
-- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
-  `verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
-  These produced the numbers the production claims rest on, so they matter —
-  but they are analysis scripts, not maintained code.
-- **`docs/offset_investigation.md`** — an open investigation, not a feature.
-
----
-
-## 2. What to use when
-
-| you want to… | use | notes |
-|---|---|---|
-| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
-| Know whether ACH is on | `GET /device/call_home` | ~2 s.  **Not** `/device/events`. |
-| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
-| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
-| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
-| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
-| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
-
-The single most useful habit: **the cheap probes are cheap and the event walk
-is not.**  Reaching for `/device/events` to answer a yes/no question about a
-unit is the mistake that motivated the v0.31.0 webapp changes.
-
----
-
-## 3. Known issues
-
-| issue | impact | status |
-|---|---|---|
-| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`.  Needs a BW capture of a spanning event to fix properly. |
-| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
-| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial.  `/docs` itself is unguarded. |
-| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug.  Surfaced in the UI as "unreliable". |
-| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint.  Must be POST-starts-job / GET-polls before any remote lab. |
-| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known.  Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
-| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
-| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
-
----
-
-## 4. What stands between this and a real tool
-
-Roughly in dependency order — each unblocks the ones below it.
-
-**1. Authentication.**  Everything else is gated on this.  SFM has none, and
-the modem IP whitelist gives zero protection because SFM *is* the whitelisted
-origin.  The agreed design delegates rather than builds: Terra-View becomes the
-authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
-auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
-
-**2. Async long operations.**  POST starts a job, GET polls.  Retrofitting this
-after building a remote lab on top of synchronous endpoints would be far worse
-than designing for it now.
-
-**3. Confirm-guards on the remaining destructive endpoints.**  Auth answers
-*who*, not *did you mean it*.  The webapp's erase is guarded; the other seven
-destructive POSTs and `/docs` are not.
-
-**4. The 5A page-boundary fix.**  Until this lands, live event download is
-unreliable on exactly the units most likely to need attention — the ones that
-have been recording heavily.  Wants a Blastware capture of an event spanning a
-page boundary before the chunk-addressing half is trustworthy.
-
-**5. A live Thor / Micromate client.**  The device side is MiniMate-only.
-Series-4 units can only be read from forwarded files, so half the fleet has no
-live path at all.
-
-**6. Test coverage that runs from a clean checkout.**  15 of 16 current
-failures are missing fixture bundles.  A test suite that cannot go green on a
-fresh clone cannot gate anything.
-
-**7. The SDM rename.**  Cosmetic relative to the above, but the longer `sfm/`
-holds the data-side code the more the tiers blur.  ~30–50 files here, ~10–15 in
-Terra-View, plus a Docker volume migration.  Do it when the codebase is quiet.
-
----
-
-## The short version
-
-The **data side is a real tool already**.  The **device side is a set of sharp
-instruments** that work in the hands of the person who wrote them, with the
-runbook open.  The gap between those two states is mostly **auth, async, and
-guardrails** — not protocol work.  The protocol is the part that is actually
-finished.

minimateplus/event_file_io.py

@@ -296,16 +296,6 @@ def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
         event.sample_rate = report.sample_rate_sps
     if report.record_time_s is not None:
         event.rectime_seconds = report.record_time_s
-    # The report's event_datetime is Blastware's exact trigger time (parsed
-    # from Event Time + Event Date).  Prefer it over the binary footer's stop
-    # time so a report-paired import matches BW to the second.
-    edt = report.event_datetime
-    if edt is not None:
-        event.timestamp = Timestamp(
-            raw=b"", flag=0x10,
-            year=edt.year, unknown_byte=0, month=edt.month, day=edt.day,
-            hour=edt.hour, minute=edt.minute, second=edt.second,
-        )
 
 
 def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
@@ -818,30 +808,6 @@ def derive_record_type_from_filename(filename, default: str = "Waveform") -> str
     return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
 
 
-# Marker for the recording-setup config block, and the offset of the record-time
-# float32 within it.  The configured post-trigger record time (seconds) is a
-# big-endian float32 exactly 30 bytes before the "Standard Recording Setup"
-# label.  Verified across the corpus reading 1.0 / 2.0 / 3.0 s on different
-# setups — and ts2 - record_time reproduces Blastware's trigger to the second
-# (N844LQHB: stop 10:33:32 - 3.0 = 10:33:29).
-_RECSETUP_MARKER = b"Standard Recording Setup"
-_RECTIME_OFFSET_BEFORE_MARKER = 30
-
-
-def _parse_record_time_seconds(raw: bytes) -> Optional[float]:
-    """The configured post-trigger record time in seconds, from the recording-
-    setup config block, or None when absent / implausible."""
-    a = raw.find(_RECSETUP_MARKER)
-    if a < _RECTIME_OFFSET_BEFORE_MARKER:
-        return None
-    off = a - _RECTIME_OFFSET_BEFORE_MARKER
-    try:
-        rt = struct.unpack(">f", raw[off:off + 4])[0]
-    except struct.error:
-        return None
-    return rt if 0.05 <= rt <= 600.0 else None
-
-
 def read_blastware_file(path: Union[str, Path]) -> Event:
     """
     Parse a Blastware waveform file into an Event.
@@ -951,10 +917,6 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
     # rest of the event (timestamp, waveform_key, project strings) is
     # still recoverable and useful.
     decoded = decode_waveform_v2(body)
-    # Discriminator for the timestamp logic below: a waveform (trigger) event
-    # vs a histogram window.  Keyed on the codec, not the filename — the
-    # save_imported_bw path passes a tmp ".bw" name whose extension lies.
-    is_waveform_body = decoded is not None
     if decoded is None:
         decoded = decode_histogram_body(body)
     if decoded is None:
@@ -986,31 +948,7 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
     ev.total_samples   = strt_fields.get("total_samples")
     ev.pretrig_samples = strt_fields.get("pretrig_samples")
 
-    # Event timestamp.  The footer's two timestamps mean different things by
-    # record type:
-    #   * Waveform: ts1 = the monitoring-SESSION start (shared across every
-    #     event that day — a unit arming at 06:00 stamps 06:00 on all of them),
-    #     ts2 = THIS event's recording STOP.  Blastware's Date/Time is the
-    #     TRIGGER = ts2 - record time, and the record time is a float32 in the
-    #     recording-setup config block (see _parse_record_time_seconds), so the
-    #     exact trigger is recoverable from the binary alone.  Falls back to ts2
-    #     (the stop, within the record duration) if the config block is absent.
-    #     (Stamping ts1 showed the session start, hours off.)
-    #   * Histogram / undecodable: ts1 = the window start, which IS the event
-    #     time — keep it.
-    # Discriminate by ``is_waveform_body`` (the codec), not the filename.
-    if is_waveform_body and ts2 is not None:
-        _stop = datetime.datetime(ts2.year, ts2.month, ts2.day,
-                                  ts2.hour, ts2.minute, ts2.second)
-        _rt = _parse_record_time_seconds(raw)
-        _trig = _stop - datetime.timedelta(seconds=_rt) if _rt is not None else _stop
-        ev.timestamp = Timestamp(
-            raw=footer[10:18],
-            flag=0x10,
-            year=_trig.year, unknown_byte=0, month=_trig.month, day=_trig.day,
-            hour=_trig.hour, minute=_trig.minute, second=_trig.second,
-        )
-    elif ts1 is not None:
+    if ts1 is not None:
         ev.timestamp = Timestamp(
             raw=footer[2:10],
             flag=0x10,

sfm/sfm_webapp.html

@@ -108,12 +108,6 @@
       color: var(--text);
     }
     .btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
-    .btn-danger   { background: var(--red);    color: #fff; }
-    .btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
-    .diag-result  { display:block; margin-top:6px; font-size:12px; opacity:.85;
-                    white-space:pre-wrap; word-break:break-word; }
-    .diag-result.ok    { color: var(--green); }
-    .diag-result.error { color: var(--red); }
     .btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
 
     /* #connect-btn styles moved to #live-connect-bar block */
@@ -916,7 +910,6 @@
   <button class="tab-btn"        data-tab="events"    onclick="switchTab('events')">Events</button>
   <button class="tab-btn"        data-tab="config"    onclick="switchTab('config')">Config</button>
   <button class="tab-btn"        data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
-  <button class="tab-btn"        data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
 </div>
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -945,10 +938,6 @@
 <div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
 
   <div class="event-toolbar">
-    <button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
-            title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
-      ⟳ Load events
-    </button>
     <button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
     <button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
             title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
@@ -1216,77 +1205,6 @@
 
 </div><!-- end #tab-call-home -->
 
-<!-- ════════════════════════════════════════════════════════════════
-     TAB: Diagnostics
-═══════════════════════════════════════════════════════════════════ -->
-<div id="tab-diagnostics" class="tab-pane">
-
-  <div class="cfg-grid">
-
-    <div class="cfg-section">
-      <div class="cfg-section-title">Device State</div>
-      <div class="hint" style="margin-bottom:10px">
-        Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
-      </div>
-      <div class="dev-table" id="diag-table"></div>
-      <div class="cfg-actions" style="margin-top:12px">
-        <button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
-        <span id="diag-status"></span>
-      </div>
-    </div>
-
-    <div class="cfg-section">
-      <div class="cfg-section-title">Actions</div>
-
-      <div class="cfg-field">
-        <label>Stop Monitoring</label>
-        <button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
-        <div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
-        <span class="diag-result" id="diag-stop-result"></span>
-      </div>
-
-      <div class="cfg-field">
-        <label>Disable Auto Call Home</label>
-        <button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
-        <div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
-        <span class="diag-result" id="diag-ach-result"></span>
-      </div>
-
-      <div class="cfg-field">
-        <label>Erase All Events</label>
-        <input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
-               oninput="diagCheckEraseConfirm()" autocomplete="off" />
-        <button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
-        <div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
-        <span class="diag-result" id="diag-erase-result"></span>
-      </div>
-    </div>
-
-    <div class="cfg-section">
-      <div class="cfg-section-title">Unresponsive Unit</div>
-      <div class="hint" style="margin-bottom:10px">
-        The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
-        to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
-        <code>ach_server</code> and answer its call — before racing it with these.
-      </div>
-
-      <div class="cfg-field">
-        <label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
-        <button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
-        <div class="hint">Success is <code>bytes_received &gt; 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
-        <span class="diag-result" id="diag-drip-result"></span>
-      </div>
-
-      <div class="cfg-field">
-        <label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
-        <button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
-        <span class="diag-result" id="diag-blind-result"></span>
-      </div>
-    </div>
-
-  </div>
-</div><!-- end #tab-diagnostics -->
-
 </div><!-- end #section-live -->
 
 <!-- ════════════════════════════════════════════════════════════════
@@ -1443,8 +1361,6 @@
 // ── State ──────────────────────────────────────────────────────────────────────
 let unitInfo      = null;
 let eventList     = [];
-let storageInfo   = null;   // /device/events/storage_range — cheap, read on connect
-let eventsLoaded  = false;  // the event chain walk is opt-in; see loadEventList()
 let currentEvent  = 0;
 let charts        = {};
 let geoAdcScale      = 6.206;
@@ -1542,7 +1458,6 @@ function switchTab(name) {
   if (name === 'units')    { if (!unitsLoaded)  loadUnits(); }
   if (name === 'monlog')   { if (!monlogLoaded) loadMonitorLog(); }
   if (name === 'sessions') { if (!sessLoaded)   loadSessions(); }
-  if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
 }
 
 // ── Connect ────────────────────────────────────────────────────────────────────
@@ -1563,13 +1478,18 @@ async function connectUnit() {
     btn.disabled = false; btn.textContent = 'Connect'; return;
   }
 
-  // Connecting deliberately does NOT walk the event chain.  That walk reads
-  // every event header over the cellular link and can take minutes — or fail
-  // outright on a unit whose buffer has wrapped past 0xFFFF.  Use the ~2 s
-  // probes instead; the event list is opt-in via loadEventList().
-  eventList = []; eventsLoaded = false;
-  setStatus('Reading device state…', 'loading');
-  storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
+  setStatus('Fetching event list…', 'loading');
+  try {
+    const r = await fetch(`${api()}/device/events?${deviceParams()}`);
+    if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
+    const evData = await r.json();
+    eventList = evData.events || [];
+    // Merge compliance from /device/events response (it re-reads it)
+    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
+  } catch (e) {
+    setStatus(`Event fetch failed: ${e.message}`, 'error');
+    btn.disabled = false; btn.textContent = 'Reconnect'; return;
+  }
 
   populateDeviceBar();
   populateDeviceTab();
@@ -1578,9 +1498,11 @@ async function connectUnit() {
 
   document.getElementById('device-bar').style.display    = 'flex';
   document.getElementById('monitor-panel').style.display = 'flex';
-  setEventButtonsEnabled();
-  document.getElementById('load-events-btn').disabled = false;
-  setDiagButtonsEnabled(true);
+  document.getElementById('load-btn').disabled     = eventList.length === 0;
+  document.getElementById('save-btn').disabled     = eventList.length === 0;
+  document.getElementById('download-btn').disabled = eventList.length === 0;
+  document.getElementById('prev-btn').disabled     = true;
+  document.getElementById('next-btn').disabled     = eventList.length <= 1;
   document.getElementById('cfg-read-btn').disabled  = false;
   document.getElementById('cfg-write-btn').disabled = false;
   document.getElementById('ch-read-btn').disabled   = false;
@@ -1588,9 +1510,7 @@ async function connectUnit() {
 
   btn.disabled = false; btn.textContent = 'Reconnect';
 
-  setStatus(storageInfo && storageInfo.is_empty
-    ? 'Connected — no events stored.'
-    : 'Connected. Event list not loaded (Events → Load events).', 'ok');
+  setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
 
   // Fetch monitor status in background (non-blocking)
   refreshMonitorStatus().catch(() => {});
@@ -1602,48 +1522,6 @@ async function connectUnit() {
   }
 }
 
-// ── Shared fetch helper ────────────────────────────────────────────────────────
-async function fetchJson(path, opts) {
-  const sep = path.includes('?') ? '&' : '?';
-  const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
-  const body = await r.json().catch(() => ({}));
-  if (!r.ok) throw new Error(body.detail || r.statusText);
-  return body;
-}
-
-function setEventButtonsEnabled() {
-  const n = eventList.length;
-  document.getElementById('load-btn').disabled     = n === 0;
-  document.getElementById('save-btn').disabled     = n === 0;
-  document.getElementById('download-btn').disabled = n === 0;
-  document.getElementById('prev-btn').disabled     = true;
-  document.getElementById('next-btn').disabled     = n <= 1;
-}
-
-// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
-async function loadEventList() {
-  if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
-  const btn = document.getElementById('load-events-btn');
-  btn.disabled = true;
-  setStatus('Walking the event chain — this can take a while…', 'loading');
-  try {
-    const evData = await fetchJson('/device/events');
-    eventList = evData.events || [];
-    eventsLoaded = true;
-    // /device/events re-reads compliance; fold it in.
-    if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
-  } catch (e) {
-    setStatus(`Event fetch failed: ${e.message}`, 'error');
-    btn.disabled = false; return;
-  }
-  populateDeviceBar();
-  populateDeviceTab();
-  populateEventChips();
-  setEventButtonsEnabled();
-  btn.disabled = false;
-  setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
-}
-
 // ── Device bar ─────────────────────────────────────────────────────────────────
 function populateDeviceBar() {
   qs('di-serial').textContent   = unitInfo.serial || '—';
@@ -1652,7 +1530,7 @@ function populateDeviceBar() {
   qs('di-sr').textContent       = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
   qs('di-rt').textContent       = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
   qs('di-trig').textContent     = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
-  qs('di-count').textContent    = eventsLoaded ? eventList.length : '—';
+  qs('di-count').textContent    = eventList.length;
   qs('di-project').textContent  = cc.project || '—';
   qs('di-client').textContent   = cc.client  || '—';
   qs('di-operator').textContent = cc.operator || '—';
@@ -1782,8 +1660,7 @@ function populateDeviceTab() {
     { label:'DSP',            value: unitInfo.dsp_version || '—' },
     { label:'Model',          value: unitInfo.model || '—' },
     { label:'Manufacturer',   value: unitInfo.manufacturer || '—' },
-    { label:'Stored Events',  value: eventsLoaded ? eventList.length : 'not loaded' },
-    { label:'Storage Used',   value: storageUsedLabel() },
+    { label:'Stored Events',  value: eventList.length },
   ];
   for (const {label, value} of cardData) {
     const c = document.createElement('div');
@@ -1830,158 +1707,6 @@ function renderTable(id, rows) {
   }
 }
 
-// ── Diagnostics ────────────────────────────────────────────────────────────────
-// Everything here is a cheap probe (POLL + one read) or a single write.  None of
-// it walks the event chain.  See docs/runbooks/wedged_unit_recovery.md.
-
-function storageUsedLabel() {
-  if (!storageInfo) return '—';
-  if (storageInfo.is_empty) return 'empty';
-  const f = storageInfo.first_key, l = storageInfo.last_key;
-  return (f && l) ? `${f} → ${l}` : '—';
-}
-
-function setDiagButtonsEnabled(on) {
-  for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
-                    'diag-drip-btn','diag-blind-btn']) {
-    const el = document.getElementById(id);
-    if (el) el.disabled = !on;
-  }
-  diagCheckEraseConfirm();
-}
-
-// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
-function diagCheckEraseConfirm() {
-  const box = document.getElementById('diag-erase-confirm');
-  const btn = document.getElementById('diag-erase-btn');
-  if (!box || !btn) return;
-  const serial = (unitInfo && unitInfo.serial) || '';
-  btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
-}
-
-function diagResult(id, text, cls) {
-  const el = document.getElementById(id);
-  if (!el) return;
-  el.textContent = text;
-  el.className = 'diag-result' + (cls ? ' ' + cls : '');
-}
-
-async function refreshDiagnostics() {
-  if (!devHost()) return;
-  const st = document.getElementById('diag-status');
-  if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
-
-  const [mon, store, idx] = await Promise.all([
-    fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
-    fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
-    fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
-  ]);
-
-  if (!store._err) storageInfo = store;
-
-  const err = v => `<span style="color:var(--red)">${v}</span>`;
-  const rows = [];
-
-  rows.push(['Monitoring', mon._err ? err(mon._err)
-    : (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
-  if (!mon._err) {
-    rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
-    if (mon.memory_total_bytes) {
-      const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
-      const pct  = (used / mon.memory_total_bytes * 100).toFixed(1);
-      rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
-    }
-  }
-
-  rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
-  if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
-
-  // SUB 0x08.  Known to report 0 on units with years of history — suspected
-  // field-offset bug in the decode, so show it but do not trust it.
-  rows.push(['Lifetime events', idx._err ? err(idx._err)
-    : `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
-
-  renderTable('diag-table', rows);
-  populateDeviceTab();
-  if (st) { st.textContent = ''; st.className = ''; }
-}
-
-async function diagStopMonitoring() {
-  const btn = document.getElementById('diag-stop-btn');
-  btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
-  try {
-    await fetchJson('/device/monitor/stop', { method: 'POST' });
-    diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
-    refreshDiagnostics();
-  } catch (e) {
-    diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
-  }
-  btn.disabled = false;
-}
-
-async function diagDisableAch() {
-  const btn = document.getElementById('diag-ach-btn');
-  btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
-  try {
-    const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
-    const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
-    diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
-  } catch (e) {
-    diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
-  }
-  btn.disabled = false;
-}
-
-async function diagEraseEvents() {
-  const serial = (unitInfo && unitInfo.serial) || 'this unit';
-  if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
-  const btn = document.getElementById('diag-erase-btn');
-  btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
-  try {
-    await fetchJson('/device/events/erase', { method: 'POST' });
-    diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
-    document.getElementById('diag-erase-confirm').value = '';
-    eventList = []; eventsLoaded = false;
-    setEventButtonsEnabled(); populateEventChips();
-    refreshDiagnostics();
-  } catch (e) {
-    diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
-  }
-  diagCheckEraseConfirm();
-}
-
-async function diagSlowDrip() {
-  const btn = document.getElementById('diag-drip-btn');
-  btn.disabled = true;
-  diagResult('diag-drip-result', 'Holding a session for 120 s…');
-  try {
-    const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
-                              { method: 'POST' });
-    const good = (r.bytes_received || 0) > 0;
-    diagResult('diag-drip-result',
-      `drips ${r.drips_sent}  ·  held ${r.duration_s}s  ·  bytes back ${r.bytes_received}` +
-      (r.send_error ? `  ·  ${r.send_error}` : '') +
-      (good ? '  → device responded' : '  → no response; the modem may not be bridging'),
-      good ? 'ok' : 'error');
-  } catch (e) {
-    diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
-  }
-  btn.disabled = false;
-}
-
-async function diagBlindStop() {
-  const btn = document.getElementById('diag-blind-btn');
-  btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
-  try {
-    const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
-    diagResult('diag-blind-result',
-      `Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
-  } catch (e) {
-    diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
-  }
-  btn.disabled = false;
-}
-
 // ── Config form ────────────────────────────────────────────────────────────────
 function populateConfigFromDeviceInfo() {
   if (!unitInfo) return;

tests/test_event_timestamp.py

@@ -1,54 +0,0 @@
-"""Event timestamp decode — waveform trigger/stop vs histogram window start.
-
-The Blastware footer holds two timestamps: ts1 = footer[2:10], ts2 = footer[10:18].
-Their meaning depends on record type:
-
-  * Waveform: ts1 is the monitoring-SESSION start (e.g. 06:00 for a unit that
-    arms at 06:00 daily — shared across every event that day), and ts2 is THIS
-    event's recording STOP.  read_blastware_file used to stamp events with ts1 →
-    every waveform showed the session start (~4.5 h off).  Binary-only, the best
-    estimate is ts2 (the stop); the exact trigger BW displays (= ts2 - record
-    duration) comes from the paired report's event_datetime, since the binary
-    STRT record-time byte is a misparsed record-type marker.
-  * Histogram: ts1/ts2 are the ~24 h window [start, stop]; the event time is the
-    window start = ts1 (unchanged).
-"""
-import datetime
-from pathlib import Path
-
-from minimateplus.event_file_io import read_blastware_file, apply_report_to_event
-from minimateplus.bw_ascii_report import BwAsciiReport
-from minimateplus.models import Event
-
-FIX = Path(__file__).parent / "fixtures"
-WAVEFORM = FIX / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"   # footer ts2 = 2026-08-25 10:33:32
-HISTOGRAM = FIX / "ts-fix" / "K441LKZU.C30H"                 # window start 2026-05-10 19:04:50
-
-
-def _tuple(ts):
-    return (ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second)
-
-
-def test_waveform_timestamp_is_exact_trigger_from_binary():
-    ev = read_blastware_file(WAVEFORM)
-    # The EXACT Blastware trigger, from the binary alone: ts2 (stop 10:33:32)
-    # minus the config record time (3.0 s) = 10:33:29 — NOT the 06:00:13
-    # monitoring-session start the old decode used.
-    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29), _tuple(ev.timestamp)
-
-
-def test_histogram_timestamp_is_window_start_unchanged():
-    ev = read_blastware_file(HISTOGRAM)
-    # Histogram event time = the window start (ts1); must NOT get the waveform
-    # ts2 treatment (that would land ~24 h off).
-    assert _tuple(ev.timestamp) == (2026, 5, 10, 19, 4, 50), _tuple(ev.timestamp)
-
-
-def test_report_event_datetime_is_authoritative_over_binary():
-    # The binary already yields the exact trigger, but a paired report stays
-    # authoritative (e.g. if the unit clock had drifted) — applying it wins.
-    ev = read_blastware_file(WAVEFORM)
-    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29)   # exact, from binary
-    apply_report_to_event(ev, BwAsciiReport(
-        event_datetime=datetime.datetime(2026, 8, 25, 10, 35, 0)))
-    assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 35, 0)    # report wins