fix(sfm): read the serial family prefix from the file, enabling BlastMates

The BW filename encodes only the serial NUMBER — `<letter><3 digits>`, so `L895…` is 10895 and nothing more. The two-letter family prefix is not in it: "BE" is a MiniMate Plus, "BA" a BlastMate. Both are Series III and their files are byte-identical in every way that matters — all 1,493 BlastMate binaries in the DL2 archive decode through the existing codec at 100%, same four channels — so the serial string was the only thing standing between SFM and BlastMate support. Two sites synthesised the prefix and got it wrong: - waveform_store `_serial_from_bw_filename` returned f"BE{num}" on import, so a BlastMate event was filed under a unit that does not exist, silently, and Terra-View read it straight through. Split into `_serial_number_from_bw_filename` (the number, which the filename really does carry) and a new `_serial_from_bw_bytes` that reads the serial out of the body and accepts it only when its numeric part agrees with the filename. save_imported_bw now prefers hint -> body -> filename guess. Verified against real archive bytes for BA9229, BA10060, BA10895, BA15957 and BE9558/BE11529/BE18003. - client `_decode_0a_partial_header` searched for a literal b"BE" in the monitor-log partial record. On a BlastMate that returns -1 and skips the whole block, so the geo threshold went missing along with the serial. Now matches any two-letter prefix, and requires the NUL terminator — stricter than the bare two-byte search it replaces. Nothing to migrate: no BlastMate events are in prod. The archive's BA units last recorded 2018-10 (BA9229, BA15957), 2023-08 (BA10895) and 2023-11 (BA10060), and the prod backfill only reaches back to ~May 2025. 21 tests. Suite: 309 passed, same 16 pre-existing failures as at HEAD (15 missing ASCII fixtures + one peak_values assertion, all untouched here). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
fix(offset): read the real serial from the file body, not "BE" + the number
2026-09-06 07:58:00 +00:00 · 2026-09-06 07:48:54 +00:00 · 2026-09-05 02:08:36 +00:00 · 2026-09-04 21:12:04 +00:00 · 2026-09-03 21:08:04 +00:00 · 2026-09-03 20:53:57 +00:00
27 changed files with 2517 additions and 119 deletions
@@ -6,6 +6,127 @@ All notable changes to seismo-relay are documented here.
 ## [Unreleased]
 ### Added
 - **`events.false_trigger_reason` — optional FT cause.** A nullable `TEXT`
  column recording *why* an event is a false trigger (e.g. `"offset"`), as a
  subtype of the FT flag: setting a reason via the sidecar review PATCH implies
  `false_trigger=1`, and the reason is cleared whenever FT ends up 0
  (confirm-real, clear-FT, `set_false_trigger(false)`). `propagate_review_to_twins`
  carries the reason to the histogram/waveform twin alongside the flag.
  Auto-migrated (`_SCHEMA` + `_migrate` ADD COLUMN — not the Migration-1
  rebuild); exposed via `/db/events`. Terra-View surfaces it as a manual
  "Flag as offset" action + an `FT · offset` badge.
 ---
 ## v0.28.0 — 2026-09-02
 **Offset (DC-baseline) false-trigger detector.** Productionizes the validated
 pre-trigger detector: a geophone event whose baseline sits off zero and stays
 flat across the record (sensor bumped / settled / drifted) is now flagged and
 surfaced in Terra-View as an `offset` false-trigger reason — catching offsets the
 crest/near-peak spike rule misses (an offset is low-crest and flat).
 ### Added
 - `shape_metrics.offset_from_samples` / `offset_from_h5`: per geophone channel,
  `|median(pre-trigger)| ≥ 0.025 in/s` AND `pre/mid/end spread ≤ 0.02` → offset;
  the consistency test rejects transients (a real event moves one third). Reads
  the `.h5` samples + the `pretrig_samples` attr, range-aware via the in/s float
  samples. Constants `OFFSET_FLOOR` / `OFFSET_MAX_SPREAD` are tunable.
 - `events.shape_offset` / `shape_offset_axis` / `shape_offset_pre` /
  `shape_offset_spread` columns (auto-migrated: `_SCHEMA` + the `_migrate`
  ADD COLUMN loop), computed at all three ingest paths and by
  `backfill_event_shape.py`, exposed via `/db/events`.
 Requires the shape/offset backfill on the prod store to populate existing events:
 `python scripts/backfill_event_shape.py --db-path … --store-root …`.
 ---
 ## v0.27.0 — 2026-08-28
 **Per-sample decoder verification at scale, plus the offset investigation.**
 The series-3 codec is now verified sample-by-sample against **14,338** preserved
 Blastware ASCII exports — 1,249 waveform and 13,089 histogram, spanning 45 units
 and files back to 2018. That is 11x the ground truth the production store
 carried, and it found one real codec bug (below).
 ### Fixed
 - **Sub-minute histograms with a partial final block decoded to nothing**
  (`histogram_codec.detect_multi_interval_stride`). The stride search confirmed
  itself on a third block header whenever the body was long enough to hold one —
  but a body can exceed two strides and still contain only two real blocks, because
  a *partial* final block leaves trailing padding. BE18193 `T193L0XM.CI0H` (51
  intervals at 2 s = one full 30-interval block plus a 21-interval remainder, in a
  2787-byte body) therefore had its correct stride of 612 discarded and produced an
  empty decode. A missing third header now means end-of-stream rather than
  disqualification; the block-counter check, which is what actually prevents the
  false positives that once mis-dispatched 9,082 files, is unchanged.
  Found by decoding the full DL2 archive against its preserved Blastware ASCII
  exports. Across **63,535 unique** histogram binaries the fix recovers **4 files** —
  `K440HJCN.3C0H` and `K557IF1U.8K0H` (stride 252), `T191HVNP.0S0H` (92) and
  `T193L0XM.CI0H` (612) — with **zero** files regressed. Verification over all
  14,340 archive pairs goes 14,337 → 14,338 exact, the only remainder being two
  series-4 IDF files that belong to a different codec.
  (The DL2 export keeps a byte-identical `Sent/` mirror of its root, so a naive
  walk double-counts every binary — 127,035 paths are 63,535 distinct files. The
  ASCII exports are *not* mirrored, so the 14,340 pair count is already distinct.)
  **No prod backfill is required for this.** Verified after the fact: all four
  recovered files are archive-only — none exists in the production store or the
  events DB — and re-running stride detection over the production store's
  **10,215** histogram binaries shows **0 files whose decode changes**. The fix
  matters for future ingests of sub-minute histograms with a partial final block,
  not for anything already stored.
  (`TOOL_VERSION` moves with the release, so whenever a backfill *is* next run for
  some other reason it will regenerate the whole store rather than skipping. That
  is harmless — the output is byte-identical for every currently-stored file — but
  it means the run takes its full ~2 hours on the NAS.)
 - **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real
  trigger is recorded twice — as a triggered waveform (stamped at the trigger instant)
  and inside the scheduled histogram whose interval contains it (stamped at the 7am/7pm
  interval start) — so the two twins can be **hours apart**. The old ±5-minute window
  silently missed them, which broke review propagation (flagging one twin didn't flag its
  twin). Twins are now matched by same serial + identical `peak_vector_sum` + opposite
  record type + the waveform falling within the histogram's interval (bounded by the next
  same-serial histogram). `window_seconds` is retained but ignored. Fixes terra-view #102
  sub-task 2.
 - **`/health` reported a hard-coded `0.1.0`** instead of the real service version.
  `sfm/server.py` now derives its version from `minimateplus.event_file_io.TOOL_VERSION`,
  making that constant the single source of truth for the service version and the
  sidecar stamp alike — one place to bump at release.
 - **`CLAUDE.md` had 793 NUL bytes appended** after its last line, which made `grep`
  treat the file as binary and silently skip it. Present since at least v0.21.0.
  Stripped.
 ### Added
 - **`docs/offset_investigation.md`** — a dated journal of the "offset" hardware
  fault: base rate, detector design, per-unit case files, ruled-out hypotheses
  (each kept with the evidence that killed it), and Instantel's own autozero
  procedure with its 2027–2069 acceptance window.
 - **`scratch/verify_against_ascii.py`** — decodes a corpus of BW binaries and
  diffs every sample against the paired `_ASCII.TXT`. Includes a saturation
  carve-out: BW clamps clipped events to the range maximum and writes `OORANGE`,
  while the decoder faithfully reports counts past nominal full scale.
 - **`scratch/offset_scan3.py`** — offset detector. Measures the resting floor in
  the *pre-trigger* window (definitionally quiet) and requires it to hold across
  pre / middle / end. Result: **5 of 45 units (11%)**, stable across a 2x
  threshold range. Supersedes `offset_scan.py` and `offset_scan2.py`, both kept
  as the reasoning trail.
 ### Verified
 - **19,244 healthy channel-events sit at a pre-trigger floor of exactly 0.000
  (62.7%), 94.5% within ±1 quantisation unit, median +0.0000.** No systematic
  zero-point bias in the decoder — an independent confirmation of the
  32000-count geo full scale, arrived at from a different direction than the
  ASCII sample comparisons.
 ---
 ## v0.26.0 — 2026-08-27
@@ -2,21 +2,28 @@
 Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
 managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
-(Sierra Wireless RV50 / RV55). Current version: **v0.26.0**.
+(Sierra Wireless RV50 / RV55). Current version: **v0.27.0**.
 Stack-level context — which repo owns what, and how the three project versions
 pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
 `~/CLAUDE.md`.
 ---
-## Where things stand (updated 2026-08-27)
+## Where things stand (updated 2026-08-28)
 Read this first when picking the project back up.
- **Series-3 decode is correct and verified.** All 11,603 series-3 binaries in
+- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
-  the prod snapshot pass every check (channel lengths, peaks vs the device's
+  archive decodes **14,338 / 14,338** paired files exactly against their
-  own reported PPV, nothing above full scale, length vs declared record time).
+  preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
-  Ground truth: 1,211/1,211 histograms exact per-interval and 75/75 waveform
+  units, files back to 2018. That is 11x the ground truth the prod store
-  sample counts exact against preserved Blastware ASCII exports.
+  carried, and it supersedes the old "per-sample on 11%, peak-only on 89%"
-  ⚠ That is per-sample proof on 11% of files and peak-only consistency on the
+  caveat. Harness: `scratch/verify_against_ascii.py` (note its saturation
-  other 89% — see `docs/instantel_protocol_reference.md` §7.6.1.
+  carve-out — BW clamps clipped events, the decoder reports true counts).
  Independent corroboration of the 32000-count scale: 19,244 healthy
  channel-events sit at a pre-trigger floor of exactly 0.000 (62.7%), 94.5%
  within ±1 quantisation unit, median +0.0000 — no zero-point bias.
 - **Series-4 (Thor / Micromate) is NOT verified.** UM-series sits at ~48%
  against device peaks with a ~1.7% systematic bias and a near-zero tail.
  Thor IDFW is pinned to `decode_waveform_legacy` deliberately.
@@ -24,11 +31,24 @@ Read this first when picking the project back up.
  (= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
  inserts DB rows for store files that have none (one-time per store) and the
  dry-run does not report that count.
- **After any codec change, regenerate the store** — `backfill_sidecars.py
+- **After any codec change, regenerate the store** — `backfill_sidecars.py`
-  --force` then `backfill_event_shape.py`, DB backup first. Stored `.h5` files
+  then `backfill_event_shape.py`, DB backup first. Stored `.h5` files do not
-  do not update themselves.
+  update themselves. No `--force` needed as long as `TOOL_VERSION` was bumped
- **Parked:** the "offset" hardware-fault investigation (Appendix E of the
+  (it gates regeneration). ⚠ On the office NAS this takes **~2 hours**
-  protocol reference) pending the multi-year BW archive.
+  (~1.5 files/sec vs 85/sec on the dev box — gzip-4 in `sfm/event_hdf5.py`
  against a Synology CPU). Budget it up front.
  **v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
  fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
  files are archive-only and were never ingested).
 - **The "offset" hardware fault has its own journal** --
  `docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
  **persistent** — it stays until the geophone is serviced. Detect it with
  `scratch/offset_scan3.py`: the resting floor in the **pre-trigger** window,
  required to hold across pre/middle/end. Never score only the dominant-peak
  axis and never use the mean — both produce false recoveries (see the
  retraction banner in the journal). Instantel's autozero procedure and its
  2027-2069 acceptance window are recorded there too. Best open lead is
  `SUB 0x0E` (unimplemented), which may carry those very numbers.
 When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
@@ -1803,4 +1823,4 @@ body) because writing a dial string may require DLE escaping for embedded contro
 To parse BW TX captures: use `bridges/captures/` scripts or adapt the `find_write_frames()` pattern
 in `/tmp/analyze_write_payload.py` — it correctly handles `0x10 0x03` DLE-escaped ETX bytes
-inside write frame data (the naive parser terminates early at the escaped `0x03`).
+inside write frame data (the naive parser terminates early at the escaped `0x03`).
@@ -1,4 +1,4 @@
-# seismo-relay  `v0.26.0`
+# seismo-relay  `v0.27.0`
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing seismographs.  Supports both the **MiniMate Plus
@@ -3524,6 +3524,11 @@ of them was, for a while.
 ### E.1 The "offset" fault
 > **The full investigation now lives in `docs/offset_investigation.md`** --
 > base rate, detector definition, per-unit case files, ruled-out hypotheses,
 > and Instantel's own autozero procedure with its 2027-2069 acceptance
 > window.  This appendix is kept as the protocol-side summary.
 **Symptom.**  One geophone channel's baseline steps away from zero and
 stays there.  The trace still carries the real AC signal, but it rides
 on a DC pedestal of a few tenths of an in/s.  Operators call this an
@@ -0,0 +1,686 @@
 # The "offset" fault — investigation journal
 > ## ⚠ CORRECTED 2026-08-28 (same day) — the v1 detector was wrong
 >
 > Brian pushed back on the finding that offsets "come and go": in the field,
 > once a unit develops one it stays broken until the geophone is replaced.
 > He was right, and the challenge exposed **two real flaws** in the v1 detector:
 >
 > 1. **It scored only the axis with the largest peak.** A real event on one axis
 >    hid a persistent pedestal on another. BE12599 on 2026-08-21 read "clean"
 >    solely because Long had a 1.065 in/s event — Tran was sitting at
 >    **+0.4732 in/s** at that moment and was never examined.
 > 2. **It used the MEAN**, which a real transient perturbs. The **median** is the
 >    resting baseline — most samples sit at it, so a blast does not move it.
 >    Same event, Long channel: mean **+0.0783** vs median **-0.0050**.
 >
 > Both flaws manufactured false recoveries. The corrected detector
 > (`scratch/offset_scan2.py`, per-channel median) shows the pedestal is
 > **persistent**, exactly as the field experience says. See §2b and §3b.
 >
 > **Then Brian proposed a better detector still** — measure the floor during
 > the *pre-trigger* window, and require it to hold across pre/middle/end.
 > That is now the detector of record (§2c). Final answer: **5 of 45 units
 > (11%)**, stable across a 2x threshold range.
 >
 > Sections below that were written against v1 are marked; v1 numbers are kept
 > for the reasoning trail, not as current fact.
 A running record of the **offset** hardware fault on Instantel Series III
 seismographs: a geophone channel whose trace sits displaced from zero rather
 than centred on it.
 This is a *journal*, not a spec. Findings are dated, dead ends are kept with
 the reason they died, and every number says where it came from. When something
 here is superseded, strike it and say why rather than deleting it — the point
 is that a future session can tell what was actually established from what was
 merely believed at the time.
 Companion material:
 - `scratch/offset_scan.py` — the detector
 - `scratch/verify_against_ascii.py` — decoder verification harness
 - `docs/instantel_protocol_reference.md` — wire protocol, incl. the
  unimplemented `SUB 0x0E` this investigation now wants
 ---
 ## TL;DR (current state, 2026-08-28)
 - **It is real device data, not a decode bug.** Settled early and confirmed
  against Blastware's own ASCII exports.
 - **Base rate: 5–6 of 45 units (11–13%)** across the full DL2 archive,
  2018–2026. This *confirms* the earlier 2-of-21 (9.5%) estimate from the much
  smaller Terra-View DB — survivorship bias from deleted events had **not**
  concealed a wave of cases.
 - **The fault is bimodal, not a drift continuum.** A unit is either clean or
  grossly off. Loosening the amplitude threshold 11× adds no new units.
 - **The unit's own sensor check cannot see it.** 102 offset events, zero
  sensor-check failures. Do not try to use it as a screen.
 - **Cause is still unsettled.** Instantel's autozero fixes the minority of
  cases; the rest are hardware. We cannot yet tell which is which remotely.
 - **The histogram corpus (63,535 files, 9.7x the waveforms) is now scanned too** —
  see §8b. It independently confirms BE18438 and BE9558 with a clean 2.5x
  separation, but detects only **2 of the 5** confirmed units, cannot attribute a
  channel, and resolves time to ~a month. **A negative histogram result is not
  evidence of health** — DC leakage into the interval peak varies 45x between units.
 - **`offset_scan3.py` has a label defect** (§8b): its spread gate discards 18.8% of
  high-|pre| rows onto units currently counted as clean. Re-cut before quoting any
  precision number again.
 - **Best open lead:** `SUB 0x0E` (channel sensor data, 8 channels × 10 bytes,
  unimplemented) may carry the very numbers Instantel says to check against
  **2027–2069**. Untested.
 ---
 ## 1. What the fault looks like
 A healthy geophone trace is centred on zero. An offset channel is parked away
 from zero, so the channel **mean approaches its own peak**. In Blastware the
 signature is "parallel lines above or below the zero line" (Instantel's own
 wording).
 Consequences observed in the field:
 - The unit can **self-trigger on its own offset** when the displacement exceeds
  the geo trigger level, producing streams of junk events with no ground
  motion. Instantel has a separate FAQ for this symptom (13-0-22, *"Unit
  triggers continuously without activity"*).
 - Recorded PPV for that channel is meaningless while the fault persists.
 ---
 ## 2. The detector
 Implemented in `scratch/offset_scan.py`. Operates on raw BW binaries only — no
 DB, no sidecars.
 ```
 for each series-3 waveform binary:
    decode -> per-channel ADC counts
    dominant axis = channel with the largest |peak|
    flag when   |mean| / peak  >  0.70
          and   |mean|         >= 0.90 x the unit's geo trigger level
 episodes = per-serial runs of flagged events, split on a >12 h gap
 ```
 Why each term:
 | term | purpose |
 |---|---|
 | `\|mean\|/peak > 0.7` | the discriminator. A DC-parked trace has mean ≈ peak. |
 | `\|mean\| >= 0.9 × trigger` | amplitude floor — suppresses quiet traces where mean and peak are both tiny and the ratio is meaningless. |
 | dominant axis only | the fault is per-channel; scoring all three dilutes it. |
 | 12 h episode gap | separates deployments/visits rather than counting events. |
 Trigger level comes from a paired `_ASCII.TXT` when one exists, else the
 per-serial median learned from that unit's ASCII files, else 0.2 in/s.
 **Known limitation.** Event traces contain real ground motion, so this can only
 see offsets large enough to *dominate* the trace. A mild offset on a real blast
 is invisible. Instantel's A/D-mode check (§5) is the only thing that sees the
 mild end. Our base rate is therefore a **gross-offset** rate.
 ### 2b. Detector v2 — per-channel median (CURRENT)
 `scratch/offset_scan2.py`. Supersedes the above.
 ```
 for each series-3 waveform binary:
    for each geo channel independently:
        pedestal = median(samples)          # resting baseline, robust to blasts
        flag the CHANNEL when |pedestal| >= 0.025 in/s   (5 A/D counts)
 a unit has a real fault when a channel is flagged on >=3 CONSECUTIVE events
 ```
 Why median: a DC pedestal shifts every sample, so it moves the median. A real
 event moves only a minority of samples, so it does not. This removes the need
 for the `m/p` ratio guard entirely — that guard existed only to compensate for
 using the mean.
 Why per-channel: the fault is on one geophone axis. Scoring only the dominant
 axis means any event with motion elsewhere hides it.
 Why "3 consecutive": the 0.025 in/s floor is only ~2x a healthy channel's
 resting median (observed 0.010-0.015), so isolated flags are noise. Persistence
 is the discriminator — and it is what the field experience predicts.
 ---
 ## 3b. Archive results, corrected (v2)
 | | v1 (dominant axis, mean) | **v2 (per-channel median)** |
 |---|---|---|
 | units with any flagged event | 6 of 45 | 19 of 45 |
 | **units with a sustained pedestal (>=3 consecutive)** | — | **8 of 45 (18%)** |
 | runs of >=3 consecutive | — | 29 |
 | runs of 1-2 events (noise) | — | 69 |
 Units with a sustained pedestal: **BE9558, BE10895, BE11007, BE11529, BE12599,
 BE13117, BE18003, BE18438**. BE10895 and BE18003 were invisible to v1.
 **The affected channel is most often Vert**, which v1 got wrong — it named
 whichever axis had the largest peak. BE13117 and BE18438 are both Vert faults.
 Longest / clearest runs:
 | unit | ch | span | events | median in/s |
 |---|---|---|---|---|
 | BE13117 | Vert | 2023-05-03 → 05-04 | 194 | 0.035 → **1.915** |
 | BE18438 | Vert | 2026-02-25 → 02-26 | 75 | 0.180 → 0.370 |
 | BE9558 | Vert | 2020-02-11 (6 h) | 33 | 0.065 → 0.090 |
 | BE12599 | Tran | 2026-08-14 → 08-23 | 8 | 0.030 → **0.565** |
 | BE18003 | Vert | 2021-03-17 → 06-11 | 3 | 0.040 → 0.060 |
 BE12599 began **2026-08-14**, not 08-17 as v1 reported, and was still faulting
 at the last event in the archive.
 ### 2c. Detector v3 — PRE-TRIGGER floor + constant-floor test (CURRENT)
 `scratch/offset_scan3.py`. Brian's method, and better than v2 for a reason
 worth naming: **the pre-trigger window is definitionally quiet** — it is the
 buffer captured before the trigger fired — whereas a whole-record median is
 merely *robust* to the event. `pretrig_samples` comes from the STRT record.
 ```
 per channel:
    pre    = median of the first pretrig_samples samples
    mid    = median of the middle third
    end    = median of the final third
    spread = max(pre,mid,end) - min(pre,mid,end)
    offset when  |pre| >= floor  AND  spread <= 0.02 in/s
 real fault when a channel is flagged on >=3 CONSECUTIVE events
 ```
 A DC offset is a **constant floor** — present before the trigger, during, and
 after. The spread test rejects transients (settling, handling, a long event
 tail) that move one segment relative to the others, which is what v2's
 whole-record median could not do.
 **The empirical noise floor justifies the threshold.** Across 19,244
 non-flagged channel-events the pre-trigger floor distributes as:
 | floor | share |
 |---|---|
 | −1 unit (−0.005) | 18.4% |
 | **0.000** | **62.7%** |
 | +1 unit (+0.005) | 13.4% |
 **94.5% within ±1 quantisation unit; median exactly +0.0000, mean −0.0008.**
 So there is **no systematic zero-point bias in the decoder** — an independent
 confirmation of the 32000-count scale. A healthy channel really does read
 0.000, and "any constant floor that is not 0.000" is the right signal, with
 ±1 unit of slack for quantisation.
 **The result is threshold-insensitive**, which is what distinguishes a real
 signal from a tuned one:
 | floor | units flagged | sustained units |
 |---|---|---|
 | 2 units (0.010) | 34 | 15 ← into the noise |
 | 3 units (0.015) | 26 | 8 |
 | **4 units (0.020)** | 17 | **5** |
 | **5 units (0.025)** — Instantel's | 12 | **5** |
 | **8 units (0.040)** | 8 | **5** |
 ### FINAL RESULT: 5 of 45 units (11%)
 **BE9558, BE11529, BE12599, BE13117, BE18438.**
 Unchanged across a 2x threshold range. BE11007 and BE10895 drop out — the
 spread test identifies them as transients, not pedestals.
 The 11% headline happens to match v1's, but the reasoning and the unit list
 differ: v1 included BE11007 and named the wrong *channel* on most units.
 ---
 ## 3. Archive results (2026-08-28)
 Source: DL2 event export, 6,577 **unique** series-3 waveforms, 45 units.
 See [`dl2-archive`](#8-data-and-tooling) for the `Sent/` mirror trap.
 **283 suspect events, 15 episodes, 6 of 45 units (13.3%).**
 Excluding BE11007 (§4, likely not an offset at all): **5 of 45 = 11.1%**.
 ### Threshold sensitivity — the bimodality result
 Re-scoring the same corpus at a range of amplitude floors, with two
 ratio cut-offs (1 A/D count = 0.005 in/s, see §5):
 | \|offset\| floor | m/p > 0.7 | m/p > 0.9 |
 |---|---|---|
 | 5 cts (0.025 in/s) — *Instantel's own* | 333 ev / 6 units | 279 ev / **5 units** |
 | 10 cts (0.050) | 294 / 6 | 274 / 5 |
 | 20 cts (0.100) | 250 / 5 | 244 / 4 |
 | 40 cts (0.200) | 209 / 5 | 203 / 4 |
 | 80 cts (0.400) | 152 / 4 | 148 / 2 |
 | 160 cts (0.800) | 144 / 2 | 141 / 1 |
 Relaxing the floor by 11× (0.27 → 0.025 in/s) adds ~14% more events and **no
 new units**. There is no population of mild offsets hiding below our threshold
 *in event data*. Either a unit is clean or it is grossly off.
 ---
 ## 4. Per-unit case files
 Ordered by severity. `m/p` medians are on the offending channel.
 ### BE13117 — one violent day, never again
 `145 / 454 events (32%)`, **1 episode**, 2023-05-04, 6.8 h.
 Offset climbed **0.393 → 1.875 in/s within the episode**. `m/p` median
 **0.996** — the trace is almost pure DC. No recurrence in the rest of its 454
 events. No ASCII files in the archive, so no calibration history.
 ### BE18438 — recurring, months apart
 `87 / 293 (30%)`, **2 episodes**: 2025-11-15 (1.2 h, n=12, 0.279 → 0.369) and
 2026-02-25 (**28.8 h**, n=75, 0.183 → 0.366). `m/p` median 0.967.
 Clean across all 196 events preceding its 2025-08-12 calibration.
 ### BE9558 — six years apart
 `38 / 196 (19%)`, **4 episodes**: 2020-02-11 (6.3 h, n=33, but only
 0.068 → 0.086 — very mild), then 2026-04-14, 2026-04-29, 2026-05-04
 (0.28–0.45). `m/p` median 0.919. Calibrated 2026-06-26; 0/7 events flagged
 after, but n=7 is far too small to call it fixed.
 ### BE12599 — the live case ⚠
 `6 / 77 (8%)`, **6 single-event episodes, one per day at exactly 05:00**,
 2026-08-17 → 2026-08-23. Offset rose 0.383 → 0.565 then fell back to 0.345.
 `m/p` ≈ 0.965, geo trigger 0.3 in/s — **the offset exceeds the trigger level,
 so the unit is triggering on its own fault**. Last calibrated 2025-08-12.
 This is the most recent and the most useful: a currently-faulting unit is the
 natural experiment for the re-zero-vs-repair question (§7).
 ### BE11529 — marginal
 `4 / 99 (4%)`, 1 episode 2025-07-08, 0.4 h, offsets only 0.051 → 0.058 in/s.
 `m/p` median 0.959, so DC-dominated, but the magnitude is near the noise of
 this method. Treat as unconfirmed.
 ### BE11007 — probably NOT an offset
 `3 / 70 (4%)`, 1 episode 2022-01-17, offsets 5.500 → 6.904 in/s — by far the
 largest. But `m/p` is only **0.719–0.738** against ≥0.9 for every other unit,
 and the peaks are 7.6–9.4 in/s on a 10 in/s range. That reads as a **large
 real blast with asymmetric ground motion**, not a parked trace. Excluded from
 the headline base rate.
 ---
 ## 5. Instantel's own procedure and thresholds
 From two Instantel technical-support FAQs supplied 2026-08-28
 (answers **13-0-21** *"How to determine offsets"* and **12-0-10** *"Removing
 offsets on an Instantel Series III monitor"*; created 2008/2007, last updated
 2009-03-06).
 ### Identifying (13-0-21)
 1. Create or use an event with the **manual minimum trigger** set for the
   connected geophone and microphone — i.e. an event that recorded no real data.
 2. Save it and open in Blastware.
 3. An offset shows as **parallel lines above or below the zero line**.
 4. Put the unit in **A/D mode** — on Series III, press and hold `OPTION`, then
   press `START MONITOR`.
 5. **Display counts higher than 5**, with no vibration or overpressure present,
   indicate an offset.
 ### Removing — the autozero (12-0-10)
 1. Be in a **quiet area with low vibration**.
 2. Power on the Blastmate III / Minimate Plus.
 3. Connect the geophone and microphone — **LINEAR mic only**.
   ⚠ *Do not connect an "A" weight microphone, regardless of what the monitor
   displays.*
 4. Press `Test`.
 5. Wait for the **Sensor Check** results to appear.
 6. Press `OPTION` and `START MONITOR` **simultaneously**.
 7. `Performing Autozero` appears; press `Enter`.
 8. Confirm the sensors are properly connected; press `Enter`.
 9. Wait for the autozero to complete.
 10. Press `Enter` twice → Main Menu, *Ready To Monitor*, offset corrected.
 ### The go/no-go number — 2027 to 2069
 > When you perform an Autozero on any Series III unit, the lists of numbers in
 > the **X1 and X8 gains should all be between 2027 and 2069**. If not, repeat
 > the Autozero. **If the numbers are extremely out of the specified range, then
 > the unit should be sent in for repair.**
 >
 > If this process does not remove the offset problem, return the unit **and
 > sensors** to Instantel for repair.
 This is the documented explanation for the field experience (Brian's dad,
 2026-08-28) that **a re-zero works maybe 10% of the time** — the autozero only
 recovers units whose zero reference is still near-correct.
 ### Scale derivation (inference, well-supported — not proven)
 2048 is 12-bit midscale. Our codec's geo full scale is 32000 internal counts =
 10 in/s, with 1 decoder unit = 16 counts = exactly 0.005 in/s
 (`geo-full-scale-is-32000-counts`). ±2000 A/D counts about 2048 therefore maps
 to ±10 in/s at **0.005 in/s per A/D count**. That makes:
 - Instantel's ">5 counts" threshold ≈ **0.025 in/s**
 - the 2027–2069 window = **±21 counts = ±0.105 in/s** of tolerated zero error
 Consistent and mutually corroborating, but we have not confirmed the A/D-count
 scale directly from a device reading.
 ---
 ## 6. Ruled out — keep these dead
 ### Condensation / humidity — DEAD (2026-08-25)
 Proposed, then killed by its own controls: BE18438 stayed flat across a 10-hour
 overnight gap, and only 2 of 21 units showed the fault while 19 sat in the same
 weather. The apparent "diurnal cycle" was an artifact of binning by hour-of-day
 across two days. See `waveform-dc-offset-is-real-device-data`.
 ### Clipping as a false-positive source — RULED OUT (2026-08-28)
 A rail-hitting trace would fake an offset (mean → peak). It isn't happening:
 median suspect peak is only **10% of full scale**, p90 is 18.6%. Only BE11007's
 3 events exceed 50% FS, and none reach 98%.
 ### The sensor check as a predictor — DOES NOT WORK (2026-08-28)
 Tested on 102 offset events across 4 units:
 | unit | state | n | failed | median ratio | median freq |
 |---|---|---|---|---|---|
 | BE11529 | offset | 4 | **0** | 3.90 | 7.6 |
 | BE11529 | clean | 14 | 0 | 3.80 | 7.5 |
 | BE12599 | offset | 6 | **0** | 4.00 | 7.4 |
 | BE12599 | clean | 13 | 0 | 4.00 | 7.6 |
 | BE18438 | offset | 87 | **0** | 3.70 | 7.6 |
 | BE18438 | clean | 25 | 0 | 3.80 | 7.5 |
 | BE9558 | offset | 5 | **0** | 3.90 | 7.8 |
 | BE9558 | clean | 44 | 0 | 3.80 | 7.5 |
 Zero failures on either side and indistinguishable ratios/frequencies. The
 swing test measures geophone frequency response and damping — it never examines
 DC zero. **A grossly offset unit passes its own self-check.** This is why the
 fault goes unnoticed until somebody looks at waveforms.
 ### "Offsets are transient / come and go on their own" — RETRACTED 2026-08-28
 v1 reported episodes lasting hours that ended spontaneously. **This was an
 artifact of the v1 detector** (see the banner at the top). With the per-channel
 median, the pedestal persists. Every clear case reads clean again only after a
 multi-day-to-multi-month gap consistent with service: BE13117 6 days, BE18438
 24 days, BE9558 63 days **with a confirmed Instantel calibration inside the
 gap**. BE12599 never reads clean — it is still faulting at the end of the
 archive. This matches the operational experience: once a unit develops an
 offset it stays broken until the geophone is replaced.
 ### "Offsets develop N months after calibration" — CONFOUNDED, NOT A FINDING
 Tempting, and it looked strong:
 | unit | suspect before latest cal | after |
 |---|---|---|
 | BE18438 | 0 / 196 | 87 / 97 |
 | BE12599 | 0 / 62 | 6 / 15 |
 | BE11529 | 0 / 82 | 4 / 17 |
 | BE9558 | 38 / 189 | 0 / 7 |
 But bucketing suspects by months-since-calibration gives **one unit per bucket**:
 `0–3mo={BE11529}`, `3–6 & 6–9mo={BE18438}`, `9–12mo={BE9558}`,
 `12–15mo={BE12599}`. The apparent "51% failure rate at 6–9 months" is entirely
 BE18438's single February 2026 episode. Five units with roughly one episode
 each cannot support a population trend. **Do not re-derive this.**
 Also note: all affected units are calibrated on a **~12–13 month cadence**, so
 "sent to Instantel" is the routine annual schedule, not evidence of a
 fault-driven return.
 ---
 ## 7. Open questions
 ### Q1 — Is it a latched bad zero or analog degradation?
 The question that decides everything. A latched zero is correctable (possibly
 over the wire); degradation means a repair. Instantel's 2027–2069 rule implies
 *both* populations exist, with the split roughly 10/90 in the field.
 **BE12599 is the natural experiment** — faulting as of 2026-08-23. Read its
 values, run the autozero, read them again.
 ### Q2 — Can we read the autozero numbers over the wire? (best lead)
 Instantel says to check *"the lists of numbers in the **X1 and X8 gains**"* —
 4 sensors × 2 gains = **8 channels**. The protocol reference already documents
 an unimplemented command with exactly that shape:
 ```
 SUB 0x0E  ->  RSP 0xF1     "channel sensor data"
  2-step read; channel selector in params[6:8] = 0x0000..0x0007
  data length 0x0A (10 bytes) per channel
 ```
 Blastware's *Unit Channel Test* sequence:
 `POLL×N → 0x15 → 0x01 → 0x08 → 0x01 → 0x0E×8 → 0x98×2 → 0x0E×8`
 — note the **second `0x0E` pass carries live ADC readings**.
 **Hypothesis (untested):** `0x0E` returns the numbers Instantel wants compared
 against 2027–2069. If true, SFM could diagnose an offset remotely *and* predict
 whether a re-zero will succeed — converting a 10%/90% shipping gamble into a
 decision made before packing a box.
 **How to test.** `bridges/ach_mitm.py` is a generic TCP proxy:
 ```bash
 python bridges/ach_mitm.py --bw-host <MODEM_IP> --bw-port 9034 --listen-port 9999
 ```
 Point Blastware at the proxy and run **Unit Channel Test**.
 ⚠ In this topology the output filenames are reversed — the tool labels the
 *connecting* side "unit", so `raw_s3_*.bin` holds Blastware's bytes and
 `raw_bw_*.bin` the unit's.
 Capture priority: (1) BE12599 while faulting, (2) a known-good unit as control,
 (3) before/after an autozero on the same unit. Eight 10-byte payloads with an
 expected value near 2048 is a very constrained puzzle.
 ### Q3 — What is the mild-offset rate?
 Unmeasurable from event files (§2). Only the A/D-mode check sees it. Would
 need a fleet sweep in A/D mode, or Q2 to succeed.
 ### Q4 — Does an offset recur on the same unit after service?
 BE9558 shows episodes in 2020 and 2026; BE18438 twice in four months. Suggestive
 of recurrence, but service records aren't in the data — only calibration dates.
 ---
 ## 8. Data and tooling
 | what | where |
 |---|---|
 | detector | `scratch/offset_scan.py` |
 | current results | `/home/serversdown/dl2-archive/offset_archive.csv` |
 | earlier candidate list (Terra-View DB, 274 events) | `scratch/offset_candidates.csv` |
 | archive working copy | `/home/serversdown/dl2-archive/files/` |
 | archive source | NAS `DeathStar` 10.0.0.2, `/volume1/Uploads/TMI/DL2-Event-backup-8-25-26/Event/autocall home/` |
 ⚠ **The DL2 export keeps a byte-identical `Sent/` mirror of its root.** 13,077
 waveform paths are 6,577 distinct files. Always dedupe by basename — this
 doubled two reported figures before it was caught.
 ---
 ## 8b. The histogram corpus — the other 90% of the archive (2026-09-04)
 Every result above §8 comes from **waveform** files. `offset_scan3.py` filters on
 `\.[A-Za-z0-9]{2}0[Ww]$`, so the corpus it scanned is 6,577 unique binaries. The
 archive also holds **63,535 unique histograms** — 9.7x more files — which the
 pre-trigger method cannot touch, because a histogram carries no samples: only a
 per-interval, per-channel peak and half-period.
 `scratch/offset_hist_scan.py` scans them. **63,505 of 63,535 decoded (99.95%),
 43 units, 77.9M intervals.** Two of the 45 units have no histograms at all.
 Output: `/home/serversdown/dl2-archive/offset_hist.csv` (190,515 channel-rows).
 ### The premise, and how far it actually holds
 A histogram file is hours of continuous monitoring, so most of its intervals are
 definitionally quiet, and a channel parked off zero cannot report a peak below
 its own displacement. The signal is real — two within-unit contrasts, siblings
 unmoved in both:
 | unit | channel | in-episode floor | outside | waveform \|pre\| same window |
 |---|---|---|---|---|
 | BE18438 | Vert | 0.0350 | 0.0050 | +0.18 .. +0.37 |
 | BE12599 | Tran | 0.0250 | 0.0050 | +0.03 .. +0.49 |
 But the **leakage from a waveform pedestal into the histogram floor is bimodal,
 not merely partial**: measured ratio ~0.9 on BE18438 Vert, ~0.7 on BE9558,
 **~0.02 on BE12599** — two orders of magnitude on one instrument. The device
 evidently measures each interval peak against a running baseline, and how much
 DC survives that varies per unit. **Consequence: a negative histogram result
 carries almost no information.** Do not read "clean in the histograms" as clean.
 ### The detector that survived
    dmin(file, ch) = min[ch] - min over the other two geo channels, SAME file
    gates (both hard):  n_intervals >= 60   AND   mic_p5 <= 5 raw counts
    day statistic:      median of dmin over that day's qualifying files
    flag day at        dmin >= 0.020 in/s (4 A/D counts)
    episode at         >= 3 CONSECUTIVE observed days
 **Result: BE18438|Vert, BE9558|Tran, BE9558|Long.** Threshold-insensitive —
 the journal's own test for a real signal against a tuned one — and this is the
 first operating point in the investigation that passes it cleanly. The identical
 answer holds across: statistic `min` or `p5`; length gate 10/30/60/120/300; mic
 gate 3/5/8/10; threshold 0.015–0.035 (a 2.3x span); persistence K = 2,3,4,5,7.
 Separation, ranked by highest floor sustained over 3 consecutive gated days
 across all 135 unit-channels:
 | unit-channel | best3 |
 |---|---|
 | BE18438 Vert | 0.1650 |
 | BE9558 Long | 0.0350 |
 | BE9558 Tran | 0.0250 |
 | *(2.5x gap)* | |
 | BE7145 Tran | 0.0100 |
 | entire rest of fleet | <= 0.0050 (one quantisation count) |
 Day-level false alarm: **37 of 99,432 gated unit-channel-days = 0.037%.**
 ### What it does NOT do — read this before trusting it
 - **It finds 2 of the 5 confirmed units, not 5.** The site-quiet gate is what
  makes it work and it is also what costs BE11529 and BE12599. BE11529's
  four-day single-axis ramp (Tran 0.025 -> 0.055, both siblings pinned at 0.005)
  is the most offset-shaped thing in the corpus outside the two detections, and
  the gate discards it.
 - **The positive class is two units.** Every threshold here is fitted to
  BE18438 and BE9558, which contribute 22 of the 37 flagged days in the entire
  corpus. No cross-validation is possible at n=2.
 - **Per-channel attribution is NOT established.** Rotating the three geo channel
  labels within each file — preserving every value, file and day, destroying
  only channel identity — reproduces the episode *count* with p = 0.769 and the
  label agreement at p = 0.038–0.077. Report a **unit and a window**; do not
  name a geophone axis on the strength of this detector alone.
 - **Timing resolution is ~1 month, not ~1 day.** A 30-day label shift still
  scores 2 of 9 episode hits; the signal dies only past ~60 days. The day-level
  series look far crisper than they are.
 - **Ground truth here is a sibling detector, not a service record.** Agreement
  between the two corpora is corroboration of a shared method. Nothing in this
  section has been checked against an actual repair, calibration or RMA.
 ### Dead ends — keep these dead
 - **Absolute floor (min / p1 / p5 / p10 / p25, thresholded alone) — RETIRED.**
  Not fleet-comparable and mostly not about the channel. Scoring each cell using
  *only the other two channels* — a statistic containing zero information about
  the suspect channel — reaches AUC 0.746 against the same labels, versus 0.872
  for the absolute floor itself. **66% of its apparent discrimination is "that
  day was noisy at that site."** Interval size alone moves its p99 7x (0.0350 at
  1 min vs 0.0050 at 2 s). And of all files with any channel above 0.025, 56.5%
  have **all three** channels above it — common-mode, i.e. the wrong physics.
 - **Zero-fraction — STRUCTURALLY IMPOSSIBLE, not merely weak.** The device never
  reports a zero histogram interval peak. The value is a max over hundreds of
  samples of a channel that always carries at least 1 count of noise, so it is
  clamped at 1 A/D count (0.005 in/s). There is no zero to count.
 - **Interval size, sample rate, geo range, firmware — refuted as confounds for
  the differential.** All four are *file-level scalars*: they move all three geo
  channels together, so they cannot produce a single-channel lift and the
  within-file differential is immune to them by construction. Geo range is
  identical across the three geo channels in **63,535 of 63,535** binaries.
  (Interval size remains fatal to the *absolute*-floor version, above.)
 ### Two findings that are independent of the histogram detector
 **1. `offset_scan3.py`'s `spread <= 0.02` gate is discarding real signal.**
 It rejects **113 of the 600 channel-rows with |pre| >= 0.025 (18.8%)**, and the
 rejections are not random — 92 of them fall across 41 unit-channels currently
 labelled NEGATIVE. Four would become sustained positives under an
 amplitude-only >=3-consecutive rule: **BE12599|Long (run of 8), BE18003|Vert
 (4), BE10895|Vert (3), BE12844|Tran (3).** Until this is re-cut, the fleet label
 is **three-state — POSITIVE / NEGATIVE / SPREAD-REJECTED(unknown)** — and the
 third state should be excluded from both TP and FP counts rather than silently
 scored as healthy. Every precision figure computed against the two-state label,
 in this section and in §3, is affected.
 **2. The waveform corpus sees ~7% of the days a unit was deployed.** 2,627
 (unit, day) observations against the histogram corpus's 35,105 — 13.4x — with a
 per-unit median ratio of 0.070. BE12599, a confirmed unit, is waveform-observed
 on 39 of its 1,666 histogram-observed days (**2.3%**). Any statement of the form
 "the fault was absent before date X" that rests on waveform coverage alone is
 much weaker than its event count suggests.
 ### BE10895 — reclassified (see also §4)
 Previously dismissed as a transient. The histogram record shows its **Vert**
 quiet-minute floor at 0.005 on 62/62 qualifying files from 2023-07-07, then
 0.010–0.015 on 48/58 files from 2023-08-03 to 08-27, while Tran moves on 2/58
 and Long on 9/58 and the site mic floor never leaves 1–3 counts. Independently,
 **42 of its 85 waveform events (49.4%) are single-axis-dominant** — one geo peak
 >= 10x both siblings and >= 0.05 in/s — the **highest rate in the 45-unit
 fleet** (BE13117 36.1%, BE18438 29.4%), and **100% of it on Vert**. Vert
 excursions of 0.1–1.5 in/s with Tran/Long at 0.005–0.035 are not ground motion.
 This is a genuine Vert-channel hardware fault, but **not the classic pedestal** —
 the differential is only one A/D count. Caveat: its entire histogram record is a
 single 52-day deployment ending 2023-08-27, so nothing says whether it
 persisted, was serviced, or resolved.
 The other six marginal units — BE11007, BE17354, BE18004, BE18104, BE9557,
 BE18003 — are **clean**. All seven cap at +0.005 to +0.007 (one A/D count)
 lifetime under the quiet-site gate, against +0.175 for BE18438 Vert and +0.062
 for BE9558 Long. Three individual waveform flags fall in windows with **zero**
 histogram coverage and are NO-DATA, not clean: BE18004|Tran 2024-10-16,
 BE9557|Tran 2021-06-28, BE9557|Vert 2025-06-12.
 ### Still open in this section
 - **The 11 thin-coverage units were not screened** (BE10202, BE11462, BE13779,
  BE15760, BE15957, BE16754, BE16758, BE8081, BE8626, BE9229, BE9887 — each
  under 20 waveform events, several with hundreds of histograms). This is the
  population most likely to hold a previously unknown offset, and it is the one
  slice of the plan that did not run. BE11462 was incidentally scored clean by
  the full-archive pass; BE10202 has no histogram files at all.
 - **No completeness audit was run** over the above.
 - Re-cutting the ground truth three-state (finding 1) and re-scoring everything
  against it.
 ---
 ## 9. Chronology
 | date | event |
 |---|---|
 | 2026-08-25 | Reported as a *waveform decode bug* — traces with a DC offset. Investigation shows the offset is **real device data**; the decoder is correct. |
 | 2026-08-25 | Brian relays his dad's description: a known hardware fault called an "offset"; usually sent to Instantel. |
 | 2026-08-25 | First detection pass over the Terra-View DB: **2 of 21 units**, 274 events, 11 months. Flagged as vulnerable to survivorship bias — flooded events were routinely deleted. |
 | 2026-08-25 | Condensation hypothesis proposed, then **killed by its own controls**. |
 | 2026-08-25 | Parked pending the multi-year archive. |
 | 2026-08-28 | DL2 archive pulled (33 GB, 546k files; 6.6 GB working set). |
 | 2026-08-28 | Archive scan: **6 of 45 units**, 283 events, 15 episodes. Prior base rate **confirmed**, not overturned. |
 | 2026-08-28 | Clipping ruled out; `m/p` established as the discriminator; BE11007 reclassified as probably a real blast. |
 | 2026-08-28 | Calibration-timing correlation attempted and **rejected as confounded**. |
 | 2026-08-28 | Instantel FAQs supplied: autozero procedure, the **2027–2069** window, the **>5 counts** threshold. Explains the ~10% re-zero success rate. |
 | 2026-08-28 | Bimodality established; sensor check proven **blind** to offsets; `SUB 0x0E` identified as the best open lead. |
 | 2026-08-28 | **v1 detector retracted.** Brian challenged the "come and go" finding against field experience. Two flaws found: dominant-axis-only scoring and mean-instead-of-median. Corrected detector shows persistent pedestals on **8 of 45 units**, and the gaps are service windows. |
 | 2026-08-28 | **Detector v3 (Brian's method):** pre-trigger floor + pre/mid/end consistency. Healthy channels proven to sit at 0.000 +/-1 unit (94.5%), confirming no decoder zero-point bias. Final: **5 of 45 units (11%)**, threshold-insensitive. |
 | 2026-09-04 | **Histogram corpus scanned** — 63,505 of 63,535 files, 43 units, 77.9M intervals (9.7x the waveform corpus). `scratch/offset_hist_scan.py`. |
 | 2026-09-04 | Absolute-floor statistic **retired**: 66% of its discrimination is a day/site confound (other-channels-only AUC 0.746 vs 0.872). Zero-fraction shown **structurally impossible** — the device clamps every interval peak at >= 1 count. |
 | 2026-09-04 | Site-quiet-gated cross-channel differential established: **BE18438 Vert, BE9558 Tran+Long**, threshold-insensitive over a 2.3x span. Finds only **2 of the 5** confirmed units — leakage into the histogram floor is bimodal (0.9 to 0.02), so a negative result carries almost no information. Per-channel attribution **not** established (channel-scramble p = 0.769). |
 | 2026-09-04 | **BE10895 reclassified** from transient to a genuine Vert fault of a different subtype — 49.4% single-axis-dominant events, the highest in the fleet, 100% on Vert. The other six marginal units are clean. |
 | 2026-09-04 | **Defect found in `offset_scan3.py`**: its `spread <= 0.02` gate discards 18.8% of rows with \|pre\| >= 0.025, concentrated on 41 negative unit-channels; 4 would be sustained positives without it. The fleet label is three-state, not two. |
@@ -30,6 +30,7 @@ from __future__ import annotations
 import datetime
 import logging
 import re
 import struct
 from typing import Optional
@@ -2532,10 +2533,17 @@ def _decode_0a_partial_header(raw_data: bytes, index: int, key4: bytes) -> Optio
        ts2 = try_ts(raw_data[ts1_end + 1:ts1_end + 1 + ts_size])
    # Extract serial and geo threshold from "BE11529\0" and "Geo: X.XXX in/s\0".
    #
    # Match any two-letter family prefix, not a literal "BE" — a BlastMate
    # reports "BA10895", and the old `find(b"BE")` returned -1 on one.  That
    # skipped this whole block, so the geo threshold went missing along with
    # the serial.  Requiring the NUL terminator in the pattern also makes the
    # match stricter than the bare two-byte search it replaces.
    serial: Optional[str] = None
    geo_ips: Optional[float] = None
-    serial_pos = raw_data.find(b"BE")
+    serial_match = re.search(rb"[A-Z]{2}\d{3,6}(?=\x00)", raw_data)
    serial_pos = serial_match.start() if serial_match else -1
    if serial_pos >= 0:
        # Read null-terminated serial starting at serial_pos.
        null_pos = raw_data.find(b"\x00", serial_pos)
@@ -50,7 +50,7 @@ SIDECAR_KIND   = "sfm.event"
 # bumped without a `pip install` re-run — leading to confusing stale
 # version stamps in sidecars.  Bump this constant and CHANGELOG.md
 # together at release time.
-TOOL_VERSION = "0.26.0"
+TOOL_VERSION = "0.28.0"
 try:
    # Best-effort: prefer the installed metadata when it's NEWER than the
@@ -413,10 +413,18 @@ def detect_multi_interval_stride(body: bytes) -> Optional[int]:
        if (_ctr(stride) - _ctr(0)) & 0xFFFF != 1:
            continue
-        # confirm on a third block when the body is long enough
+        # Confirm on a third block WHEN ONE IS ACTUALLY PRESENT.  A body can
-        if 2 * stride + _MULTI_HEADER_LEN <= len(body):
+        # be longer than two strides and still hold only two real blocks: a
-            if not _is_multi_header(body, 2 * stride):
+        # final *partial* block leaves trailing padding.  E.g. 51 intervals at
-                continue
+        # 2 s = one full 30-interval block + a 21-interval remainder, in a
        # 2787-byte body — long enough to demand a third header at 1224 that
        # does not exist.  Requiring it unconditionally threw away the correct
        # stride and the file decoded to nothing (BE18193 T193L0XM.CI0H).
        # The block-counter check above is the decisive anti-false-positive
        # test; this one is corroboration, so a missing third header means
        # end-of-stream, not disqualification.
        if (2 * stride + _MULTI_HEADER_LEN <= len(body)
                and _is_multi_header(body, 2 * stride)):
            if (_ctr(2 * stride) - _ctr(stride)) & 0xFFFF != 1:
                continue
        return stride
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "seismo-relay"
-version = "0.26.0"
+version = "0.27.0"
 description = "Python client and REST server for MiniMate Plus seismographs"
 requires-python = ">=3.10"
 dependencies = [
@@ -0,0 +1,234 @@
 #!/usr/bin/env python3
 """Offset detector — HISTOGRAM corpus (the other 90% of the archive).
 `offset_scan3.py` measures the pre-trigger floor in *waveform* samples.  That
 covers 6,577 of the archive's 70,112 unique series-3 files; the remaining
 63,535 are **histograms**, which carry no samples — only a per-interval,
 per-channel peak + half-period.  So the pre-trigger method cannot run on them.
 The histogram analogue of "the resting floor" is the **low percentile of the
 per-interval peaks**.  A histogram file is typically hours of continuous
 monitoring, so the great majority of its intervals are definitionally quiet;
 the bottom of that distribution is what the channel reads when nothing is
 happening.  A healthy channel bottoms out at 0.000-0.005 in/s.  A channel
 parked off zero cannot report a peak below its own displacement, so its floor
 is pinned up.
 ⚠ The DC leakage into the histogram peak is PARTIAL.  Measured within-unit
 against episodes already established from the waveform scan:
    BE18438  Vert  in-episode 0.0350 vs 0.0050 outside  (waveform pre = +0.18..+0.37)
    BE12599  Tran  in-episode 0.0250 vs 0.0050 outside  (waveform pre = +0.03..+0.49)
 so the device's per-interval peak is evidently measured against a running /
 AC-coupled baseline that removes most, but not all, of the DC.  The residual
 is real and channel-specific, but the margin is ~5 quantisation counts rather
 than the ~70 the waveform detector enjoys.  Do not carry the waveform
 detector's 0.025 in/s floor across unexamined — calibrate on the CSV.
 Because the absolute floor also moves with site noise (traffic, wind, a
 generator), the statistic that matters most is the **cross-channel
 differential**: a channel's floor minus the quietest of the other two geo
 channels in the same file.  Site noise lifts all three together and cancels;
 a DC offset lifts one.
 This script does not decide anything.  It emits every candidate statistic per
 (file, channel) so thresholds can be calibrated against the waveform-derived
 ground truth in `offset_v3.csv` rather than guessed.
 Usage:
    python scratch/offset_hist_scan.py --dir /home/serversdown/dl2-archive/files \
        --out /home/serversdown/dl2-archive/offset_hist.csv --jobs 4
 """
 from __future__ import annotations
 import argparse
 import csv
 import datetime
 import logging
 import re
 import statistics
 import sys
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.event_file_io import read_blastware_file  # noqa: E402
 GEO = ("Tran", "Vert", "Long")
 K = 10.0 / 32000.0          # ADC count -> in/s (see CLAUDE.md: full scale 32000)
 _HIST = re.compile(r"\.[A-Za-z0-9]{2}0[Hh]$")
 _STEM = re.compile(r"^([B-Z])(\d{3})")
 _B36 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"
 _SERIAL_RE = re.compile(rb"\b([A-Z]{2}\d{3,6})\b")
 def serial_of(name: str, path=None) -> str:
    """Real serial for a BW file.
    The filename encodes only the NUMBER: `<letter><3 digits>` where
    letter = chr(ord('B') + serial // 1000).  The two-letter family prefix
    ("BE", "BA", ...) is **not** in the filename, so it must be read out of
    the file body.  Four units in the DL2 archive are BA, not BE — assuming
    "BE" mislabels BA9229, BA10060, BA10895 and BA15957.
    """
    m = _STEM.match(name)
    if not m:
        return "?"
    num = (ord(m.group(1)) - ord("B")) * 1000 + int(m.group(2))
    if path is not None:
        try:
            for s in _SERIAL_RE.findall(Path(path).read_bytes()):
                s = s.decode()
                if s[2:].lstrip("0") == str(num):
                    return s
        except Exception:
            pass
    return f"BE{num}"          # last-resort fallback; prefix unverified
 def stem_time(name: str):
    """Decode the filename's base-36 timestamp.  Epoch 1985-01-01, 1296 s/tick.
    Preferred over the file's own footer timestamp only because it costs
    nothing; the caller falls back to the decoded event when this fails.
    """
    try:
        base, ext = name.rsplit(".", 1)
        n = 0
        for c in base[4:8].upper():
            n = n * 36 + _B36.index(c)
        ab = _B36.index(ext[0].upper()) * 36 + _B36.index(ext[1].upper())
        return datetime.datetime(1985, 1, 1) + datetime.timedelta(seconds=n * 1296 + ab)
    except Exception:
        return None
 def _pct(sorted_vals, q):
    """Nearest-rank percentile on an already-sorted list."""
    if not sorted_vals:
        return None
    i = min(len(sorted_vals) - 1, max(0, int(len(sorted_vals) * q / 100.0)))
    return sorted_vals[i]
 def scan(path_str: str):
    logging.disable(logging.WARNING)   # per-worker: the codec warns on undecodables
    p = Path(path_str)
    try:
        ev = read_blastware_file(p)
    except Exception:
        return None
    s = ev.raw_samples or {}
    if not any(s.get(c) for c in GEO):
        return None
    ts = stem_time(p.name) or ev.timestamp
    stamp = ""
    if ts is not None:
        stamp = (f"{ts.year:04d}-{ts.month:02d}-{ts.day:02d}T"
                 f"{ts.hour:02d}:{ts.minute:02d}:{ts.second:02d}")
    # Per-channel floor candidates, in in/s.
    stats = {}
    for ch in GEO:
        v = sorted(s.get(ch) or [])
        if not v:
            continue
        stats[ch] = {
            "n": len(v),
            "min": v[0] * K,
            "p1": _pct(v, 1) * K,
            "p5": _pct(v, 5) * K,
            "p10": _pct(v, 10) * K,
            "p25": _pct(v, 25) * K,
            "med": statistics.median(v) * K,
            "peak": v[-1] * K,
            "zeros": sum(1 for x in v if x == 0) / len(v),
        }
    if len(stats) < 2:      # need at least one sibling channel for the differential
        return None
    # Mic floor as a site-noise proxy (raw counts; the dB conversion is not
    # needed — only its relative movement matters here).
    mic = sorted(s.get("MicL") or [])
    mic_p5 = _pct(mic, 5) if mic else ""
    rows = []
    for ch, st in stats.items():
        others = [stats[o]["p5"] for o in stats if o != ch]
        rows.append({
            "serial": serial_of(p.name, p),
            "timestamp": stamp,
            "filename": p.name,
            "channel": ch,
            "n_intervals": st["n"],
            "min": round(st["min"], 4),
            "p1": round(st["p1"], 4),
            "p5": round(st["p5"], 4),
            "p10": round(st["p10"], 4),
            "p25": round(st["p25"], 4),
            "median": round(st["med"], 4),
            "peak": round(st["peak"], 4),
            "frac_zero": round(st["zeros"], 4),
            # the site-noise-cancelling statistic: this channel's floor above
            # the quietest sibling geo channel in the same file
            "diff_p5": round(st["p5"] - min(others), 4),
            "mic_p5": mic_p5,
        })
    return rows
 COLS = ["serial", "timestamp", "filename", "channel", "n_intervals",
        "min", "p1", "p5", "p10", "p25", "median", "peak", "frac_zero",
        "diff_p5", "mic_p5"]
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--dir", required=True)
    ap.add_argument("--out", required=True)
    ap.add_argument("--jobs", type=int, default=4)
    ap.add_argument("--limit", type=int, default=0, help="stop after N files (smoke test)")
    a = ap.parse_args()
    # Dedupe by basename — the DL2 export keeps a byte-identical `Sent/`
    # mirror of its root, which doubled two figures before it was caught.
    seen, files = set(), []
    for q in sorted(Path(a.dir).rglob("*")):
        if q.is_file() and _HIST.search(q.name) and q.name not in seen:
            seen.add(q.name)
            files.append(str(q))
    if a.limit:
        files = files[:a.limit]
    print(f"unique histogram binaries: {len(files)}", flush=True)
    rows, undecodable = [], 0
    with ProcessPoolExecutor(max_workers=a.jobs) as ex:
        futs = [ex.submit(scan, f) for f in files]
        for i, fut in enumerate(as_completed(futs), 1):
            r = fut.result()
            if r:
                rows.extend(r)
            else:
                undecodable += 1
            if i % 5000 == 0:
                print(f"  {i}/{len(files)}", flush=True)
    with open(a.out, "w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=COLS)
        w.writeheader()
        w.writerows(rows)
    files_ok = len({r["filename"] for r in rows})
    units = len({r["serial"] for r in rows})
    ivals = sum(r["n_intervals"] for r in rows) // 3
    print(f"\ndecoded {files_ok}/{len(files)} files "
          f"({undecodable} undecodable), {units} units, ~{ivals/1e6:.1f}M intervals")
    print(f"wrote {a.out}  ({len(rows)} channel-rows)")
 if __name__ == "__main__":
    main()
@@ -0,0 +1,171 @@
 #!/usr/bin/env python3
 """Scan series-3 waveform binaries for the 'offset' hardware fault.
 A healthy geophone trace is centred on zero.  An offset unit sits displaced,
 so the channel mean approaches its own peak.  Detector (unchanged from the
 2026-08-25 run, see memory note `offset-archive-analysis-backlog`):
    dominant-axis  |mean| / peak > 0.7
    AND            |mean| >= 0.9 * the unit's geo trigger level
 Trigger level is read from a paired _ASCII.TXT where one exists, otherwise
 from a per-serial median learned across that unit's ASCII files, otherwise
 --default-trigger.
 Serial is decoded from the BW filename: prefix letter encodes thousands
 (chr(ord('B') + n)), next 3 digits the remainder -- T193 -> BE18193.
 Usage:
  python scratch/offset_scan.py --dir <path> [--jobs N] --out offsets.csv
 """
 from __future__ import annotations
 import argparse, csv, json, re, sys
 from collections import defaultdict
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.event_file_io import read_blastware_file
 from minimateplus.bw_ascii_report import parse_report
 GEO = ("Tran", "Vert", "Long")
 _GEO_FS_COUNTS = 32000.0
 _WAVE_RE = re.compile(r"\.[A-Za-z0-9]{2}0[Ww]$")
 _STEM_RE = re.compile(r"^([B-Z])(\d{3})")
 MEAN_OVER_PEAK_MIN = 0.7
 TRIGGER_FRACTION   = 0.9
 def serial_from_name(name: str):
    m = _STEM_RE.match(name)
    if not m:
        return None
    letter, digits = m.group(1), m.group(2)
    return f"BE{(ord(letter) - ord('B')) * 1000 + int(digits)}"
 def counts_to_ips(c, gr):
    return c * (gr or 10.0) / _GEO_FS_COUNTS
 def scan_one(path_str: str, default_trigger: float) -> dict | None:
    p = Path(path_str)
    try:
        gr, trig = 10.0, None
        ap = p.with_name(p.name.replace(".", "_", 1) + "_ASCII.TXT") \
            if False else p.parent / (p.stem + "_" + p.suffix.lstrip(".") + "_ASCII.TXT")
        if ap.exists():
            rep = parse_report(ap.read_text(errors="replace"))
            gr = rep.geo_range_ips or 10.0
            trig = rep.geo_trigger_level_ips
        ev = read_blastware_file(p)
        s = ev.raw_samples or {}
        if not all(s.get(c) for c in GEO):
            return None
        best = None
        for ch in GEO:
            arr = s[ch]
            n = len(arr)
            if n == 0:
                continue
            mean = sum(arr) / n
            peak = max(abs(v) for v in arr)
            if peak == 0:
                continue
            ratio = abs(mean) / peak
            if best is None or peak > best["peak_counts"]:
                best = {"channel": ch, "mean_counts": mean,
                        "peak_counts": peak, "ratio": ratio}
        if best is None:
            return None
        ts = ev.timestamp
        return {
            "serial": serial_from_name(p.name) or "?",
            "timestamp": (f"{ts.year:04d}-{ts.month:02d}-{ts.day:02d}T"
                          f"{ts.hour:02d}:{ts.minute:02d}:{ts.second:02d}") if ts else "",
            "filename": p.name,
            "channel": best["channel"],
            "offset_ips": round(counts_to_ips(best["mean_counts"], gr), 4),
            "peak_ips": round(counts_to_ips(best["peak_counts"], gr), 4),
            "mean_over_peak": round(best["ratio"], 3),
            "trigger_level_ips": trig if trig is not None else "",
            "geo_range_ips": gr,
        }
    except Exception:
        return None
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--dir", required=True)
    ap.add_argument("--jobs", type=int, default=4)
    ap.add_argument("--limit", type=int, default=0)
    ap.add_argument("--default-trigger", type=float, default=0.2)
    ap.add_argument("--out", required=True)
    a = ap.parse_args()
    # The DL2 export keeps a byte-identical `Sent/` mirror of the root, so
    # enumerate paths but keep only the first occurrence of each basename —
    # otherwise every event is counted twice.
    seen = set()
    files = []
    for q in sorted(Path(a.dir).rglob("*")):
        if q.is_file() and _WAVE_RE.search(q.name) and q.name not in seen:
            seen.add(q.name)
            files.append(q)
    if a.limit:
        files = files[: a.limit]
    print(f"waveform binaries to scan: {len(files)}", flush=True)
    rows = []
    with ProcessPoolExecutor(max_workers=a.jobs) as ex:
        futs = [ex.submit(scan_one, str(p), a.default_trigger) for p in files]
        for n, f in enumerate(as_completed(futs), 1):
            r = f.result()
            if r:
                rows.append(r)
            if n % 2000 == 0:
                print(f"  {n}/{len(files)}", flush=True)
    # learn per-serial trigger levels from the rows that had an ASCII
    by_serial = defaultdict(list)
    for r in rows:
        if r["trigger_level_ips"] != "":
            by_serial[r["serial"]].append(float(r["trigger_level_ips"]))
    med = {}
    for k, v in by_serial.items():
        v.sort()
        med[k] = v[len(v) // 2]
    for r in rows:
        if r["trigger_level_ips"] == "":
            r["trigger_level_ips"] = med.get(r["serial"], a.default_trigger)
        r["suspect"] = int(
            r["mean_over_peak"] > MEAN_OVER_PEAK_MIN
            and abs(r["offset_ips"]) >= TRIGGER_FRACTION * float(r["trigger_level_ips"])
        )
    cols = ["serial", "timestamp", "filename", "channel", "offset_ips", "peak_ips",
            "mean_over_peak", "trigger_level_ips", "geo_range_ips", "suspect"]
    with open(a.out, "w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=cols)
        w.writeheader()
        w.writerows(rows)
    sus = [r for r in rows if r["suspect"]]
    print(f"\nscanned {len(rows)} decodable waveforms")
    print(f"suspect events: {len(sus)}")
    per = defaultdict(int)
    for r in sus:
        per[r["serial"]] += 1
    print(f"units with >=1 suspect event: {len(per)} of {len({r['serial'] for r in rows})}")
    for s, n in sorted(per.items(), key=lambda x: -x[1])[:20]:
        print(f"  {s:10} {n}")
    print(f"\nwrote {a.out}")
 if __name__ == "__main__":
    main()
@@ -0,0 +1,108 @@
 #!/usr/bin/env python3
 """Offset detector v2 — per-channel MEDIAN pedestal.
 Supersedes the dominant-axis / mean detector in offset_scan.py, which had two
 flaws that manufactured false "recoveries":
  1. It scored only the axis with the largest peak, so a real event on one axis
     hid a persistent pedestal on another.  BE12599 2026-08-21 read "clean"
     because Long had a 1.065 in/s event, while Tran sat at +0.47 in/s.
  2. It used the MEAN, which a real transient perturbs.  The median is the
     resting baseline: most samples sit at it, so a blast does not move it.
     Same event, Long: mean +0.0783 vs median -0.0050.
 Flags a CHANNEL when |median| >= --floor in/s (default 0.025 = 5 A/D counts,
 Instantel's own criterion; 1 A/D count = 0.005 in/s).
 Emits one row per (event, channel) so persistence can be tracked per channel.
 """
 from __future__ import annotations
 import argparse, csv, re, statistics, sys
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.event_file_io import read_blastware_file
 GEO = ("Tran", "Vert", "Long")
 K = 10.0 / 32000.0                      # ADC counts -> in/s at the 10 in/s range
 _WAVE_RE = re.compile(r"\.[A-Za-z0-9]{2}0[Ww]$")
 _STEM_RE = re.compile(r"^([B-Z])(\d{3})")
 def serial_from_name(n):
    m = _STEM_RE.match(n)
    return f"BE{(ord(m.group(1))-ord('B'))*1000+int(m.group(2))}" if m else "?"
 def scan_one(ps):
    p = Path(ps)
    try:
        ev = read_blastware_file(p)
        s = ev.raw_samples or {}
        if not all(s.get(c) for c in GEO):
            return None
        ts = ev.timestamp
        stamp = (f"{ts.year:04d}-{ts.month:02d}-{ts.day:02d}T"
                 f"{ts.hour:02d}:{ts.minute:02d}:{ts.second:02d}") if ts else ""
        out = []
        for ch in GEO:
            a = s[ch]
            out.append({
                "serial": serial_from_name(p.name), "timestamp": stamp,
                "filename": p.name, "channel": ch,
                "median_ips": round(statistics.median(a) * K, 4),
                "mean_ips":   round(statistics.fmean(a) * K, 4),
                "peak_ips":   round(max(abs(v) for v in a) * K, 4),
            })
        return out
    except Exception:
        return None
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--dir", required=True)
    ap.add_argument("--jobs", type=int, default=4)
    ap.add_argument("--floor", type=float, default=0.025)
    ap.add_argument("--out", required=True)
    a = ap.parse_args()
    seen, files = set(), []
    for q in sorted(Path(a.dir).rglob("*")):
        if q.is_file() and _WAVE_RE.search(q.name) and q.name not in seen:
            seen.add(q.name); files.append(str(q))
    print(f"unique waveform binaries: {len(files)}", flush=True)
    rows = []
    with ProcessPoolExecutor(max_workers=a.jobs) as ex:
        for n, f in enumerate(as_completed([ex.submit(scan_one, p) for p in files]), 1):
            r = f.result()
            if r: rows.extend(r)
            if n % 2000 == 0: print(f"  {n}/{len(files)}", flush=True)
    for r in rows:
        r["offset"] = int(abs(r["median_ips"]) >= a.floor)
    cols = ["serial","timestamp","filename","channel","median_ips","mean_ips","peak_ips","offset"]
    with open(a.out, "w", newline="") as fh:
        w = csv.DictWriter(fh, fieldnames=cols); w.writeheader(); w.writerows(rows)
    from collections import defaultdict
    ev_flagged = {(r["serial"], r["filename"]) for r in rows if r["offset"]}
    ev_all     = {(r["serial"], r["filename"]) for r in rows}
    per = defaultdict(set)
    for r in rows:
        if r["offset"]: per[r["serial"]].add(r["filename"])
    tot = defaultdict(set)
    for r in rows: tot[r["serial"]].add(r["filename"])
    print(f"\nfloor = {a.floor} in/s ({a.floor/0.005:.0f} A/D counts)")
    print(f"events with >=1 offset channel: {len(ev_flagged)} of {len(ev_all)}")
    print(f"units affected: {len(per)} of {len(tot)}")
    for s in sorted(per, key=lambda s: -len(per[s])):
        print(f"  {s:9} {len(per[s]):4} / {len(tot[s]):4} events")
    print(f"\nwrote {a.out}")
 if __name__ == "__main__":
    main()
@@ -0,0 +1,117 @@
 #!/usr/bin/env python3
 """Offset detector v3 — pre-trigger floor, with pre/mid/end consistency.
 Brian's method, and better than v2's whole-record median for one reason: the
 pre-trigger window is *definitionally* quiet (it is the buffer captured before
 the trigger fired), whereas a whole-record median is merely robust to the event.
 Per channel:
    pre  = median of the first `pretrig_samples` samples (STRT record)
    mid  = median of the middle third
    end  = median of the final third
    spread = max(pre,mid,end) - min(pre,mid,end)
 A DC offset is a *constant floor*: |pre| at or above the floor AND a small
 spread. A transient (settling, handling, a long-tailed event) moves one segment
 relative to the others and is rejected by the spread test.
 Floor default 0.025 in/s = 5 A/D counts (Instantel's own criterion; 1 count =
 0.005 in/s).  Quantisation is 0.005 in/s, so `spread` is measured in units of it.
 """
 from __future__ import annotations
 import argparse, csv, re, statistics, sys
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.event_file_io import read_blastware_file
 GEO=("Tran","Vert","Long"); K=10.0/32000.0
 _WAVE=re.compile(r"\.[A-Za-z0-9]{2}0[Ww]$"); _STEM=re.compile(r"^([B-Z])(\d{3})")
 _SERIAL_RE = re.compile(rb"\b([A-Z]{2}\d{3,6})\b")
 def serial_of(name: str, path=None) -> str:
    """Real serial for a BW file.
    The filename encodes only the NUMBER: `<letter><3 digits>` where
    letter = chr(ord('B') + serial // 1000).  The two-letter family prefix
    ("BE", "BA", ...) is **not** in the filename, so it must be read out of
    the file body.  Four units in the DL2 archive are BA, not BE — assuming
    "BE" mislabels BA9229, BA10060, BA10895 and BA15957.
    """
    m = _STEM.match(name)
    if not m:
        return "?"
    num = (ord(m.group(1)) - ord("B")) * 1000 + int(m.group(2))
    if path is not None:
        try:
            for s in _SERIAL_RE.findall(Path(path).read_bytes()):
                s = s.decode()
                if s[2:].lstrip("0") == str(num):
                    return s
        except Exception:
            pass
    return f"BE{num}"          # last-resort fallback; prefix unverified
 def scan(ps):
    p=Path(ps)
    try:
        ev=read_blastware_file(p); s=ev.raw_samples or {}
        if not all(s.get(c) for c in GEO): return None
        pre_n=ev.pretrig_samples
        ts=ev.timestamp
        stamp=(f"{ts.year:04d}-{ts.month:02d}-{ts.day:02d}T"
               f"{ts.hour:02d}:{ts.minute:02d}:{ts.second:02d}") if ts else ""
        out=[]
        for ch in GEO:
            a=s[ch]; n=len(a); t=n//3
            pre = a[:pre_n] if (pre_n and 0 < pre_n < n) else a[:t]
            mid, end = a[t:2*t], a[2*t:]
            if not pre or not mid or not end: continue
            v=[statistics.median(x)*K for x in (pre,mid,end)]
            out.append({"serial":serial_of(p.name, p),"timestamp":stamp,
                        "filename":p.name,"channel":ch,
                        "pretrig_n": pre_n or 0,
                        "pre":round(v[0],4),"mid":round(v[1],4),"end":round(v[2],4),
                        "spread":round(max(v)-min(v),4),
                        "peak":round(max(abs(x) for x in a)*K,4)})
        return out
    except Exception:
        return None
 def main():
    ap=argparse.ArgumentParser()
    ap.add_argument("--dir",required=True); ap.add_argument("--jobs",type=int,default=4)
    ap.add_argument("--floor",type=float,default=0.025)
    ap.add_argument("--max-spread",type=float,default=0.02)
    ap.add_argument("--out",required=True)
    a=ap.parse_args()
    seen=set(); files=[]
    for q in sorted(Path(a.dir).rglob("*")):
        if q.is_file() and _WAVE.search(q.name) and q.name not in seen:
            seen.add(q.name); files.append(str(q))
    print(f"unique waveform binaries: {len(files)}",flush=True)
    rows=[]
    with ProcessPoolExecutor(max_workers=a.jobs) as ex:
        for i,f in enumerate(as_completed([ex.submit(scan,p) for p in files]),1):
            r=f.result()
            if r: rows.extend(r)
            if i%2000==0: print(f"  {i}/{len(files)}",flush=True)
    for r in rows:
        r["offset"]=int(abs(r["pre"])>=a.floor and r["spread"]<=a.max_spread)
    cols=["serial","timestamp","filename","channel","pretrig_n","pre","mid","end","spread","peak","offset"]
    with open(a.out,"w",newline="") as fh:
        w=csv.DictWriter(fh,fieldnames=cols); w.writeheader(); w.writerows(rows)
    from collections import defaultdict
    per=defaultdict(set); tot=defaultdict(set)
    for r in rows:
        tot[r["serial"]].add(r["filename"])
        if r["offset"]: per[r["serial"]].add(r["filename"])
    print(f"\nfloor={a.floor} in/s ({a.floor/0.005:.0f} counts)  max spread={a.max_spread}")
    print(f"units affected: {len(per)} of {len(tot)}")
    for s in sorted(per,key=lambda s:-len(per[s])):
        print(f"  {s:9} {len(per[s]):4} / {len(tot[s]):4} events")
    print(f"\nwrote {a.out}")
 if __name__=="__main__": main()
@@ -0,0 +1,221 @@
 #!/usr/bin/env python3
 """Verify the series-3 decoder against preserved Blastware ASCII exports.
 Pairs each `<stem>_<ext>_ASCII.TXT` with its binary `<stem>.<ext>`, decodes the
 binary with the production codec, and compares against BW's own export:
  waveform  — per-channel sample counts, then every sample value
  histogram — interval count, then every per-interval channel peak
 ADC counts convert as  ips = counts * geo_range_ips / 32000  (1 decoder unit =
 16 counts = 0.005 in/s at the 10 in/s range; see CLAUDE.md).
 Usage:
  python scratch/verify_against_ascii.py --dir <path> [--limit N] [--jobs N]
                                         [--out results.json] [--kind w|h|all]
 """
 from __future__ import annotations
 import argparse, json, re, sys, traceback
 from concurrent.futures import ProcessPoolExecutor, as_completed
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.event_file_io import read_blastware_file
 from minimateplus.bw_ascii_report import parse_report_file, parse_report
 GEO = ("Tran", "Vert", "Long")
 _GEO_FS_COUNTS = 32000.0
 _ASCII_SUFFIX_RE = re.compile(r"_ASCII\.TXT$", re.IGNORECASE)
 def binary_for(ascii_path: Path) -> Path:
    """H907KXOW_WC0H_ASCII.TXT -> H907KXOW.WC0H"""
    stem = _ASCII_SUFFIX_RE.sub("", ascii_path.name)
    if "_" not in stem:
        return ascii_path.with_name(stem)
    head, _, ext = stem.rpartition("_")
    return ascii_path.with_name(f"{head}.{ext}")
 def counts_to_ips(counts, geo_range_ips):
    r = geo_range_ips if geo_range_ips else 10.0
    return counts * r / _GEO_FS_COUNTS
 def agrees(got, exp, geo_range_ips, tol=0.0006):
    """True when decoded `got` matches BW's exported `exp`.
    Saturation carve-out: when an event clips, BW clamps its export to the
    channel's range maximum (and writes OORANGE for the summary PPV), while
    the decoder faithfully reproduces raw counts that can sit a decoder unit
    or two past nominal full scale (32016 counts observed = 10.005 in/s on
    the 10 in/s range).  Same sign and both at/above the ceiling is agreement,
    not a decode error.
    """
    if abs(got - exp) <= tol:
        return True
    r = geo_range_ips if geo_range_ips else 10.0
    if abs(exp) >= r - tol and abs(got) >= r - tol and (got >= 0) == (exp >= 0):
        return True
    return False
 def parse_interval_table(text: str):
    """Histogram interval rows: time, Tpk, Tfq, Vpk, Vfq, Lpk, Lfq, PVS, ..., micdB, micfq"""
    rows = []
    seen_header = False
    for line in text.splitlines():
        if "\t" not in line:
            continue
        cols = [c.strip().strip('"') for c in line.split("\t")]
        cols = [c for c in cols if c != ""]
        if not seen_header:
            if any(c in ("Tran", "Vert", "Long") for c in cols):
                seen_header = True
            continue
        if len(cols) < 7:
            continue
        if not re.match(r"^\d{1,2}:\d{2}:\d{2}$", cols[0]):
            continue
        def num(s):
            try:
                return float(s)
            except ValueError:
                return None
        rows.append({"time": cols[0], "Tran": num(cols[1]),
                     "Vert": num(cols[3]), "Long": num(cols[5])})
    return rows
 def check_one(ascii_path_str: str) -> dict:
    ap = Path(ascii_path_str)
    bp = binary_for(ap)
    res = {"ascii": ap.name, "binary": bp.name, "status": "?",
           "kind": None, "detail": ""}
    try:
        if not bp.exists():
            res["status"] = "no_binary"
            return res
        text = ap.read_text(errors="replace")
        rep = parse_report(text, parse_samples=True)
        ev = read_blastware_file(bp)
        gr = rep.geo_range_ips
        res["kind"] = kind = ("histogram"
                              if (rep.event_type or "").lower().startswith(("full histogram", "histogram"))
                              else "waveform")
        samples = ev.raw_samples or {}
        dec_n = {c: len(samples.get(c) or []) for c in GEO}
        if kind == "histogram":
            rows = parse_interval_table(text)
            res["n_ascii"] = len(rows)
            res["n_decoded"] = dec_n["Tran"]
            if not rows:
                res["status"] = "no_ascii_table"
                return res
            if dec_n["Tran"] == 0:
                res["status"] = "decode_empty"
                return res
            if dec_n["Tran"] != len(rows):
                res["status"] = "count_mismatch"
                res["detail"] = f"decoded {dec_n['Tran']} vs ascii {len(rows)}"
                return res
            bad = 0
            worst = 0.0
            for i, row in enumerate(rows):
                for ch in GEO:
                    exp = row[ch]
                    if exp is None:
                        continue
                    got = counts_to_ips(samples[ch][i], gr)
                    if not agrees(got, exp, gr):
                        bad += 1
                        worst = max(worst, abs(got - exp))
            res["worst_abs"] = round(worst, 6)
            res["status"] = "exact" if bad == 0 else "value_mismatch"
            if bad:
                res["detail"] = f"{bad} interval-channel values off"
            return res
        # waveform
        asc = rep.samples or []
        res["n_ascii"] = len(asc)
        res["n_decoded"] = dec_n["Tran"]
        if not asc:
            res["status"] = "no_ascii_table"
            return res
        if dec_n["Tran"] == 0:
            res["status"] = "decode_empty"
            return res
        if len({dec_n[c] for c in GEO}) != 1:
            res["status"] = "channel_len_mismatch"
            res["detail"] = str(dec_n)
            return res
        if dec_n["Tran"] != len(asc):
            res["status"] = "count_mismatch"
            res["detail"] = f"decoded {dec_n['Tran']} vs ascii {len(asc)}"
            return res
        bad = 0
        worst = 0.0
        for i, quad in enumerate(asc):
            for j, ch in enumerate(GEO):
                exp = quad[j]
                got = counts_to_ips(samples[ch][i], gr)
                if not agrees(got, exp, gr):
                    bad += 1
                    worst = max(worst, abs(got - exp))
        res["worst_abs"] = round(worst, 6)
        res["status"] = "exact" if bad == 0 else "value_mismatch"
        if bad:
            res["detail"] = f"{bad} sample values off"
        return res
    except Exception as e:
        res["status"] = "error"
        res["detail"] = f"{type(e).__name__}: {e}"
        return res
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--dir", required=True)
    ap.add_argument("--limit", type=int, default=0)
    ap.add_argument("--jobs", type=int, default=8)
    ap.add_argument("--kind", choices=["w", "h", "all"], default="all")
    ap.add_argument("--out", default=None)
    a = ap.parse_args()
    root = Path(a.dir)
    files = sorted(p for p in root.rglob("*")
                   if p.is_file() and p.name.upper().endswith("_ASCII.TXT"))
    if a.kind != "all":
        want = "0W" if a.kind == "w" else "0H"
        files = [p for p in files
                 if _ASCII_SUFFIX_RE.sub("", p.name).upper().endswith(want)]
    if a.limit:
        files = files[: a.limit]
    print(f"pairs to check: {len(files)}", flush=True)
    out = []
    from collections import Counter
    tally = Counter()
    with ProcessPoolExecutor(max_workers=a.jobs) as ex:
        futs = {ex.submit(check_one, str(p)): p for p in files}
        for n, f in enumerate(as_completed(futs), 1):
            r = f.result()
            out.append(r)
            tally[(r["kind"], r["status"])] += 1
            if n % 500 == 0:
                print(f"  {n}/{len(files)}", flush=True)
    print("\n=== results ===")
    for (kind, status), n in sorted(tally.items(), key=lambda x: -x[1]):
        print(f"  {str(kind):10} {status:22} {n}")
    if a.out:
        Path(a.out).write_text(json.dumps(out, indent=1))
        print(f"\nwrote {a.out}")
 if __name__ == "__main__":
    main()
@@ -1,12 +1,12 @@
 #!/usr/bin/env python3
-"""Backfill events.shape_* from each event's .h5 waveform samples. Idempotent."""
+"""Backfill events.shape_* and shape_offset_* from each event's .h5 samples. Idempotent."""
 from __future__ import annotations
 import argparse, logging, sys
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from sfm.database import SeismoDb
 from sfm.waveform_store import WaveformStore
-from sfm.shape_metrics import shape_from_h5
+from sfm.shape_metrics import shape_from_h5, offset_from_h5
 log = logging.getLogger("backfill_event_shape")
@@ -21,6 +21,7 @@ def backfill_shape(db: SeismoDb, store: WaveformStore, *, dry_run: bool = False)
        if not h5_path.exists():
            counts["skipped_no_h5"] += 1; continue
        shape = shape_from_h5(h5_path)
        offset = offset_from_h5(h5_path)
        if shape is None:
            # The .h5 can no longer yield a shape (fewer than 2 samples, or a
            # flat trace).  Clear any previously stored value rather than
@@ -28,22 +29,32 @@ def backfill_shape(db: SeismoDb, store: WaveformStore, *, dry_run: bool = False)
            # from and silently feeds the false-trigger detector.  Seen after
            # a decoder fix shrinks an event: 493 rows in the prod snapshot
            # were carrying metrics from a superseded decode (2026-08-25).
-            if row.get("shape_crest_factor") is not None:
+            if (row.get("shape_crest_factor") is not None
                    or row.get("shape_offset") is not None):
                if not dry_run:
                    with db._connect() as conn:
                        conn.execute(
                            "UPDATE events SET shape_crest_factor=NULL, "
                            "shape_near_peak_count=NULL, shape_sample_count=NULL, "
-                            "shape_axis=NULL WHERE id=?", (row["id"],))
+                            "shape_axis=NULL, shape_offset=NULL, shape_offset_axis=NULL, "
                            "shape_offset_pre=NULL, shape_offset_spread=NULL WHERE id=?",
                            (row["id"],))
                counts["cleared_stale"] += 1
            counts["skipped_no_samples"] += 1; continue
        if not dry_run:
            with db._connect() as conn:
                conn.execute(
                    "UPDATE events SET shape_crest_factor=?, shape_near_peak_count=?, "
-                    "shape_sample_count=?, shape_axis=? WHERE id=?",
+                    "shape_sample_count=?, shape_axis=?, shape_offset=?, "
                    "shape_offset_axis=?, shape_offset_pre=?, shape_offset_spread=? "
                    "WHERE id=?",
                    (shape["crest_factor"], shape["near_peak_count"],
-                     shape["sample_count"], shape["axis"], row["id"]))
+                     shape["sample_count"], shape["axis"],
                     (1 if offset["offset"] else 0) if offset else None,
                     offset["axis"] if offset else None,
                     offset["pre"] if offset else None,
                     offset["spread"] if offset else None,
                     row["id"]))
        counts["updated"] += 1
    log.info("backfill_shape: %s", counts)
    return counts
@@ -82,6 +82,7 @@ CREATE TABLE IF NOT EXISTS events (
    record_type             TEXT,                   -- "single_shot" | "continuous"
    false_trigger           INTEGER NOT NULL DEFAULT 0,  -- 0=no, 1=yes (manual flag)
    reviewed_real           INTEGER NOT NULL DEFAULT 0,  -- 0=no, 1=operator-confirmed real (mutually exclusive with false_trigger)
    false_trigger_reason    TEXT,                   -- optional FT cause ("offset", ...); NULL = none. Only meaningful when false_trigger=1.
    blastware_filename      TEXT,                   -- event file within waveform store; extension is per-event (AB0T encodes timestamp)
    blastware_filesize      INTEGER,                -- bytes; NULL if no event file saved
    a5_pickle_filename      TEXT,                   -- "<filename>.a5.pkl" sidecar
@@ -99,6 +100,10 @@ CREATE TABLE IF NOT EXISTS events (
    shape_near_peak_count   INTEGER,                -- samples >= 0.5 * peak (FT: few; real: many)
    shape_sample_count      INTEGER,                -- total samples (to normalize near_peak_count)
    shape_axis              TEXT,                   -- geophone channel measured ("Tran"/"Vert"/"Long")
    shape_offset            INTEGER,                -- 1 = DC-offset false trigger (pre-trigger baseline off zero + flat). Meaningful for waveforms only.
    shape_offset_axis       TEXT,                   -- geo channel the offset was measured on
    shape_offset_pre        REAL,                   -- pre-trigger baseline median (in/s)
    shape_offset_spread     REAL,                   -- max(pre,mid,end) - min(...) in in/s; small = constant/DC
    created_at              TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
    UNIQUE(serial, timestamp)
 );
@@ -225,7 +230,12 @@ class SeismoDb:
            ("shape_near_peak_count", "INTEGER"),
            ("shape_sample_count",    "INTEGER"),
            ("shape_axis",            "TEXT"),
            ("shape_offset",          "INTEGER"),
            ("shape_offset_axis",     "TEXT"),
            ("shape_offset_pre",      "REAL"),
            ("shape_offset_spread",   "REAL"),
            ("reviewed_real",         "INTEGER NOT NULL DEFAULT 0"),
            ("false_trigger_reason",  "TEXT"),
        ):
            if col not in existing_cols:
                log.info("_migrate: events ADD COLUMN %s %s", col, ddl)
@@ -430,9 +440,11 @@ class SeismoDb:
                             tran_zc_above_range, vert_zc_above_range,
                             long_zc_above_range, mic_zc_above_range,
                             shape_crest_factor, shape_near_peak_count,
-                             shape_sample_count, shape_axis)
+                             shape_sample_count, shape_axis,
                             shape_offset, shape_offset_axis,
                             shape_offset_pre, shape_offset_spread)
                        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
-                                ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                                ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
                        """,
                        (
                            self._new_id(), serial, key, session_id, ts,
@@ -464,6 +476,10 @@ class SeismoDb:
                            rec.get("shape_near_peak_count"),
                            rec.get("shape_sample_count"),
                            rec.get("shape_axis"),
                            rec.get("shape_offset"),
                            rec.get("shape_offset_axis"),
                            rec.get("shape_offset_pre"),
                            rec.get("shape_offset_spread"),
                        ),
                    )
                    inserted += 1
@@ -517,7 +533,11 @@ class SeismoDb:
                               shape_crest_factor    = COALESCE(?, shape_crest_factor),
                               shape_near_peak_count = COALESCE(?, shape_near_peak_count),
                               shape_sample_count    = COALESCE(?, shape_sample_count),
-                               shape_axis            = COALESCE(?, shape_axis)
+                               shape_axis            = COALESCE(?, shape_axis),
                               shape_offset          = COALESCE(?, shape_offset),
                               shape_offset_axis     = COALESCE(?, shape_offset_axis),
                               shape_offset_pre      = COALESCE(?, shape_offset_pre),
                               shape_offset_spread   = COALESCE(?, shape_offset_spread)
                         WHERE serial = ? AND timestamp = ?
                        """,
                        (
@@ -549,6 +569,10 @@ class SeismoDb:
                            rec.get("shape_near_peak_count") if rec else None,
                            rec.get("shape_sample_count")    if rec else None,
                            rec.get("shape_axis")            if rec else None,
                            rec.get("shape_offset")          if rec else None,
                            rec.get("shape_offset_axis")     if rec else None,
                            rec.get("shape_offset_pre")      if rec else None,
                            rec.get("shape_offset_spread")   if rec else None,
                            serial,
                            ts,
                        ),
@@ -603,61 +627,116 @@ class SeismoDb:
            ).fetchall()
        return [dict(r) for r in rows]
-    def find_twins(self, event_id: str, *, window_seconds: int = 300) -> list[dict]:
+    def find_twins(self, event_id: str, *, window_seconds: int | None = None) -> list[dict]:
        """
-        Find this event's histogram/waveform twin(s): rows sharing the same
+        Find this event's histogram/waveform twin(s): the SAME physical event
-        serial and an identical peak_vector_sum, whose timestamp falls
+        recorded both as a scheduled histogram and as a triggered waveform.
        within ``window_seconds`` of this event's timestamp.  Excludes the
        event itself.  Returns [] if the event or any required field
        (serial / peak_vector_sum / timestamp) is missing.
-        Caveat: identical-PVS matching is a proxy for "same physical event
+        A real trigger is captured twice — once as a triggered waveform (stamped
-        recorded twice," not a guarantee. In the rare case where the device
+        at the trigger instant) and once inside the scheduled histogram whose
-        clamps/saturates PVS (clamped to sqrt(3) * geo_range), two distinct
+        interval contains it (stamped at the histogram's interval start, e.g. the
-        saturated events on the same serial within the window can share the
+        7am/7pm call-in). The two can be HOURS apart in time yet report the same
-        same clamped PVS value and be matched as twins even though they are
+        serial and identical peak_vector_sum. Twins are therefore matched by:
-        different events. This is harmless in practice — false_trigger/
+
-        reviewed_real are derived/index columns re-derivable from the
+          * same serial,
-        sidecar source of truth — but worth knowing if twin counts look
+          * identical peak_vector_sum,
-        surprising on a saturated/clamped run.
+          * OPPOSITE record type (one histogram, one waveform), and
          * the waveform's timestamp falls within the histogram's interval —
            from a histogram's timestamp up to the next histogram (same serial).
        This replaces the old ±``window_seconds`` heuristic, which silently
        missed twins more than a few minutes apart (a histogram's interval-start
        stamp and the trigger instant routinely differ by hours). ``window_seconds``
        is still accepted for backward compatibility but is ignored.
        Returns [] if the event or a required field (serial / peak_vector_sum /
        timestamp) is missing.
        Caveat: identical-PVS matching remains a proxy for "same physical event"
        — if the device clamps/saturates PVS (to sqrt(3) * geo_range), two
        distinct saturated events could share a PVS. The added opposite-type and
        interval constraints make a false pairing far less likely than the old
        time-window match, and false_trigger/reviewed_real stay re-derivable from
        the sidecar source of truth.
        """
        def _parse(ts):
            if not ts:
                return None
            try:
                return datetime.datetime.fromisoformat(str(ts).replace(" ", "T"))
            except ValueError:
                return None
        def _is_hist(rt):
            return str(rt or "").lower().startswith("hist")
        row = self.get_event(event_id)
        if not row:
            return []
-        serial = row.get("serial"); pvs = row.get("peak_vector_sum"); ts = row.get("timestamp")
+        serial = row.get("serial"); pvs = row.get("peak_vector_sum")
-        if serial is None or pvs is None or not ts:
+        t_target = _parse(row.get("timestamp"))
        if serial is None or pvs is None or t_target is None:
            return []
-        try:
+        target_hist = _is_hist(row.get("record_type"))
            t = datetime.datetime.fromisoformat(ts.replace(" ", "T"))
        except ValueError:
            return []
        lo = (t - datetime.timedelta(seconds=window_seconds)).isoformat()
        hi = (t + datetime.timedelta(seconds=window_seconds)).isoformat()
        with self._connect() as conn:
            rows = conn.execute(
                "SELECT * FROM events WHERE serial=? AND id!=? AND peak_vector_sum=? "
                "AND timestamp BETWEEN ? AND ?",
                (serial, event_id, pvs, lo, hi),
            ).fetchall()
        return [dict(r) for r in rows]
-    def propagate_review_to_twins(self, event_id: str, *, window_seconds: int = 300) -> list[str]:
+        with self._connect() as conn:
            cand_rows = [dict(r) for r in conn.execute(
                "SELECT * FROM events WHERE serial=? AND id!=? AND peak_vector_sum=?",
                (serial, event_id, pvs)).fetchall()]
            hist_ts = [r["timestamp"] for r in conn.execute(
                "SELECT timestamp FROM events WHERE serial=? AND lower(record_type) LIKE 'hist%'",
                (serial,)).fetchall()]
        # Histogram interval-start times for this serial, sorted, to bound intervals.
        starts = sorted(x for x in (_parse(t) for t in hist_ts) if x is not None)
        def _interval_end(h_start):
            # The next histogram strictly after h_start bounds the interval; else open-ended.
            for x in starts:
                if x > h_start:
                    return x
            return None
        def _covers(h_start, w_time):
            end = _interval_end(h_start)
            return h_start <= w_time and (end is None or w_time < end)
        twins = []
        for c in cand_rows:
            if _is_hist(c.get("record_type")) == target_hist:
                continue  # twins are strictly cross-type (one histogram, one waveform)
            c_time = _parse(c.get("timestamp"))
            if c_time is None:
                continue
            h_start, w_time = (t_target, c_time) if target_hist else (c_time, t_target)
            if _covers(h_start, w_time):
                twins.append(c)
        return twins
    def propagate_review_to_twins(self, event_id: str, *, window_seconds: int | None = None) -> list[str]:
        """
-        Copy this event's `false_trigger`/`reviewed_real` columns onto each
+        Copy this event's `false_trigger`/`reviewed_real`/`false_trigger_reason`
-        of its histogram/waveform twins (see `find_twins`), so flagging one
+        columns onto each of its histogram/waveform twins (see `find_twins`), so
-        twin flags both.  Returns the list of twin ids updated.
+        flagging one twin flags both.  Returns the list of twin ids updated.
        ``window_seconds`` is accepted for backward compatibility but ignored;
        twin matching is now interval-based (see `find_twins`).
        """
        row = self.get_event(event_id)
        if not row:
            return []
        ft = 1 if row.get("false_trigger") else 0
        real = 1 if row.get("reviewed_real") else 0
-        twins = self.find_twins(event_id, window_seconds=window_seconds)
+        # The reason is a subtype of the FT flag — carry it only when the source
        # is actually a false trigger, so a confirmed-real twin never keeps one.
        reason = row.get("false_trigger_reason") if ft else None
        twins = self.find_twins(event_id)
        moved = []
        with self._connect() as conn:
            for tw in twins:
-                conn.execute("UPDATE events SET false_trigger=?, reviewed_real=? WHERE id=?",
+                conn.execute(
-                             (ft, real, tw["id"]))
+                    "UPDATE events SET false_trigger=?, reviewed_real=?, false_trigger_reason=? WHERE id=?",
                    (ft, real, reason, tw["id"]))
                moved.append(tw["id"])
        return moved
@@ -678,7 +757,7 @@ class SeismoDb:
                )
            else:
                cur = conn.execute(
-                    "UPDATE events SET false_trigger=0 WHERE id=?",
+                    "UPDATE events SET false_trigger=0, false_trigger_reason=NULL WHERE id=?",
                    (event_id,),
                )
        return cur.rowcount > 0
@@ -772,7 +851,8 @@ class SeismoDb:
            return False
        has_ft = "false_trigger" in review
        has_real = "reviewed_real" in review
-        if not has_ft and not has_real:
+        has_reason = "false_trigger_reason" in review
        if not has_ft and not has_real and not has_reason:
            # Nothing derived to update; just confirm the row exists.
            with self._connect() as conn:
                row = conn.execute(
@@ -785,11 +865,19 @@ class SeismoDb:
            sets["false_trigger"] = 1 if review.get("false_trigger") else 0
        if has_real:
            sets["reviewed_real"] = 1 if review.get("reviewed_real") else 0
        if has_reason:
            reason = review.get("false_trigger_reason") or None
            sets["false_trigger_reason"] = reason
            if reason:                      # a reason is a subtype of FT → implies FT
                sets["false_trigger"] = 1
        # mutual exclusivity: a true in one forces the other column to 0
        if sets.get("false_trigger") == 1:
            sets["reviewed_real"] = 0
        if sets.get("reviewed_real") == 1:
            sets["false_trigger"] = 0
        # the reason is only meaningful while flagged FT — clear it if FT ends up 0
        if sets.get("false_trigger") == 0:
            sets["false_trigger_reason"] = None
        assign = ", ".join(f"{k}=?" for k in sets)
        params = list(sets.values()) + [event_id]
        with self._connect() as conn:
@@ -67,6 +67,7 @@ from minimateplus.blastware_file import write_blastware_file, blastware_filename
 from minimateplus.client import _decode_a5_metadata_into, _decode_a5_waveform, _decode_event_count
 from minimateplus.framing import build_bw_write_frame, SESSION_RESET, POLL_PROBE, POLL_DATA
 from minimateplus.protocol import SUB_STOP_MONITORING
 from minimateplus.event_file_io import TOOL_VERSION as SFM_VERSION  # single source for the service version (release-bumped)
 from sfm import event_hdf5
 from sfm.cache import SFMCache, get_cache
 from sfm.database import SeismoDb
@@ -90,7 +91,7 @@ app = FastAPI(
        "Implements the minimateplus RS-232 protocol library.\n"
        "Proxied by terra-view at /api/sfm/*."
    ),
-    version="0.26.0",
+    version=SFM_VERSION,
 )
 # Allow requests from the waveform viewer opened as a local file (file://)
@@ -371,7 +372,7 @@ def _backfill_events(events: list, info: "DeviceInfo") -> None:
@app.get("/health")
 def health() -> dict:
    """Service heartbeat.  No device I/O."""
-    return {"status": "ok", "service": "sfm", "version": "0.1.0"}
+    return {"status": "ok", "service": "sfm", "version": SFM_VERSION}
@app.get("/", response_class=FileResponse)
@@ -47,6 +47,60 @@ def shape_from_samples(chans: dict) -> dict | None:
    return s
 # ── Offset (DC-baseline) detection ────────────────────────────────────────────
 # A DC offset is a false trigger where the geophone baseline sits at a constant
 # non-zero floor (sensor bumped / settled / drifted) instead of oscillating
 # around zero. Brian's method (validated in scratch/offset_scan3.py): the
 # pre-trigger window is definitionally quiet, so a true offset shows |pre| off
 # zero AND stays flat across the record (pre ≈ mid ≈ end). A transient moves one
 # third relative to the others and is rejected by the spread test.
 # Thresholds are in in/s (the .h5 samples are already range-scaled); validated at
 # Normal range (10 in/s) — the only range in the fleet.
 OFFSET_FLOOR = 0.025        # |pre| at/above this reads as an off-zero baseline (5 A/D counts)
 OFFSET_MAX_SPREAD = 0.02    # max(pre,mid,end) - min(...) at/below this reads as flat/constant
 def _channel_offset(x, pretrig_n):
    """Return (pre, spread, is_offset) for one channel, or None if unusable."""
    x = np.asarray(x, dtype=float)
    n = x.size
    if n < 3:
        return None
    t = n // 3
    pre = x[:pretrig_n] if (pretrig_n and 0 < pretrig_n < n) else x[:t]
    mid, end = x[t:2 * t], x[2 * t:]
    if pre.size == 0 or mid.size == 0 or end.size == 0:
        return None
    vals = [float(np.median(seg)) for seg in (pre, mid, end)]
    spread = max(vals) - min(vals)
    is_offset = abs(vals[0]) >= OFFSET_FLOOR and spread <= OFFSET_MAX_SPREAD
    return vals[0], spread, is_offset
 def offset_from_samples(chans: dict, pretrig_n) -> dict | None:
    """Detect a DC-offset false trigger across the geophone channels.
    An event is offset if ANY geo channel's pre-trigger baseline is off zero and
    flat across the record. Reports the tripping axis (or, if none trips, the
    most-offset-like axis) with its ``pre``/``spread`` for transparency + tuning.
    Returns None when no geo channel is usable.
    """
    results = []
    for ax in _GEO_CHANNELS:
        x = chans.get(ax)
        if x is None:
            continue
        r = _channel_offset(x, pretrig_n)
        if r is not None:
            results.append((ax, r[0], r[1], r[2]))
    if not results:
        return None
    offenders = [r for r in results if r[3]]
    ax, pre, spread, _ = max(offenders or results, key=lambda r: abs(r[1]))
    return {"offset": bool(offenders), "axis": ax,
            "pre": round(pre, 6), "spread": round(spread, 6)}
 def shape_from_h5(path) -> dict | None:
    import h5py
    try:
@@ -56,3 +110,18 @@ def shape_from_h5(path) -> dict | None:
    except Exception:
        return None
    return shape_from_samples(chans)
 def offset_from_h5(path) -> dict | None:
    """offset_from_samples fed from an event's .h5 (float32 in/s geo samples +
    the pretrig_samples attribute)."""
    import h5py
    try:
        with h5py.File(path, "r") as f:
            chans = {ax: f[f"samples/{ax}"][:] for ax in _GEO_CHANNELS
                     if f"samples/{ax}" in f}
            pretrig_n = f.attrs.get("pretrig_samples")
    except Exception:
        return None
    pretrig_n = int(pretrig_n) if pretrig_n is not None else 0
    return offset_from_samples(chans, pretrig_n)
@@ -32,6 +32,7 @@ from __future__ import annotations
 import datetime
 import logging
 import pickle
 import re
 import shutil
 from pathlib import Path
 from typing import Optional, Union
@@ -41,7 +42,7 @@ from minimateplus.blastware_file import blastware_filename, write_blastware_file
 from minimateplus.framing import S3Frame
 from minimateplus.models import Event
 from sfm import event_hdf5
-from sfm.shape_metrics import shape_from_h5
+from sfm.shape_metrics import shape_from_h5, offset_from_h5
 log = logging.getLogger("sfm.waveform_store")
@@ -270,6 +271,13 @@ class WaveformStore:
            "shape_sample_count":    _shape["sample_count"],
            "shape_axis":            _shape["axis"],
        } if _shape else {}
        _offset = offset_from_h5(hdf5_path) if hdf5_filename else None
        _offset_rec = {
            "shape_offset":        1 if _offset["offset"] else 0,
            "shape_offset_axis":   _offset["axis"],
            "shape_offset_pre":    _offset["pre"],
            "shape_offset_spread": _offset["spread"],
        } if _offset else {}
        return {
            "filename":            filename,
            "filesize":            filesize,
@@ -278,6 +286,7 @@ class WaveformStore:
            "hdf5_filename":       hdf5_filename,
            "sidecar_filename":    sidecar_path.name,
            **_shape_rec,
            **_offset_rec,
        }
    def save_imported_bw(
@@ -371,8 +380,16 @@ class WaveformStore:
        # Resolve serial.  blastware_filename derives a 4-char prefix from
        # the numeric serial (e.g. BE11529 → M529); we go the other way
-        # via the source filename if a hint wasn't given.
+        # if a hint wasn't given.  The filename carries only the NUMBER,
-        serial = serial_hint or _serial_from_bw_filename(source_path.name) or "UNKNOWN"
+        # so read the family prefix out of the body first — a BlastMate
        # ("BA") filed as "BE" is a unit that does not exist.  The
        # filename-only decoder stays as the last resort.
        serial = (
            serial_hint
            or _serial_from_bw_bytes(bw_bytes, source_path.name)
            or _serial_from_bw_filename(source_path.name)
            or "UNKNOWN"
        )
        # Use the source filename verbatim — it already encodes timestamp
        # + record type per BW's AB0T scheme, and we want to preserve it
@@ -461,6 +478,13 @@ class WaveformStore:
            "shape_sample_count":    _shape["sample_count"],
            "shape_axis":            _shape["axis"],
        } if _shape else {}
        _offset = offset_from_h5(hdf5_path) if hdf5_filename else None
        _offset_rec = {
            "shape_offset":        1 if _offset["offset"] else 0,
            "shape_offset_axis":   _offset["axis"],
            "shape_offset_pre":    _offset["pre"],
            "shape_offset_spread": _offset["spread"],
        } if _offset else {}
        return ev, {
            "filename":           filename,
            "filesize":           filesize,
@@ -470,6 +494,7 @@ class WaveformStore:
            "sidecar_filename":   sidecar_path.name,
            "serial":             serial,
            **_shape_rec,
            **_offset_rec,
        }
    def save_imported_idf(
@@ -751,6 +776,13 @@ class WaveformStore:
            "shape_sample_count":    _shape["sample_count"],
            "shape_axis":            _shape["axis"],
        } if _shape else {}
        _offset = offset_from_h5(hdf5_path) if hdf5_filename else None
        _offset_rec = {
            "shape_offset":        1 if _offset["offset"] else 0,
            "shape_offset_axis":   _offset["axis"],
            "shape_offset_pre":    _offset["pre"],
            "shape_offset_spread": _offset["spread"],
        } if _offset else {}
        return ev, {
            "filename":           filename,
            "filesize":           filesize,
@@ -760,6 +792,7 @@ class WaveformStore:
            "sidecar_filename":   sidecar_path.name,
            "serial":             serial,
            **_shape_rec,
            **_offset_rec,
        }
    def load_a5(self, serial: str, filename: str) -> Optional[list[S3Frame]]:
@@ -816,20 +849,24 @@ class WaveformStore:
 # ── helpers ─────────────────────────────────────────────────────────────────────
-def _serial_from_bw_filename(name: str) -> Optional[str]:
+def _serial_number_from_bw_filename(name: str) -> Optional[int]:
    """
-    Reverse of `blastware_filename`'s serial-prefix encoding.
+    Reverse of `blastware_filename`'s serial-prefix encoding — the NUMBER only.
    BW filename format (V10.72): `<P><serial3><stem4>.<ext>`
    where P = chr(ord('B') + floor(serial // 1000))
    and serial3 = f"{serial % 1000:03d}".
    Examples (from CLAUDE.md verification archive):
-      P036... → BE14036       H907... → BE6907
+      P036... → 14036       H907... → 6907
-      M529... → BE11529       T003... → BE18003
+      M529... → 11529       T003... → 18003
      L895... → 10895
-    Returns the inferred BE-prefix serial (e.g. "BE11529") or None when
+    ⚠ The filename encodes **only the number**.  The two-letter family
-    the filename doesn't match the expected pattern.
+    prefix is NOT in it — "BE" is a MiniMate Plus, "BA" a BlastMate — so
    the prefix has to come from the file body (`_serial_from_bw_bytes`)
    or from an explicit hint.  Returns None when the filename doesn't
    match the expected pattern.
    """
    if not name:
        return None
@@ -842,5 +879,43 @@ def _serial_from_bw_filename(name: str) -> Optional[str]:
    if prefix_letter < "B":
        return None
    thousands = ord(prefix_letter) - ord("B")
-    serial_num = thousands * 1000 + int(base[1:4])
+    return thousands * 1000 + int(base[1:4])
-    return f"BE{serial_num}"
+
 _BW_SERIAL_RE = re.compile(rb"[A-Z]{2}\d{3,6}")
 def _serial_from_bw_bytes(data: bytes, name: str) -> Optional[str]:
    """
    Read the real serial — prefix included — out of a BW file body.
    The body carries the serial as a plain ASCII string ("BE9558",
    "BA10895").  We accept a candidate only when its numeric part matches
    the number the filename encodes, which keeps a stray byte sequence in
    the sample stream from being mistaken for a serial.
    Returns None when the filename number can't be derived or no
    candidate in the body agrees with it — the caller then falls back.
    """
    num = _serial_number_from_bw_filename(name)
    if num is None or not data:
        return None
    for match in _BW_SERIAL_RE.findall(data):
        candidate = match.decode("ascii", errors="replace")
        if candidate[2:].lstrip("0") == str(num):
            return candidate
    return None
 def _serial_from_bw_filename(name: str) -> Optional[str]:
    """
    Best-effort serial from the filename alone.
    ⚠ The family prefix is a **guess** — the filename does not carry it.
    "BE" is right for every MiniMate Plus but wrong for a BlastMate, whose
    serials start "BA".  Prefer `_serial_from_bw_bytes` whenever the file
    body is at hand; this exists for callers that only have a name
    (log lines, dry-run output).
    """
    num = _serial_number_from_bw_filename(name)
    return None if num is None else f"BE{num}"
@@ -1,33 +1,71 @@
-import datetime
+import sqlite3
 from sfm.database import SeismoDb
 from minimateplus.models import Event, Timestamp
-def _ins(db, key, serial, pvs, ts):
+def _ins(db, key, serial, pvs, ts, record_type="Waveform"):
    ev = Event(index=0)
    ev._waveform_key = bytes.fromhex(key)
    ev.timestamp = ts
    # peak_vector_sum comes from peak_values; simplest: insert then UPDATE pvs directly
    db.insert_events([ev], serial=serial)
    row = [r for r in db.query_events(serial=serial) if r["waveform_key"] == key][0]
    import sqlite3
    with sqlite3.connect(db.db_path) as c:
-        c.execute("UPDATE events SET peak_vector_sum=? WHERE id=?", (pvs, row["id"]))
+        c.execute("UPDATE events SET peak_vector_sum=?, record_type=? WHERE id=?",
                  (pvs, record_type, row["id"]))
    return row["id"]
-def test_find_twins_matches_same_serial_pvs_near_time(tmp_path):
+def _ts(hour, minute, second=0, day=25):
    return Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=day,
                     hour=hour, minute=minute, second=second)
 def test_histogram_and_waveform_twin_across_hours(tmp_path):
    # The real UM12947 case: histogram stamped at its 7pm interval start, the
    # triggered waveform 75 min later — same serial + identical PVS. The old
    # ±5-min window missed this; interval matching catches it, both directions.
    db = SeismoDb(tmp_path / "s.db")
-    base = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=5)
+    hist_pm = _ins(db, "01110001", "BE1", 0.4763, _ts(19, 31, 17), "Histogram")
-    twin = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=45)
+    wave    = _ins(db, "01110002", "BE1", 0.4763, _ts(20, 46, 44), "Waveform")
-    far = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=21, minute=0, second=0)
+    _ins(db, "01110003", "BE1", 0.0100, _ts(7, 0, 0, day=26), "Histogram")  # bounds the interval
-    # d needs a timestamp distinct from `twin` (UNIQUE(serial, timestamp) would
+    assert {r["id"] for r in db.find_twins(hist_pm)} == {wave}
-    # otherwise collide with b and UPSERT onto its row instead of inserting a
+    assert {r["id"] for r in db.find_twins(wave)} == {hist_pm}
-    # new one) while staying near `base` in time.
+
-    near = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=44)
+
-    a = _ins(db, "01110001", "BE1", 0.4763, base)
+def test_same_type_not_twinned(tmp_path):
-    b = _ins(db, "01110002", "BE1", 0.4763, twin)   # twin: same serial+pvs, 40s apart
+    # Two waveforms, same serial + PVS, seconds apart → NOT twins (cross-type only).
-    c = _ins(db, "01110003", "BE1", 0.4763, far)    # same pvs but >window away
+    db = SeismoDb(tmp_path / "s.db")
-    d = _ins(db, "01110004", "BE1", 0.9999, near)   # near time but different pvs
+    a = _ins(db, "01110001", "BE1", 0.4763, _ts(20, 19, 5), "Waveform")
-    ids = {r["id"] for r in db.find_twins(a, window_seconds=300)}
+    _ins(db, "01110002", "BE1", 0.4763, _ts(20, 19, 45), "Waveform")
-    assert ids == {b}
+    assert db.find_twins(a) == []
 def test_waveform_matches_only_the_containing_interval(tmp_path):
    # Two overnight intervals with the same PVS; a waveform in the SECOND interval
    # must twin with that histogram, never the first — even though PVS matches both.
    db = SeismoDb(tmp_path / "s.db")
    h1 = _ins(db, "01110001", "BE1", 0.4763, _ts(19, 0, 0, day=25), "Histogram")
    h2 = _ins(db, "01110002", "BE1", 0.4763, _ts(7, 0, 0, day=26), "Histogram")
    w = _ins(db, "01110003", "BE1", 0.4763, _ts(8, 0, 0, day=26), "Waveform")
    assert {r["id"] for r in db.find_twins(w)} == {h2}
    assert w not in {r["id"] for r in db.find_twins(h1)}
 def test_different_pvs_not_twinned(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    h = _ins(db, "01110001", "BE1", 0.4763, _ts(19, 0, 0), "Histogram")
    _ins(db, "01110002", "BE1", 0.9999, _ts(20, 0, 0), "Waveform")  # different PVS
    assert db.find_twins(h) == []
 def test_open_ended_latest_interval(tmp_path):
    # A waveform after the latest histogram (nothing bounds the interval) still twins.
    db = SeismoDb(tmp_path / "s.db")
    h = _ins(db, "01110001", "BE1", 0.4763, _ts(19, 0, 0), "Histogram")
    w = _ins(db, "01110002", "BE1", 0.4763, _ts(23, 30, 0), "Waveform")
    assert {r["id"] for r in db.find_twins(h)} == {w}
 def test_missing_fields_returns_empty(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    assert db.find_twins("nonexistent-id") == []
@@ -0,0 +1,103 @@
 import sqlite3
 from sfm.database import SeismoDb
 from minimateplus.models import Event, Timestamp
 def _ev(db, key="0111aaaa", serial="BE1"):
    ev = Event(index=0)
    ev._waveform_key = bytes.fromhex(key)
    ev.timestamp = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0,
                             month=6, day=25, hour=8, minute=0, second=0)
    ev.record_type = "Waveform"
    db.insert_events([ev], serial=serial)
    return [r for r in db.query_events(serial=serial) if r["waveform_key"] == key][0]["id"]
 def test_flag_offset_reason_implies_ft(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger_reason": "offset"})
    row = db.get_event(eid)
    assert row["false_trigger"] == 1          # a reason is a subtype of FT
    assert row["false_trigger_reason"] == "offset"
    assert row["reviewed_real"] == 0
 def test_plain_ft_leaves_reason_null(tmp_path):
    # Reason is OPTIONAL — flagging FT without one records no reason.
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger": True})
    row = db.get_event(eid)
    assert row["false_trigger"] == 1
    assert row["false_trigger_reason"] is None
 def test_confirm_real_clears_reason(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger_reason": "offset"})
    db.update_event_review(eid, {"reviewed_real": True})
    row = db.get_event(eid)
    assert row["reviewed_real"] == 1
    assert row["false_trigger"] == 0
    assert row["false_trigger_reason"] is None
 def test_clear_ft_clears_reason(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger_reason": "offset"})
    db.update_event_review(eid, {"false_trigger": False})
    row = db.get_event(eid)
    assert row["false_trigger"] == 0
    assert row["false_trigger_reason"] is None
 def test_set_false_trigger_false_clears_reason(tmp_path):
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger_reason": "offset"})
    assert db.set_false_trigger(eid, False) is True
    row = db.get_event(eid)
    assert row["false_trigger"] == 0
    assert row["false_trigger_reason"] is None
 def test_reason_can_be_cleared_without_clearing_ft(tmp_path):
    # Setting reason to None removes the reason but leaves the FT flag intact.
    db = SeismoDb(tmp_path / "s.db")
    eid = _ev(db)
    db.update_event_review(eid, {"false_trigger_reason": "offset"})
    db.update_event_review(eid, {"false_trigger_reason": None})
    row = db.get_event(eid)
    assert row["false_trigger"] == 1
    assert row["false_trigger_reason"] is None
 def _ts(h, m, d=25):
    return Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0,
                     month=2, day=d, hour=h, minute=m, second=0)
 def test_offset_reason_propagates_to_twin(tmp_path):
    # Flag a waveform as offset → its histogram twin also becomes FT with reason=offset.
    db = SeismoDb(tmp_path / "s.db")
    def ins(key, ts, rt):
        ev = Event(index=0); ev._waveform_key = bytes.fromhex(key); ev.timestamp = ts
        db.insert_events([ev], serial="BE1")
        rid = [r for r in db.query_events(serial="BE1") if r["waveform_key"] == key][0]["id"]
        with sqlite3.connect(db.db_path) as c:
            c.execute("UPDATE events SET peak_vector_sum=0.4763, record_type=? WHERE id=?", (rt, rid))
        return rid
    hist = ins("01110001", _ts(19, 31), "Histogram")   # interval start
    wave = ins("01110002", _ts(20, 46), "Waveform")     # trigger inside the interval
    db.update_event_review(wave, {"false_trigger_reason": "offset"})
    db.propagate_review_to_twins(wave)
    row = db.get_event(hist)
    assert row["false_trigger"] == 1
    assert row["false_trigger_reason"] == "offset"
@@ -0,0 +1,17 @@
 """The /health version must track the release, not a stale literal.
 terra-view's SFM Admin page displays whatever `/health` reports. It was
 hardcoded to "0.1.0" and never bumped, so the page showed 0.1.0 while the
 service was actually 0.26.0. These guard against that regression — and run
 without httpx (they call the endpoint function directly, no TestClient).
 """
 from minimateplus.event_file_io import TOOL_VERSION
 from sfm.server import app, health
 def test_health_reports_current_tool_version():
    assert health()["version"] == TOOL_VERSION
 def test_openapi_version_matches_tool_version():
    assert app.version == TOOL_VERSION
@@ -643,3 +643,38 @@ def test_multi_interval_matches_blastware_ascii_exactly():
                assert hz is None
            elif not cell.startswith("<"):
                assert hz is not None and abs(hz - float(cell)) <= max(0.55, float(cell) * 0.02)
 def test_partial_final_block_is_not_disqualified_by_missing_third_header():
    """A body can exceed two strides yet hold only two real blocks.
    Regression for BE18193 `T193L0XM.CI0H` — 51 intervals at 2 s = one full
    30-interval block plus a 21-interval remainder, in a body long enough to
    demand a third block header at ``2 * stride`` that does not exist.  The
    third-block confirmation used to be mandatory whenever the body was long
    enough, so the correct stride was discarded and the file decoded to
    nothing.  A missing third header means end-of-stream, not disqualification;
    the block-counter check is the decisive anti-false-positive test.
    """
    full = [(1, 1, 2, 2, 3, 3, 4, 4)] * 30
    partial = [(5, 5, 6, 6, 7, 7, 8, 8)] * 21
    body = (_mk_multi_block(full, ctr=256)
            + _mk_multi_block(partial, ctr=257)
            + b"\xff" * 700)      # trailing padding past 2 * stride
    stride = 12 + 20 * 30
    assert 2 * stride + 6 <= len(body), "padding must reach past two strides"
    # the whole point: a third header is absent, and that must not disqualify
    assert detect_multi_interval_stride(body) == stride
    recs = walk_multi_interval_blocks(body)
    assert len(recs) == 51
    assert recs[0]["t_peak"] == 1
    assert recs[-1]["t_peak"] == 5
 def test_third_block_still_rejects_a_mismatched_counter():
    """The corroboration must still bite when a third block IS present."""
    ivs = [(1, 1, 2, 2, 3, 3, 4, 4)] * 4
    body = (_mk_multi_block(ivs, ctr=256)
            + _mk_multi_block(ivs, ctr=257)
            + _mk_multi_block(ivs, ctr=999))   # counter jumps — not consecutive
    assert detect_multi_interval_stride(body) != 12 + 20 * 4
@@ -0,0 +1,94 @@
 import numpy as np
 import h5py
 from sfm.shape_metrics import offset_from_samples, offset_from_h5
 def test_flags_constant_dc_floor():
    # A geophone channel sitting at a constant +0.05 in/s across the whole record
    # is a DC offset: baseline off zero AND flat across pre/mid/end thirds.
    n = 300
    chans = {"Tran": np.full(n, 0.05), "Vert": np.zeros(n), "Long": np.zeros(n)}
    r = offset_from_samples(chans, pretrig_n=50)
    assert r["offset"] is True
    assert r["axis"] == "Tran"
    assert abs(r["pre"] - 0.05) < 1e-6
    assert r["spread"] < 0.02
 def test_transient_rejected_by_spread():
    # Off-zero pre-trigger but the baseline SETTLES back over the record — a
    # transient, not a constant offset. The spread test must reject it.
    x = np.concatenate([np.full(100, 0.05), np.full(100, 0.025), np.zeros(100)])
    chans = {"Tran": x, "Vert": np.zeros(300), "Long": np.zeros(300)}
    r = offset_from_samples(chans, pretrig_n=100)
    assert r["offset"] is False
 def test_clean_oscillation_not_offset():
    t = np.arange(300)
    x = 0.4 * np.sin(2 * np.pi * t / 20)   # oscillates around zero — baseline IS zero
    chans = {"Tran": x, "Vert": np.zeros(300), "Long": np.zeros(300)}
    r = offset_from_samples(chans, pretrig_n=50)
    assert r["offset"] is False
 def test_below_floor_not_offset_but_reports_pre():
    # A flat baseline below the floor is not an offset; still report the axis/pre
    # for tuning transparency.
    n = 300
    chans = {"Tran": np.full(n, 0.01), "Vert": np.zeros(n), "Long": np.zeros(n)}
    r = offset_from_samples(chans, pretrig_n=50)
    assert r["offset"] is False
    assert r["axis"] == "Tran"
    assert abs(r["pre"] - 0.01) < 1e-6
 def test_none_when_no_geo_channels():
    assert offset_from_samples({"MicL": np.full(300, 0.05)}, pretrig_n=50) is None
 def test_pretrig_fallback_when_invalid():
    # pretrig_n of 0 (missing/unusable) falls back to the first third.
    n = 300
    chans = {"Tran": np.full(n, 0.05), "Vert": np.zeros(n), "Long": np.zeros(n)}
    r = offset_from_samples(chans, pretrig_n=0)
    assert r["offset"] is True
 def test_flags_offset_on_any_axis():
    # Offset on Vert alone still flags the event, and Vert is reported.
    n = 300
    chans = {"Tran": np.zeros(n), "Vert": np.full(n, -0.06), "Long": np.zeros(n)}
    r = offset_from_samples(chans, pretrig_n=50)
    assert r["offset"] is True
    assert r["axis"] == "Vert"
 def _write_h5(path, chans, pretrig_n):
    with h5py.File(path, "w") as f:
        g = f.create_group("samples")
        for k, v in chans.items():
            g.create_dataset(k, data=np.asarray(v, dtype="float32"))
        if pretrig_n is not None:
            f.attrs["pretrig_samples"] = pretrig_n
 def test_offset_from_h5_reads_pretrig_attr(tmp_path):
    p = tmp_path / "ev.h5"
    n = 300
    _write_h5(p, {"Tran": np.full(n, 0.05), "Vert": np.zeros(n), "Long": np.zeros(n)},
              pretrig_n=50)
    r = offset_from_h5(str(p))
    assert r["offset"] is True and r["axis"] == "Tran"
 def test_offset_from_h5_missing_pretrig_attr_falls_back(tmp_path):
    p = tmp_path / "noattr.h5"
    n = 300
    _write_h5(p, {"Tran": np.full(n, 0.05), "Vert": np.zeros(n), "Long": np.zeros(n)},
              pretrig_n=None)
    assert offset_from_h5(str(p))["offset"] is True   # falls back to first-third
 def test_offset_from_h5_missing_file_is_none(tmp_path):
    assert offset_from_h5(str(tmp_path / "nope.h5")) is None
@@ -0,0 +1,64 @@
 from __future__ import annotations
 from pathlib import Path
 import numpy as np, h5py
 from sfm.database import SeismoDb
 from sfm.waveform_store import WaveformStore
 from scripts.backfill_event_shape import backfill_shape
 from minimateplus.models import Event, Timestamp, PeakValues
 _FIX = Path(__file__).parent / "fixtures/histogram-extension-re/events-5-21-26/K558LL8B.7I0W"
 def _event(waveform_key="0111abcd"):
    ev = Event(index=0)
    ev._waveform_key = bytes.fromhex(waveform_key)
    ev.timestamp = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0,
                             month=6, day=25, hour=8, minute=50, second=0)
    ev.record_type = "Waveform"
    ev.peak_values = PeakValues(tran=0.075, vert=0.220, long=0.045,
                                 peak_vector_sum=0.231, micl=0.01)
    return ev
 def test_insert_stores_offset_from_record(tmp_path: Path):
    db = SeismoDb(tmp_path / "s.db")
    ev = _event()
    rec = {ev._waveform_key.hex(): {
        "filename": "F.CE0W", "filesize": 10,
        "shape_offset": 1, "shape_offset_axis": "Tran",
        "shape_offset_pre": 0.05, "shape_offset_spread": 0.001}}
    db.insert_events([ev], serial="BE1", waveform_records=rec)
    row = db.query_events(serial="BE1")[0]
    assert row["shape_offset"] == 1
    assert row["shape_offset_axis"] == "Tran"
    assert abs(row["shape_offset_pre"] - 0.05) < 1e-6
    assert abs(row["shape_offset_spread"] - 0.001) < 1e-6
 def test_save_imported_bw_attaches_offset(tmp_path: Path):
    store = WaveformStore(tmp_path / "waveforms")
    ev, rec = store.save_imported_bw(_FIX.read_bytes(), source_path=_FIX, serial_hint="BE9558")
    assert rec["shape_offset"] in (0, 1)
    assert rec["shape_offset_axis"] in ("Tran", "Vert", "Long")
    assert "shape_offset_pre" in rec and "shape_offset_spread" in rec
 def test_backfill_updates_offset(tmp_path: Path):
    db = SeismoDb(tmp_path / "s.db")
    store = WaveformStore(tmp_path / "waveforms")
    ev = Event(index=0); ev._waveform_key = bytes.fromhex("0111abcd")
    db.insert_events([ev], serial="BE1",
                     waveform_records={ev._waveform_key.hex(): {"filename": "F.CE0W", "filesize": 10}})
    p = store.hdf5_path_for("BE1", "F.CE0W")
    with h5py.File(p, "w") as f:
        g = f.create_group("samples")
        g.create_dataset("Tran", data=np.full(300, 0.05, "float32"))
        g.create_dataset("Vert", data=np.zeros(300, "float32"))
        g.create_dataset("Long", data=np.zeros(300, "float32"))
        f.attrs["pretrig_samples"] = 50
    backfill_shape(db, store)
    row = db.query_events(serial="BE1")[0]
    assert row["shape_offset"] == 1
    assert row["shape_offset_axis"] == "Tran"
@@ -0,0 +1,101 @@
 """The BW filename encodes the serial NUMBER, never the family prefix.
 "BE" is a MiniMate Plus; "BA" is a BlastMate.  Both are Series III and their
 files are byte-compatible — the whole archive's 1,493 BlastMate binaries
 decode through the same codec at 100% — so the only thing that distinguishes
 them downstream is the serial string, and that lives in the file body.
 Synthesising the prefix as "BE" files a BlastMate under a unit that does not
 exist.  Four units in the DL2 archive are affected: BA9229, BA10060, BA10895
 and BA15957.
 """
 from __future__ import annotations
 import pytest
 from minimateplus.client import _decode_0a_partial_header
 from sfm.waveform_store import (
    _serial_from_bw_bytes,
    _serial_from_bw_filename,
    _serial_number_from_bw_filename,
 )
 # ── the filename gives a number, and only a number ──────────────────────────
@pytest.mark.parametrize("name,num", [
    ("P036L318.C80H", 14036),   # BE14036
    ("H907KWRK.WB0H", 6907),    # BE6907
    ("M529LKIQ.G10",  11529),   # BE11529
    ("T003LQ9K.OE0H", 18003),   # BE18003
    ("L895K63F.GE0W", 10895),   # BA10895 — a BlastMate
    ("K229HGQI.XO0W", 9229),    # BA9229  — a BlastMate
 ])
 def test_number_from_filename(name, num):
    assert _serial_number_from_bw_filename(name) == num
@pytest.mark.parametrize("name", ["", "not_a_bw_file.bin", "AB12", "1234ABCD.XX0W"])
 def test_number_from_filename_rejects_junk(name):
    assert _serial_number_from_bw_filename(name) is None
 def test_filename_only_decoder_is_a_guess():
    """It still answers "BE" — that is why it must not be the first choice."""
    assert _serial_from_bw_filename("L895K63F.GE0W") == "BE10895"
    assert _serial_from_bw_filename("M529LKIQ.G10") == "BE11529"
    assert _serial_from_bw_filename("nonsense") is None
 # ── the body carries the truth ──────────────────────────────────────────────
 def _body(serial: bytes) -> bytes:
    return b"\x00" * 32 + b"STRT" + b"\xff\xfe" + serial + b"\x00Geo: 0.254 in/s\x00"
 def test_body_wins_for_a_blastmate():
    assert _serial_from_bw_bytes(_body(b"BA10895"), "L895K63F.GE0W") == "BA10895"
 def test_body_wins_for_a_minimate():
    assert _serial_from_bw_bytes(_body(b"BE11529"), "M529LKIQ.G10") == "BE11529"
 def test_body_candidate_must_match_the_filename_number():
    """A serial-shaped byte run that disagrees with the filename is ignored."""
    assert _serial_from_bw_bytes(_body(b"XX99999"), "L895K63F.GE0W") is None
 def test_body_tolerates_a_leading_zero():
    assert _serial_from_bw_bytes(_body(b"BA09229"), "K229HGQI.XO0W") == "BA09229"
@pytest.mark.parametrize("data,name", [
    (b"", "L895K63F.GE0W"),          # no bytes
    (_body(b"BA10895"), "junk.bin"),  # no derivable number
 ])
 def test_body_returns_none_when_it_cannot_decide(data, name):
    assert _serial_from_bw_bytes(data, name) is None
 # ── the live monitor-log path ───────────────────────────────────────────────
 def _partial_record(serial: bytes) -> bytes:
    """0x2C partial record: type, prefix, two 9-byte timestamps, then ASCII."""
    ts = bytes([11, 0x10, 4, 0x07, 0xE9, 0, 16, 2, 0])   # 2025-04-11 16:02:00
    return (bytes([0x2C]) + b"\x00" * 10 + ts + ts
            + b"\x00\x00\x00\x00" + serial + b"\x00Geo: 0.254 in/s\x00")
@pytest.mark.parametrize("serial", [b"BE11529", b"BA10895", b"UM11719"])
 def test_monitor_log_reads_any_family_prefix(serial):
    entry = _decode_0a_partial_header(_partial_record(serial), 0, b"\x01\x11\x00\x00")
    assert entry is not None
    assert entry.serial == serial.decode()
 def test_monitor_log_geo_threshold_survives_a_blastmate():
    """The old find(b"BE") skipped the whole block, losing geo too."""
    entry = _decode_0a_partial_header(_partial_record(b"BA10895"), 0, b"\x01\x11\x00\x00")
    assert entry is not None
    assert entry.geo_threshold_ips == pytest.approx(0.254)
@@ -3,31 +3,34 @@ from sfm.database import SeismoDb
 from minimateplus.models import Event, Timestamp
-def _ins(db, key, serial, pvs, ts):
+def _ins(db, key, serial, pvs, ts, record_type="Waveform"):
    ev = Event(index=0)
    ev._waveform_key = bytes.fromhex(key)
    ev.timestamp = ts
    # peak_vector_sum comes from peak_values; simplest: insert then UPDATE pvs directly
    db.insert_events([ev], serial=serial)
    row = [r for r in db.query_events(serial=serial) if r["waveform_key"] == key][0]
    with sqlite3.connect(db.db_path) as c:
-        c.execute("UPDATE events SET peak_vector_sum=? WHERE id=?", (pvs, row["id"]))
+        c.execute("UPDATE events SET peak_vector_sum=?, record_type=? WHERE id=?",
                  (pvs, record_type, row["id"]))
    return row["id"]
-def test_propagate_copies_flags_to_twins(tmp_path):
+def _ts(hour, minute, second=0, day=25):
    return Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=day,
                     hour=hour, minute=minute, second=second)
 def test_propagate_copies_flags_across_hours_apart_twins(tmp_path):
    # Flagging the waveform FT propagates to its histogram twin 75 min earlier
    # (the interval matcher pairs them; the old ±5-min window would have missed it).
    db = SeismoDb(tmp_path / "s.db")
-    base = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=5)
+    hist  = _ins(db, "01110001", "BE1", 0.4763, _ts(19, 31, 17), "Histogram")
-    twin = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=45)
+    wave  = _ins(db, "01110002", "BE1", 0.4763, _ts(20, 46, 44), "Waveform")   # twin, 75 min later
-    other = Timestamp(raw=b"", flag=0x10, year=2026, unknown_byte=0, month=2, day=25, hour=20, minute=19, second=44)
+    other = _ins(db, "01110003", "BE1", 0.9999, _ts(20, 20, 0),  "Waveform")   # different pvs
-    primary_id = _ins(db, "01110001", "BE1", 0.4763, base)
+    db.update_event_review(wave, {"false_trigger": True})
-    twin_id = _ins(db, "01110002", "BE1", 0.4763, twin)        # twin: same serial+pvs, 40s apart
+    moved = db.propagate_review_to_twins(wave)
    non_twin_id = _ins(db, "01110003", "BE1", 0.9999, other)   # near time but different pvs
-    db.update_event_review(primary_id, {"false_trigger": True})
+    assert hist in moved
-    moved = db.propagate_review_to_twins(primary_id)
+    assert db.get_event(hist)["false_trigger"] == 1
-
+    assert db.get_event(other)["false_trigger"] == 0
    assert twin_id in moved
    assert db.get_event(twin_id)["false_trigger"] == 1
    assert db.get_event(non_twin_id)["false_trigger"] == 0