chore(release): v0.27.0 — decoder verified at scale; offset investigation

Bumps pyproject, TOOL_VERSION, README and CLAUDE.md to 0.27.0. sfm/server.py
now derives its version from TOOL_VERSION (c8c4ec2), so that constant is the
single source of truth for the service version and the sidecar stamp alike.

What ships:
  - histogram partial-final-block fix (4 files recovered, 0 regressed)
  - interval-based find_twins matching (terra-view #102 sub-task 2)
  - /health no longer reports a hard-coded 0.1.0
  - 793 NUL bytes stripped from CLAUDE.md (made grep skip it as binary)
  - docs/offset_investigation.md, and the offset detectors
  - scratch/verify_against_ascii.py

Verification: the series-3 codec now decodes 14,338 / 14,338 archive pairs
exactly against their Blastware ASCII exports (1,249 waveform + 13,089
histogram, 45 units, back to 2018) — 11x the ground truth the prod store
carried, and it supersedes the old "per-sample on 11% of files" caveat.

Independent check on the scale: 19,244 healthy channel-events sit at a
pre-trigger floor of exactly 0.000 (62.7%), 94.5% within one quantisation
unit, median +0.0000. No zero-point bias in the decoder.

⚠ TOOL_VERSION moved, so the next prod backfill regenerates the whole store
(~2 hours on the NAS). That is intended — it is what publishes the 4 recovered
histograms — but it is not a no-op; schedule it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
2026-08-28 22:21:47 +00:00
co-authored by Claude Opus 5
parent ad84a04404
commit 8e808b09d4
5 changed files with 72 additions and 19 deletions
+28 -16
View File
@@ -2,21 +2,24 @@
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.26.0**.
(Sierra Wireless RV50 / RV55). Current version: **v0.27.0**.
---
## Where things stand (updated 2026-08-27)
## Where things stand (updated 2026-08-28)
Read this first when picking the project back up.
- **Series-3 decode is correct and verified.** All 11,603 series-3 binaries in
the prod snapshot pass every check (channel lengths, peaks vs the device's
own reported PPV, nothing above full scale, length vs declared record time).
Ground truth: 1,211/1,211 histograms exact per-interval and 75/75 waveform
sample counts exact against preserved Blastware ASCII exports.
⚠ That is per-sample proof on 11% of files and peak-only consistency on the
other 89% — see `docs/instantel_protocol_reference.md` §7.6.1.
- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
archive decodes **14,338 / 14,338** paired files exactly against their
preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
units, files back to 2018. That is 11x the ground truth the prod store
carried, and it supersedes the old "per-sample on 11%, peak-only on 89%"
caveat. Harness: `scratch/verify_against_ascii.py` (note its saturation
carve-out — BW clamps clipped events, the decoder reports true counts).
Independent corroboration of the 32000-count scale: 19,244 healthy
channel-events sit at a pre-trigger floor of exactly 0.000 (62.7%), 94.5%
within ±1 quantisation unit, median +0.0000 — no zero-point bias.
- **Series-4 (Thor / Micromate) is NOT verified.** UM-series sits at ~48%
against device peaks with a ~1.7% systematic bias and a near-zero tail.
Thor IDFW is pinned to `decode_waveform_legacy` deliberately.
@@ -24,14 +27,23 @@ Read this first when picking the project back up.
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
inserts DB rows for store files that have none (one-time per store) and the
dry-run does not report that count.
- **After any codec change, regenerate the store** — `backfill_sidecars.py
--force` then `backfill_event_shape.py`, DB backup first. Stored `.h5` files
do not update themselves.
- **After any codec change, regenerate the store** — `backfill_sidecars.py`
then `backfill_event_shape.py`, DB backup first. Stored `.h5` files do not
update themselves. No `--force` needed as long as `TOOL_VERSION` was bumped
(it gates regeneration). ⚠ On the office NAS this takes **~2 hours**
(~1.5 files/sec vs 85/sec on the dev box — gzip-4 in `sfm/event_hdf5.py`
against a Synology CPU). Budget it up front.
**v0.27.0 owes prod a backfill:** the partial-final-block fix recovers 4
histograms that are still empty in the store.
- **The "offset" hardware fault has its own journal** --
`docs/offset_investigation.md`. Base rate settled at 5-6 of 45 units
(11-13%) across the DL2 archive; Instantel's own autozero procedure and
its 2027-2069 go/no-go window are recorded there. Best open lead is
`SUB 0x0E` (unimplemented), which may carry those numbers.
`docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
**persistent** — it stays until the geophone is serviced. Detect it with
`scratch/offset_scan3.py`: the resting floor in the **pre-trigger** window,
required to hold across pre/middle/end. Never score only the dominant-peak
axis and never use the mean — both produce false recoveries (see the
retraction banner in the journal). Instantel's autozero procedure and its
2027-2069 acceptance window are recorded there too. Best open lead is
`SUB 0x0E` (unimplemented), which may carry those very numbers.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document