chore(release): v0.27.0 — decoder verified at scale; offset investigation
Bumps pyproject, TOOL_VERSION, README and CLAUDE.md to 0.27.0. sfm/server.py
now derives its version from TOOL_VERSION (c8c4ec2), so that constant is the
single source of truth for the service version and the sidecar stamp alike.
What ships:
- histogram partial-final-block fix (4 files recovered, 0 regressed)
- interval-based find_twins matching (terra-view #102 sub-task 2)
- /health no longer reports a hard-coded 0.1.0
- 793 NUL bytes stripped from CLAUDE.md (made grep skip it as binary)
- docs/offset_investigation.md, and the offset detectors
- scratch/verify_against_ascii.py
Verification: the series-3 codec now decodes 14,338 / 14,338 archive pairs
exactly against their Blastware ASCII exports (1,249 waveform + 13,089
histogram, 45 units, back to 2018) — 11x the ground truth the prod store
carried, and it supersedes the old "per-sample on 11% of files" caveat.
Independent check on the scale: 19,244 healthy channel-events sit at a
pre-trigger floor of exactly 0.000 (62.7%), 94.5% within one quantisation
unit, median +0.0000. No zero-point bias in the decoder.
⚠ TOOL_VERSION moved, so the next prod backfill regenerates the whole store
(~2 hours on the NAS). That is intended — it is what publishes the 4 recovered
histograms — but it is not a no-op; schedule it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -2,21 +2,24 @@
|
||||
|
||||
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
|
||||
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.26.0**.
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.27.0**.
|
||||
|
||||
---
|
||||
|
||||
## Where things stand (updated 2026-08-27)
|
||||
## Where things stand (updated 2026-08-28)
|
||||
|
||||
Read this first when picking the project back up.
|
||||
|
||||
- **Series-3 decode is correct and verified.** All 11,603 series-3 binaries in
|
||||
the prod snapshot pass every check (channel lengths, peaks vs the device's
|
||||
own reported PPV, nothing above full scale, length vs declared record time).
|
||||
Ground truth: 1,211/1,211 histograms exact per-interval and 75/75 waveform
|
||||
sample counts exact against preserved Blastware ASCII exports.
|
||||
⚠ That is per-sample proof on 11% of files and peak-only consistency on the
|
||||
other 89% — see `docs/instantel_protocol_reference.md` §7.6.1.
|
||||
- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
|
||||
archive decodes **14,338 / 14,338** paired files exactly against their
|
||||
preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
|
||||
units, files back to 2018. That is 11x the ground truth the prod store
|
||||
carried, and it supersedes the old "per-sample on 11%, peak-only on 89%"
|
||||
caveat. Harness: `scratch/verify_against_ascii.py` (note its saturation
|
||||
carve-out — BW clamps clipped events, the decoder reports true counts).
|
||||
Independent corroboration of the 32000-count scale: 19,244 healthy
|
||||
channel-events sit at a pre-trigger floor of exactly 0.000 (62.7%), 94.5%
|
||||
within ±1 quantisation unit, median +0.0000 — no zero-point bias.
|
||||
- **Series-4 (Thor / Micromate) is NOT verified.** UM-series sits at ~48%
|
||||
against device peaks with a ~1.7% systematic bias and a near-zero tail.
|
||||
Thor IDFW is pinned to `decode_waveform_legacy` deliberately.
|
||||
@@ -24,14 +27,23 @@ Read this first when picking the project back up.
|
||||
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
|
||||
inserts DB rows for store files that have none (one-time per store) and the
|
||||
dry-run does not report that count.
|
||||
- **After any codec change, regenerate the store** — `backfill_sidecars.py
|
||||
--force` then `backfill_event_shape.py`, DB backup first. Stored `.h5` files
|
||||
do not update themselves.
|
||||
- **After any codec change, regenerate the store** — `backfill_sidecars.py`
|
||||
then `backfill_event_shape.py`, DB backup first. Stored `.h5` files do not
|
||||
update themselves. No `--force` needed as long as `TOOL_VERSION` was bumped
|
||||
(it gates regeneration). ⚠ On the office NAS this takes **~2 hours**
|
||||
(~1.5 files/sec vs 85/sec on the dev box — gzip-4 in `sfm/event_hdf5.py`
|
||||
against a Synology CPU). Budget it up front.
|
||||
**v0.27.0 owes prod a backfill:** the partial-final-block fix recovers 4
|
||||
histograms that are still empty in the store.
|
||||
- **The "offset" hardware fault has its own journal** --
|
||||
`docs/offset_investigation.md`. Base rate settled at 5-6 of 45 units
|
||||
(11-13%) across the DL2 archive; Instantel's own autozero procedure and
|
||||
its 2027-2069 go/no-go window are recorded there. Best open lead is
|
||||
`SUB 0x0E` (unimplemented), which may carry those numbers.
|
||||
`docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
|
||||
**persistent** — it stays until the geophone is serviced. Detect it with
|
||||
`scratch/offset_scan3.py`: the resting floor in the **pre-trigger** window,
|
||||
required to hold across pre/middle/end. Never score only the dominant-peak
|
||||
axis and never use the mean — both produce false recoveries (see the
|
||||
retraction banner in the journal). Instantel's autozero procedure and its
|
||||
2027-2069 acceptance window are recorded there too. Best open lead is
|
||||
`SUB 0x0E` (unimplemented), which may carry those very numbers.
|
||||
|
||||
|
||||
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
|
||||
|
||||
Reference in New Issue
Block a user