fix(series4): Thor/Micromate decoder is now per-sample exact

Verified against Thor's own CSV exports, which carry a per-sample
four-column block beside every binary (CSV/<name>.IDFW.csv). Those 1,012
paired files were in the corpus all along; the decoder had been pinned to
a superseded walker on the stated grounds that "Thor has no ASCII ground
truth in the corpus and its geo scaling is separately suspect". Both
premises were false.

  IDFW per-sample exact      39.1%  -> 100.000% (1,057,536/1,057,536)
  IDFW files fully exact     0/153  -> 153/153
  IDFW PPV median error      -3.32% -> -0.002%
  IDFH within 2% of Thor PPV 51.1%  -> 100.0% (858/858)
  prod IDFW, 8 units         -3.3%  -> -0.001%

Four independent root causes:

- Geo LSB was 0.0003, the 4-dp *display rounding* of the real
  0.000310308 mistaken for the LSB, so every series-4 geophone sample
  read 3.3% low. Pinned to +-6e-11 by intersecting 991,415 rounding
  constraints; corroborated by the +-full-scale seed (+-32226) left in
  unwritten IDFH slots. IDFH had a separate, also wrong, 10.0/32768.

- IDFH histograms were capped at 250 intervals: the segment validator
  required the interval counter's high byte to be zero, but the counter
  is a uint16 cumulative index, so every segment past interval 255 was
  rejected. Runs over ~4 hours lost their tail, often the peak.
  540/858 corpus files affected.

- Record mode 00 00 (raw int16, 10-byte header) was unhandled and fell
  through the dispatch, silently dropping each channel's first 512
  samples -- the long-standing "loud events truncate" symptom.
  MODE_ABSOLUTE is now also accepted as a segment-0 preamble.

- The body-offset search matched 00 02 00 *inside* record headers,
  selecting a candidate part-way down the chain and decoding a
  rotation-shifted body. It now anchors on record headers and takes the
  chain head (6 ms/file).

Also fixes the separately tracked "UM-series decodes ~1000x low" bug.
Series-3 re-verified unchanged at 14,338/14,338 exact after the shared
waveform_codec change.

Known open: 41/575 prod IDFW files (7%, mostly UM12947/UM20147) decode
with unequal channel lengths and also fail metadata extraction -- a
different header variant with no Thor export in the store.

NOTE: this is a codec change; the Thor store owes a regeneration via
scripts/backfill_thor_events.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
2026-09-10 18:06:14 +00:00
co-authored by Claude Opus 5
parent 91b9b4578c
commit 726c2ce1b5
8 changed files with 949 additions and 66 deletions
+13 -2
View File
@@ -595,8 +595,19 @@ class WaveformStore:
)
# Binary-derived peaks fill in when the .txt didn't supply them.
# They're ~3% low vs the device-authoritative .txt values (residual
# codec drift), so .txt always wins when present.
#
# The old justification for this precedence -- "binary peaks are ~3%
# low vs the .txt" -- was a decoder bug (geo LSB 0.0003 instead of
# 0.000310308) and was fixed 2026-09-10; the binary now agrees with
# Thor's own export per-sample. The .txt still wins when present
# because it is what the operator sees in Thor's report.
#
# ⚠ One case where the .txt is the *less* accurate of the two:
# Thor floors displayed histogram PPV at 0.0050 in/s, so on quiet
# IDFH events the .txt reports 0.0050 while the binary decodes the
# true ~0.0025. 41.4% of prod IDFH sidecars carry a component PPV
# larger than their own vector sum because of it. Left as-is
# deliberately, so stored peaks keep matching Thor's report.
if binary_peaks is not None:
if binary_peaks.transverse_ips and not report_dict.get("tran_ppv"):
report_dict["tran_ppv"] = binary_peaks.transverse_ips