seismo-relay

519 Commits 6 Branches 5 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
serversdownandClaude Opus 5	904522a9c5	fix(codec): 40 NN int16 blocks are not capped at NN=8 data_block_len() rejected any `40 NN` block with NN > 0x08. That guard had no evidence behind it: every corpus available when it was written used only NN in {1,2,3,4,8}, so it was never exercised. Loud UM12947 events use NN of 12, 16, 20 ... up to 196. Because walk_body/run stop at the first unrecognised tag rather than raising, rejecting those blocks surfaced as silently short channels -- e.g. Tran 1812 / Vert 2132 / Long 2324 on a file whose export carries 2324 for all three. The real bound is the buffer; the caller additionally clamps to the record end. Verified against Thor's own CSV exports for UM12947 (2025-07-14 .. 09-25, 167 waveforms, supplied as CSV.zip): length mismatches 22 -> 0 per-sample exact 1,476,242 / 1,476,249 These are NOT truncated recordings, which was the competing hypothesis -- the exports carry the full sample count. tests/test_waveform_codec.py asserted the cap as intended behaviour. That assertion encoded an assumption, not a verified fact, and is replaced with one pinning the opposite plus the evidence. Across all three ground-truth corpora: 459 waveform files, 3,807,158 / 3,807,165 samples exact. Production IDFW is now 575/575 with zero truncations and zero decode failures (median PPV error -0.0007% across 8 units). Series-3 re-verified unchanged at 14,338/14,338. The 7 residual samples each differ by one 4th-decimal tick and are Thor's own rounding: intersecting the per-sample rounding constraints over that corpus is infeasible (binding pair contradict by 2.3e-11, 7e-5 relative), so no single linear LSB reproduces every printed value. _GEO_LSB_IPS is already pinned to ~1e-11; do not retune it to chase these. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL	2026-09-11 05:05:46 +00:00
serversdownandClaude Opus 5	c07aaa552c	fix(series4): support mic-disabled (3-channel) Thor units Verified against a second Thor corpus (9-10-26-csv-req: UM11402, UM12947, UM20147) with per-sample CSV exports: 139/139 waveforms exact (1,273,380/1,273,380 samples) and 877/877 histograms within 2% of Thor's reported PPV -- up from 66.9% and 56.6%. Some units run with the microphone disabled, which changes two structural things that were both hardcoded to the 4-channel shape: - Waveform body head sat below the scan floor. A 3-channel unit has a shorter fixed header and puts its record chain head at 0x0dba, under the old _BODY_SCAN_FLOOR of 0x0E00. The scan could not see it and fell through to the Vert segment-0 record, decoding a body shifted one position around the channel rotation -- Vert came up exactly 512 samples short. Floor lowered to 0x0C00. The body-offset scoring also had to stop requiring four channels, or `equal` is permanently False for these events and the pick falls back to raw sample count. - Histogram interval record is 56 bytes, not 72. It is 16 * n_channels + 8, and is not inferable from the segment length alone. The interval count now comes from the segment's cumulative counter (n = counter - prev_counter) and the stride is derived from it. Assuming 72 read 7 intervals out of every 10-interval segment, then walked off alignment into garbage that decoded as ~10 in/s peaks -- inflating some files' PPV by up to 191,000%. Also recovers 4 files that previously decoded no intervals at all. Combined across both corpora: 292/292 waveform files, 2,330,916/2,330,916 samples exact. Production IDFW truncations 41 -> 22. Series-3 unaffected (no shared-codec change in this commit; last full run 14,338/14,338). Known open, diagnosed but NOT verified: the remaining 22 unequal + 1 failing production IDFW files (all UM12947, 2025-07-14..09-23) stop the block walker on tag 40 0c. data_block_len() caps the 40 NN int16 block at NN > 0x08 while those files use NN up to 196. Both verified corpora only ever use NN in {1,2,3,4,8}, so the cap is untested there and lifting it leaves both at 100.000% -- which is not evidence it decodes these correctly. Deliberately not shipped; needs Thor CSV exports for UM12947 in that date range. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL	2026-09-10 20:00:56 +00:00
serversdownandClaude Opus 5	726c2ce1b5	fix(series4): Thor/Micromate decoder is now per-sample exact Verified against Thor's own CSV exports, which carry a per-sample four-column block beside every binary (CSV/<name>.IDFW.csv). Those 1,012 paired files were in the corpus all along; the decoder had been pinned to a superseded walker on the stated grounds that "Thor has no ASCII ground truth in the corpus and its geo scaling is separately suspect". Both premises were false. IDFW per-sample exact 39.1% -> 100.000% (1,057,536/1,057,536) IDFW files fully exact 0/153 -> 153/153 IDFW PPV median error -3.32% -> -0.002% IDFH within 2% of Thor PPV 51.1% -> 100.0% (858/858) prod IDFW, 8 units -3.3% -> -0.001% Four independent root causes: - Geo LSB was 0.0003, the 4-dp display rounding of the real 0.000310308 mistaken for the LSB, so every series-4 geophone sample read 3.3% low. Pinned to +-6e-11 by intersecting 991,415 rounding constraints; corroborated by the +-full-scale seed (+-32226) left in unwritten IDFH slots. IDFH had a separate, also wrong, 10.0/32768. - IDFH histograms were capped at 250 intervals: the segment validator required the interval counter's high byte to be zero, but the counter is a uint16 cumulative index, so every segment past interval 255 was rejected. Runs over ~4 hours lost their tail, often the peak. 540/858 corpus files affected. - Record mode 00 00 (raw int16, 10-byte header) was unhandled and fell through the dispatch, silently dropping each channel's first 512 samples -- the long-standing "loud events truncate" symptom. MODE_ABSOLUTE is now also accepted as a segment-0 preamble. - The body-offset search matched 00 02 00 inside record headers, selecting a candidate part-way down the chain and decoding a rotation-shifted body. It now anchors on record headers and takes the chain head (6 ms/file). Also fixes the separately tracked "UM-series decodes ~1000x low" bug. Series-3 re-verified unchanged at 14,338/14,338 exact after the shared waveform_codec change. Known open: 41/575 prod IDFW files (7%, mostly UM12947/UM20147) decode with unequal channel lengths and also fail metadata extraction -- a different header variant with no Thor export in the store. NOTE: this is a codec change; the Thor store owes a regeneration via scripts/backfill_thor_events.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL	2026-09-10 18:06:14 +00:00

Author

SHA1

Message

Date

serversdownandClaude Opus 5

904522a9c5

fix(codec): 40 NN int16 blocks are not capped at NN=8

data_block_len() rejected any `40 NN` block with NN > 0x08. That guard had
no evidence behind it: every corpus available when it was written used only
NN in {1,2,3,4,8}, so it was never exercised. Loud UM12947 events use NN of
12, 16, 20 ... up to 196.

Because walk_body/run stop at the first unrecognised tag rather than
raising, rejecting those blocks surfaced as silently short channels -- e.g.
Tran 1812 / Vert 2132 / Long 2324 on a file whose export carries 2324 for
all three. The real bound is the buffer; the caller additionally clamps to
the record end.

Verified against Thor's own CSV exports for UM12947 (2025-07-14 .. 09-25,
167 waveforms, supplied as CSV.zip):

  length mismatches   22 -> 0
  per-sample exact    1,476,242 / 1,476,249

These are NOT truncated recordings, which was the competing hypothesis --
the exports carry the full sample count.

tests/test_waveform_codec.py asserted the cap as intended behaviour. That
assertion encoded an assumption, not a verified fact, and is replaced with
one pinning the opposite plus the evidence.

Across all three ground-truth corpora: 459 waveform files,
3,807,158 / 3,807,165 samples exact. Production IDFW is now 575/575 with
zero truncations and zero decode failures (median PPV error -0.0007% across
8 units). Series-3 re-verified unchanged at 14,338/14,338.

The 7 residual samples each differ by one 4th-decimal tick and are Thor's
own rounding: intersecting the per-sample rounding constraints over that
corpus is infeasible (binding pair contradict by 2.3e-11, 7e-5 relative),
so no single linear LSB reproduces every printed value. _GEO_LSB_IPS is
already pinned to ~1e-11; do not retune it to chase these.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL

2026-09-11 05:05:46 +00:00

serversdownandClaude Opus 5

c07aaa552c

fix(series4): support mic-disabled (3-channel) Thor units

Verified against a second Thor corpus (9-10-26-csv-req: UM11402, UM12947,
UM20147) with per-sample CSV exports: 139/139 waveforms exact
(1,273,380/1,273,380 samples) and 877/877 histograms within 2% of Thor's
reported PPV -- up from 66.9% and 56.6%.

Some units run with the microphone disabled, which changes two structural
things that were both hardcoded to the 4-channel shape:

- Waveform body head sat below the scan floor. A 3-channel unit has a
  shorter fixed header and puts its record chain head at 0x0dba, under the
  old _BODY_SCAN_FLOOR of 0x0E00. The scan could not see it and fell
  through to the Vert segment-0 record, decoding a body shifted one
  position around the channel rotation -- Vert came up exactly 512 samples
  short. Floor lowered to 0x0C00. The body-offset scoring also had to stop
  requiring four channels, or `equal` is permanently False for these events
  and the pick falls back to raw sample count.

- Histogram interval record is 56 bytes, not 72. It is
  16 * n_channels + 8, and is not inferable from the segment length alone.
  The interval count now comes from the segment's cumulative counter
  (n = counter - prev_counter) and the stride is derived from it. Assuming
  72 read 7 intervals out of every 10-interval segment, then walked off
  alignment into garbage that decoded as ~10 in/s peaks -- inflating some
  files' PPV by up to 191,000%. Also recovers 4 files that previously
  decoded no intervals at all.

Combined across both corpora: 292/292 waveform files,
2,330,916/2,330,916 samples exact. Production IDFW truncations 41 -> 22.
Series-3 unaffected (no shared-codec change in this commit; last full run
14,338/14,338).

Known open, diagnosed but NOT verified: the remaining 22 unequal + 1 failing
production IDFW files (all UM12947, 2025-07-14..09-23) stop the block walker
on tag 40 0c. data_block_len() caps the 40 NN int16 block at NN > 0x08 while
those files use NN up to 196. Both verified corpora only ever use
NN in {1,2,3,4,8}, so the cap is untested there and lifting it leaves both at
100.000% -- which is not evidence it decodes these correctly. Deliberately
not shipped; needs Thor CSV exports for UM12947 in that date range.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL

2026-09-10 20:00:56 +00:00

serversdownandClaude Opus 5

726c2ce1b5

fix(series4): Thor/Micromate decoder is now per-sample exact

Verified against Thor's own CSV exports, which carry a per-sample
four-column block beside every binary (CSV/<name>.IDFW.csv). Those 1,012
paired files were in the corpus all along; the decoder had been pinned to
a superseded walker on the stated grounds that "Thor has no ASCII ground
truth in the corpus and its geo scaling is separately suspect". Both
premises were false.

  IDFW per-sample exact      39.1%  -> 100.000% (1,057,536/1,057,536)
  IDFW files fully exact     0/153  -> 153/153
  IDFW PPV median error      -3.32% -> -0.002%
  IDFH within 2% of Thor PPV 51.1%  -> 100.0% (858/858)
  prod IDFW, 8 units         -3.3%  -> -0.001%

Four independent root causes:

- Geo LSB was 0.0003, the 4-dp *display rounding* of the real
  0.000310308 mistaken for the LSB, so every series-4 geophone sample
  read 3.3% low. Pinned to +-6e-11 by intersecting 991,415 rounding
  constraints; corroborated by the +-full-scale seed (+-32226) left in
  unwritten IDFH slots. IDFH had a separate, also wrong, 10.0/32768.

- IDFH histograms were capped at 250 intervals: the segment validator
  required the interval counter's high byte to be zero, but the counter
  is a uint16 cumulative index, so every segment past interval 255 was
  rejected. Runs over ~4 hours lost their tail, often the peak.
  540/858 corpus files affected.

- Record mode 00 00 (raw int16, 10-byte header) was unhandled and fell
  through the dispatch, silently dropping each channel's first 512
  samples -- the long-standing "loud events truncate" symptom.
  MODE_ABSOLUTE is now also accepted as a segment-0 preamble.

- The body-offset search matched 00 02 00 *inside* record headers,
  selecting a candidate part-way down the chain and decoding a
  rotation-shifted body. It now anchors on record headers and takes the
  chain head (6 ms/file).

Also fixes the separately tracked "UM-series decodes ~1000x low" bug.
Series-3 re-verified unchanged at 14,338/14,338 exact after the shared
waveform_codec change.

Known open: 41/575 prod IDFW files (7%, mostly UM12947/UM20147) decode
with unequal channel lengths and also fail metadata extraction -- a
different header variant with no Thor export in the store.

NOTE: this is a codec change; the Thor store owes a regeneration via
scripts/backfill_thor_events.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL

2026-09-10 18:06:14 +00:00