fix(series4): support mic-disabled (3-channel) Thor units

Verified against a second Thor corpus (9-10-26-csv-req: UM11402, UM12947,
UM20147) with per-sample CSV exports: 139/139 waveforms exact
(1,273,380/1,273,380 samples) and 877/877 histograms within 2% of Thor's
reported PPV -- up from 66.9% and 56.6%.

Some units run with the microphone disabled, which changes two structural
things that were both hardcoded to the 4-channel shape:

- Waveform body head sat below the scan floor. A 3-channel unit has a
  shorter fixed header and puts its record chain head at 0x0dba, under the
  old _BODY_SCAN_FLOOR of 0x0E00. The scan could not see it and fell
  through to the Vert segment-0 record, decoding a body shifted one
  position around the channel rotation -- Vert came up exactly 512 samples
  short. Floor lowered to 0x0C00. The body-offset scoring also had to stop
  requiring four channels, or `equal` is permanently False for these events
  and the pick falls back to raw sample count.

- Histogram interval record is 56 bytes, not 72. It is
  16 * n_channels + 8, and is not inferable from the segment length alone.
  The interval count now comes from the segment's cumulative counter
  (n = counter - prev_counter) and the stride is derived from it. Assuming
  72 read 7 intervals out of every 10-interval segment, then walked off
  alignment into garbage that decoded as ~10 in/s peaks -- inflating some
  files' PPV by up to 191,000%. Also recovers 4 files that previously
  decoded no intervals at all.

Combined across both corpora: 292/292 waveform files,
2,330,916/2,330,916 samples exact. Production IDFW truncations 41 -> 22.
Series-3 unaffected (no shared-codec change in this commit; last full run
14,338/14,338).

Known open, diagnosed but NOT verified: the remaining 22 unequal + 1 failing
production IDFW files (all UM12947, 2025-07-14..09-23) stop the block walker
on tag 40 0c. data_block_len() caps the 40 NN int16 block at NN > 0x08 while
those files use NN up to 196. Both verified corpora only ever use
NN in {1,2,3,4,8}, so the cap is untested there and lifting it leaves both at
100.000% -- which is not evidence it decodes these correctly. Deliberately
not shipped; needs Thor CSV exports for UM12947 in that date range.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
2026-09-10 20:00:56 +00:00
co-authored by Claude Opus 5
parent 726c2ce1b5
commit c07aaa552c
5 changed files with 240 additions and 37 deletions
+30 -4
View File
@@ -64,10 +64,36 @@ quiet files the decoder is now *more* accurate than that reference.
New: `scratch/verify_thor_against_csv.py`, `tests/test_idf_binary_codec.py`
(10 tests, fixtures under `tests/fixtures/thor-idf/`).
**Known open:** 41/575 production IDFW files (7%, mostly UM12947/UM20147)
still decode with unequal channel lengths and also fail metadata extraction —
a different header variant with no Thor export in the store. Pull their CSV
exports before attempting a fix.
### Fixed — mic-disabled (3-channel) units
Verified on a second corpus (`9-10-26-csv-req`: UM11402, UM12947, UM20147) —
**139/139 waveforms per-sample exact (1,273,380 samples), 877/877 histograms
within 2%** (was 66.9% and 56.6%).
- **Waveform body head sat below the scan floor.** A 3-channel unit's shorter
header puts the record chain head at `0x0dba`, under the old
`_BODY_SCAN_FLOOR` of `0x0E00`. The scan couldn't see it and fell through
to the Vert segment-0 record, decoding a body shifted one position around
the channel rotation — Vert came up exactly 512 samples short. Floor
lowered to `0x0C00`; body-offset scoring now accepts 3 channels as "equal"
instead of demanding 4.
- **Histogram interval record is 56 bytes, not 72.** It is
`16 × n_channels + 8`, so mic-disabled units pack 56. Assuming 72 read 7
intervals out of every 10-interval segment then walked off alignment into
garbage decoding as ~10 in/s peaks (errors up to +191,000%). The interval
count now comes from the segment's cumulative counter and the stride is
derived from it; also recovers 4 files that decoded no intervals at all.
Combined across both corpora: **292/292 waveform files, 2,330,916/2,330,916
samples exact.** Production IDFW truncations 41 → 22.
**Known open — diagnosed but NOT verified:** 23/575 production IDFW files
(4%), all UM12947 between 2025-07-14 and 2025-09-23, stop the block walker on
tag `40 0c`. `data_block_len()` caps the `40 NN` int16 block at `NN > 0x08`,
but these files use NN up to 196. Both verified corpora only ever use
NN ∈ {1,2,3,4,8}, so the cap is untested there and lifting it leaves both at
100.000% — which is *not* evidence it decodes these correctly. Needs Thor CSV
exports for UM12947 in that date range (the 9-10-26 upload starts 2025-09-25).
---