fix(series4): support mic-disabled (3-channel) Thor units

Verified against a second Thor corpus (9-10-26-csv-req: UM11402, UM12947,
UM20147) with per-sample CSV exports: 139/139 waveforms exact
(1,273,380/1,273,380 samples) and 877/877 histograms within 2% of Thor's
reported PPV -- up from 66.9% and 56.6%.

Some units run with the microphone disabled, which changes two structural
things that were both hardcoded to the 4-channel shape:

- Waveform body head sat below the scan floor. A 3-channel unit has a
  shorter fixed header and puts its record chain head at 0x0dba, under the
  old _BODY_SCAN_FLOOR of 0x0E00. The scan could not see it and fell
  through to the Vert segment-0 record, decoding a body shifted one
  position around the channel rotation -- Vert came up exactly 512 samples
  short. Floor lowered to 0x0C00. The body-offset scoring also had to stop
  requiring four channels, or `equal` is permanently False for these events
  and the pick falls back to raw sample count.

- Histogram interval record is 56 bytes, not 72. It is
  16 * n_channels + 8, and is not inferable from the segment length alone.
  The interval count now comes from the segment's cumulative counter
  (n = counter - prev_counter) and the stride is derived from it. Assuming
  72 read 7 intervals out of every 10-interval segment, then walked off
  alignment into garbage that decoded as ~10 in/s peaks -- inflating some
  files' PPV by up to 191,000%. Also recovers 4 files that previously
  decoded no intervals at all.

Combined across both corpora: 292/292 waveform files,
2,330,916/2,330,916 samples exact. Production IDFW truncations 41 -> 22.
Series-3 unaffected (no shared-codec change in this commit; last full run
14,338/14,338).

Known open, diagnosed but NOT verified: the remaining 22 unequal + 1 failing
production IDFW files (all UM12947, 2025-07-14..09-23) stop the block walker
on tag 40 0c. data_block_len() caps the 40 NN int16 block at NN > 0x08 while
those files use NN up to 196. Both verified corpora only ever use
NN in {1,2,3,4,8}, so the cap is untested there and lifting it leaves both at
100.000% -- which is not evidence it decodes these correctly. Deliberately
not shipped; needs Thor CSV exports for UM12947 in that date range.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
2026-09-10 20:00:56 +00:00
co-authored by Claude Opus 5
parent 726c2ce1b5
commit c07aaa552c
5 changed files with 240 additions and 37 deletions
+14 -3
View File
@@ -38,9 +38,20 @@ Read this first when picking the project back up.
matched `00 02 00` *inside* record headers, decoding a rotation-shifted
body. IDFW is no longer pinned to `decode_waveform_legacy`.
Series-3 re-verified unchanged at 14,338/14,338 after the shared-codec
change. **Still open:** 41/575 prod IDFW files (7%, mostly UM12947) decode
with unequal channel lengths and have no Thor export — pull their CSVs
before touching it.
change.
- **Mic-disabled (3-channel) units are a distinct shape (2026-09-10).**
Verified on a second corpus (`~/thor-csv-req`, UM11402/UM12947/UM20147):
**139/139** waveforms per-sample exact, **877/877** histograms within 2%.
Two structural differences: the shorter header puts the waveform record
chain head at `0x0dba` (below the old `_BODY_SCAN_FLOOR` of `0x0E00`, so it
was invisible and Vert came up exactly 512 short), and the histogram
interval record is **56 bytes, not 72** — `16 × n_channels + 8`, derived per
segment from the cumulative interval counter, never assumed.
**Still open:** 23/575 prod IDFW files (4%), all UM12947 2025-07-14..09-23,
stop the walker on tag `40 0c` — `data_block_len()` caps `40 NN` at NN>8 but
these use up to 196. **Diagnosed, NOT verified** — no Thor export exists for
those dates. Do not ship the cap change without one; see
`docs/idf_protocol_reference.md`.
- **Open, not blocking:** 14 sensitive-range files show an exact 8x
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
inserts DB rows for store files that have none (one-time per store) and the