histogram_codec: peak count is uint8 (not uint16 LE) — properly cracks

the BE9558 / BE18003 extension-byte case

The bytes at [7]/[11]/[15]/[19] are an annotation field (purpose still
unclear — empirically non-zero on intervals with sub-Hz or unmeasurable
freq), NOT the high byte of the peak count.  The N844 fixture corpus
the original RE was done against had zero values in those bytes for
every block, so uint8 and uint16 LE were equivalent there — but on
real BE9558 Tran-drift events and BE18003 Histogram+Continuous events
the uint16 LE interpretation produced peaks up to 268 in/s and 35×
inflated PVS sums.

Cross-correlated against BW's per-interval ASCII export on:
  - K558LKZU/LL1P/LL3K  → 100% T/V/L/M peak match (1435 blocks each)
  - T003LKZR/LL0O/LL1M  → 100% T/V/L, 99.3% M (0.05 dB rounding only)
  - N599LKZS/LL0L        → 100% all channels
  - N844 fixture corpus  → 100% all channels (unchanged)

Annotations preserved on every record for future RE; the defensive
_MAX_PEAK_COUNT bound is no longer needed (uint8 maxes at 1.275 in/s,
well below any physical limit).

Synthetic regression test added using the verbatim K558LKZU.RE0H
interval-12 block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-21 06:05:19 +00:00
parent e949232875
commit d506ebc103
4 changed files with 138 additions and 51 deletions
+35 -5
View File
@@ -12,7 +12,21 @@ implementation lives in `minimateplus/histogram_codec.py`.
in-repo histogram fixture corpus decodes byte-exact against BW's
ASCII export.
24 regression tests pass against ~3,500 blocks across 5 fixtures.
26 regression tests pass against ~3,500 blocks across 5 in-repo
fixtures, plus a synthetic regression block taken from a real
BE9558 prod event to lock in the uint8-peak interpretation.
**Important correction (2026-05-21):** the per-channel peak count
is `uint8` at byte[6]/[10]/[14]/[18], NOT `uint16 LE` at byte[6:8]
etc. The N844 fixture corpus the original RE was done against has
zero values in bytes [7]/[11]/[15]/[19] for every block, so the
two interpretations happened to be equivalent. Cross-correlating
non-N844 events (BE9558 Tran-drift, BE18003 Histogram+Continuous)
against BW's per-interval ASCII export — 4 channels × ~1400 blocks
per event × multiple events = 100% byte-exact only when the peak
is read as uint8. Reading as uint16 LE produced peaks up to 268
in/s per channel and 35× inflated PVS sums when first deployed to
prod (rolled back, root-caused, and fixed in commit 7183b95+1).
## Body format
@@ -27,15 +41,21 @@ Each block represents one histogram interval. Block layout:
[1] segment_id (uint8) 0x00..0x03 — 256 blocks per segment
[2:4] block_ctr (uint16 LE) resets each segment (0x0100, 0x0101, …)
[4:6] 0x000a (uint16 LE) constant marker (= 10)
[6:8] T_peak_count uint16 LE Tran peak (count × 0.005 → in/s at Normal)
[6] T_peak_count uint8 Tran peak (count × 0.005 → in/s at Normal,
max 1.275 in/s — fits in uint8)
[7] T_annotation uint8 empirically non-zero on intervals with sub-Hz
or unmeasurable freq; meaning not fully RE'd
[8:10] T_halfperiod uint16 LE Tran half-period in samples
(freq_Hz = 512 / halfp; ≤ 5 means ">100 Hz")
[10:12] V_peak_count uint16 LE Vert peak
[10] V_peak_count uint8 Vert peak
[11] V_annotation uint8
[12:14] V_halfperiod uint16 LE Vert freq half-period
[14:16] L_peak_count uint16 LE Long peak
[14] L_peak_count uint8 Long peak
[15] L_annotation uint8
[16:18] L_halfperiod uint16 LE Long freq half-period
[18:20] M_peak_count uint16 LE MicL peak count
[18] M_peak_count uint8 MicL peak count
(dB via waveform_codec.mic_count_to_db)
[19] M_annotation uint8
[20:22] M_halfperiod uint16 LE MicL freq half-period
[22:24] 0x00 0x00 constant
[24:28] 4-byte variable purpose unknown — possibly CRC,
@@ -99,6 +119,16 @@ slot[8] = 9 → 512/9 = 56.9 → 57 Hz ✓ M_freq
## What's NOT yet decoded
- **Annotation bytes (`block[7]/[11]/[15]/[19]`)**. Empirically
non-zero on intervals where the per-channel ZC frequency comes
out as `N/A` or sub-Hz (`<1.0`, `1.X`). Hypothesis tested in the
RE session: byte != 0 ↔ sub-Hz freq. Only ~50% correlation
across the K558 corpus, so the relationship is more complex.
Possibilities: time-of-peak-within-interval, halfp extension for
very-long-period signals, or a debug/diagnostic field the firmware
writes opportunistically. Doesn't affect peak amplitudes or
waveform reconstruction. Captured as `record["annotations"]` for
future RE.
- **4-byte variable metadata field (bytes 24:28)**. Not needed for
waveform reconstruction. Speculation: per-block CRC, sub-second
timestamp offset, or a Mic psi(L) count not in the 9 samples.