fix(histogram): block is big-endian + terminal block tail — 1/1196 to 1211/1211
Two errors in the series-3 histogram block model, both found by diffing against the per-interval data table in the preserved Blastware ASCII exports (1211 files in the prod snapshot — far stronger ground truth than the header PPV used previously). 1. The block is uniformly BIG-ENDIAN. Peaks and half-periods are uint16 BE (T_peak [5:7], T_halfperiod [7:9], V_peak [9:11], V_halfperiod [11:13], L_peak [13:15], L_halfperiod [15:17], M_peak [17:19], M_halfperiod [19:21]); only block_ctr [2:4] is little-endian. The old uint8-peak model silently CLIPPED any peak above 1.275 in/s: the final interval of BE18193/T193LQ9K.OE0H reads 8.270 in/s in BW's export (1654 counts = 0x0676) and decoded as 0x76 = 118 = 0.590. The byte documented as a per-channel "annotation" was never an annotation — it is the half-period's high byte, which is exactly why it was non-zero on the sub-Hz intervals BW renders as "<1.0". The marker is block[4] alone. Testing [4:6] as a uint16 LE marker forced block[5] == 0, which is what capped the peak at one byte. 2. The final block of each stream carries tail 9c 06 00 42 instead of 1e 0a 00 00, and holds arbitrary bytes at [21:23]. Rejecting it dropped the last interval of nearly every histogram — frequently the interval holding the event peak, so the file's PPV read low. Verified end to end through the production path: 1211/1211 histograms decode exactly (interval count + every per-interval peak), plus 842,442 per-interval frequency comparisons with zero mismatches. Previously 1 of 1196 files was fully correct. decode_histogram_body_full records expose `is_terminal` in place of the removed `annotations` tuple. +6 tests. No regressions: full-suite failure list unchanged from baseline. NOTE: stored histogram .h5 files need regenerating to pick this up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -1,3 +1,30 @@
|
||||
> ## SUPERSEDED 2026-08-25 — the block is uniformly BIG-ENDIAN
|
||||
>
|
||||
> The `uint8` peak / `annotation` byte model described below is wrong,
|
||||
> though it decoded quiet data correctly. The real layout:
|
||||
>
|
||||
> - **Every per-channel field is `uint16` big-endian.** `T_peak` is
|
||||
> `[5:7]`, `T_halfperiod` `[7:9]`, `V_peak` `[9:11]`, and so on.
|
||||
> Only `block_ctr` at `[2:4]` is little-endian.
|
||||
> - The **marker is `block[4]` alone**, not a `uint16 LE` at `[4:6]`.
|
||||
> Testing `[4:6] == 10` forced `block[5] == 0`, which is exactly what
|
||||
> capped every geo peak at one byte (255 counts = 1.275 in/s).
|
||||
> - The **"annotation" byte was never an annotation** — it is the high
|
||||
> byte of the big-endian half-period. That is why it was non-zero
|
||||
> precisely on the sub-Hz intervals Blastware renders as `<1.0`.
|
||||
> - The **final block of the stream carries tail `9c 06 00 42`** instead
|
||||
> of `1e 0a 00 00`, and arbitrary bytes at `[21:23]`. Rejecting it
|
||||
> dropped the last interval of nearly every histogram — often the one
|
||||
> holding the event peak, so the file's PPV read low.
|
||||
>
|
||||
> Verified against 1211 production histograms paired with their Blastware
|
||||
> ASCII exports: **1211/1211 decode exactly** (interval count plus every
|
||||
> per-interval peak), and 842,442 per-interval frequency comparisons match
|
||||
> with zero mismatches. The uint8 model scored 1204/1211 — the seven
|
||||
> failures are exactly the files containing a peak above 1.275 in/s.
|
||||
>
|
||||
> The section below is retained as the reasoning trail.
|
||||
|
||||
# Histogram body codec — FULLY DECODED (2026-05-20)
|
||||
|
||||
Clean working status doc for the MiniMate Plus histogram-mode event
|
||||
|
||||
Reference in New Issue
Block a user