fix(histogram): partial final block no longer discards the correct stride

detect_multi_interval_stride() confirmed a candidate stride on a third block
header whenever the body was long enough to contain one. But a body can exceed
two strides and still hold only two real blocks: a partial final block leaves
trailing padding. BE18193 T193L0XM.CI0H — 51 intervals at 2 s, i.e. one full
30-interval block plus a 21-interval remainder in a 2787-byte body — had every
decisive check pass at stride 612 (header at 0, header at 612, block counter
256 -> 257) and was then rejected for the absent third header at 1224. It
decoded to nothing.

A missing third header now means end-of-stream rather than disqualification.
The block-counter check is untouched — that is the test that prevents the
false positives which once handed 9,082 standard-block files to the
multi-interval walker.

Found by running the full DL2 archive against its preserved Blastware ASCII
exports (14,340 paired files, 11x the previous ground-truth corpus).

Measured over 127,035 archive histogram binaries:
  recovered 8 files (strides 92, 252, 612; BE18193, BE18191, BE9557, BE9440)
  regressed 0 files
Full-corpus verification: 14,337 -> 14,338 exact of 14,338 decodable pairs
(the 2 excluded are series-4 IDF, a different codec).

Also adds scratch/verify_against_ascii.py (per-sample decoder verification
against BW exports, with a saturation carve-out — BW clamps clipped events to
the range max while the decoder reports true counts) and scratch/offset_scan.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
2026-08-28 20:19:30 +00:00
co-authored by Claude Opus 5
parent 75ac610c61
commit 14e997b20c
5 changed files with 453 additions and 4 deletions
+35
View File
@@ -643,3 +643,38 @@ def test_multi_interval_matches_blastware_ascii_exactly():
assert hz is None
elif not cell.startswith("<"):
assert hz is not None and abs(hz - float(cell)) <= max(0.55, float(cell) * 0.02)
def test_partial_final_block_is_not_disqualified_by_missing_third_header():
"""A body can exceed two strides yet hold only two real blocks.
Regression for BE18193 `T193L0XM.CI0H` — 51 intervals at 2 s = one full
30-interval block plus a 21-interval remainder, in a body long enough to
demand a third block header at ``2 * stride`` that does not exist. The
third-block confirmation used to be mandatory whenever the body was long
enough, so the correct stride was discarded and the file decoded to
nothing. A missing third header means end-of-stream, not disqualification;
the block-counter check is the decisive anti-false-positive test.
"""
full = [(1, 1, 2, 2, 3, 3, 4, 4)] * 30
partial = [(5, 5, 6, 6, 7, 7, 8, 8)] * 21
body = (_mk_multi_block(full, ctr=256)
+ _mk_multi_block(partial, ctr=257)
+ b"\xff" * 700) # trailing padding past 2 * stride
stride = 12 + 20 * 30
assert 2 * stride + 6 <= len(body), "padding must reach past two strides"
# the whole point: a third header is absent, and that must not disqualify
assert detect_multi_interval_stride(body) == stride
recs = walk_multi_interval_blocks(body)
assert len(recs) == 51
assert recs[0]["t_peak"] == 1
assert recs[-1]["t_peak"] == 5
def test_third_block_still_rejects_a_mismatched_counter():
"""The corroboration must still bite when a third block IS present."""
ivs = [(1, 1, 2, 2, 3, 3, 4, 4)] * 4
body = (_mk_multi_block(ivs, ctr=256)
+ _mk_multi_block(ivs, ctr=257)
+ _mk_multi_block(ivs, ctr=999)) # counter jumps — not consecutive
assert detect_multi_interval_stride(body) != 12 + 20 * 4