fix(histogram): partial final block no longer discards the correct stride
detect_multi_interval_stride() confirmed a candidate stride on a third block header whenever the body was long enough to contain one. But a body can exceed two strides and still hold only two real blocks: a partial final block leaves trailing padding. BE18193 T193L0XM.CI0H — 51 intervals at 2 s, i.e. one full 30-interval block plus a 21-interval remainder in a 2787-byte body — had every decisive check pass at stride 612 (header at 0, header at 612, block counter 256 -> 257) and was then rejected for the absent third header at 1224. It decoded to nothing. A missing third header now means end-of-stream rather than disqualification. The block-counter check is untouched — that is the test that prevents the false positives which once handed 9,082 standard-block files to the multi-interval walker. Found by running the full DL2 archive against its preserved Blastware ASCII exports (14,340 paired files, 11x the previous ground-truth corpus). Measured over 127,035 archive histogram binaries: recovered 8 files (strides 92, 252, 612; BE18193, BE18191, BE9557, BE9440) regressed 0 files Full-corpus verification: 14,337 -> 14,338 exact of 14,338 decodable pairs (the 2 excluded are series-4 IDF, a different codec). Also adds scratch/verify_against_ascii.py (per-sample decoder verification against BW exports, with a saturation carve-out — BW clamps clipped events to the range max while the decoder reports true counts) and scratch/offset_scan.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -643,3 +643,38 @@ def test_multi_interval_matches_blastware_ascii_exactly():
|
||||
assert hz is None
|
||||
elif not cell.startswith("<"):
|
||||
assert hz is not None and abs(hz - float(cell)) <= max(0.55, float(cell) * 0.02)
|
||||
|
||||
|
||||
def test_partial_final_block_is_not_disqualified_by_missing_third_header():
|
||||
"""A body can exceed two strides yet hold only two real blocks.
|
||||
|
||||
Regression for BE18193 `T193L0XM.CI0H` — 51 intervals at 2 s = one full
|
||||
30-interval block plus a 21-interval remainder, in a body long enough to
|
||||
demand a third block header at ``2 * stride`` that does not exist. The
|
||||
third-block confirmation used to be mandatory whenever the body was long
|
||||
enough, so the correct stride was discarded and the file decoded to
|
||||
nothing. A missing third header means end-of-stream, not disqualification;
|
||||
the block-counter check is the decisive anti-false-positive test.
|
||||
"""
|
||||
full = [(1, 1, 2, 2, 3, 3, 4, 4)] * 30
|
||||
partial = [(5, 5, 6, 6, 7, 7, 8, 8)] * 21
|
||||
body = (_mk_multi_block(full, ctr=256)
|
||||
+ _mk_multi_block(partial, ctr=257)
|
||||
+ b"\xff" * 700) # trailing padding past 2 * stride
|
||||
stride = 12 + 20 * 30
|
||||
assert 2 * stride + 6 <= len(body), "padding must reach past two strides"
|
||||
# the whole point: a third header is absent, and that must not disqualify
|
||||
assert detect_multi_interval_stride(body) == stride
|
||||
recs = walk_multi_interval_blocks(body)
|
||||
assert len(recs) == 51
|
||||
assert recs[0]["t_peak"] == 1
|
||||
assert recs[-1]["t_peak"] == 5
|
||||
|
||||
|
||||
def test_third_block_still_rejects_a_mismatched_counter():
|
||||
"""The corroboration must still bite when a third block IS present."""
|
||||
ivs = [(1, 1, 2, 2, 3, 3, 4, 4)] * 4
|
||||
body = (_mk_multi_block(ivs, ctr=256)
|
||||
+ _mk_multi_block(ivs, ctr=257)
|
||||
+ _mk_multi_block(ivs, ctr=999)) # counter jumps — not consecutive
|
||||
assert detect_multi_interval_stride(body) != 12 + 20 * 4
|
||||
|
||||
Reference in New Issue
Block a user