fix(histogram): partial final block no longer discards the correct stride
detect_multi_interval_stride() confirmed a candidate stride on a third block header whenever the body was long enough to contain one. But a body can exceed two strides and still hold only two real blocks: a partial final block leaves trailing padding. BE18193 T193L0XM.CI0H — 51 intervals at 2 s, i.e. one full 30-interval block plus a 21-interval remainder in a 2787-byte body — had every decisive check pass at stride 612 (header at 0, header at 612, block counter 256 -> 257) and was then rejected for the absent third header at 1224. It decoded to nothing. A missing third header now means end-of-stream rather than disqualification. The block-counter check is untouched — that is the test that prevents the false positives which once handed 9,082 standard-block files to the multi-interval walker. Found by running the full DL2 archive against its preserved Blastware ASCII exports (14,340 paired files, 11x the previous ground-truth corpus). Measured over 127,035 archive histogram binaries: recovered 8 files (strides 92, 252, 612; BE18193, BE18191, BE9557, BE9440) regressed 0 files Full-corpus verification: 14,337 -> 14,338 exact of 14,338 decodable pairs (the 2 excluded are series-4 IDF, a different codec). Also adds scratch/verify_against_ascii.py (per-sample decoder verification against BW exports, with a saturation carve-out — BW clamps clipped events to the range max while the decoder reports true counts) and scratch/offset_scan.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -413,10 +413,18 @@ def detect_multi_interval_stride(body: bytes) -> Optional[int]:
|
||||
|
||||
if (_ctr(stride) - _ctr(0)) & 0xFFFF != 1:
|
||||
continue
|
||||
# confirm on a third block when the body is long enough
|
||||
if 2 * stride + _MULTI_HEADER_LEN <= len(body):
|
||||
if not _is_multi_header(body, 2 * stride):
|
||||
continue
|
||||
# Confirm on a third block WHEN ONE IS ACTUALLY PRESENT. A body can
|
||||
# be longer than two strides and still hold only two real blocks: a
|
||||
# final *partial* block leaves trailing padding. E.g. 51 intervals at
|
||||
# 2 s = one full 30-interval block + a 21-interval remainder, in a
|
||||
# 2787-byte body — long enough to demand a third header at 1224 that
|
||||
# does not exist. Requiring it unconditionally threw away the correct
|
||||
# stride and the file decoded to nothing (BE18193 T193L0XM.CI0H).
|
||||
# The block-counter check above is the decisive anti-false-positive
|
||||
# test; this one is corroboration, so a missing third header means
|
||||
# end-of-stream, not disqualification.
|
||||
if (2 * stride + _MULTI_HEADER_LEN <= len(body)
|
||||
and _is_multi_header(body, 2 * stride)):
|
||||
if (_ctr(2 * stride) - _ctr(stride)) & 0xFFFF != 1:
|
||||
continue
|
||||
return stride
|
||||
|
||||
Reference in New Issue
Block a user