Big event bugfix. see details:
## v0.13.0 — 2026-05-01 ### Fixed - **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.** `read_bulk_waveform_stream` was walking the chunk counter past the actual end of the event, picking up post-event circular-buffer garbage that corrupted reconstructed Blastware files for any waveform > ~1 sec. The loop now extracts the event's `end_offset` from the STRT record at `data[23:27]` of the probe response and stops the chunk walk when the next counter would step past it. Verified against three BW MITM captures (4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7 bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9. ### Added - `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)` — computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the formula confirmed across all 3 BW captures. Not yet wired into `read_bulk_waveform_stream` (the legacy TERM is still used to preserve the existing `blastware_file.write_blastware_file` frame-structure expectations); available for the next iteration that switches to BW's 0x0200 chunk step. - `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer from the STRT record in an A5 response payload.
This commit is contained in:
+130
-18
@@ -123,8 +123,11 @@ def build_5a_frame(offset_word: int, raw_params: bytes) -> bytes:
|
||||
Returns:
|
||||
Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX]
|
||||
"""
|
||||
if len(raw_params) not in (10, 11):
|
||||
raise ValueError(f"raw_params must be 10 or 11 bytes, got {len(raw_params)}")
|
||||
if len(raw_params) not in (10, 11, 12):
|
||||
# 10 = termination params; 11 = regular probe / chunk params;
|
||||
# 12 = metadata-page params (extra trailing 0x00 — BW byte-perfect quirk
|
||||
# for the two fixed metadata reads at counter=0x1002 and 0x1004).
|
||||
raise ValueError(f"raw_params must be 10/11/12 bytes, got {len(raw_params)}")
|
||||
|
||||
# Build stuffed section between STX and checksum
|
||||
s = bytearray()
|
||||
@@ -398,28 +401,21 @@ def bulk_waveform_params(key4: bytes, counter: int, *, is_probe: bool = False) -
|
||||
|
||||
def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
|
||||
"""
|
||||
Build the 10-byte params block for the SUB 5A termination request.
|
||||
DEPRECATED 2026-05-01 — see bulk_waveform_term_v2().
|
||||
|
||||
The termination request uses offset=0x005A and a DIFFERENT params layout —
|
||||
the leading 0x00 byte is dropped, key4[0:2] shifts to params[0:2], and the
|
||||
counter high byte is at params[2]:
|
||||
Build the 10-byte params block for the SUB 5A termination request, OLD layout
|
||||
(used in conjunction with the fixed offset_word=0x005A). Kept for backward
|
||||
compatibility — produces a tiny ~100-byte device-side terminator response
|
||||
rather than the proper partial-last-chunk + footer payload that BW gets.
|
||||
|
||||
params[0] = key4[0]
|
||||
params[1] = key4[1]
|
||||
params[2] = (counter >> 8) & 0xFF
|
||||
params[3:] = zeros
|
||||
|
||||
Counter for the termination request = last_regular_counter + 0x0400.
|
||||
|
||||
Confirmed from 1-2-26 BW TX capture: final request (frame 83) uses
|
||||
offset=0x005A, params[0:3] = key4[0:2] + term_counter_hi.
|
||||
|
||||
Args:
|
||||
key4: 4-byte waveform key.
|
||||
counter: Termination counter (= last regular counter + 0x0400).
|
||||
|
||||
Returns:
|
||||
10-byte params block.
|
||||
Use bulk_waveform_term_v2() for new code — it computes the verified
|
||||
offset_word + params from end_offset (extracted from STRT) and the last
|
||||
chunk counter.
|
||||
"""
|
||||
if len(key4) != 4:
|
||||
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
|
||||
@@ -430,6 +426,123 @@ def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
|
||||
return bytes(p)
|
||||
|
||||
|
||||
def bulk_waveform_term_v2(
|
||||
key4: bytes,
|
||||
end_offset: int,
|
||||
last_chunk_counter: int,
|
||||
) -> tuple[int, bytes]:
|
||||
"""
|
||||
Compute the SUB 5A TERM frame's offset_word and 10-byte params block.
|
||||
|
||||
Confirmed across 3 events (4-27-26 + 5-1-26 captures):
|
||||
|
||||
next_boundary = last_chunk_counter + 0x0200
|
||||
offset_word = end_offset - next_boundary (residual byte count)
|
||||
params[0] = key4[0] (= 0x01 on every observed device)
|
||||
params[1] = key4[1] (= 0x11)
|
||||
params[2] = (next_boundary >> 8) & 0xFF
|
||||
params[3] = next_boundary & 0xFF
|
||||
params[4:10] = zeros
|
||||
|
||||
Verification:
|
||||
| end_offset | last_chunk | next_boundary | offset_word | params[2:4] |
|
||||
| 0x1ABE | 0x1800 | 0x1A00 | 0x00BE | 1A 00 |
|
||||
| 0x21F2 | 0x1E00 | 0x2000 | 0x01F2 | 20 00 |
|
||||
| 0x417E | 0x3E38 | 0x4038 | 0x0146 | 40 38 |
|
||||
|
||||
The device receives `requested_address = (params[2] << 8) | offset_word`
|
||||
and replies with `(end_offset - next_boundary)` bytes of waveform tail
|
||||
starting at `next_boundary` — including the 26-byte file footer.
|
||||
|
||||
Args:
|
||||
key4: 4-byte waveform key for this event.
|
||||
end_offset: Event-end pointer (= `(end_key[2] << 8) | end_key[3]`
|
||||
from the STRT record at data[23:27] of A5[0]).
|
||||
last_chunk_counter: Counter of the last full 0x0200-byte chunk fetched
|
||||
(the chunk that covers [last_chunk_counter,
|
||||
last_chunk_counter + 0x0200)).
|
||||
|
||||
Returns:
|
||||
(offset_word, params10) tuple. Pass as
|
||||
`build_5a_frame(offset_word, params)`.
|
||||
|
||||
Raises:
|
||||
ValueError: on inconsistent inputs.
|
||||
"""
|
||||
if len(key4) != 4:
|
||||
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
|
||||
next_boundary = last_chunk_counter + 0x0200
|
||||
if next_boundary > 0xFFFF:
|
||||
raise ValueError(
|
||||
f"next_boundary 0x{next_boundary:04X} exceeds uint16; check inputs"
|
||||
)
|
||||
if end_offset <= last_chunk_counter:
|
||||
raise ValueError(
|
||||
f"end_offset 0x{end_offset:04X} must be > "
|
||||
f"last_chunk_counter 0x{last_chunk_counter:04X}"
|
||||
)
|
||||
offset_word = end_offset - next_boundary
|
||||
if offset_word < 0:
|
||||
# Last chunk overshot end_offset; caller should have stopped one chunk
|
||||
# earlier. Treat as zero residual.
|
||||
offset_word = 0
|
||||
if offset_word > 0xFFFF:
|
||||
raise ValueError(
|
||||
f"offset_word 0x{offset_word:04X} exceeds uint16"
|
||||
)
|
||||
p = bytearray(10)
|
||||
p[0] = key4[0]
|
||||
p[1] = key4[1]
|
||||
p[2] = (next_boundary >> 8) & 0xFF
|
||||
p[3] = next_boundary & 0xFF
|
||||
return offset_word, bytes(p)
|
||||
|
||||
|
||||
# ── End-offset extraction from STRT record ────────────────────────────────────
|
||||
|
||||
STRT_MARKER = b"STRT"
|
||||
|
||||
|
||||
def parse_strt_end_offset(a5_data: bytes) -> Optional[int]:
|
||||
"""
|
||||
Extract the event-end offset from the STRT record in an A5 response payload.
|
||||
|
||||
The first A5 response (the probe response, or the first chunk for events
|
||||
with non-zero start_key[2:4]) contains a STRT record at byte offset 17 of
|
||||
`data`. Layout:
|
||||
|
||||
data[17:21] "STRT"
|
||||
data[21:23] ff fe sentinel
|
||||
data[23:27] end_key ← 4-byte key of where this event ENDS
|
||||
data[27:31] start_key
|
||||
...
|
||||
|
||||
Returns `(end_key[2] << 8) | end_key[3]` — the absolute device-buffer
|
||||
address where the event ends. Use this to bound the chunk loop and to
|
||||
compute the TERM frame.
|
||||
|
||||
Verified end_offset values:
|
||||
| event start_key | end_key | end_offset |
|
||||
| 01110000 | 01111ABE | 0x1ABE |
|
||||
| 01110000 | 011121F2 | 0x21F2 |
|
||||
| 011121F2 | 0111417E | 0x417E |
|
||||
|
||||
Args:
|
||||
a5_data: The `data` field of an A5 response frame (frame.data).
|
||||
|
||||
Returns:
|
||||
The end_offset (uint16) if STRT is found, else None.
|
||||
"""
|
||||
pos = a5_data.find(STRT_MARKER)
|
||||
if pos < 0 or pos + 10 > len(a5_data):
|
||||
return None
|
||||
# data[pos+4:pos+6] is "ff fe"; data[pos+6:pos+10] is end_key.
|
||||
end_key = a5_data[pos + 6 : pos + 10]
|
||||
if len(end_key) < 4:
|
||||
return None
|
||||
return (end_key[2] << 8) | end_key[3]
|
||||
|
||||
|
||||
# ── Pre-built POLL frames ─────────────────────────────────────────────────────
|
||||
#
|
||||
# POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the
|
||||
@@ -470,7 +583,6 @@ class S3Frame:
|
||||
|
||||
|
||||
# ── Streaming S3 frame parser ─────────────────────────────────────────────────
|
||||
|
||||
class S3FrameParser:
|
||||
"""
|
||||
Incremental byte-stream parser for S3→BW response frames.
|
||||
|
||||
+45
-10
@@ -35,6 +35,8 @@ from .framing import (
|
||||
token_params,
|
||||
bulk_waveform_params,
|
||||
bulk_waveform_term_params,
|
||||
bulk_waveform_term_v2,
|
||||
parse_strt_end_offset,
|
||||
POLL_PROBE,
|
||||
POLL_DATA,
|
||||
SESSION_RESET,
|
||||
@@ -122,16 +124,21 @@ DATA_LENGTHS: dict[int, int] = {
|
||||
}
|
||||
|
||||
# SUB 5A (BULK_WAVEFORM_STREAM) protocol constants.
|
||||
# Confirmed from 1-2-26 BW TX capture analysis (2026-04-02).
|
||||
_BULK_CHUNK_OFFSET = 0x1004 # offset field for probe + all regular chunk requests ✅
|
||||
_BULK_TERM_OFFSET = 0x005A # offset field for termination request ✅
|
||||
_BULK_COUNTER_STEP = 0x0400 # chunk counter increment per chunk ✅
|
||||
# Chunk counter formula: key4[2:4] + (chunk_num - 1) * 0x0400
|
||||
# where key4[2:4] is the event's circular-buffer base offset ((key4[2]<<8)|key4[3]).
|
||||
# Earlier captures showed 0x1004 for chunk 1 of key 01110000 — that was a Blastware
|
||||
# artifact. For keys where key4[2:4] != 0x0000 (e.g. key 01111884) the old
|
||||
# "n * 0x0400" formula sends counters from the wrong buffer region and the device
|
||||
# returns data from a different event. Confirmed correct 2026-04-24.
|
||||
#
|
||||
# 2026-05-01 minimal-fix: the chunk-counter walk is now bounded by the event's
|
||||
# `end_offset` extracted from the STRT record at data[23:27] of the probe
|
||||
# response. Without this bound the loop kept asking for chunks past the event
|
||||
# end and the device responded with post-event circular-buffer garbage,
|
||||
# corrupting reconstructed Blastware files for events ≥ 2 sec.
|
||||
#
|
||||
# We keep the OLD 0x0400 chunk step here (BW actually uses 0x0200 — see §7.8.5
|
||||
# of the protocol reference for the corrected understanding) because the
|
||||
# existing blastware_file.py builder relies on the 0x0400-step frame structure
|
||||
# to produce valid files. Switching to BW's 0x0200 step is a separate task
|
||||
# that also requires updating the file builder.
|
||||
_BULK_CHUNK_OFFSET = 0x1004 # offset_word for probe + all chunk requests
|
||||
_BULK_TERM_OFFSET = 0x005A # offset_word for the legacy terminator
|
||||
_BULK_COUNTER_STEP = 0x0400 # chunk counter increment
|
||||
|
||||
# Default timeout values (seconds).
|
||||
# MiniMate Plus is a slow device — keep these generous.
|
||||
@@ -610,6 +617,24 @@ class MiniMateProtocol:
|
||||
frames_data.append(rsp)
|
||||
log.debug("5A A5[0] page_key=0x%04X %d bytes", rsp.page_key, len(rsp.data))
|
||||
|
||||
# ── Parse STRT end_offset from probe response (NEW 2026-05-01) ────────
|
||||
# The first A5 response contains a STRT record at data[17:]. The
|
||||
# bytes at data[23:27] are the event's end-key, whose low 16 bits
|
||||
# are the absolute device-buffer address where the event ends. Use
|
||||
# this to bound the chunk loop and stop the over-read past event end.
|
||||
# See docs/instantel_protocol_reference.md §7.8.5 and CLAUDE.md
|
||||
# "SUB 5A — STRT record encodes end_offset".
|
||||
_end_offset = parse_strt_end_offset(rsp.data)
|
||||
if _end_offset is None:
|
||||
# Defensive fallback — let max_chunks cap the walk.
|
||||
log.warning("5A: STRT not found in probe; cannot bound chunk loop")
|
||||
_end_offset = 0xFFFF
|
||||
else:
|
||||
log.debug(
|
||||
"5A STRT start_offset=0x%04X end_offset=0x%04X size=0x%04X",
|
||||
_key4_offset, _end_offset, _end_offset - _key4_offset,
|
||||
)
|
||||
|
||||
# ── Step 2: chunk loop ───────────────────────────────────────────────
|
||||
# Counter formula: _chunk_base + (chunk_num - 1) * 0x0400
|
||||
# where _chunk_base = max(key4[2:4], 0x0400).
|
||||
@@ -629,6 +654,16 @@ class MiniMateProtocol:
|
||||
_chunk_base = max(_key4_offset, _BULK_COUNTER_STEP)
|
||||
for chunk_num in range(1, max_chunks + 1):
|
||||
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
|
||||
# Stop when we'd step past the event end (NEW 2026-05-01). Without
|
||||
# this, the device returns post-event circular-buffer data which
|
||||
# corrupts the reconstructed file for events ≥ 2 sec.
|
||||
if counter >= _end_offset:
|
||||
log.debug(
|
||||
"5A chunk loop done at counter=0x%04X (end=0x%04X); "
|
||||
"%d chunks fetched",
|
||||
counter, _end_offset, len(frames_data),
|
||||
)
|
||||
break
|
||||
params = bulk_waveform_params(key4, counter)
|
||||
log.debug("5A chunk %d counter=0x%04X", chunk_num, counter)
|
||||
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
|
||||
|
||||
Reference in New Issue
Block a user