16 Commits

Author SHA1 Message Date
claude 7b62c790a9 fix(seismo-lab): remove duplicate capture history list 2026-05-04 14:30:46 -04:00
claude b66cc9d075 fix(blastware_file): update TERM detection logic and strip duplicate header blocks for accurate file writing 2026-05-04 14:28:11 -04:00
claude 45e61fbcaf big refactor of waveform protocol. 2026-05-03 01:20:21 -04:00
claude d758825c67 fix(protocol): correct continuous-mode record header classification for accurate timestamp extraction 2026-05-01 20:28:55 -04:00
claude 0fbb39c21a Big event bugfix. see details:
## v0.13.0 — 2026-05-01

### Fixed

- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
  `read_bulk_waveform_stream` was walking the chunk counter past the actual
  end of the event, picking up post-event circular-buffer garbage that
  corrupted reconstructed Blastware files for any waveform > ~1 sec.  The
  loop now extracts the event's `end_offset` from the STRT record at
  `data[23:27]` of the probe response and stops the chunk walk when the next
  counter would step past it.  Verified against three BW MITM captures
  (4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
  bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.

### Added

- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)` —
  computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
  formula confirmed across all 3 BW captures.  Not yet wired into
  `read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
  existing `blastware_file.write_blastware_file` frame-structure expectations);
  available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
  from the STRT record in an A5 response payload.
2026-05-01 18:37:34 -04:00
claude 738b39f3cb Manually Merged seismo lab persistent connection branch into the new direct download branch, creating a new branch called seismo-lab-new 2026-05-01 15:13:50 -04:00
Claude 625b0a4dfc feat(seismo_lab): add Download tab that captures wire bytes during event download
Adds a new CapturingTransport wrapper in minimateplus.transport that mirrors
every TX/RX byte to two raw .bin files using the same on-wire format as
bridges/ach_mitm.py, so the resulting captures are byte-for-byte compatible
with the existing Blastware MITM captures and load directly in the Analyzer.

A new "Download" tab in seismo_lab.py lets the user connect to a device over
TCP or serial and run connect / list-keys / download-events while the wrapper
saves raw_bw_<ts>.bin (our TX) and raw_s3_<ts>.bin (device TX) into a
seismo_dl_<ts>[_<label>]/ session directory. On completion, the panel hands
both files to the Analyzer and switches tabs, mirroring the UX of the
existing Bridge capture flow.
2026-05-01 00:12:02 +00:00
Claude b14f31f3b0 Include capture label in TCP raw filename
Matches serial bridge naming: raw_bw_{ts}_{label}.bin / raw_s3_{ts}_{label}.bin

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:48:10 +00:00
Claude b9ab368934 Fix TCP capture: write files only when capture is active
Previously every Blastware connection auto-created files.
Now TCP mode works the same as serial mode:
- Start Bridge: proxy listens and forwards silently, no files written
- New Capture: opens raw_bw/raw_s3 files; pipe threads write to them
- Stop Capture: flushes and closes files, fires Analyzer callback
- No connection = no file; multiple captures per bridge session work correctly

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:26:31 +00:00
Claude 9004241846 Restore multi-capture Bridge design + TCP mode
Brings back the protocol-exp BridgePanel design:
- Single bridge session stays up; New Capture / Stop Capture create
  labelled raw-file segments on demand (no files created at bridge start)
- Capture history listbox shows all segments; double-click reloads in Analyzer
- On capture complete: Analyzer auto-populates and runs analysis

TCP mode integrated into same tab (Serial/TCP radio toggle):
- Each incoming Blastware connection is automatically a capture segment
- Session appears in history list; Analyzer wires up live on connect
- Stop Capture disconnects current TCP session

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:20:43 +00:00
Claude 6861d9ed97 Merge TCP mode into Bridge tab (Serial/TCP radio toggle)
Removes the separate 'TCP Capture' tab and folds TCP MITM capture directly
into the existing Bridge tab.  A Serial/TCP radio selector at the top swaps
the connection fields (COM ports vs. listen port + device host:port) while
keeping the same Start Bridge / Stop Bridge / Add Mark buttons, capture
checkboxes, log dir, and live log — identical UX for both modes.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 23:01:45 +00:00
claude 5cd5652560 Merge branch 'seismo-lab' of https://github.com/serversdwn/seismo-relay into seismo-lab 2026-04-26 18:16:52 -04:00
Claude 897ac8a3f3 Add TCP MITM capture tab (TcpBridgePanel)
New 'TCP Capture' tab in seismo_lab.py: listens on a configurable local
port for an incoming Blastware connection, transparently forwards all
traffic to the real seismograph device, and saves both directions to
raw_bw_<ts>.bin / raw_s3_<ts>.bin in the same format the Analyzer already
understands.  Session start wires up Analyzer live mode automatically via
the same on_bridge_started callback as the COM-port bridge.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 22:10:48 +00:00
serversdown 310fc5986c Merge pull request 'seismo-lab2' (#7) from seismo-lab2 into seismo-lab
Reviewed-on: #7
2026-04-26 16:49:28 -04:00
Claude e1150b30aa fix(analyzer): name A5/5A frames; revert S3 checksum validation
Add 0x5A (BULK_WAVEFORM_STREAM) and 0xA5 (BULK_WAVEFORM_RESPONSE) to
SUB_TABLE so they display with real names instead of UNKNOWN_5A/A5.

Revert S3 checksum validation to checksum_valid=None (the original
intentional behavior). Large S3 frames (A5 bulk waveform, E5 compliance
config) embed inner DLE+ETX sub-frame delimiters; the trailing 0x03 of
the last inner delimiter can land where the parser expects the SUM8
checksum byte, causing false BAD CHK on every valid A5 frame.
protocol.py _validate_frame documents and ignores exactly this issue.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 20:40:45 +00:00
Claude 9bbecea70f fix(parser): correct S3 frame terminator — bare ETX, not DLE+ETX
parse_s3 had the S3 terminator logic inverted vs the real S3FrameParser
in framing.py. It was terminating on DLE+ETX and treating bare ETX as
payload, which caused every bare 0x03 to be swallowed — bundling multiple
real S3 frames into one giant body until a DLE+ETX sequence happened to
appear. Result: 583-byte POLL_RESPONSE 'frames' containing many real
frames concatenated, all showing BAD CHK.

Fix: mirror S3FrameParser exactly —
  - Bare ETX (0x03) = real frame terminator
  - DLE+ETX (0x10 0x03) = inner-frame literal data (A4/E5 sub-frames),
    appended to body and parsing continues

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 20:23:18 +00:00
13 changed files with 1605 additions and 503 deletions
+106
View File
@@ -4,6 +4,112 @@ All notable changes to seismo-relay are documented here.
---
## v0.13.2 — 2026-05-01
### Fixed
- **`_extract_record_type` — third 0C-record header format ("short", 8 bytes).**
A live SFM download against BE11529 produced files named `M5290000.000`
(zero-stamped) because the 0C waveform record's first bytes were
`01 05 07 ea ...` — neither the 9-byte single-shot layout (`0x10` at byte 1)
nor the 10-byte continuous layout (`0x10` at bytes 0 and 2). Investigation
showed this is a third format observed in the wild: an 8-byte header with no
marker bytes at all (`[day][month][year_BE:2][unknown][hour][min][sec]`).
The detection logic now scans the year (uint16 BE) at byte 2 / byte 3 / byte
4 and picks whichever offset returns a sensible year (20152050) — each
format has the year at a unique position so this disambiguates cleanly.
- New format → `event.record_type = "Waveform (Short)"`,
`Timestamp.from_short_record()`.
- Existing single-shot and continuous parsers unchanged.
- The user's event from May 1, 2026 13:21:37 now correctly resolves to a
filename like `M529LKIQ.G10` instead of `M5290000.000`.
### Added
- `Timestamp.from_short_record(data)` — decodes the 8-byte header.
- `_detect_record_format(data)` — internal helper returning
`"single_shot" / "continuous" / "short" / None` via year-position scan.
---
## v0.13.1 — 2026-05-01
### Fixed
- **`_extract_record_type` — Continuous-mode record headers misclassified as Unknown.**
In single-shot mode the 0C waveform record's 9-byte header puts the sub_code
marker `0x10` at byte 1, with the day at byte 0. In Continuous mode the
header is 10 bytes with the marker at byte 0 *and* byte 2, and the day at
byte 1. Previous logic only inspected byte 1 and treated any value other
than `0x10` / `0x03` as `"Unknown"`, which prevented `event.timestamp` from
being populated for any continuous-mode event whose day-of-month wasn't
exactly 3 or 16. As a downstream effect, `blastware_filename()` saw
`event.timestamp == None`, fell back to `stem="0000"` / `ab="00"`, and
produced filenames like `M5290000.000`. Discovered from a live SFM run on
BE11529 in continuous mode (day-of-month = 5).
Now disambiguates by checking BOTH byte 0 and byte 2: if both are `0x10`,
it's the 10-byte continuous header; else if byte 1 is `0x10`, it's the
9-byte single-shot header. Day-of-month no longer matters.
*Superseded by v0.13.2 — the user's actual record uses a third 8-byte format
with no `0x10` markers, which v0.13.1 still misclassified.*
---
## v0.13.0 — 2026-05-01
### Fixed
- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
`read_bulk_waveform_stream` was walking the chunk counter past the actual
end of the event, picking up post-event circular-buffer garbage that
corrupted reconstructed Blastware files for any waveform > ~1 sec. The
loop now extracts the event's `end_offset` from the STRT record at
`data[23:27]` of the probe response and stops the chunk walk when the next
counter would step past it. Verified against three BW MITM captures
(4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.
### Added
- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`
computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
formula confirmed across all 3 BW captures. Not yet wired into
`read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
existing `blastware_file.write_blastware_file` frame-structure expectations);
available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
from the STRT record in an A5 response payload.
### Documentation
- **CLAUDE.md and `docs/instantel_protocol_reference.md` extensively
rewritten** to reflect the corrected SUB 5A protocol. See:
- CLAUDE.md "SUB 5A — chunk counter formula (REWRITTEN 2026-05-01)"
- CLAUDE.md "SUB 5A — STRT record encodes end_offset"
- CLAUDE.md "SUB 5A — TERM frame formula"
- CLAUDE.md "SUB 5A — fixed metadata pages 0x1002 and 0x1004"
- CLAUDE.md "SUB 0A — WAVEHDR response length distinguishes events from
boundaries" (0x46 = real event, 0x2C = boundary marker)
- protocol reference §7.8.5 / §7.8.6 / §7.8.7 / §7.8.8
- The previous chunk-counter formula (`max(key4[2:4], 0x0400) + (chunk-1) *
0x0400`) is now marked DEPRECATED and explicitly tagged WRONG with
pointers to the new sections, so future work doesn't re-derive it.
### Known minor diffs vs Blastware (deferred to a follow-up)
- We still use the OLD 0x0400 chunk step rather than BW's 0x0200; switching
also requires updating `blastware_file.write_blastware_file`'s skip values
and "extra chunk after metadata" logic, which depends on a fresh capture
to verify.
- We still use the legacy fixed `offset_word=0x005A` TERM frame rather than
BW's `end_offset - next_boundary` formula, for the same reason.
- Two fixed metadata pages at counter `0x1002` and `0x1004` are not yet
read explicitly; under the current 0x0400 walk their content is reachable
via the sample chunk that covers buffer addresses `[0x1000, 0x1400)`.
---
## v0.12.5 — 2026-04-21
### Changed
+2 -32
View File
@@ -2,7 +2,7 @@
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.12.3**.
(Sierra Wireless RV50 / RV55). Current version: **v0.13.2**.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
@@ -41,7 +41,7 @@ Full read pipeline + write pipeline + erase pipeline + monitor log + call home c
| Event header / first key | 1E | ✅ |
| Waveform header | 0A | ✅ |
| Waveform record (peaks, timestamp, project) | 0C | ✅ |
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 |
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ over-read bug fixed v0.13.0 (chunk loop bounded by STRT end_offset); minor wire diffs vs BW deferred — see "SUB 5A — chunk counter formula" |
| Event advance / next key | 1F | ✅ |
| **Write commands (push config to device)** | **6883** | ✅ new v0.8.0 |
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
@@ -347,36 +347,6 @@ Do NOT use fixed absolute offsets for sample_rate or record_time.
Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to
`S3FrameParser`.
**SUB 5A (bulk waveform) TCP frame splitting — confirmed 2026-04-27:**
Over TCP via cellular modem, each 5A chunk request that produces a single ~1100-byte
A5 response over direct RS-232 may arrive as **two separate, complete S3 frames** of
~550 bytes each ("2-frame mode"). The modem's Data Forwarding Timeout (~100-150 ms)
can split the RS-232 response into two TCP segments, each parsed as a complete S3 frame.
Under different modem/timing conditions the full ~1100-byte response arrives as **one
S3 frame** ("1-frame mode").
**Both modes require `extra_chunks_after_metadata=1`** (the extra chunk at metadata_counter
+ 0x0400). The device's waveform footer data lives at circular-buffer address 0x1C00 for
this event; the terminator frame must be sent at 0x1C00 (not 0x1800) to receive it.
Example for a 2-second Continuous event (BE11529, key=01110000) via TCP:
- **2-frame mode:** 1 probe frame (554 B) + 5 chunks × 2 frames (556-573 B) + 1 extra chunk × 2 frames + 1 terminator (208 B) = **14 A5 frames** → 6864-byte file
- **1-frame mode:** 1 probe frame (~1097 B) + 5 chunks × 1 frame (~1079-1113 B) + 1 extra chunk × 1 frame (smaller, tail of event) + 1 terminator → **8 A5 frames** → 6864-byte file
- All frames contribute body data; using all of them gives the correct file.
**Fix (confirmed 2026-04-27):** `_recv_5a_batch()` in `protocol.py` collects ALL
A5 frames per chunk request before the next request is sent, using a 0.5 s batch
timeout after the first frame to catch the ~150 ms delayed second frame. `write_blastware_file()`
includes ALL body frames without skipping — the extra chunk's frames are part of the
body data, NOT padding to be discarded.
**WRONG earlier hypothesis (do not re-introduce):** An attempt was made to auto-detect
1-frame vs 2-frame mode from the probe frame size and skip the extra chunk when
`probe_data_len >= 700`. This was wrong — the extra chunk is always needed to advance
the device's internal state to the footer address. The `_probe_is_large` branch was
removed 2026-04-27.
### Required ACEmanager settings (Sierra Wireless RV50/RV55)
| Setting | Value | Why |
+10 -2
View File
@@ -21,7 +21,15 @@ Typical usage (TCP / modem):
from .client import MiniMateClient
from .models import DeviceInfo, Event, MonitorLogEntry
from .transport import SerialTransport, TcpTransport
from .transport import CapturingTransport, SerialTransport, TcpTransport
__version__ = "0.1.0"
__all__ = ["MiniMateClient", "DeviceInfo", "Event", "MonitorLogEntry", "SerialTransport", "TcpTransport"]
__all__ = [
"MiniMateClient",
"DeviceInfo",
"Event",
"MonitorLogEntry",
"SerialTransport",
"TcpTransport",
"CapturingTransport",
]
+58 -27
View File
@@ -672,11 +672,10 @@ def write_blastware_file(
# Do NOT use a5_frames[-1] — if _a5_frames contains stray frames from a
# subsequent event (a known get_events side-effect), the last frame will
# not be the terminator and the footer will be mis-identified.
# TERM detection (v0.14.0): last frame if page_key != 0x0010 (sample marker)
term_idx: Optional[int] = None
for _i, _f in enumerate(a5_frames):
if _f.page_key == 0x0000:
term_idx = _i
break
if a5_frames and a5_frames[-1].page_key != 0x0010:
term_idx = len(a5_frames) - 1
if term_idx is not None:
body_frames = a5_frames[:term_idx]
@@ -685,34 +684,28 @@ def write_blastware_file(
body_frames = a5_frames
term_frame = None
log.warning(
"write_blastware_file: %d body_frames term_idx=%s",
len(body_frames),
str(term_idx) if term_idx is not None else "None",
# Frame contribution loop (v0.14.0 BW-exact walk).
# Skip values:
# probe (fi=0): probe_skip
# meta@0x1002 (fi=1): 13 (6-byte inner header)
# meta@0x1004 (fi=2): 13 (6-byte inner header)
# sample chunks (fi=3+): 12 (5-byte inner header)
last_fi = len(body_frames) - 1
log.debug(
"write_blastware_file: %d body_frames last_fi=%d",
len(body_frames), last_fi,
)
all_bytes = bytearray()
for fi, frame in enumerate(body_frames):
# All body frames contribute to the waveform body — no frames are skipped.
#
# Over TCP via cellular modem, _recv_5a_batch() correctly collects all
# A5 frames per chunk request (the device's ~1100-byte RS-232 response
# is forwarded as ~2 TCP segments of ~550 bytes each, each parsed as a
# separate S3 frame). ALL of these frames contain ADC body data and
# must be included in the file — confirmed from 4-27-26 TCP capture
# analysis: contributions from all 14 frames → 6821 bytes → file 6864 bytes.
#
# Skip amounts (offsets into frame.data):
# fi=0 (probe): probe_skip — skips the type_tag header + STRT record
# fi=1: 13 — 7-byte frame.data prefix + 6 inner header bytes
# fi>=2: 12 — 7-byte frame.data prefix + 5 inner header bytes
if fi == 0:
skip = probe_skip
elif fi == 1:
skip = 13
elif fi in (1, 2):
skip = 13 # metadata pages
else:
skip = 12
skip = 12 # sample chunks
contribution = _frame_body_bytes(frame, skip)
log.warning("write_blastware_file: fi=%d skip=%d raw_data=%d contribution=%d",
@@ -739,11 +732,49 @@ def write_blastware_file(
bytes(all_bytes[-28:]).hex() if len(all_bytes) >= 28 else bytes(all_bytes).hex(),
)
if len(all_bytes) >= 26:
# Strip embedded "duplicate header+STRT" blocks from body (v0.14.1).
# Chunk@0x1000 sometimes lands on the device's metadata-mirror page,
# whose response includes a 25-byte "00 12 03 00 STRT ..." block that
# mirrors the file's own header + STRT record. BW treats embedded STRT
# markers as second-event starts and rejects the file. Replace these
# blocks with zeros to preserve file size + alignment.
needle = b"\x00\x12\x03\x00STRT"
pos = bytes(all_bytes).find(needle)
while pos >= 0:
end = pos + 25
if end <= len(all_bytes):
all_bytes[pos:end] = b"\x00" * 25
log.warning(
"write_blastware_file: stripped duplicate header+STRT at "
"all_bytes[%d:%d] (replaced with 25 zero-bytes)",
pos, end,
)
pos = bytes(all_bytes).find(needle, end)
# Find the first valid 0e 08 footer marker (v0.14.0).
footer_pos = -1
pos = 0
while True:
pos = bytes(all_bytes).find(b"\x0e\x08", pos)
if pos < 0 or pos + 26 > len(all_bytes):
break
yr = (all_bytes[pos + 4] << 8) | all_bytes[pos + 5]
if 2015 <= yr <= 2050:
footer_pos = pos
break
pos += 1
if footer_pos >= 0:
body = bytes(all_bytes[:footer_pos])
footer = bytes(all_bytes[footer_pos:footer_pos + 26])
log.warning(
"write_blastware_file: real 0e 08 footer at all_bytes[%d]; "
"truncating %d post-footer bytes",
footer_pos, len(all_bytes) - footer_pos - 26,
)
elif len(all_bytes) >= 26:
body = bytes(all_bytes[:-26])
footer = bytes(all_bytes[-26:])
else:
# Fallback: no terminator or very short stream → build footer from event metadata
body = bytes(all_bytes)
start_dt = _ts_from_model(event.timestamp)
stop_dt: Optional[datetime.datetime] = None
@@ -754,7 +785,7 @@ def write_blastware_file(
+ _encode_ts_be(start_dt)
+ _encode_ts_be(stop_dt)
+ b"\x00\x01\x00\x02\x00\x00"
+ b"\x00\x00" # CRC placeholder
+ b"\x00\x00"
)
# ── Write file ───────────────────────────────────────────────────────────
+66 -22
View File
@@ -1345,6 +1345,11 @@ def _decode_waveform_record_into(data: bytes, event: Event) -> None:
event.timestamp = Timestamp.from_continuous_record(data)
except Exception as exc:
log.warning("continuous record timestamp decode failed: %s", exc)
elif event.record_type == "Waveform (Short)":
try:
event.timestamp = Timestamp.from_short_record(data)
except Exception as exc:
log.warning("short record timestamp decode failed: %s", exc)
# ── Peak values (per-channel PPV + Peak Vector Sum) ───────────────────────
try:
@@ -1636,34 +1641,73 @@ def _decode_a5_waveform(
}
def _detect_record_format(data: bytes) -> Optional[str]:
"""
Detect which timestamp-header format a 210-byte 0C waveform record uses.
THREE formats observed on BE11529 firmware S338.17:
"single_shot" — 9-byte header:
[day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
sub_code=0x10 at byte [1]. Year at [3:5].
"continuous" — 10-byte header:
[0x10] [day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
marker 0x10 at byte [0] AND byte [2]. Year at [4:6].
"short" — 8-byte header (NEW 2026-05-01):
[day] [month] [year_BE:2] [unknown] [hour] [min] [sec]
No marker bytes. Year at [2:4].
Each format has the year (uint16 BE) at a UNIQUE byte position, so we can
disambiguate by scanning each candidate position and picking the one
where the year falls in a sane range (2015..2050).
Returns "single_shot" / "continuous" / "short" or None if no format matches.
"""
if len(data) < 8:
return None
def _sane_year(hi: int, lo: int) -> bool:
y = (hi << 8) | lo
return 2015 <= y <= 2050
# Order matters: prefer formats with stronger marker-byte evidence first.
if data[1] == 0x10 and len(data) >= 9 and _sane_year(data[3], data[4]):
return "single_shot"
if (data[0] == 0x10 and data[2] == 0x10
and len(data) >= 10 and _sane_year(data[4], data[5])):
return "continuous"
if _sane_year(data[2], data[3]):
return "short"
return None
def _extract_record_type(data: bytes) -> Optional[str]:
"""
Decode the recording mode from byte[1] of the 210-byte waveform record.
Return a human-readable name for the waveform record format detected
in the first bytes of a 210-byte 0C record.
Byte[1] is the sub-record code that immediately follows the day byte in the
9-byte timestamp header at the start of each waveform record:
[day:1] [sub_code:1] [month:1] [year:2 BE] ...
Confirmed codes (✅ 2026-04-01):
0x10 → "Waveform" (continuous / single-shot mode)
Histogram mode code is not yet confirmed — a histogram event must be
captured with debug=true to identify it. Returns None for unknown codes.
Maps to the format codes returned by _detect_record_format():
"single_shot""Waveform"
"continuous""Waveform (Continuous)"
"short""Waveform (Short)"
None → "Unknown(XX.YY.ZZ)"
"""
if len(data) < 2:
return None
code = data[1]
if code == 0x10:
fmt = _detect_record_format(data)
if fmt == "single_shot":
return "Waveform"
if code == 0x03:
# Continuous mode waveform record (confirmed by user — NOT a monitor log).
# The byte layout differs from 0x10 single-shot records: the timestamp
# fields decode as garbage under the 0x10 waveform layout.
# TODO: confirm correct timestamp layout for 0x03 records from a known-time event.
if fmt == "continuous":
return "Waveform (Continuous)"
log.warning("_extract_record_type: unknown sub_code=0x%02X", code)
return f"Unknown(0x{code:02X})"
if fmt == "short":
return "Waveform (Short)"
if len(data) >= 3:
log.warning(
"_extract_record_type: unrecognized header: data[0:3]=%02X %02X %02X",
data[0], data[1], data[2],
)
return f"Unknown({data[0]:02X}.{data[1]:02X}.{data[2]:02X})"
return None
def _extract_peak_floats(data: bytes) -> Optional[PeakValues]:
"""
+130 -18
View File
@@ -123,8 +123,11 @@ def build_5a_frame(offset_word: int, raw_params: bytes) -> bytes:
Returns:
Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX]
"""
if len(raw_params) not in (10, 11):
raise ValueError(f"raw_params must be 10 or 11 bytes, got {len(raw_params)}")
if len(raw_params) not in (10, 11, 12):
# 10 = termination params; 11 = regular probe / chunk params;
# 12 = metadata-page params (extra trailing 0x00 — BW byte-perfect quirk
# for the two fixed metadata reads at counter=0x1002 and 0x1004).
raise ValueError(f"raw_params must be 10/11/12 bytes, got {len(raw_params)}")
# Build stuffed section between STX and checksum
s = bytearray()
@@ -398,28 +401,21 @@ def bulk_waveform_params(key4: bytes, counter: int, *, is_probe: bool = False) -
def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
"""
Build the 10-byte params block for the SUB 5A termination request.
DEPRECATED 2026-05-01 — see bulk_waveform_term_v2().
The termination request uses offset=0x005A and a DIFFERENT params layout
the leading 0x00 byte is dropped, key4[0:2] shifts to params[0:2], and the
counter high byte is at params[2]:
Build the 10-byte params block for the SUB 5A termination request, OLD layout
(used in conjunction with the fixed offset_word=0x005A). Kept for backward
compatibility — produces a tiny ~100-byte device-side terminator response
rather than the proper partial-last-chunk + footer payload that BW gets.
params[0] = key4[0]
params[1] = key4[1]
params[2] = (counter >> 8) & 0xFF
params[3:] = zeros
Counter for the termination request = last_regular_counter + 0x0400.
Confirmed from 1-2-26 BW TX capture: final request (frame 83) uses
offset=0x005A, params[0:3] = key4[0:2] + term_counter_hi.
Args:
key4: 4-byte waveform key.
counter: Termination counter (= last regular counter + 0x0400).
Returns:
10-byte params block.
Use bulk_waveform_term_v2() for new code — it computes the verified
offset_word + params from end_offset (extracted from STRT) and the last
chunk counter.
"""
if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
@@ -430,6 +426,123 @@ def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
return bytes(p)
def bulk_waveform_term_v2(
key4: bytes,
end_offset: int,
last_chunk_counter: int,
) -> tuple[int, bytes]:
"""
Compute the SUB 5A TERM frame's offset_word and 10-byte params block.
Confirmed across 3 events (4-27-26 + 5-1-26 captures):
next_boundary = last_chunk_counter + 0x0200
offset_word = end_offset - next_boundary (residual byte count)
params[0] = key4[0] (= 0x01 on every observed device)
params[1] = key4[1] (= 0x11)
params[2] = (next_boundary >> 8) & 0xFF
params[3] = next_boundary & 0xFF
params[4:10] = zeros
Verification:
| end_offset | last_chunk | next_boundary | offset_word | params[2:4] |
| 0x1ABE | 0x1800 | 0x1A00 | 0x00BE | 1A 00 |
| 0x21F2 | 0x1E00 | 0x2000 | 0x01F2 | 20 00 |
| 0x417E | 0x3E38 | 0x4038 | 0x0146 | 40 38 |
The device receives `requested_address = (params[2] << 8) | offset_word`
and replies with `(end_offset - next_boundary)` bytes of waveform tail
starting at `next_boundary` — including the 26-byte file footer.
Args:
key4: 4-byte waveform key for this event.
end_offset: Event-end pointer (= `(end_key[2] << 8) | end_key[3]`
from the STRT record at data[23:27] of A5[0]).
last_chunk_counter: Counter of the last full 0x0200-byte chunk fetched
(the chunk that covers [last_chunk_counter,
last_chunk_counter + 0x0200)).
Returns:
(offset_word, params10) tuple. Pass as
`build_5a_frame(offset_word, params)`.
Raises:
ValueError: on inconsistent inputs.
"""
if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
next_boundary = last_chunk_counter + 0x0200
if next_boundary > 0xFFFF:
raise ValueError(
f"next_boundary 0x{next_boundary:04X} exceeds uint16; check inputs"
)
if end_offset <= last_chunk_counter:
raise ValueError(
f"end_offset 0x{end_offset:04X} must be > "
f"last_chunk_counter 0x{last_chunk_counter:04X}"
)
offset_word = end_offset - next_boundary
if offset_word < 0:
# Last chunk overshot end_offset; caller should have stopped one chunk
# earlier. Treat as zero residual.
offset_word = 0
if offset_word > 0xFFFF:
raise ValueError(
f"offset_word 0x{offset_word:04X} exceeds uint16"
)
p = bytearray(10)
p[0] = key4[0]
p[1] = key4[1]
p[2] = (next_boundary >> 8) & 0xFF
p[3] = next_boundary & 0xFF
return offset_word, bytes(p)
# ── End-offset extraction from STRT record ────────────────────────────────────
STRT_MARKER = b"STRT"
def parse_strt_end_offset(a5_data: bytes) -> Optional[int]:
"""
Extract the event-end offset from the STRT record in an A5 response payload.
The first A5 response (the probe response, or the first chunk for events
with non-zero start_key[2:4]) contains a STRT record at byte offset 17 of
`data`. Layout:
data[17:21] "STRT"
data[21:23] ff fe sentinel
data[23:27] end_key ← 4-byte key of where this event ENDS
data[27:31] start_key
...
Returns `(end_key[2] << 8) | end_key[3]` — the absolute device-buffer
address where the event ends. Use this to bound the chunk loop and to
compute the TERM frame.
Verified end_offset values:
| event start_key | end_key | end_offset |
| 01110000 | 01111ABE | 0x1ABE |
| 01110000 | 011121F2 | 0x21F2 |
| 011121F2 | 0111417E | 0x417E |
Args:
a5_data: The `data` field of an A5 response frame (frame.data).
Returns:
The end_offset (uint16) if STRT is found, else None.
"""
pos = a5_data.find(STRT_MARKER)
if pos < 0 or pos + 10 > len(a5_data):
return None
# data[pos+4:pos+6] is "ff fe"; data[pos+6:pos+10] is end_key.
end_key = a5_data[pos + 6 : pos + 10]
if len(end_key) < 4:
return None
return (end_key[2] << 8) | end_key[3]
# ── Pre-built POLL frames ─────────────────────────────────────────────────────
#
# POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the
@@ -470,7 +583,6 @@ class S3Frame:
# ── Streaming S3 frame parser ─────────────────────────────────────────────────
class S3FrameParser:
"""
Incremental byte-stream parser for S3→BW response frames.
+52
View File
@@ -201,6 +201,58 @@ class Timestamp:
second=second,
)
@classmethod
def from_short_record(cls, data: bytes) -> "Timestamp":
"""
Decode an 8-byte timestamp header from a 210-byte waveform record.
Wire layout (✅ CONFIRMED 2026-05-01 against live SFM run on BE11529 in
Continuous mode, day-of-month = 1 May, raw: 01 05 07 ea 00 0d 15 25):
byte[0]: day (uint8)
byte[1]: month (uint8)
bytes[2-3]: year (big-endian uint16)
byte[4]: unknown (0x00 in observed sample)
byte[5]: hour (uint8)
byte[6]: minute (uint8)
byte[7]: second (uint8)
This is a third format observed in the wild — distinct from the 9-byte
(single-shot, sub_code=0x10 at [1]) and 10-byte (continuous, 0x10 at
[0] AND [2]) layouts. No marker bytes; disambiguated by where the
year lands when scanned at byte 2/3/4.
Args:
data: at least 8 bytes; only the first 8 are consumed.
Returns:
Decoded Timestamp.
Raises:
ValueError: if data is fewer than 8 bytes.
"""
if len(data) < 8:
raise ValueError(
f"Short record timestamp requires at least 8 bytes, got {len(data)}"
)
day = data[0]
month = data[1]
year = struct.unpack_from(">H", data, 2)[0]
unknown_byte = data[4]
hour = data[5]
minute = data[6]
second = data[7]
return cls(
raw=bytes(data[:8]),
flag=0,
year=year,
unknown_byte=unknown_byte,
month=month,
day=day,
hour=hour,
minute=minute,
second=second,
)
@property
def clock_set(self) -> bool:
"""False when year == 1995 (factory default / battery-lost state)."""
+217 -229
View File
@@ -35,6 +35,8 @@ from .framing import (
token_params,
bulk_waveform_params,
bulk_waveform_term_params,
bulk_waveform_term_v2,
parse_strt_end_offset,
POLL_PROBE,
POLL_DATA,
SESSION_RESET,
@@ -122,16 +124,22 @@ DATA_LENGTHS: dict[int, int] = {
}
# SUB 5A (BULK_WAVEFORM_STREAM) protocol constants.
# Confirmed from 1-2-26 BW TX capture analysis (2026-04-02).
_BULK_CHUNK_OFFSET = 0x1004 # offset field for probe + all regular chunk requests ✅
_BULK_TERM_OFFSET = 0x005A # offset field for termination request ✅
_BULK_COUNTER_STEP = 0x0400 # chunk counter increment per chunk ✅
# Chunk counter formula: key4[2:4] + (chunk_num - 1) * 0x0400
# where key4[2:4] is the event's circular-buffer base offset ((key4[2]<<8)|key4[3]).
# Earlier captures showed 0x1004 for chunk 1 of key 01110000 — that was a Blastware
# artifact. For keys where key4[2:4] != 0x0000 (e.g. key 01111884) the old
# "n * 0x0400" formula sends counters from the wrong buffer region and the device
# returns data from a different event. Confirmed correct 2026-04-24.
#
# 2026-05-01 minimal-fix: the chunk-counter walk is now bounded by the event's
# `end_offset` extracted from the STRT record at data[23:27] of the probe
# response. Without this bound the loop kept asking for chunks past the event
# end and the device responded with post-event circular-buffer garbage,
# corrupting reconstructed Blastware files for events ≥ 2 sec.
#
# We keep the OLD 0x0400 chunk step here (BW actually uses 0x0200 — see §7.8.5
# of the protocol reference for the corrected understanding) because the
# existing blastware_file.py builder relies on the 0x0400-step frame structure
# to produce valid files. Switching to BW's 0x0200 step is a separate task
# that also requires updating the file builder.
# BW-exact protocol values (v0.14.0). Verified against 4-27-26 + 5-1-26 captures.
_BULK_CHUNK_OFFSET = 0x1002 # offset_word for probe + all chunk requests
_BULK_TERM_OFFSET = 0x005A # offset_word for the legacy terminator (fallback only)
_BULK_COUNTER_STEP = 0x0200 # chunk counter increment (matches chunk payload size)
# Default timeout values (seconds).
# MiniMate Plus is a slow device — keep these generous.
@@ -526,223 +534,260 @@ class MiniMateProtocol:
self,
key4: bytes,
*,
stop_after_metadata: bool = True,
max_chunks: int = 32,
stop_after_metadata: bool = True, # DEPRECATED — no-op under BW-exact walk
max_chunks: int = 256, # safety cap only; loop is bounded by end_offset
include_terminator: bool = False,
extra_chunks_after_metadata: int = 1,
extra_chunks_after_metadata: int = 1, # DEPRECATED — no-op
) -> list[S3Frame]:
"""
Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event.
Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event using
Blastware's exact protocol. REWRITTEN 2026-05-02 (v0.14.0).
The bulk waveform stream carries both raw ADC samples (large) and
event-time metadata strings ("Project:", "Client:", "User Name:",
"Seis Loc:", "Extended Notes") embedded in one of the middle frames
(confirmed: A5[7] of 9 for 1-2-26 capture).
Algorithm (matches BW captures across 2-sec / 3-sec / event-2):
Protocol is request-per-chunk, NOT a continuous stream:
1. Probe (offset=_BULK_CHUNK_OFFSET, is_probe=True, counter=0x0000)
2. Chunks (offset=_BULK_CHUNK_OFFSET, is_probe=False, counter+=0x0400)
3. Loop until metadata found (stop_after_metadata=True) or max_chunks
4. Termination (offset=_BULK_TERM_OFFSET, counter=last+_BULK_COUNTER_STEP)
Device responds with a final A5 frame (page_key=0x0000).
1. Probe
- For events at start_key[2:4] = 0x0000 (first event after erase
/ wrap): probe at counter=0x0000 with full key in params.
- For continuation events (start_key[2:4] != 0): first chunk at
counter = start_key[2:4] + 0x0046; acts as both probe and
first sample chunk; response carries STRT.
By default the termination frame (page_key=0x0000) is NOT included in the
returned list. Pass include_terminator=True to append it; the blastware_file
writer needs the terminator frame's body to reconstruct the waveform file footer.
2. Parse end_offset from STRT record at data[23:27] of the probe response.
Args:
key4: 4-byte waveform key from EVENT_HEADER (1E).
stop_after_metadata: If True (default), send termination as soon as
b"Project:" is found in a frame's data — avoids
downloading the full ADC waveform payload (several
hundred KB). Set False to download everything.
max_chunks: Safety cap on the number of chunk requests sent
(default 32; a typical event uses 9 large frames).
include_terminator: If True, append the terminator A5 frame
(page_key=0x0000) to the returned list. The
terminator carries the waveform file footer bytes.
Default False preserves existing caller behaviour.
3. Read two fixed metadata pages at counter=0x1002 and counter=0x1004
— global session metadata (Project / Client / User Name / Seis Loc
/ Extended Notes ASCII strings). Event 1 only; continuation
events skip these (BW caches them across the session).
4. Walk sample chunks at 0x0200 increments, starting from 0x0600 for
event 1 or `start + 0x0046 + 0x0200` for continuation events.
Stop when `next_chunk + 0x0200 > end_offset`.
5. Send TERM frame with offset_word and params computed by
`bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`.
The TERM response contains the partial last chunk (residual =
end_offset - next_boundary) including the 26-byte 0e 08 file
footer.
Returns:
List of S3Frame objects from each A5 response frame. Frame indices
match the request sequence: index 0 = probe response, index 1 = first
chunk, etc. If include_terminator=True, the last element is the
terminator frame (page_key=0x0000).
List of S3Frame objects from each A5 response (probe, metadata
pages, sample chunks, optional TERM response). Caller passes
`include_terminator=True` (e.g. write_blastware_file) to keep the
TERM response in the list — it's required to reconstruct the
file footer.
Deprecated kwargs:
stop_after_metadata: legacy "Project:"-string-based stop condition.
No-op under the BW-exact walk; the loop is
deterministically bounded by end_offset from
STRT. Accepted for backward compat.
extra_chunks_after_metadata: same.
Raises:
ProtocolError: on timeout, bad checksum, or unexpected SUB.
Confirmed from 1-2-26 BW TX/RX captures (2026-04-02):
- probe + 8 regular chunks + 1 termination = 10 TX frames
- 9 large A5 responses + 1 terminator A5 = 10 RX frames
- page_key=0x0010 on large frames; page_key=0x0000 on terminator ✅
- "Project:" metadata at A5[7].data[626] ✅
ProtocolError: on timeout / bad checksum / unexpected SUB.
"""
if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xFF - 0x5A = 0xA5
# Quietly accept and warn on deprecated kwargs.
if not stop_after_metadata:
log.debug("5A: stop_after_metadata=False is no-op under BW-exact walk")
if extra_chunks_after_metadata not in (0, 1):
log.debug("5A: extra_chunks_after_metadata=%d is no-op under BW-exact walk",
extra_chunks_after_metadata)
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xA5
frames_data: list[S3Frame] = []
counter = 0
# BW counter formula (confirmed from 4-3-26 capture for key 0111245a,
# and empirical live-device test 2026-04-06 for key 01110000):
# counter for chunk n = max(key4[2:4], 0x0400) + (n - 1) * 0x0400
# key4[2:4] is the event's circular-buffer base offset. The max() guard
# ensures chunk 1 never uses counter=0x0000 (which equals the probe address
# and causes the device to re-return STRT record data for the first chunk).
_key4_offset = (key4[2] << 8) | key4[3]
start_offset = (key4[2] << 8) | key4[3]
is_event_1 = (start_offset == 0)
# ── Step 1: probe ────────────────────────────────────────────────────
log.debug("5A probe key=%s key4_offset=0x%04X", key4.hex(), _key4_offset)
params = bulk_waveform_params(key4, 0, is_probe=True)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
self._parser.reset() # reset bytes_fed counter before probe recv
# ── Step 1: probe / first chunk ──────────────────────────────────────
if is_event_1:
probe_counter = 0
probe_params = bulk_waveform_params(key4, 0, is_probe=True)
log.debug("5A probe (event-1) key=%s counter=0x0000", key4.hex())
else:
# Continuation events: first 5A request lands at start+0x0046,
# acting as both probe and first sample chunk. Confirmed from
# 5-1-26 "copy 2nd address event" capture.
probe_counter = start_offset + 0x0046
probe_params = bulk_waveform_params(key4, probe_counter)
log.debug(
"5A probe (event-N) key=%s counter=0x%04X (start+0x46)",
key4.hex(), probe_counter,
)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, probe_params))
self._parser.reset()
try:
probe_batch = self._recv_5a_batch(rsp_sub)
rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False)
except TimeoutError:
log.warning(
"5A probe TIMED OUT for key=%s"
"%d raw bytes received (no complete A5 frame assembled)",
"5A probe TIMED OUT for key=%s%d raw bytes received",
key4.hex(), self._parser.bytes_fed,
)
raise
frames_data.extend(probe_batch)
log.debug(
"5A probe: %d frame(s) page_keys=%s",
len(probe_batch),
[f"0x{f.page_key:04X}" for f in probe_batch],
)
# Log probe frame size for diagnostics.
# The device always needs extra_chunks_after_metadata chunks after the
# metadata frame before termination to prime the valid waveform footer.
# This holds regardless of TCP frame size (1-frame vs 2-frame mode).
_effective_extra_chunks = extra_chunks_after_metadata
frames_data.append(rsp)
log.debug("5A A5[0] (probe) page_key=0x%04X %d bytes",
rsp.page_key, len(rsp.data))
# ── Step 2: parse STRT end_offset from probe response ────────────────
end_offset = parse_strt_end_offset(rsp.data)
if end_offset is None:
log.warning(
"5A probe data_len=%d effective_extra_chunks=%d",
len(probe_batch[0].data),
_effective_extra_chunks,
"5A probe response did not contain a STRT record; "
"cannot bound chunk loop — falling back to max_chunks=%d cap",
max_chunks,
)
end_offset = 0xFFFF # impossible value → loop runs to max_chunks
else:
log.info(
"5A STRT start_offset=0x%04X end_offset=0x%04X size=0x%04X",
start_offset, end_offset, end_offset - start_offset,
)
# ── Step 2: chunk loop ───────────────────────────────────────────────
# Counter formula: _chunk_base + (chunk_num - 1) * 0x0400
# where _chunk_base = max(key4[2:4], 0x0400).
#
# For events with key4[2:4] != 0 (e.g. key 0111245a, offset 0x245a):
# _chunk_base = 0x245a → chunk 1=0x245a, chunk 2=0x285a, ...
# Confirmed from 4-3-26 capture.
#
# For events with key4[2:4] == 0 (e.g. key 01110000):
# _chunk_base = max(0, 0x0400) = 0x0400
# → chunk 1=0x0400, chunk 2=0x0800, ... (= old chunk_num*0x0400)
# CRITICAL: counter=0x0000 (same as the probe) causes the device to
# re-return the STRT record data for chunk 1, making frame 1 look like
# a second probe response (confirmed from server log: frame 1 len=1097,
# contains STRT\xff\xfe, contributes zero body bytes after DLE-strip).
# counter=0x0400 for chunk 1 confirmed working (empirical test 2026-04-06).
_chunk_base = max(_key4_offset, _BULK_COUNTER_STEP)
for chunk_num in range(1, max_chunks + 1):
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
params = bulk_waveform_params(key4, counter)
log.debug("5A chunk %d counter=0x%04X", chunk_num, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
self._parser.reset() # reset bytes_fed for accurate per-chunk count
# ── Step 3: metadata pages 0x1002 + 0x1004 (event 1 only) ────────────
# Confirmed from BW captures: BW reads these two fixed device-buffer
# pages immediately after the probe for events at start_key[2:4]=0.
# Continuation events skip them (BW caches across the session).
# Their content is global compliance-setup metadata: Project, Client,
# User Name, Seis Loc, Extended Notes.
if is_event_1:
for meta_counter in (0x1002, 0x1004):
# Metadata page params have an extra trailing 0x00 byte
# (12-byte params instead of 11) — empirical from BW captures.
# Checksum-neutral but matches BW byte-for-byte.
meta_params = bytes([
0x00,
key4[0], key4[1],
(meta_counter >> 8) & 0xFF,
meta_counter & 0xFF,
0, 0, 0, 0, 0, 0, 0,
])
log.debug("5A metadata page counter=0x%04X", meta_counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, meta_params))
self._parser.reset()
try:
# Collect ALL frames from this chunk response.
# Over TCP via modem, a single large A5 device response (~1100 bytes
# RS-232) is split across ~2 TCP segments, each parsed as its own
# complete S3 frame. _recv_5a_batch gathers all of them so that
# every subsequent chunk request is paired with the correct response.
batch = self._recv_5a_batch(rsp_sub, first_timeout=10.0)
meta_rsp = self._recv_one(
expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
)
except TimeoutError:
log.warning(
"5A metadata page 0x%04X TIMED OUT — continuing",
meta_counter,
)
continue
frames_data.append(meta_rsp)
log.debug(
"5A meta@0x%04X page_key=0x%04X %d bytes",
meta_counter, meta_rsp.page_key, len(meta_rsp.data),
)
# ── Step 4: sample chunk loop, bounded by end_offset ─────────────────
# Sample chunks start at:
# event 1: counter = 0x0600
# event N (>0): counter = probe_counter + 0x0200
# (probe was the first sample chunk)
if is_event_1:
counter = 0x0600
else:
counter = probe_counter + _BULK_COUNTER_STEP
last_chunk_counter: Optional[int] = (
probe_counter if not is_event_1 else None
)
chunks_fetched = 0
while chunks_fetched < max_chunks:
# Stop when next chunk would straddle the event end.
if counter + _BULK_COUNTER_STEP > end_offset:
log.debug(
"5A chunk loop done at counter=0x%04X (end=0x%04X); "
"%d chunks fetched",
counter, end_offset, chunks_fetched,
)
break
params = bulk_waveform_params(key4, counter)
log.debug("5A chunk #%d counter=0x%04X", chunks_fetched + 1, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
self._parser.reset()
try:
rsp = self._recv_one(
expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
)
except TimeoutError:
raw = self._parser.bytes_fed
log.warning(
"5A TIMEOUT chunk=%d counter=0x%04X raw_bytes=%d",
chunk_num, counter, raw,
chunks_fetched + 1, counter, raw,
)
if raw > 0 and frames_data:
# Device sent a partial byte (likely a bare DLE/ETX end-of-stream
# signal) but never completed a full frame. Treat as graceful
# stream end and fall through to the termination step.
log.warning(
"5A end-of-stream detected at chunk=%d (raw_bytes=%d, "
"frames_collected=%d) — proceeding to termination",
chunk_num, raw, len(frames_data),
"5A unexpected end-of-stream — proceeding to TERM",
)
break
raise
# Process all frames from this batch.
metadata_found = False
for rsp in batch:
log.warning(
"5A RX chunk=%d page_key=0x%04X data_len=%d contains_Project=%s",
chunk_num, rsp.page_key, len(rsp.data), b"Project:" in rsp.data,
)
if rsp.page_key == 0x0000:
# Device unexpectedly terminated mid-stream.
log.debug("5A page_key=0x0000 — device terminated early")
if include_terminator:
frames_data.append(rsp)
return frames_data
frames_data.append(rsp)
if stop_after_metadata and b"Project:" in rsp.data:
metadata_found = True
if metadata_found:
# Download extra_chunks_after_metadata more chunks after metadata.
# This primes the device to return the valid waveform footer in the
# termination response — without it the terminator carries too few bytes
# (confirmed 2026-04-23). The extra chunk data also belongs in the
# file body (confirmed from TCP capture analysis 2026-04-27).
log.debug("5A metadata found — fetching %d more chunk(s)",
_effective_extra_chunks)
for _extra_n in range(_effective_extra_chunks):
chunk_num += 1
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
params = bulk_waveform_params(key4, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
try:
extra_batch = self._recv_5a_batch(rsp_sub, first_timeout=10.0)
for ef in extra_batch:
log.debug(
"5A extra chunk page_key=0x%04X data_len=%d",
ef.page_key, len(ef.data),
"5A RX chunk=%d page_key=0x%04X data_len=%d",
chunks_fetched + 1, rsp.page_key, len(rsp.data),
)
if rsp.page_key == 0x0000:
# Device terminated mid-stream unexpectedly.
log.warning(
"5A unexpected page_key=0x0000 mid-stream at counter=0x%04X",
counter,
)
if ef.page_key == 0x0000:
if include_terminator:
frames_data.append(ef)
frames_data.append(rsp)
return frames_data
frames_data.append(ef)
except TimeoutError:
log.debug("5A extra chunk %d timed out — end of stream", _extra_n + 1)
break
break
frames_data.append(rsp)
last_chunk_counter = counter
counter += _BULK_COUNTER_STEP
chunks_fetched += 1
else:
log.warning(
"5A reached max_chunks=%d without end-of-stream; sending termination",
max_chunks,
"5A reached max_chunks=%d at counter=0x%04X (end=0x%04X)",
max_chunks, counter, end_offset,
)
# ── Step 3: termination ──────────────────────────────────────────────
term_counter = counter + _BULK_COUNTER_STEP
term_params = bulk_waveform_term_params(key4, term_counter)
log.debug(
"5A termination term_counter=0x%04X offset=0x%04X",
term_counter, _BULK_TERM_OFFSET,
# ── Step 5: TERM with proper end_offset-derived formula ──────────────
if last_chunk_counter is None or end_offset == 0xFFFF:
# No STRT or no chunks fetched — fall back to legacy TERM.
log.warning(
"5A using legacy TERM (offset_word=0x005A); "
"end_offset unavailable or no chunks fetched",
)
legacy_counter = (last_chunk_counter or probe_counter) + _BULK_COUNTER_STEP
term_offset_word = _BULK_TERM_OFFSET # 0x005A
term_params = bulk_waveform_term_params(key4, legacy_counter)
else:
term_offset_word, term_params = bulk_waveform_term_v2(
key4, end_offset, last_chunk_counter,
)
self._send(build_5a_frame(_BULK_TERM_OFFSET, term_params))
try:
term_rsp = self._recv_one(expected_sub=rsp_sub)
log.debug(
"5A termination response page_key=0x%04X %d bytes",
"5A TERM offset_word=0x%04X params[2:4]=%s end=0x%04X "
"last_chunk=0x%04X",
term_offset_word, term_params[2:4].hex(),
end_offset, last_chunk_counter,
)
self._send(build_5a_frame(term_offset_word, term_params))
try:
term_rsp = self._recv_one(expected_sub=rsp_sub, timeout=10.0)
log.info(
"5A TERM response page_key=0x%04X %d bytes",
term_rsp.page_key, len(term_rsp.data),
)
if include_terminator:
frames_data.append(term_rsp)
except TimeoutError:
log.debug("5A no termination response — device may have already closed")
log.warning("5A no TERM response (timeout)")
return frames_data
@@ -1403,63 +1448,6 @@ class MiniMateProtocol:
log.debug("TX %d bytes: %s", len(frame), frame.hex())
self._transport.write(frame)
def _recv_5a_batch(
self,
expected_sub: int,
first_timeout: float = 10.0,
batch_timeout: float = 0.5,
) -> list[S3Frame]:
"""
Collect all S3 frames that arrive as part of one device response.
Over TCP via cellular modem, a single device A5 response (~1100 bytes of
RS-232 data) is forwarded in multiple TCP segments due to the modem's
data-forwarding timeout (~100-150 ms per segment). Each TCP segment
contains a complete, valid S3 frame (~550 bytes). Calling _recv_one()
once returns only the first segment's frame and misses the rest, causing
the chunk request/response pairing to cascade out of alignment.
This helper collects ALL frames before returning, by trying additional
short-timeout receives after the first frame arrives.
The caller must call self._parser.reset() before this method to ensure
bytes_fed is accurate; this method always uses reset_parser=False.
Args:
expected_sub: Expected SUB byte for validation.
first_timeout: Timeout for the mandatory first frame. Should be
generous (default 10 s) since the device may be slow.
batch_timeout: Short timeout for subsequent frames. Default 0.5 s
— comfortably longer than the modem forwarding gap
(~150 ms) but short enough to avoid stalling when
only one frame is expected (probe, terminator).
Returns:
List of S3Frame objects in arrival order (at least one).
Raises:
TimeoutError: If no frame arrives within first_timeout.
UnexpectedResponse: If any frame has the wrong SUB byte.
"""
frames: list[S3Frame] = []
first = self._recv_one(
expected_sub=expected_sub,
reset_parser=False,
timeout=first_timeout,
)
frames.append(first)
while True:
try:
extra = self._recv_one(
expected_sub=expected_sub,
reset_parser=False,
timeout=batch_timeout,
)
frames.append(extra)
except TimeoutError:
break
return frames
def _recv_one(
self,
expected_sub: Optional[int] = None,
+99
View File
@@ -454,3 +454,102 @@ class SocketTransport(TcpTransport):
def __repr__(self) -> str:
return f"SocketTransport(peer={self.host!r})"
# ── Capturing transport (MITM-style raw byte mirror) ──────────────────────────
class CapturingTransport(BaseTransport):
"""
Wraps another BaseTransport and mirrors every byte to two raw capture files:
raw_bw_<...>.bin — bytes WE wrote to the device (BW-side TX)
raw_s3_<...>.bin — bytes the device wrote back (S3-side TX)
The file naming and on-wire byte layout are identical to the captures
produced by `bridges/ach_mitm.py`, so the resulting `.bin` files can be
loaded directly by the Analyzer (File > Open Capture) and parsed by the
same tooling used for genuine Blastware MITM captures.
All BaseTransport methods are forwarded to the inner transport; the only
side-effect is that successful read/write byte streams are appended to the
two open binary files.
Args:
inner: An already-built BaseTransport (SerialTransport / TcpTransport).
bw_path: File path for the "BW TX" stream (bytes we send). Opened "wb".
s3_path: File path for the "S3 TX" stream (bytes the device sends).
Opened "wb".
Example:
with CapturingTransport(TcpTransport("1.2.3.4", 9034),
"raw_bw.bin", "raw_s3.bin") as t:
client = MiniMateClient(transport=t)
client.connect()
client.get_events()
# both .bin files now hold the full bidirectional capture.
"""
def __init__(self, inner: BaseTransport, bw_path: str, s3_path: str) -> None:
self._inner = inner
self._bw_path = bw_path
self._s3_path = s3_path
self._bw_fh = None
self._s3_fh = None
# Forward inner attrs so callers can introspect (e.g. .host, .port).
self.host = getattr(inner, "host", None)
self.port = getattr(inner, "port", None)
# ── BaseTransport interface ───────────────────────────────────────────────
def connect(self) -> None:
if self._bw_fh is None:
self._bw_fh = open(self._bw_path, "wb", buffering=0)
if self._s3_fh is None:
self._s3_fh = open(self._s3_path, "wb", buffering=0)
self._inner.connect()
def disconnect(self) -> None:
try:
self._inner.disconnect()
finally:
for fh_attr in ("_bw_fh", "_s3_fh"):
fh = getattr(self, fh_attr)
if fh is not None:
try:
fh.flush()
fh.close()
except Exception:
pass
setattr(self, fh_attr, None)
@property
def is_connected(self) -> bool:
return self._inner.is_connected
def write(self, data: bytes) -> None:
self._inner.write(data)
if data and self._bw_fh is not None:
try:
self._bw_fh.write(data)
except Exception:
pass
def read(self, n: int) -> bytes:
got = self._inner.read(n)
if got and self._s3_fh is not None:
try:
self._s3_fh.write(got)
except Exception:
pass
return got
@property
def bw_path(self) -> str:
return self._bw_path
@property
def s3_path(self) -> str:
return self._s3_path
def __repr__(self) -> str:
return f"CapturingTransport({self._inner!r}, bw={self._bw_path!r}, s3={self._s3_path!r})"
+2
View File
@@ -53,7 +53,9 @@ SUB_TABLE: dict[int, tuple[str, str, str]] = {
0x82: ("TRIGGER_CONFIG_WRITE", "BW→S3", "0x1C bytes; trigger config block; mirrors SUB 1C"),
0x83: ("TRIGGER_WRITE_CONFIRM", "BW→S3", "Short frame; commit step after 0x82"),
# S3→BW responses
0x5A: ("BULK_WAVEFORM_STREAM", "BW→S3", "Bulk waveform chunk request; response is A5 stream"),
0xA4: ("POLL_RESPONSE", "S3→BW", "Response to SUB 5B poll"),
0xA5: ("BULK_WAVEFORM_RESPONSE", "S3→BW", "Response to SUB 5A; waveform chunks + metadata"),
0xFE: ("FULL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 01"),
0xF9: ("CHANNEL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 06"),
0xF7: ("EVENT_INDEX_RESPONSE", "S3→BW", "Response to SUB 08; contains backlight/power-save"),
+31 -34
View File
@@ -33,7 +33,7 @@ STX = 0x02
ETX = 0x03
ACK = 0x41
__version__ = "0.2.3"
__version__ = "0.2.5"
@dataclass
@@ -186,7 +186,7 @@ def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]:
IDLE = 0
IN_FRAME = 1
AFTER_DLE = 2
IN_FRAME_DLE = 2 # saw DLE inside frame — waiting for next byte
state = IDLE
body = bytearray()
@@ -206,66 +206,63 @@ def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]:
state = IN_FRAME
i += 2
continue
# ACK bytes, boot strings, garbage — silently ignored
elif state == IN_FRAME:
if b == DLE:
state = AFTER_DLE
state = IN_FRAME_DLE
i += 1
continue
body.append(b)
else: # AFTER_DLE
if b == DLE:
body.append(DLE)
state = IN_FRAME
i += 1
continue
if b == ETX:
# Bare ETX = real S3 frame terminator (confirmed from S3FrameParser)
end_offset = i + 1
trailer_start = i + 1
trailer_end = trailer_start + trailer_len
trailer = blob[trailer_start:trailer_end]
chk_valid = None
chk_type = None
chk_hex = None
payload = bytes(body)
if len(body) >= 1:
received_chk = body[-1]
computed_chk = checksum8_sum(bytes(body[:-1]))
if computed_chk == received_chk:
chk_valid = True
chk_type = "SUM8"
chk_hex = f"{received_chk:02x}"
payload = bytes(body[:-1])
else:
chk_valid = False
# S3 checksums are deliberately not validated here.
# Large S3 responses (A5 bulk waveform, E5 compliance) embed
# inner DLE+ETX sub-frame terminators whose trailing 0x03 byte
# lands where the parser would expect the SUM8 checksum, causing
# false failures. The live protocol (protocol.py _validate_frame)
# also skips S3 checksum enforcement for the same reason.
frames.append(Frame(
index=idx,
start_offset=start_offset,
end_offset=end_offset,
payload_raw=bytes(body),
payload=payload,
payload=bytes(body),
trailer=trailer,
checksum_valid=chk_valid,
checksum_type=chk_type,
checksum_hex=chk_hex
checksum_valid=None,
checksum_type=None,
checksum_hex=None
))
idx += 1
state = IDLE
i = trailer_end
continue
body.append(b)
else: # IN_FRAME_DLE
if b == DLE:
# DLE DLE → literal 0x10 in payload
body.append(DLE)
state = IN_FRAME
i += 1
continue
if b == ETX:
# DLE+ETX inside a frame = inner-frame terminator (A4/E5 sub-frames).
# Treat as literal data, NOT the outer frame end.
body.append(DLE)
body.append(ETX)
state = IN_FRAME
i += 1
continue
# Unexpected DLE + byte → treat as literal data
body.append(DLE)
body.append(b)
state = IN_FRAME
i += 1
continue
i += 1
+792 -100
View File
File diff suppressed because it is too large Load Diff
+19 -18
View File
@@ -37,6 +37,7 @@ from __future__ import annotations
import datetime
import logging
import sys
import tempfile
import threading
import time
from pathlib import Path
@@ -863,8 +864,8 @@ def device_event_blastware_file(
Supply either *port* (serial) or *host* (TCP/modem).
The file is written to /tmp and streamed back as a binary download.
Blastware can open it directly filename encodes serial + timestamp.
The file is written to the OS temp directory and streamed back as a binary
download. Blastware can open it directly filename encodes serial + timestamp.
Filename format: <prefix><serial3><stem><AB>0<W|H>
- prefix letter = chr(ord('B') + floor(serial_numeric / 1000))
@@ -885,23 +886,13 @@ def device_event_blastware_file(
def _do():
with _build_client(port, baud, host, tcp_port, timeout=120.0) as client:
info = client.connect()
# Use stop_after_metadata=True (full_waveform=False) with 1 extra
# chunk after "Project:". The extra chunk primes the device so that
# the termination response carries the full waveform footer bytes.
# Without it the terminator returns only ~90 bytes (no useful footer).
#
# The extra chunk's ADC data IS part of the Blastware file body —
# confirmed from 4-27-26 TCP capture: all 14 A5 frames (including the
# extra chunk's 2 TCP sub-frames) contribute to the correct 6864-byte
# output. write_blastware_file() includes all frames unconditionally.
#
# full_waveform=True (natural end-of-stream) downloads ALL chunks
# including post-event silence (35+ chunks for a 9-sec event at
# 1024 sps) — this produces 24KB+ files that Blastware rejects.
# Under v0.14.0 BW-exact 5A walk, the chunk loop is bounded by
# the event end_offset extracted from STRT. No more
# stop_after_metadata / extra_chunks gymnastics — these
# kwargs are now no-ops.
events = client.get_events(
full_waveform=False,
stop_after_index=index,
extra_chunks_after_metadata=1,
)
matching = [ev for ev in events if ev.index == index]
return matching[0] if matching else None, info
@@ -937,8 +928,18 @@ def device_event_blastware_file(
# Build filename using the same algorithm Blastware uses
filename = blastware_filename(ev, serial)
# Write to /tmp so FastAPI can stream it back
out_path = Path("/tmp") / filename
# Write to OS temp dir (cross-platform: /tmp on Linux/macOS,
# %TEMP% on Windows) so FastAPI can stream it back via FileResponse.
out_path = Path(tempfile.gettempdir()) / filename
# Delete any stale file at this path before writing. On Windows we have
# observed the new (smaller) file getting trailing zero-bytes from the
# previous (larger) file when filesystem semantics around open(...,"wb")
# don't truncate cleanly (e.g. through a synced folder). Explicit unlink
# eliminates that ambiguity.
try:
out_path.unlink()
except FileNotFoundError:
pass
write_blastware_file(ev, a5_frames, out_path)
log.info(
"blastware_file: wrote %s (%d A5 frames, serial=%s)",