16 Commits

Author SHA1 Message Date
claude 7b62c790a9 fix(seismo-lab): remove duplicate capture history list 2026-05-04 14:30:46 -04:00
claude b66cc9d075 fix(blastware_file): update TERM detection logic and strip duplicate header blocks for accurate file writing 2026-05-04 14:28:11 -04:00
claude 45e61fbcaf big refactor of waveform protocol. 2026-05-03 01:20:21 -04:00
claude d758825c67 fix(protocol): correct continuous-mode record header classification for accurate timestamp extraction 2026-05-01 20:28:55 -04:00
claude 0fbb39c21a Big event bugfix. see details:
## v0.13.0 — 2026-05-01

### Fixed

- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
  `read_bulk_waveform_stream` was walking the chunk counter past the actual
  end of the event, picking up post-event circular-buffer garbage that
  corrupted reconstructed Blastware files for any waveform > ~1 sec.  The
  loop now extracts the event's `end_offset` from the STRT record at
  `data[23:27]` of the probe response and stops the chunk walk when the next
  counter would step past it.  Verified against three BW MITM captures
  (4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
  bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.

### Added

- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)` —
  computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
  formula confirmed across all 3 BW captures.  Not yet wired into
  `read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
  existing `blastware_file.write_blastware_file` frame-structure expectations);
  available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
  from the STRT record in an A5 response payload.
2026-05-01 18:37:34 -04:00
claude 738b39f3cb Manually Merged seismo lab persistent connection branch into the new direct download branch, creating a new branch called seismo-lab-new 2026-05-01 15:13:50 -04:00
Claude 625b0a4dfc feat(seismo_lab): add Download tab that captures wire bytes during event download
Adds a new CapturingTransport wrapper in minimateplus.transport that mirrors
every TX/RX byte to two raw .bin files using the same on-wire format as
bridges/ach_mitm.py, so the resulting captures are byte-for-byte compatible
with the existing Blastware MITM captures and load directly in the Analyzer.

A new "Download" tab in seismo_lab.py lets the user connect to a device over
TCP or serial and run connect / list-keys / download-events while the wrapper
saves raw_bw_<ts>.bin (our TX) and raw_s3_<ts>.bin (device TX) into a
seismo_dl_<ts>[_<label>]/ session directory. On completion, the panel hands
both files to the Analyzer and switches tabs, mirroring the UX of the
existing Bridge capture flow.
2026-05-01 00:12:02 +00:00
Claude b14f31f3b0 Include capture label in TCP raw filename
Matches serial bridge naming: raw_bw_{ts}_{label}.bin / raw_s3_{ts}_{label}.bin

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:48:10 +00:00
Claude b9ab368934 Fix TCP capture: write files only when capture is active
Previously every Blastware connection auto-created files.
Now TCP mode works the same as serial mode:
- Start Bridge: proxy listens and forwards silently, no files written
- New Capture: opens raw_bw/raw_s3 files; pipe threads write to them
- Stop Capture: flushes and closes files, fires Analyzer callback
- No connection = no file; multiple captures per bridge session work correctly

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:26:31 +00:00
Claude 9004241846 Restore multi-capture Bridge design + TCP mode
Brings back the protocol-exp BridgePanel design:
- Single bridge session stays up; New Capture / Stop Capture create
  labelled raw-file segments on demand (no files created at bridge start)
- Capture history listbox shows all segments; double-click reloads in Analyzer
- On capture complete: Analyzer auto-populates and runs analysis

TCP mode integrated into same tab (Serial/TCP radio toggle):
- Each incoming Blastware connection is automatically a capture segment
- Session appears in history list; Analyzer wires up live on connect
- Stop Capture disconnects current TCP session

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-27 20:20:43 +00:00
Claude 6861d9ed97 Merge TCP mode into Bridge tab (Serial/TCP radio toggle)
Removes the separate 'TCP Capture' tab and folds TCP MITM capture directly
into the existing Bridge tab.  A Serial/TCP radio selector at the top swaps
the connection fields (COM ports vs. listen port + device host:port) while
keeping the same Start Bridge / Stop Bridge / Add Mark buttons, capture
checkboxes, log dir, and live log — identical UX for both modes.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 23:01:45 +00:00
claude 5cd5652560 Merge branch 'seismo-lab' of https://github.com/serversdwn/seismo-relay into seismo-lab 2026-04-26 18:16:52 -04:00
Claude 897ac8a3f3 Add TCP MITM capture tab (TcpBridgePanel)
New 'TCP Capture' tab in seismo_lab.py: listens on a configurable local
port for an incoming Blastware connection, transparently forwards all
traffic to the real seismograph device, and saves both directions to
raw_bw_<ts>.bin / raw_s3_<ts>.bin in the same format the Analyzer already
understands.  Session start wires up Analyzer live mode automatically via
the same on_bridge_started callback as the COM-port bridge.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 22:10:48 +00:00
serversdown 310fc5986c Merge pull request 'seismo-lab2' (#7) from seismo-lab2 into seismo-lab
Reviewed-on: #7
2026-04-26 16:49:28 -04:00
Claude e1150b30aa fix(analyzer): name A5/5A frames; revert S3 checksum validation
Add 0x5A (BULK_WAVEFORM_STREAM) and 0xA5 (BULK_WAVEFORM_RESPONSE) to
SUB_TABLE so they display with real names instead of UNKNOWN_5A/A5.

Revert S3 checksum validation to checksum_valid=None (the original
intentional behavior). Large S3 frames (A5 bulk waveform, E5 compliance
config) embed inner DLE+ETX sub-frame delimiters; the trailing 0x03 of
the last inner delimiter can land where the parser expects the SUM8
checksum byte, causing false BAD CHK on every valid A5 frame.
protocol.py _validate_frame documents and ignores exactly this issue.

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 20:40:45 +00:00
Claude 9bbecea70f fix(parser): correct S3 frame terminator — bare ETX, not DLE+ETX
parse_s3 had the S3 terminator logic inverted vs the real S3FrameParser
in framing.py. It was terminating on DLE+ETX and treating bare ETX as
payload, which caused every bare 0x03 to be swallowed — bundling multiple
real S3 frames into one giant body until a DLE+ETX sequence happened to
appear. Result: 583-byte POLL_RESPONSE 'frames' containing many real
frames concatenated, all showing BAD CHK.

Fix: mirror S3FrameParser exactly —
  - Bare ETX (0x03) = real frame terminator
  - DLE+ETX (0x10 0x03) = inner-frame literal data (A4/E5 sub-frames),
    appended to body and parsing continues

https://claude.ai/code/session_014NczSHUz9uTzCAf4cVASTJ
2026-04-26 20:23:18 +00:00
13 changed files with 1605 additions and 503 deletions
+106
View File
@@ -4,6 +4,112 @@ All notable changes to seismo-relay are documented here.
--- ---
## v0.13.2 — 2026-05-01
### Fixed
- **`_extract_record_type` — third 0C-record header format ("short", 8 bytes).**
A live SFM download against BE11529 produced files named `M5290000.000`
(zero-stamped) because the 0C waveform record's first bytes were
`01 05 07 ea ...` — neither the 9-byte single-shot layout (`0x10` at byte 1)
nor the 10-byte continuous layout (`0x10` at bytes 0 and 2). Investigation
showed this is a third format observed in the wild: an 8-byte header with no
marker bytes at all (`[day][month][year_BE:2][unknown][hour][min][sec]`).
The detection logic now scans the year (uint16 BE) at byte 2 / byte 3 / byte
4 and picks whichever offset returns a sensible year (20152050) — each
format has the year at a unique position so this disambiguates cleanly.
- New format → `event.record_type = "Waveform (Short)"`,
`Timestamp.from_short_record()`.
- Existing single-shot and continuous parsers unchanged.
- The user's event from May 1, 2026 13:21:37 now correctly resolves to a
filename like `M529LKIQ.G10` instead of `M5290000.000`.
### Added
- `Timestamp.from_short_record(data)` — decodes the 8-byte header.
- `_detect_record_format(data)` — internal helper returning
`"single_shot" / "continuous" / "short" / None` via year-position scan.
---
## v0.13.1 — 2026-05-01
### Fixed
- **`_extract_record_type` — Continuous-mode record headers misclassified as Unknown.**
In single-shot mode the 0C waveform record's 9-byte header puts the sub_code
marker `0x10` at byte 1, with the day at byte 0. In Continuous mode the
header is 10 bytes with the marker at byte 0 *and* byte 2, and the day at
byte 1. Previous logic only inspected byte 1 and treated any value other
than `0x10` / `0x03` as `"Unknown"`, which prevented `event.timestamp` from
being populated for any continuous-mode event whose day-of-month wasn't
exactly 3 or 16. As a downstream effect, `blastware_filename()` saw
`event.timestamp == None`, fell back to `stem="0000"` / `ab="00"`, and
produced filenames like `M5290000.000`. Discovered from a live SFM run on
BE11529 in continuous mode (day-of-month = 5).
Now disambiguates by checking BOTH byte 0 and byte 2: if both are `0x10`,
it's the 10-byte continuous header; else if byte 1 is `0x10`, it's the
9-byte single-shot header. Day-of-month no longer matters.
*Superseded by v0.13.2 — the user's actual record uses a third 8-byte format
with no `0x10` markers, which v0.13.1 still misclassified.*
---
## v0.13.0 — 2026-05-01
### Fixed
- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
`read_bulk_waveform_stream` was walking the chunk counter past the actual
end of the event, picking up post-event circular-buffer garbage that
corrupted reconstructed Blastware files for any waveform > ~1 sec. The
loop now extracts the event's `end_offset` from the STRT record at
`data[23:27]` of the probe response and stops the chunk walk when the next
counter would step past it. Verified against three BW MITM captures
(4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.
### Added
- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`
computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
formula confirmed across all 3 BW captures. Not yet wired into
`read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
existing `blastware_file.write_blastware_file` frame-structure expectations);
available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
from the STRT record in an A5 response payload.
### Documentation
- **CLAUDE.md and `docs/instantel_protocol_reference.md` extensively
rewritten** to reflect the corrected SUB 5A protocol. See:
- CLAUDE.md "SUB 5A — chunk counter formula (REWRITTEN 2026-05-01)"
- CLAUDE.md "SUB 5A — STRT record encodes end_offset"
- CLAUDE.md "SUB 5A — TERM frame formula"
- CLAUDE.md "SUB 5A — fixed metadata pages 0x1002 and 0x1004"
- CLAUDE.md "SUB 0A — WAVEHDR response length distinguishes events from
boundaries" (0x46 = real event, 0x2C = boundary marker)
- protocol reference §7.8.5 / §7.8.6 / §7.8.7 / §7.8.8
- The previous chunk-counter formula (`max(key4[2:4], 0x0400) + (chunk-1) *
0x0400`) is now marked DEPRECATED and explicitly tagged WRONG with
pointers to the new sections, so future work doesn't re-derive it.
### Known minor diffs vs Blastware (deferred to a follow-up)
- We still use the OLD 0x0400 chunk step rather than BW's 0x0200; switching
also requires updating `blastware_file.write_blastware_file`'s skip values
and "extra chunk after metadata" logic, which depends on a fresh capture
to verify.
- We still use the legacy fixed `offset_word=0x005A` TERM frame rather than
BW's `end_offset - next_boundary` formula, for the same reason.
- Two fixed metadata pages at counter `0x1002` and `0x1004` are not yet
read explicitly; under the current 0x0400 walk their content is reachable
via the sample chunk that covers buffer addresses `[0x1000, 0x1400)`.
---
## v0.12.5 — 2026-04-21 ## v0.12.5 — 2026-04-21
### Changed ### Changed
+2 -32
View File
@@ -2,7 +2,7 @@
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.12.3**. (Sierra Wireless RV50 / RV55). Current version: **v0.13.2**.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
@@ -41,7 +41,7 @@ Full read pipeline + write pipeline + erase pipeline + monitor log + call home c
| Event header / first key | 1E | ✅ | | Event header / first key | 1E | ✅ |
| Waveform header | 0A | ✅ | | Waveform header | 0A | ✅ |
| Waveform record (peaks, timestamp, project) | 0C | ✅ | | Waveform record (peaks, timestamp, project) | 0C | ✅ |
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 | | **Bulk waveform stream (event-time metadata)** | **5A** | ✅ over-read bug fixed v0.13.0 (chunk loop bounded by STRT end_offset); minor wire diffs vs BW deferred — see "SUB 5A — chunk counter formula" |
| Event advance / next key | 1F | ✅ | | Event advance / next key | 1F | ✅ |
| **Write commands (push config to device)** | **6883** | ✅ new v0.8.0 | | **Write commands (push config to device)** | **6883** | ✅ new v0.8.0 |
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 | | **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
@@ -347,36 +347,6 @@ Do NOT use fixed absolute offsets for sample_rate or record_time.
Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to
`S3FrameParser`. `S3FrameParser`.
**SUB 5A (bulk waveform) TCP frame splitting — confirmed 2026-04-27:**
Over TCP via cellular modem, each 5A chunk request that produces a single ~1100-byte
A5 response over direct RS-232 may arrive as **two separate, complete S3 frames** of
~550 bytes each ("2-frame mode"). The modem's Data Forwarding Timeout (~100-150 ms)
can split the RS-232 response into two TCP segments, each parsed as a complete S3 frame.
Under different modem/timing conditions the full ~1100-byte response arrives as **one
S3 frame** ("1-frame mode").
**Both modes require `extra_chunks_after_metadata=1`** (the extra chunk at metadata_counter
+ 0x0400). The device's waveform footer data lives at circular-buffer address 0x1C00 for
this event; the terminator frame must be sent at 0x1C00 (not 0x1800) to receive it.
Example for a 2-second Continuous event (BE11529, key=01110000) via TCP:
- **2-frame mode:** 1 probe frame (554 B) + 5 chunks × 2 frames (556-573 B) + 1 extra chunk × 2 frames + 1 terminator (208 B) = **14 A5 frames** → 6864-byte file
- **1-frame mode:** 1 probe frame (~1097 B) + 5 chunks × 1 frame (~1079-1113 B) + 1 extra chunk × 1 frame (smaller, tail of event) + 1 terminator → **8 A5 frames** → 6864-byte file
- All frames contribute body data; using all of them gives the correct file.
**Fix (confirmed 2026-04-27):** `_recv_5a_batch()` in `protocol.py` collects ALL
A5 frames per chunk request before the next request is sent, using a 0.5 s batch
timeout after the first frame to catch the ~150 ms delayed second frame. `write_blastware_file()`
includes ALL body frames without skipping — the extra chunk's frames are part of the
body data, NOT padding to be discarded.
**WRONG earlier hypothesis (do not re-introduce):** An attempt was made to auto-detect
1-frame vs 2-frame mode from the probe frame size and skip the extra chunk when
`probe_data_len >= 700`. This was wrong — the extra chunk is always needed to advance
the device's internal state to the footer address. The `_probe_is_large` branch was
removed 2026-04-27.
### Required ACEmanager settings (Sierra Wireless RV50/RV55) ### Required ACEmanager settings (Sierra Wireless RV50/RV55)
| Setting | Value | Why | | Setting | Value | Why |
+10 -2
View File
@@ -21,7 +21,15 @@ Typical usage (TCP / modem):
from .client import MiniMateClient from .client import MiniMateClient
from .models import DeviceInfo, Event, MonitorLogEntry from .models import DeviceInfo, Event, MonitorLogEntry
from .transport import SerialTransport, TcpTransport from .transport import CapturingTransport, SerialTransport, TcpTransport
__version__ = "0.1.0" __version__ = "0.1.0"
__all__ = ["MiniMateClient", "DeviceInfo", "Event", "MonitorLogEntry", "SerialTransport", "TcpTransport"] __all__ = [
"MiniMateClient",
"DeviceInfo",
"Event",
"MonitorLogEntry",
"SerialTransport",
"TcpTransport",
"CapturingTransport",
]
+58 -27
View File
@@ -672,11 +672,10 @@ def write_blastware_file(
# Do NOT use a5_frames[-1] — if _a5_frames contains stray frames from a # Do NOT use a5_frames[-1] — if _a5_frames contains stray frames from a
# subsequent event (a known get_events side-effect), the last frame will # subsequent event (a known get_events side-effect), the last frame will
# not be the terminator and the footer will be mis-identified. # not be the terminator and the footer will be mis-identified.
# TERM detection (v0.14.0): last frame if page_key != 0x0010 (sample marker)
term_idx: Optional[int] = None term_idx: Optional[int] = None
for _i, _f in enumerate(a5_frames): if a5_frames and a5_frames[-1].page_key != 0x0010:
if _f.page_key == 0x0000: term_idx = len(a5_frames) - 1
term_idx = _i
break
if term_idx is not None: if term_idx is not None:
body_frames = a5_frames[:term_idx] body_frames = a5_frames[:term_idx]
@@ -685,34 +684,28 @@ def write_blastware_file(
body_frames = a5_frames body_frames = a5_frames
term_frame = None term_frame = None
log.warning( # Frame contribution loop (v0.14.0 BW-exact walk).
"write_blastware_file: %d body_frames term_idx=%s", # Skip values:
len(body_frames), # probe (fi=0): probe_skip
str(term_idx) if term_idx is not None else "None", # meta@0x1002 (fi=1): 13 (6-byte inner header)
# meta@0x1004 (fi=2): 13 (6-byte inner header)
# sample chunks (fi=3+): 12 (5-byte inner header)
last_fi = len(body_frames) - 1
log.debug(
"write_blastware_file: %d body_frames last_fi=%d",
len(body_frames), last_fi,
) )
all_bytes = bytearray() all_bytes = bytearray()
for fi, frame in enumerate(body_frames): for fi, frame in enumerate(body_frames):
# All body frames contribute to the waveform body — no frames are skipped.
#
# Over TCP via cellular modem, _recv_5a_batch() correctly collects all
# A5 frames per chunk request (the device's ~1100-byte RS-232 response
# is forwarded as ~2 TCP segments of ~550 bytes each, each parsed as a
# separate S3 frame). ALL of these frames contain ADC body data and
# must be included in the file — confirmed from 4-27-26 TCP capture
# analysis: contributions from all 14 frames → 6821 bytes → file 6864 bytes.
#
# Skip amounts (offsets into frame.data):
# fi=0 (probe): probe_skip — skips the type_tag header + STRT record
# fi=1: 13 — 7-byte frame.data prefix + 6 inner header bytes
# fi>=2: 12 — 7-byte frame.data prefix + 5 inner header bytes
if fi == 0: if fi == 0:
skip = probe_skip skip = probe_skip
elif fi == 1: elif fi in (1, 2):
skip = 13 skip = 13 # metadata pages
else: else:
skip = 12 skip = 12 # sample chunks
contribution = _frame_body_bytes(frame, skip) contribution = _frame_body_bytes(frame, skip)
log.warning("write_blastware_file: fi=%d skip=%d raw_data=%d contribution=%d", log.warning("write_blastware_file: fi=%d skip=%d raw_data=%d contribution=%d",
@@ -739,11 +732,49 @@ def write_blastware_file(
bytes(all_bytes[-28:]).hex() if len(all_bytes) >= 28 else bytes(all_bytes).hex(), bytes(all_bytes[-28:]).hex() if len(all_bytes) >= 28 else bytes(all_bytes).hex(),
) )
if len(all_bytes) >= 26: # Strip embedded "duplicate header+STRT" blocks from body (v0.14.1).
# Chunk@0x1000 sometimes lands on the device's metadata-mirror page,
# whose response includes a 25-byte "00 12 03 00 STRT ..." block that
# mirrors the file's own header + STRT record. BW treats embedded STRT
# markers as second-event starts and rejects the file. Replace these
# blocks with zeros to preserve file size + alignment.
needle = b"\x00\x12\x03\x00STRT"
pos = bytes(all_bytes).find(needle)
while pos >= 0:
end = pos + 25
if end <= len(all_bytes):
all_bytes[pos:end] = b"\x00" * 25
log.warning(
"write_blastware_file: stripped duplicate header+STRT at "
"all_bytes[%d:%d] (replaced with 25 zero-bytes)",
pos, end,
)
pos = bytes(all_bytes).find(needle, end)
# Find the first valid 0e 08 footer marker (v0.14.0).
footer_pos = -1
pos = 0
while True:
pos = bytes(all_bytes).find(b"\x0e\x08", pos)
if pos < 0 or pos + 26 > len(all_bytes):
break
yr = (all_bytes[pos + 4] << 8) | all_bytes[pos + 5]
if 2015 <= yr <= 2050:
footer_pos = pos
break
pos += 1
if footer_pos >= 0:
body = bytes(all_bytes[:footer_pos])
footer = bytes(all_bytes[footer_pos:footer_pos + 26])
log.warning(
"write_blastware_file: real 0e 08 footer at all_bytes[%d]; "
"truncating %d post-footer bytes",
footer_pos, len(all_bytes) - footer_pos - 26,
)
elif len(all_bytes) >= 26:
body = bytes(all_bytes[:-26]) body = bytes(all_bytes[:-26])
footer = bytes(all_bytes[-26:]) footer = bytes(all_bytes[-26:])
else: else:
# Fallback: no terminator or very short stream → build footer from event metadata
body = bytes(all_bytes) body = bytes(all_bytes)
start_dt = _ts_from_model(event.timestamp) start_dt = _ts_from_model(event.timestamp)
stop_dt: Optional[datetime.datetime] = None stop_dt: Optional[datetime.datetime] = None
@@ -754,7 +785,7 @@ def write_blastware_file(
+ _encode_ts_be(start_dt) + _encode_ts_be(start_dt)
+ _encode_ts_be(stop_dt) + _encode_ts_be(stop_dt)
+ b"\x00\x01\x00\x02\x00\x00" + b"\x00\x01\x00\x02\x00\x00"
+ b"\x00\x00" # CRC placeholder + b"\x00\x00"
) )
# ── Write file ─────────────────────────────────────────────────────────── # ── Write file ───────────────────────────────────────────────────────────
+66 -22
View File
@@ -1345,6 +1345,11 @@ def _decode_waveform_record_into(data: bytes, event: Event) -> None:
event.timestamp = Timestamp.from_continuous_record(data) event.timestamp = Timestamp.from_continuous_record(data)
except Exception as exc: except Exception as exc:
log.warning("continuous record timestamp decode failed: %s", exc) log.warning("continuous record timestamp decode failed: %s", exc)
elif event.record_type == "Waveform (Short)":
try:
event.timestamp = Timestamp.from_short_record(data)
except Exception as exc:
log.warning("short record timestamp decode failed: %s", exc)
# ── Peak values (per-channel PPV + Peak Vector Sum) ─────────────────────── # ── Peak values (per-channel PPV + Peak Vector Sum) ───────────────────────
try: try:
@@ -1636,34 +1641,73 @@ def _decode_a5_waveform(
} }
def _detect_record_format(data: bytes) -> Optional[str]:
"""
Detect which timestamp-header format a 210-byte 0C waveform record uses.
THREE formats observed on BE11529 firmware S338.17:
"single_shot" 9-byte header:
[day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
sub_code=0x10 at byte [1]. Year at [3:5].
"continuous" 10-byte header:
[0x10] [day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
marker 0x10 at byte [0] AND byte [2]. Year at [4:6].
"short" 8-byte header (NEW 2026-05-01):
[day] [month] [year_BE:2] [unknown] [hour] [min] [sec]
No marker bytes. Year at [2:4].
Each format has the year (uint16 BE) at a UNIQUE byte position, so we can
disambiguate by scanning each candidate position and picking the one
where the year falls in a sane range (2015..2050).
Returns "single_shot" / "continuous" / "short" or None if no format matches.
"""
if len(data) < 8:
return None
def _sane_year(hi: int, lo: int) -> bool:
y = (hi << 8) | lo
return 2015 <= y <= 2050
# Order matters: prefer formats with stronger marker-byte evidence first.
if data[1] == 0x10 and len(data) >= 9 and _sane_year(data[3], data[4]):
return "single_shot"
if (data[0] == 0x10 and data[2] == 0x10
and len(data) >= 10 and _sane_year(data[4], data[5])):
return "continuous"
if _sane_year(data[2], data[3]):
return "short"
return None
def _extract_record_type(data: bytes) -> Optional[str]: def _extract_record_type(data: bytes) -> Optional[str]:
""" """
Decode the recording mode from byte[1] of the 210-byte waveform record. Return a human-readable name for the waveform record format detected
in the first bytes of a 210-byte 0C record.
Byte[1] is the sub-record code that immediately follows the day byte in the Maps to the format codes returned by _detect_record_format():
9-byte timestamp header at the start of each waveform record: "single_shot" "Waveform"
[day:1] [sub_code:1] [month:1] [year:2 BE] ... "continuous" "Waveform (Continuous)"
"short" "Waveform (Short)"
Confirmed codes ( 2026-04-01): None "Unknown(XX.YY.ZZ)"
0x10 "Waveform" (continuous / single-shot mode)
Histogram mode code is not yet confirmed a histogram event must be
captured with debug=true to identify it. Returns None for unknown codes.
""" """
if len(data) < 2: fmt = _detect_record_format(data)
return None if fmt == "single_shot":
code = data[1]
if code == 0x10:
return "Waveform" return "Waveform"
if code == 0x03: if fmt == "continuous":
# Continuous mode waveform record (confirmed by user — NOT a monitor log).
# The byte layout differs from 0x10 single-shot records: the timestamp
# fields decode as garbage under the 0x10 waveform layout.
# TODO: confirm correct timestamp layout for 0x03 records from a known-time event.
return "Waveform (Continuous)" return "Waveform (Continuous)"
log.warning("_extract_record_type: unknown sub_code=0x%02X", code) if fmt == "short":
return f"Unknown(0x{code:02X})" return "Waveform (Short)"
if len(data) >= 3:
log.warning(
"_extract_record_type: unrecognized header: data[0:3]=%02X %02X %02X",
data[0], data[1], data[2],
)
return f"Unknown({data[0]:02X}.{data[1]:02X}.{data[2]:02X})"
return None
def _extract_peak_floats(data: bytes) -> Optional[PeakValues]: def _extract_peak_floats(data: bytes) -> Optional[PeakValues]:
""" """
+130 -18
View File
@@ -123,8 +123,11 @@ def build_5a_frame(offset_word: int, raw_params: bytes) -> bytes:
Returns: Returns:
Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX] Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX]
""" """
if len(raw_params) not in (10, 11): if len(raw_params) not in (10, 11, 12):
raise ValueError(f"raw_params must be 10 or 11 bytes, got {len(raw_params)}") # 10 = termination params; 11 = regular probe / chunk params;
# 12 = metadata-page params (extra trailing 0x00 — BW byte-perfect quirk
# for the two fixed metadata reads at counter=0x1002 and 0x1004).
raise ValueError(f"raw_params must be 10/11/12 bytes, got {len(raw_params)}")
# Build stuffed section between STX and checksum # Build stuffed section between STX and checksum
s = bytearray() s = bytearray()
@@ -398,28 +401,21 @@ def bulk_waveform_params(key4: bytes, counter: int, *, is_probe: bool = False) -
def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes: def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
""" """
Build the 10-byte params block for the SUB 5A termination request. DEPRECATED 2026-05-01 see bulk_waveform_term_v2().
The termination request uses offset=0x005A and a DIFFERENT params layout Build the 10-byte params block for the SUB 5A termination request, OLD layout
the leading 0x00 byte is dropped, key4[0:2] shifts to params[0:2], and the (used in conjunction with the fixed offset_word=0x005A). Kept for backward
counter high byte is at params[2]: compatibility produces a tiny ~100-byte device-side terminator response
rather than the proper partial-last-chunk + footer payload that BW gets.
params[0] = key4[0] params[0] = key4[0]
params[1] = key4[1] params[1] = key4[1]
params[2] = (counter >> 8) & 0xFF params[2] = (counter >> 8) & 0xFF
params[3:] = zeros params[3:] = zeros
Counter for the termination request = last_regular_counter + 0x0400. Use bulk_waveform_term_v2() for new code it computes the verified
offset_word + params from end_offset (extracted from STRT) and the last
Confirmed from 1-2-26 BW TX capture: final request (frame 83) uses chunk counter.
offset=0x005A, params[0:3] = key4[0:2] + term_counter_hi.
Args:
key4: 4-byte waveform key.
counter: Termination counter (= last regular counter + 0x0400).
Returns:
10-byte params block.
""" """
if len(key4) != 4: if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}") raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
@@ -430,6 +426,123 @@ def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
return bytes(p) return bytes(p)
def bulk_waveform_term_v2(
key4: bytes,
end_offset: int,
last_chunk_counter: int,
) -> tuple[int, bytes]:
"""
Compute the SUB 5A TERM frame's offset_word and 10-byte params block.
Confirmed across 3 events (4-27-26 + 5-1-26 captures):
next_boundary = last_chunk_counter + 0x0200
offset_word = end_offset - next_boundary (residual byte count)
params[0] = key4[0] (= 0x01 on every observed device)
params[1] = key4[1] (= 0x11)
params[2] = (next_boundary >> 8) & 0xFF
params[3] = next_boundary & 0xFF
params[4:10] = zeros
Verification:
| end_offset | last_chunk | next_boundary | offset_word | params[2:4] |
| 0x1ABE | 0x1800 | 0x1A00 | 0x00BE | 1A 00 |
| 0x21F2 | 0x1E00 | 0x2000 | 0x01F2 | 20 00 |
| 0x417E | 0x3E38 | 0x4038 | 0x0146 | 40 38 |
The device receives `requested_address = (params[2] << 8) | offset_word`
and replies with `(end_offset - next_boundary)` bytes of waveform tail
starting at `next_boundary` including the 26-byte file footer.
Args:
key4: 4-byte waveform key for this event.
end_offset: Event-end pointer (= `(end_key[2] << 8) | end_key[3]`
from the STRT record at data[23:27] of A5[0]).
last_chunk_counter: Counter of the last full 0x0200-byte chunk fetched
(the chunk that covers [last_chunk_counter,
last_chunk_counter + 0x0200)).
Returns:
(offset_word, params10) tuple. Pass as
`build_5a_frame(offset_word, params)`.
Raises:
ValueError: on inconsistent inputs.
"""
if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
next_boundary = last_chunk_counter + 0x0200
if next_boundary > 0xFFFF:
raise ValueError(
f"next_boundary 0x{next_boundary:04X} exceeds uint16; check inputs"
)
if end_offset <= last_chunk_counter:
raise ValueError(
f"end_offset 0x{end_offset:04X} must be > "
f"last_chunk_counter 0x{last_chunk_counter:04X}"
)
offset_word = end_offset - next_boundary
if offset_word < 0:
# Last chunk overshot end_offset; caller should have stopped one chunk
# earlier. Treat as zero residual.
offset_word = 0
if offset_word > 0xFFFF:
raise ValueError(
f"offset_word 0x{offset_word:04X} exceeds uint16"
)
p = bytearray(10)
p[0] = key4[0]
p[1] = key4[1]
p[2] = (next_boundary >> 8) & 0xFF
p[3] = next_boundary & 0xFF
return offset_word, bytes(p)
# ── End-offset extraction from STRT record ────────────────────────────────────
STRT_MARKER = b"STRT"
def parse_strt_end_offset(a5_data: bytes) -> Optional[int]:
"""
Extract the event-end offset from the STRT record in an A5 response payload.
The first A5 response (the probe response, or the first chunk for events
with non-zero start_key[2:4]) contains a STRT record at byte offset 17 of
`data`. Layout:
data[17:21] "STRT"
data[21:23] ff fe sentinel
data[23:27] end_key 4-byte key of where this event ENDS
data[27:31] start_key
...
Returns `(end_key[2] << 8) | end_key[3]` the absolute device-buffer
address where the event ends. Use this to bound the chunk loop and to
compute the TERM frame.
Verified end_offset values:
| event start_key | end_key | end_offset |
| 01110000 | 01111ABE | 0x1ABE |
| 01110000 | 011121F2 | 0x21F2 |
| 011121F2 | 0111417E | 0x417E |
Args:
a5_data: The `data` field of an A5 response frame (frame.data).
Returns:
The end_offset (uint16) if STRT is found, else None.
"""
pos = a5_data.find(STRT_MARKER)
if pos < 0 or pos + 10 > len(a5_data):
return None
# data[pos+4:pos+6] is "ff fe"; data[pos+6:pos+10] is end_key.
end_key = a5_data[pos + 6 : pos + 10]
if len(end_key) < 4:
return None
return (end_key[2] << 8) | end_key[3]
# ── Pre-built POLL frames ───────────────────────────────────────────────────── # ── Pre-built POLL frames ─────────────────────────────────────────────────────
# #
# POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the # POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the
@@ -470,7 +583,6 @@ class S3Frame:
# ── Streaming S3 frame parser ───────────────────────────────────────────────── # ── Streaming S3 frame parser ─────────────────────────────────────────────────
class S3FrameParser: class S3FrameParser:
""" """
Incremental byte-stream parser for S3BW response frames. Incremental byte-stream parser for S3BW response frames.
+52
View File
@@ -201,6 +201,58 @@ class Timestamp:
second=second, second=second,
) )
@classmethod
def from_short_record(cls, data: bytes) -> "Timestamp":
"""
Decode an 8-byte timestamp header from a 210-byte waveform record.
Wire layout ( CONFIRMED 2026-05-01 against live SFM run on BE11529 in
Continuous mode, day-of-month = 1 May, raw: 01 05 07 ea 00 0d 15 25):
byte[0]: day (uint8)
byte[1]: month (uint8)
bytes[2-3]: year (big-endian uint16)
byte[4]: unknown (0x00 in observed sample)
byte[5]: hour (uint8)
byte[6]: minute (uint8)
byte[7]: second (uint8)
This is a third format observed in the wild distinct from the 9-byte
(single-shot, sub_code=0x10 at [1]) and 10-byte (continuous, 0x10 at
[0] AND [2]) layouts. No marker bytes; disambiguated by where the
year lands when scanned at byte 2/3/4.
Args:
data: at least 8 bytes; only the first 8 are consumed.
Returns:
Decoded Timestamp.
Raises:
ValueError: if data is fewer than 8 bytes.
"""
if len(data) < 8:
raise ValueError(
f"Short record timestamp requires at least 8 bytes, got {len(data)}"
)
day = data[0]
month = data[1]
year = struct.unpack_from(">H", data, 2)[0]
unknown_byte = data[4]
hour = data[5]
minute = data[6]
second = data[7]
return cls(
raw=bytes(data[:8]),
flag=0,
year=year,
unknown_byte=unknown_byte,
month=month,
day=day,
hour=hour,
minute=minute,
second=second,
)
@property @property
def clock_set(self) -> bool: def clock_set(self) -> bool:
"""False when year == 1995 (factory default / battery-lost state).""" """False when year == 1995 (factory default / battery-lost state)."""
+225 -237
View File
@@ -35,6 +35,8 @@ from .framing import (
token_params, token_params,
bulk_waveform_params, bulk_waveform_params,
bulk_waveform_term_params, bulk_waveform_term_params,
bulk_waveform_term_v2,
parse_strt_end_offset,
POLL_PROBE, POLL_PROBE,
POLL_DATA, POLL_DATA,
SESSION_RESET, SESSION_RESET,
@@ -122,16 +124,22 @@ DATA_LENGTHS: dict[int, int] = {
} }
# SUB 5A (BULK_WAVEFORM_STREAM) protocol constants. # SUB 5A (BULK_WAVEFORM_STREAM) protocol constants.
# Confirmed from 1-2-26 BW TX capture analysis (2026-04-02). #
_BULK_CHUNK_OFFSET = 0x1004 # offset field for probe + all regular chunk requests ✅ # 2026-05-01 minimal-fix: the chunk-counter walk is now bounded by the event's
_BULK_TERM_OFFSET = 0x005A # offset field for termination request ✅ # `end_offset` extracted from the STRT record at data[23:27] of the probe
_BULK_COUNTER_STEP = 0x0400 # chunk counter increment per chunk ✅ # response. Without this bound the loop kept asking for chunks past the event
# Chunk counter formula: key4[2:4] + (chunk_num - 1) * 0x0400 # end and the device responded with post-event circular-buffer garbage,
# where key4[2:4] is the event's circular-buffer base offset ((key4[2]<<8)|key4[3]). # corrupting reconstructed Blastware files for events ≥ 2 sec.
# Earlier captures showed 0x1004 for chunk 1 of key 01110000 — that was a Blastware #
# artifact. For keys where key4[2:4] != 0x0000 (e.g. key 01111884) the old # We keep the OLD 0x0400 chunk step here (BW actually uses 0x0200 — see §7.8.5
# "n * 0x0400" formula sends counters from the wrong buffer region and the device # of the protocol reference for the corrected understanding) because the
# returns data from a different event. Confirmed correct 2026-04-24. # existing blastware_file.py builder relies on the 0x0400-step frame structure
# to produce valid files. Switching to BW's 0x0200 step is a separate task
# that also requires updating the file builder.
# BW-exact protocol values (v0.14.0). Verified against 4-27-26 + 5-1-26 captures.
_BULK_CHUNK_OFFSET = 0x1002 # offset_word for probe + all chunk requests
_BULK_TERM_OFFSET = 0x005A # offset_word for the legacy terminator (fallback only)
_BULK_COUNTER_STEP = 0x0200 # chunk counter increment (matches chunk payload size)
# Default timeout values (seconds). # Default timeout values (seconds).
# MiniMate Plus is a slow device — keep these generous. # MiniMate Plus is a slow device — keep these generous.
@@ -526,223 +534,260 @@ class MiniMateProtocol:
self, self,
key4: bytes, key4: bytes,
*, *,
stop_after_metadata: bool = True, stop_after_metadata: bool = True, # DEPRECATED — no-op under BW-exact walk
max_chunks: int = 32, max_chunks: int = 256, # safety cap only; loop is bounded by end_offset
include_terminator: bool = False, include_terminator: bool = False,
extra_chunks_after_metadata: int = 1, extra_chunks_after_metadata: int = 1, # DEPRECATED — no-op
) -> list[S3Frame]: ) -> list[S3Frame]:
""" """
Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event. Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event using
Blastware's exact protocol. REWRITTEN 2026-05-02 (v0.14.0).
The bulk waveform stream carries both raw ADC samples (large) and Algorithm (matches BW captures across 2-sec / 3-sec / event-2):
event-time metadata strings ("Project:", "Client:", "User Name:",
"Seis Loc:", "Extended Notes") embedded in one of the middle frames
(confirmed: A5[7] of 9 for 1-2-26 capture).
Protocol is request-per-chunk, NOT a continuous stream: 1. Probe
1. Probe (offset=_BULK_CHUNK_OFFSET, is_probe=True, counter=0x0000) - For events at start_key[2:4] = 0x0000 (first event after erase
2. Chunks (offset=_BULK_CHUNK_OFFSET, is_probe=False, counter+=0x0400) / wrap): probe at counter=0x0000 with full key in params.
3. Loop until metadata found (stop_after_metadata=True) or max_chunks - For continuation events (start_key[2:4] != 0): first chunk at
4. Termination (offset=_BULK_TERM_OFFSET, counter=last+_BULK_COUNTER_STEP) counter = start_key[2:4] + 0x0046; acts as both probe and
Device responds with a final A5 frame (page_key=0x0000). first sample chunk; response carries STRT.
By default the termination frame (page_key=0x0000) is NOT included in the 2. Parse end_offset from STRT record at data[23:27] of the probe response.
returned list. Pass include_terminator=True to append it; the blastware_file
writer needs the terminator frame's body to reconstruct the waveform file footer.
Args: 3. Read two fixed metadata pages at counter=0x1002 and counter=0x1004
key4: 4-byte waveform key from EVENT_HEADER (1E). global session metadata (Project / Client / User Name / Seis Loc
stop_after_metadata: If True (default), send termination as soon as / Extended Notes ASCII strings). Event 1 only; continuation
b"Project:" is found in a frame's data — avoids events skip these (BW caches them across the session).
downloading the full ADC waveform payload (several
hundred KB). Set False to download everything. 4. Walk sample chunks at 0x0200 increments, starting from 0x0600 for
max_chunks: Safety cap on the number of chunk requests sent event 1 or `start + 0x0046 + 0x0200` for continuation events.
(default 32; a typical event uses 9 large frames). Stop when `next_chunk + 0x0200 > end_offset`.
include_terminator: If True, append the terminator A5 frame
(page_key=0x0000) to the returned list. The 5. Send TERM frame with offset_word and params computed by
terminator carries the waveform file footer bytes. `bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`.
Default False preserves existing caller behaviour. The TERM response contains the partial last chunk (residual =
end_offset - next_boundary) including the 26-byte 0e 08 file
footer.
Returns: Returns:
List of S3Frame objects from each A5 response frame. Frame indices List of S3Frame objects from each A5 response (probe, metadata
match the request sequence: index 0 = probe response, index 1 = first pages, sample chunks, optional TERM response). Caller passes
chunk, etc. If include_terminator=True, the last element is the `include_terminator=True` (e.g. write_blastware_file) to keep the
terminator frame (page_key=0x0000). TERM response in the list it's required to reconstruct the
file footer.
Deprecated kwargs:
stop_after_metadata: legacy "Project:"-string-based stop condition.
No-op under the BW-exact walk; the loop is
deterministically bounded by end_offset from
STRT. Accepted for backward compat.
extra_chunks_after_metadata: same.
Raises: Raises:
ProtocolError: on timeout, bad checksum, or unexpected SUB. ProtocolError: on timeout / bad checksum / unexpected SUB.
Confirmed from 1-2-26 BW TX/RX captures (2026-04-02):
- probe + 8 regular chunks + 1 termination = 10 TX frames
- 9 large A5 responses + 1 terminator A5 = 10 RX frames
- page_key=0x0010 on large frames; page_key=0x0000 on terminator
- "Project:" metadata at A5[7].data[626]
""" """
if len(key4) != 4: if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}") raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xFF - 0x5A = 0xA5 # Quietly accept and warn on deprecated kwargs.
if not stop_after_metadata:
log.debug("5A: stop_after_metadata=False is no-op under BW-exact walk")
if extra_chunks_after_metadata not in (0, 1):
log.debug("5A: extra_chunks_after_metadata=%d is no-op under BW-exact walk",
extra_chunks_after_metadata)
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xA5
frames_data: list[S3Frame] = [] frames_data: list[S3Frame] = []
counter = 0
# BW counter formula (confirmed from 4-3-26 capture for key 0111245a, start_offset = (key4[2] << 8) | key4[3]
# and empirical live-device test 2026-04-06 for key 01110000): is_event_1 = (start_offset == 0)
# counter for chunk n = max(key4[2:4], 0x0400) + (n - 1) * 0x0400
# key4[2:4] is the event's circular-buffer base offset. The max() guard
# ensures chunk 1 never uses counter=0x0000 (which equals the probe address
# and causes the device to re-return STRT record data for the first chunk).
_key4_offset = (key4[2] << 8) | key4[3]
# ── Step 1: probe ──────────────────────────────────────────────────── # ── Step 1: probe / first chunk ──────────────────────────────────────
log.debug("5A probe key=%s key4_offset=0x%04X", key4.hex(), _key4_offset) if is_event_1:
params = bulk_waveform_params(key4, 0, is_probe=True) probe_counter = 0
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params)) probe_params = bulk_waveform_params(key4, 0, is_probe=True)
self._parser.reset() # reset bytes_fed counter before probe recv log.debug("5A probe (event-1) key=%s counter=0x0000", key4.hex())
else:
# Continuation events: first 5A request lands at start+0x0046,
# acting as both probe and first sample chunk. Confirmed from
# 5-1-26 "copy 2nd address event" capture.
probe_counter = start_offset + 0x0046
probe_params = bulk_waveform_params(key4, probe_counter)
log.debug(
"5A probe (event-N) key=%s counter=0x%04X (start+0x46)",
key4.hex(), probe_counter,
)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, probe_params))
self._parser.reset()
try: try:
probe_batch = self._recv_5a_batch(rsp_sub) rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False)
except TimeoutError: except TimeoutError:
log.warning( log.warning(
"5A probe TIMED OUT for key=%s" "5A probe TIMED OUT for key=%s%d raw bytes received",
"%d raw bytes received (no complete A5 frame assembled)",
key4.hex(), self._parser.bytes_fed, key4.hex(), self._parser.bytes_fed,
) )
raise raise
frames_data.extend(probe_batch)
log.debug(
"5A probe: %d frame(s) page_keys=%s",
len(probe_batch),
[f"0x{f.page_key:04X}" for f in probe_batch],
)
# Log probe frame size for diagnostics. frames_data.append(rsp)
# The device always needs extra_chunks_after_metadata chunks after the log.debug("5A A5[0] (probe) page_key=0x%04X %d bytes",
# metadata frame before termination to prime the valid waveform footer. rsp.page_key, len(rsp.data))
# This holds regardless of TCP frame size (1-frame vs 2-frame mode).
_effective_extra_chunks = extra_chunks_after_metadata
log.warning(
"5A probe data_len=%d effective_extra_chunks=%d",
len(probe_batch[0].data),
_effective_extra_chunks,
)
# ── Step 2: chunk loop ─────────────────────────────────────────────── # ── Step 2: parse STRT end_offset from probe response ────────────────
# Counter formula: _chunk_base + (chunk_num - 1) * 0x0400 end_offset = parse_strt_end_offset(rsp.data)
# where _chunk_base = max(key4[2:4], 0x0400). if end_offset is None:
# log.warning(
# For events with key4[2:4] != 0 (e.g. key 0111245a, offset 0x245a): "5A probe response did not contain a STRT record; "
# _chunk_base = 0x245a → chunk 1=0x245a, chunk 2=0x285a, ... "cannot bound chunk loop — falling back to max_chunks=%d cap",
# Confirmed from 4-3-26 capture. max_chunks,
# )
# For events with key4[2:4] == 0 (e.g. key 01110000): end_offset = 0xFFFF # impossible value → loop runs to max_chunks
# _chunk_base = max(0, 0x0400) = 0x0400 else:
# → chunk 1=0x0400, chunk 2=0x0800, ... (= old chunk_num*0x0400) log.info(
# CRITICAL: counter=0x0000 (same as the probe) causes the device to "5A STRT start_offset=0x%04X end_offset=0x%04X size=0x%04X",
# re-return the STRT record data for chunk 1, making frame 1 look like start_offset, end_offset, end_offset - start_offset,
# a second probe response (confirmed from server log: frame 1 len=1097, )
# contains STRT\xff\xfe, contributes zero body bytes after DLE-strip).
# counter=0x0400 for chunk 1 confirmed working (empirical test 2026-04-06). # ── Step 3: metadata pages 0x1002 + 0x1004 (event 1 only) ────────────
_chunk_base = max(_key4_offset, _BULK_COUNTER_STEP) # Confirmed from BW captures: BW reads these two fixed device-buffer
for chunk_num in range(1, max_chunks + 1): # pages immediately after the probe for events at start_key[2:4]=0.
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP # Continuation events skip them (BW caches across the session).
params = bulk_waveform_params(key4, counter) # Their content is global compliance-setup metadata: Project, Client,
log.debug("5A chunk %d counter=0x%04X", chunk_num, counter) # User Name, Seis Loc, Extended Notes.
if is_event_1:
for meta_counter in (0x1002, 0x1004):
# Metadata page params have an extra trailing 0x00 byte
# (12-byte params instead of 11) — empirical from BW captures.
# Checksum-neutral but matches BW byte-for-byte.
meta_params = bytes([
0x00,
key4[0], key4[1],
(meta_counter >> 8) & 0xFF,
meta_counter & 0xFF,
0, 0, 0, 0, 0, 0, 0,
])
log.debug("5A metadata page counter=0x%04X", meta_counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, meta_params))
self._parser.reset()
try:
meta_rsp = self._recv_one(
expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
)
except TimeoutError:
log.warning(
"5A metadata page 0x%04X TIMED OUT — continuing",
meta_counter,
)
continue
frames_data.append(meta_rsp)
log.debug(
"5A meta@0x%04X page_key=0x%04X %d bytes",
meta_counter, meta_rsp.page_key, len(meta_rsp.data),
)
# ── Step 4: sample chunk loop, bounded by end_offset ─────────────────
# Sample chunks start at:
# event 1: counter = 0x0600
# event N (>0): counter = probe_counter + 0x0200
# (probe was the first sample chunk)
if is_event_1:
counter = 0x0600
else:
counter = probe_counter + _BULK_COUNTER_STEP
last_chunk_counter: Optional[int] = (
probe_counter if not is_event_1 else None
)
chunks_fetched = 0
while chunks_fetched < max_chunks:
# Stop when next chunk would straddle the event end.
if counter + _BULK_COUNTER_STEP > end_offset:
log.debug(
"5A chunk loop done at counter=0x%04X (end=0x%04X); "
"%d chunks fetched",
counter, end_offset, chunks_fetched,
)
break
params = bulk_waveform_params(key4, counter)
log.debug("5A chunk #%d counter=0x%04X", chunks_fetched + 1, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params)) self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
self._parser.reset() # reset bytes_fed for accurate per-chunk count self._parser.reset()
try: try:
# Collect ALL frames from this chunk response. rsp = self._recv_one(
# Over TCP via modem, a single large A5 device response (~1100 bytes expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
# RS-232) is split across ~2 TCP segments, each parsed as its own )
# complete S3 frame. _recv_5a_batch gathers all of them so that
# every subsequent chunk request is paired with the correct response.
batch = self._recv_5a_batch(rsp_sub, first_timeout=10.0)
except TimeoutError: except TimeoutError:
raw = self._parser.bytes_fed raw = self._parser.bytes_fed
log.warning( log.warning(
"5A TIMEOUT chunk=%d counter=0x%04X raw_bytes=%d", "5A TIMEOUT chunk=%d counter=0x%04X raw_bytes=%d",
chunk_num, counter, raw, chunks_fetched + 1, counter, raw,
) )
if raw > 0 and frames_data: if raw > 0 and frames_data:
# Device sent a partial byte (likely a bare DLE/ETX end-of-stream
# signal) but never completed a full frame. Treat as graceful
# stream end and fall through to the termination step.
log.warning( log.warning(
"5A end-of-stream detected at chunk=%d (raw_bytes=%d, " "5A unexpected end-of-stream — proceeding to TERM",
"frames_collected=%d) — proceeding to termination",
chunk_num, raw, len(frames_data),
) )
break break
raise raise
# Process all frames from this batch. log.debug(
metadata_found = False "5A RX chunk=%d page_key=0x%04X data_len=%d",
for rsp in batch: chunks_fetched + 1, rsp.page_key, len(rsp.data),
log.warning(
"5A RX chunk=%d page_key=0x%04X data_len=%d contains_Project=%s",
chunk_num, rsp.page_key, len(rsp.data), b"Project:" in rsp.data,
)
if rsp.page_key == 0x0000:
# Device unexpectedly terminated mid-stream.
log.debug("5A page_key=0x0000 — device terminated early")
if include_terminator:
frames_data.append(rsp)
return frames_data
frames_data.append(rsp)
if stop_after_metadata and b"Project:" in rsp.data:
metadata_found = True
if metadata_found:
# Download extra_chunks_after_metadata more chunks after metadata.
# This primes the device to return the valid waveform footer in the
# termination response — without it the terminator carries too few bytes
# (confirmed 2026-04-23). The extra chunk data also belongs in the
# file body (confirmed from TCP capture analysis 2026-04-27).
log.debug("5A metadata found — fetching %d more chunk(s)",
_effective_extra_chunks)
for _extra_n in range(_effective_extra_chunks):
chunk_num += 1
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
params = bulk_waveform_params(key4, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
try:
extra_batch = self._recv_5a_batch(rsp_sub, first_timeout=10.0)
for ef in extra_batch:
log.debug(
"5A extra chunk page_key=0x%04X data_len=%d",
ef.page_key, len(ef.data),
)
if ef.page_key == 0x0000:
if include_terminator:
frames_data.append(ef)
return frames_data
frames_data.append(ef)
except TimeoutError:
log.debug("5A extra chunk %d timed out — end of stream", _extra_n + 1)
break
break
else:
log.warning(
"5A reached max_chunks=%d without end-of-stream; sending termination",
max_chunks,
) )
# ── Step 3: termination ────────────────────────────────────────────── if rsp.page_key == 0x0000:
term_counter = counter + _BULK_COUNTER_STEP # Device terminated mid-stream unexpectedly.
term_params = bulk_waveform_term_params(key4, term_counter) log.warning(
log.debug( "5A unexpected page_key=0x0000 mid-stream at counter=0x%04X",
"5A termination term_counter=0x%04X offset=0x%04X", counter,
term_counter, _BULK_TERM_OFFSET, )
) if include_terminator:
self._send(build_5a_frame(_BULK_TERM_OFFSET, term_params)) frames_data.append(rsp)
try: return frames_data
term_rsp = self._recv_one(expected_sub=rsp_sub)
frames_data.append(rsp)
last_chunk_counter = counter
counter += _BULK_COUNTER_STEP
chunks_fetched += 1
else:
log.warning(
"5A reached max_chunks=%d at counter=0x%04X (end=0x%04X)",
max_chunks, counter, end_offset,
)
# ── Step 5: TERM with proper end_offset-derived formula ──────────────
if last_chunk_counter is None or end_offset == 0xFFFF:
# No STRT or no chunks fetched — fall back to legacy TERM.
log.warning(
"5A using legacy TERM (offset_word=0x005A); "
"end_offset unavailable or no chunks fetched",
)
legacy_counter = (last_chunk_counter or probe_counter) + _BULK_COUNTER_STEP
term_offset_word = _BULK_TERM_OFFSET # 0x005A
term_params = bulk_waveform_term_params(key4, legacy_counter)
else:
term_offset_word, term_params = bulk_waveform_term_v2(
key4, end_offset, last_chunk_counter,
)
log.debug( log.debug(
"5A termination response page_key=0x%04X %d bytes", "5A TERM offset_word=0x%04X params[2:4]=%s end=0x%04X "
"last_chunk=0x%04X",
term_offset_word, term_params[2:4].hex(),
end_offset, last_chunk_counter,
)
self._send(build_5a_frame(term_offset_word, term_params))
try:
term_rsp = self._recv_one(expected_sub=rsp_sub, timeout=10.0)
log.info(
"5A TERM response page_key=0x%04X %d bytes",
term_rsp.page_key, len(term_rsp.data), term_rsp.page_key, len(term_rsp.data),
) )
if include_terminator: if include_terminator:
frames_data.append(term_rsp) frames_data.append(term_rsp)
except TimeoutError: except TimeoutError:
log.debug("5A no termination response — device may have already closed") log.warning("5A no TERM response (timeout)")
return frames_data return frames_data
@@ -1403,63 +1448,6 @@ class MiniMateProtocol:
log.debug("TX %d bytes: %s", len(frame), frame.hex()) log.debug("TX %d bytes: %s", len(frame), frame.hex())
self._transport.write(frame) self._transport.write(frame)
def _recv_5a_batch(
self,
expected_sub: int,
first_timeout: float = 10.0,
batch_timeout: float = 0.5,
) -> list[S3Frame]:
"""
Collect all S3 frames that arrive as part of one device response.
Over TCP via cellular modem, a single device A5 response (~1100 bytes of
RS-232 data) is forwarded in multiple TCP segments due to the modem's
data-forwarding timeout (~100-150 ms per segment). Each TCP segment
contains a complete, valid S3 frame (~550 bytes). Calling _recv_one()
once returns only the first segment's frame and misses the rest, causing
the chunk request/response pairing to cascade out of alignment.
This helper collects ALL frames before returning, by trying additional
short-timeout receives after the first frame arrives.
The caller must call self._parser.reset() before this method to ensure
bytes_fed is accurate; this method always uses reset_parser=False.
Args:
expected_sub: Expected SUB byte for validation.
first_timeout: Timeout for the mandatory first frame. Should be
generous (default 10 s) since the device may be slow.
batch_timeout: Short timeout for subsequent frames. Default 0.5 s
comfortably longer than the modem forwarding gap
(~150 ms) but short enough to avoid stalling when
only one frame is expected (probe, terminator).
Returns:
List of S3Frame objects in arrival order (at least one).
Raises:
TimeoutError: If no frame arrives within first_timeout.
UnexpectedResponse: If any frame has the wrong SUB byte.
"""
frames: list[S3Frame] = []
first = self._recv_one(
expected_sub=expected_sub,
reset_parser=False,
timeout=first_timeout,
)
frames.append(first)
while True:
try:
extra = self._recv_one(
expected_sub=expected_sub,
reset_parser=False,
timeout=batch_timeout,
)
frames.append(extra)
except TimeoutError:
break
return frames
def _recv_one( def _recv_one(
self, self,
expected_sub: Optional[int] = None, expected_sub: Optional[int] = None,
+99
View File
@@ -454,3 +454,102 @@ class SocketTransport(TcpTransport):
def __repr__(self) -> str: def __repr__(self) -> str:
return f"SocketTransport(peer={self.host!r})" return f"SocketTransport(peer={self.host!r})"
# ── Capturing transport (MITM-style raw byte mirror) ──────────────────────────
class CapturingTransport(BaseTransport):
"""
Wraps another BaseTransport and mirrors every byte to two raw capture files:
raw_bw_<...>.bin bytes WE wrote to the device (BW-side TX)
raw_s3_<...>.bin bytes the device wrote back (S3-side TX)
The file naming and on-wire byte layout are identical to the captures
produced by `bridges/ach_mitm.py`, so the resulting `.bin` files can be
loaded directly by the Analyzer (File > Open Capture) and parsed by the
same tooling used for genuine Blastware MITM captures.
All BaseTransport methods are forwarded to the inner transport; the only
side-effect is that successful read/write byte streams are appended to the
two open binary files.
Args:
inner: An already-built BaseTransport (SerialTransport / TcpTransport).
bw_path: File path for the "BW TX" stream (bytes we send). Opened "wb".
s3_path: File path for the "S3 TX" stream (bytes the device sends).
Opened "wb".
Example:
with CapturingTransport(TcpTransport("1.2.3.4", 9034),
"raw_bw.bin", "raw_s3.bin") as t:
client = MiniMateClient(transport=t)
client.connect()
client.get_events()
# both .bin files now hold the full bidirectional capture.
"""
def __init__(self, inner: BaseTransport, bw_path: str, s3_path: str) -> None:
self._inner = inner
self._bw_path = bw_path
self._s3_path = s3_path
self._bw_fh = None
self._s3_fh = None
# Forward inner attrs so callers can introspect (e.g. .host, .port).
self.host = getattr(inner, "host", None)
self.port = getattr(inner, "port", None)
# ── BaseTransport interface ───────────────────────────────────────────────
def connect(self) -> None:
if self._bw_fh is None:
self._bw_fh = open(self._bw_path, "wb", buffering=0)
if self._s3_fh is None:
self._s3_fh = open(self._s3_path, "wb", buffering=0)
self._inner.connect()
def disconnect(self) -> None:
try:
self._inner.disconnect()
finally:
for fh_attr in ("_bw_fh", "_s3_fh"):
fh = getattr(self, fh_attr)
if fh is not None:
try:
fh.flush()
fh.close()
except Exception:
pass
setattr(self, fh_attr, None)
@property
def is_connected(self) -> bool:
return self._inner.is_connected
def write(self, data: bytes) -> None:
self._inner.write(data)
if data and self._bw_fh is not None:
try:
self._bw_fh.write(data)
except Exception:
pass
def read(self, n: int) -> bytes:
got = self._inner.read(n)
if got and self._s3_fh is not None:
try:
self._s3_fh.write(got)
except Exception:
pass
return got
@property
def bw_path(self) -> str:
return self._bw_path
@property
def s3_path(self) -> str:
return self._s3_path
def __repr__(self) -> str:
return f"CapturingTransport({self._inner!r}, bw={self._bw_path!r}, s3={self._s3_path!r})"
+2
View File
@@ -53,7 +53,9 @@ SUB_TABLE: dict[int, tuple[str, str, str]] = {
0x82: ("TRIGGER_CONFIG_WRITE", "BW→S3", "0x1C bytes; trigger config block; mirrors SUB 1C"), 0x82: ("TRIGGER_CONFIG_WRITE", "BW→S3", "0x1C bytes; trigger config block; mirrors SUB 1C"),
0x83: ("TRIGGER_WRITE_CONFIRM", "BW→S3", "Short frame; commit step after 0x82"), 0x83: ("TRIGGER_WRITE_CONFIRM", "BW→S3", "Short frame; commit step after 0x82"),
# S3→BW responses # S3→BW responses
0x5A: ("BULK_WAVEFORM_STREAM", "BW→S3", "Bulk waveform chunk request; response is A5 stream"),
0xA4: ("POLL_RESPONSE", "S3→BW", "Response to SUB 5B poll"), 0xA4: ("POLL_RESPONSE", "S3→BW", "Response to SUB 5B poll"),
0xA5: ("BULK_WAVEFORM_RESPONSE", "S3→BW", "Response to SUB 5A; waveform chunks + metadata"),
0xFE: ("FULL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 01"), 0xFE: ("FULL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 01"),
0xF9: ("CHANNEL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 06"), 0xF9: ("CHANNEL_CONFIG_RESPONSE", "S3→BW", "Response to SUB 06"),
0xF7: ("EVENT_INDEX_RESPONSE", "S3→BW", "Response to SUB 08; contains backlight/power-save"), 0xF7: ("EVENT_INDEX_RESPONSE", "S3→BW", "Response to SUB 08; contains backlight/power-save"),
+33 -36
View File
@@ -33,7 +33,7 @@ STX = 0x02
ETX = 0x03 ETX = 0x03
ACK = 0x41 ACK = 0x41
__version__ = "0.2.3" __version__ = "0.2.5"
@dataclass @dataclass
@@ -184,9 +184,9 @@ def validate_bw_body_auto(body: bytes) -> Optional[Tuple[bytes, bytes, str]]:
def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]: def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]:
frames: List[Frame] = [] frames: List[Frame] = []
IDLE = 0 IDLE = 0
IN_FRAME = 1 IN_FRAME = 1
AFTER_DLE = 2 IN_FRAME_DLE = 2 # saw DLE inside frame — waiting for next byte
state = IDLE state = IDLE
body = bytearray() body = bytearray()
@@ -206,66 +206,63 @@ def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]:
state = IN_FRAME state = IN_FRAME
i += 2 i += 2
continue continue
# ACK bytes, boot strings, garbage — silently ignored
elif state == IN_FRAME: elif state == IN_FRAME:
if b == DLE: if b == DLE:
state = AFTER_DLE state = IN_FRAME_DLE
i += 1 i += 1
continue continue
body.append(b)
else: # AFTER_DLE
if b == DLE:
body.append(DLE)
state = IN_FRAME
i += 1
continue
if b == ETX: if b == ETX:
# Bare ETX = real S3 frame terminator (confirmed from S3FrameParser)
end_offset = i + 1 end_offset = i + 1
trailer_start = i + 1 trailer_start = i + 1
trailer_end = trailer_start + trailer_len trailer_end = trailer_start + trailer_len
trailer = blob[trailer_start:trailer_end] trailer = blob[trailer_start:trailer_end]
chk_valid = None # S3 checksums are deliberately not validated here.
chk_type = None # Large S3 responses (A5 bulk waveform, E5 compliance) embed
chk_hex = None # inner DLE+ETX sub-frame terminators whose trailing 0x03 byte
payload = bytes(body) # lands where the parser would expect the SUM8 checksum, causing
# false failures. The live protocol (protocol.py _validate_frame)
if len(body) >= 1: # also skips S3 checksum enforcement for the same reason.
received_chk = body[-1]
computed_chk = checksum8_sum(bytes(body[:-1]))
if computed_chk == received_chk:
chk_valid = True
chk_type = "SUM8"
chk_hex = f"{received_chk:02x}"
payload = bytes(body[:-1])
else:
chk_valid = False
frames.append(Frame( frames.append(Frame(
index=idx, index=idx,
start_offset=start_offset, start_offset=start_offset,
end_offset=end_offset, end_offset=end_offset,
payload_raw=bytes(body), payload_raw=bytes(body),
payload=payload, payload=bytes(body),
trailer=trailer, trailer=trailer,
checksum_valid=chk_valid, checksum_valid=None,
checksum_type=chk_type, checksum_type=None,
checksum_hex=chk_hex checksum_hex=None
)) ))
idx += 1 idx += 1
state = IDLE state = IDLE
i = trailer_end i = trailer_end
continue continue
body.append(b)
else: # IN_FRAME_DLE
if b == DLE:
# DLE DLE → literal 0x10 in payload
body.append(DLE)
state = IN_FRAME
i += 1
continue
if b == ETX:
# DLE+ETX inside a frame = inner-frame terminator (A4/E5 sub-frames).
# Treat as literal data, NOT the outer frame end.
body.append(DLE)
body.append(ETX)
state = IN_FRAME
i += 1
continue
# Unexpected DLE + byte → treat as literal data # Unexpected DLE + byte → treat as literal data
body.append(DLE) body.append(DLE)
body.append(b) body.append(b)
state = IN_FRAME state = IN_FRAME
i += 1
continue
i += 1 i += 1
+803 -111
View File
File diff suppressed because it is too large Load Diff
+19 -18
View File
@@ -37,6 +37,7 @@ from __future__ import annotations
import datetime import datetime
import logging import logging
import sys import sys
import tempfile
import threading import threading
import time import time
from pathlib import Path from pathlib import Path
@@ -863,8 +864,8 @@ def device_event_blastware_file(
Supply either *port* (serial) or *host* (TCP/modem). Supply either *port* (serial) or *host* (TCP/modem).
The file is written to /tmp and streamed back as a binary download. The file is written to the OS temp directory and streamed back as a binary
Blastware can open it directly filename encodes serial + timestamp. download. Blastware can open it directly filename encodes serial + timestamp.
Filename format: <prefix><serial3><stem><AB>0<W|H> Filename format: <prefix><serial3><stem><AB>0<W|H>
- prefix letter = chr(ord('B') + floor(serial_numeric / 1000)) - prefix letter = chr(ord('B') + floor(serial_numeric / 1000))
@@ -885,23 +886,13 @@ def device_event_blastware_file(
def _do(): def _do():
with _build_client(port, baud, host, tcp_port, timeout=120.0) as client: with _build_client(port, baud, host, tcp_port, timeout=120.0) as client:
info = client.connect() info = client.connect()
# Use stop_after_metadata=True (full_waveform=False) with 1 extra # Under v0.14.0 BW-exact 5A walk, the chunk loop is bounded by
# chunk after "Project:". The extra chunk primes the device so that # the event end_offset extracted from STRT. No more
# the termination response carries the full waveform footer bytes. # stop_after_metadata / extra_chunks gymnastics — these
# Without it the terminator returns only ~90 bytes (no useful footer). # kwargs are now no-ops.
#
# The extra chunk's ADC data IS part of the Blastware file body —
# confirmed from 4-27-26 TCP capture: all 14 A5 frames (including the
# extra chunk's 2 TCP sub-frames) contribute to the correct 6864-byte
# output. write_blastware_file() includes all frames unconditionally.
#
# full_waveform=True (natural end-of-stream) downloads ALL chunks
# including post-event silence (35+ chunks for a 9-sec event at
# 1024 sps) — this produces 24KB+ files that Blastware rejects.
events = client.get_events( events = client.get_events(
full_waveform=False, full_waveform=False,
stop_after_index=index, stop_after_index=index,
extra_chunks_after_metadata=1,
) )
matching = [ev for ev in events if ev.index == index] matching = [ev for ev in events if ev.index == index]
return matching[0] if matching else None, info return matching[0] if matching else None, info
@@ -937,8 +928,18 @@ def device_event_blastware_file(
# Build filename using the same algorithm Blastware uses # Build filename using the same algorithm Blastware uses
filename = blastware_filename(ev, serial) filename = blastware_filename(ev, serial)
# Write to /tmp so FastAPI can stream it back # Write to OS temp dir (cross-platform: /tmp on Linux/macOS,
out_path = Path("/tmp") / filename # %TEMP% on Windows) so FastAPI can stream it back via FileResponse.
out_path = Path(tempfile.gettempdir()) / filename
# Delete any stale file at this path before writing. On Windows we have
# observed the new (smaller) file getting trailing zero-bytes from the
# previous (larger) file when filesystem semantics around open(...,"wb")
# don't truncate cleanly (e.g. through a synced folder). Explicit unlink
# eliminates that ambiguity.
try:
out_path.unlink()
except FileNotFoundError:
pass
write_blastware_file(ev, a5_frames, out_path) write_blastware_file(ev, a5_frames, out_path)
log.info( log.info(
"blastware_file: wrote %s (%d A5 frames, serial=%s)", "blastware_file: wrote %s (%d A5 frames, serial=%s)",