Merge pull request 'v0.14.3 - Full waveform DL pipeline tested and working.' (#15) from protocol-fix into main

Reviewed-on: #15
This commit was merged in pull request #15.
This commit is contained in:
2026-05-05 20:49:47 -04:00
10 changed files with 1049 additions and 380 deletions
+200
View File
@@ -4,6 +4,206 @@ All notable changes to seismo-relay are documented here.
--- ---
## v0.14.3 — 2026-05-05
### Fixed
- **`build_5a_frame` — DLE-stuffing rule for 0x10 bytes in params (the
long-standing >1-sec event 0 "won't open in BW" bug).**
Previously `build_5a_frame` wrote params bytes RAW with no DLE stuffing,
based on the incorrect assumption that the device handled all `0x10`
bytes in params literally. It does not. The device's actual de-stuffing
rule for the params region is:
- `10 10` → de-stuffs to `10`
- `10 02/03/04` → kept literal (inner-frame markers)
- `10 X` for other X → de-stuffs to just `X` (drops the `0x10`)
When the counter passed in params has `0x10` in the high byte (e.g.
counter=`0x1000` produces params bytes `... 10 00 ...`), the device
silently corrupts the request to counter=`0x__00` and responds with
whatever lives at that wrong address. For counter=0x1000 the wrong
address was 0x0000, so the response was a copy of the file header +
STRT record. That STRT block then got embedded in the assembled body
at file offset `0x1016`, and Blastware refused to open the file
(interprets the second STRT as a malformed multi-event file).
This explains the entire >1-sec event-0 failure pattern:
- 1-sec events have `end_offset < 0x1000`, so the chunk walk never
requests counter `0x10__` and the bug never triggers.
- 2-sec / 3-sec / longer events all need a chunk at counter `0x1000`
(and longer events also need `0x1200`, `0x1400`, etc., none of which
have `0x10` in the high byte except `0x1000`). Just one corrupted
response is enough to embed STRT in the body and break the file.
Verified against BW 5-1-26 "copy 3sec" capture: all 17 5A request
frames (probe + 2 metadata pages + 13 sample chunks + TERM) now match
BW's wire output **byte-for-byte**, including the doubled `10 10 00`
for counter=0x1000.
### Notes
- `0x10` bytes in `offset_hi` (the standalone offset field at body[5])
are still written RAW — confirmed correct per the 1-2-26 capture.
- BW's actual encoding of `10 02` / `10 04` for meta pages 0x1002 /
0x1004 is *not* doubled — it relies on the device keeping `10 02`
and `10 04` as literal pairs. This is preserved by the fix.
---
## v0.14.2 — 2026-05-04
### Fixed
- **`blastware_file.py` — removed harmful "duplicate header+STRT" strip.**
The v0.13.x strip logic was matching the byte sequence `00 12 03 00 STRT`
in legitimate waveform data — sample chunks at counter `0x1000` and
beyond often contain those bytes coincidentally — and zeroing 25 bytes
of valid samples per match. This is why event 0 (event-1 case in the
protocol) downloads of >1-sec recordings always failed in BW: the strip
destroyed real data at body offset `0x1012..0x102B` and propagated
alignment differences through the rest of the body. Sub-1-sec events
worked because their `end_offset` was below `0x1002`, so no sample
chunks landed in the metadata-page region and the strip's needle never
matched. Verified fix by re-feeding the BW 5-1-26 "copy 3sec" capture's
A5 frames into the file builder: output is now byte-identical to BW's
saved `M529LKIQ.G10` reference (8708 bytes, 0 differences).
- BW already concatenates frame contributions in stream order without
any de-duplication; SFM now does the same.
---
## v0.14.1 — 2026-05-04
### Fixed
- **`read_bulk_waveform_stream` — event-N probe counter off-by-`0x46`.**
Continuation events (start_key[2:4] != 0) were being probed at counter
`start_offset + 0x0046` instead of just `start_offset`. In the iteration
walk, `cur_key` from 1F is already the off=0x46 WAVEHDR record key, so the
earlier formula effectively double-counted the WAVEHDR offset. The probe
landed one WAVEHDR past the actual event start, the response no longer
contained the STRT record at byte 17, `parse_strt_end_offset` returned
`None`, and the chunk loop fell back to the `max_chunks=128` cap — walking
~110 chunks of post-event circular-buffer garbage. Verified against the
5-1-26 "copy 2nd address" and 5-4-26 BW 2-sec event captures: BW probes
counter=`0x2238` with key=`01112238` and STRT is present at byte 17 of
the response (end_offset=`0x417E`).
- **CLAUDE.md / docs/instantel_protocol_reference.md** — corrected the
event-N section to clarify that `start_key` in those formulas is the
off=0x46 key, not the off=0x2C boundary key, and removed the spurious
`+0x46` from the chunk-walk pseudocode.
---
## v0.13.2 — 2026-05-01
### Fixed
- **`_extract_record_type` — third 0C-record header format ("short", 8 bytes).**
A live SFM download against BE11529 produced files named `M5290000.000`
(zero-stamped) because the 0C waveform record's first bytes were
`01 05 07 ea ...` — neither the 9-byte single-shot layout (`0x10` at byte 1)
nor the 10-byte continuous layout (`0x10` at bytes 0 and 2). Investigation
showed this is a third format observed in the wild: an 8-byte header with no
marker bytes at all (`[day][month][year_BE:2][unknown][hour][min][sec]`).
The detection logic now scans the year (uint16 BE) at byte 2 / byte 3 / byte
4 and picks whichever offset returns a sensible year (20152050) — each
format has the year at a unique position so this disambiguates cleanly.
- New format → `event.record_type = "Waveform (Short)"`,
`Timestamp.from_short_record()`.
- Existing single-shot and continuous parsers unchanged.
- The user's event from May 1, 2026 13:21:37 now correctly resolves to a
filename like `M529LKIQ.G10` instead of `M5290000.000`.
### Added
- `Timestamp.from_short_record(data)` — decodes the 8-byte header.
- `_detect_record_format(data)` — internal helper returning
`"single_shot" / "continuous" / "short" / None` via year-position scan.
---
## v0.13.1 — 2026-05-01
### Fixed
- **`_extract_record_type` — Continuous-mode record headers misclassified as Unknown.**
In single-shot mode the 0C waveform record's 9-byte header puts the sub_code
marker `0x10` at byte 1, with the day at byte 0. In Continuous mode the
header is 10 bytes with the marker at byte 0 *and* byte 2, and the day at
byte 1. Previous logic only inspected byte 1 and treated any value other
than `0x10` / `0x03` as `"Unknown"`, which prevented `event.timestamp` from
being populated for any continuous-mode event whose day-of-month wasn't
exactly 3 or 16. As a downstream effect, `blastware_filename()` saw
`event.timestamp == None`, fell back to `stem="0000"` / `ab="00"`, and
produced filenames like `M5290000.000`. Discovered from a live SFM run on
BE11529 in continuous mode (day-of-month = 5).
Now disambiguates by checking BOTH byte 0 and byte 2: if both are `0x10`,
it's the 10-byte continuous header; else if byte 1 is `0x10`, it's the
9-byte single-shot header. Day-of-month no longer matters.
*Superseded by v0.13.2 — the user's actual record uses a third 8-byte format
with no `0x10` markers, which v0.13.1 still misclassified.*
---
## v0.13.0 — 2026-05-01
### Fixed
- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
`read_bulk_waveform_stream` was walking the chunk counter past the actual
end of the event, picking up post-event circular-buffer garbage that
corrupted reconstructed Blastware files for any waveform > ~1 sec. The
loop now extracts the event's `end_offset` from the STRT record at
`data[23:27]` of the probe response and stops the chunk walk when the next
counter would step past it. Verified against three BW MITM captures
(4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.
### Added
- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`
computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
formula confirmed across all 3 BW captures. Not yet wired into
`read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
existing `blastware_file.write_blastware_file` frame-structure expectations);
available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
from the STRT record in an A5 response payload.
### Documentation
- **CLAUDE.md and `docs/instantel_protocol_reference.md` extensively
rewritten** to reflect the corrected SUB 5A protocol. See:
- CLAUDE.md "SUB 5A — chunk counter formula (REWRITTEN 2026-05-01)"
- CLAUDE.md "SUB 5A — STRT record encodes end_offset"
- CLAUDE.md "SUB 5A — TERM frame formula"
- CLAUDE.md "SUB 5A — fixed metadata pages 0x1002 and 0x1004"
- CLAUDE.md "SUB 0A — WAVEHDR response length distinguishes events from
boundaries" (0x46 = real event, 0x2C = boundary marker)
- protocol reference §7.8.5 / §7.8.6 / §7.8.7 / §7.8.8
- The previous chunk-counter formula (`max(key4[2:4], 0x0400) + (chunk-1) *
0x0400`) is now marked DEPRECATED and explicitly tagged WRONG with
pointers to the new sections, so future work doesn't re-derive it.
### Known minor diffs vs Blastware (deferred to a follow-up)
- We still use the OLD 0x0400 chunk step rather than BW's 0x0200; switching
also requires updating `blastware_file.write_blastware_file`'s skip values
and "extra chunk after metadata" logic, which depends on a fresh capture
to verify.
- We still use the legacy fixed `offset_word=0x005A` TERM frame rather than
BW's `end_offset - next_boundary` formula, for the same reason.
- Two fixed metadata pages at counter `0x1002` and `0x1004` are not yet
read explicitly; under the current 0x0400 walk their content is reachable
via the sample chunk that covers buffer addresses `[0x1000, 0x1400)`.
---
## v0.12.6 — 2026-05-01 ## v0.12.6 — 2026-05-01
### Fixed ### Fixed
+127 -66
View File
@@ -2,7 +2,7 @@
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.12.3**. (Sierra Wireless RV50 / RV55). Current version: **v0.14.3**.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
@@ -27,7 +27,7 @@ CHANGELOG.md ← version history
--- ---
## Current implementation state (v0.12.3) ## Current implementation state (v0.14.3)
Full read pipeline + write pipeline + erase pipeline + monitor log + call home config working end-to-end over TCP/cellular: Full read pipeline + write pipeline + erase pipeline + monitor log + call home config working end-to-end over TCP/cellular:
@@ -41,14 +41,15 @@ Full read pipeline + write pipeline + erase pipeline + monitor log + call home c
| Event header / first key | 1E | ✅ | | Event header / first key | 1E | ✅ |
| Waveform header | 0A | ✅ | | Waveform header | 0A | ✅ |
| Waveform record (peaks, timestamp, project) | 0C | ✅ | | Waveform record (peaks, timestamp, project) | 0C | ✅ |
| **Bulk waveform stream (event-time metadata)** | **5A** | ⚠️ partial — over-reads ~5× past event end for ≥2-sec events; corrected algorithm documented but not yet implemented (see "SUB 5A — chunk counter formula" section, dated 2026-05-01) | | **Bulk waveform stream (event-time metadata + full waveform)** | **5A** | **byte-perfect against BW captures (v0.14.3, 2026-05-05)** — STRT-bounded chunk walk + correct event-N probe counter + DLE-stuffed `0x10` bytes in params + concatenate-only file body assembly. All 17 5A request frames in the 5-1-26 3-sec capture reproduce byte-for-byte. |
| Event advance / next key | 1F | ✅ | | Event advance / next key | 1F | ✅ |
| **Write commands (push config to device)** | **6883** | ✅ new v0.8.0 | | **Write commands (push config to device)** | **6883** | ✅ new v0.8.0 |
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 | | **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ new v0.10.0 | | **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ new v0.10.0 |
| **Auto Call Home config (read + write)** | **2C → 7E → 7F** | ✅ **new v0.12.3** | | **Auto Call Home config (read + write)** | **2C → 7E → 7F** | ✅ **new v0.12.3** |
`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F` `get_events()` sequence per event: `1E → 0A → 1E(arm token=0xFE) → 0C → 1F(arm) → POLL×3 → 5A → 1F(browse)`
(see "Correct iteration pattern" section below for full detail)
`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72` `push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`
@@ -115,8 +116,31 @@ S3→BW (response):
section contribute only `XX` to the running sum; lone bytes contribute normally. This section contribute only `XX` to the running sum; lone bytes contribute normally. This
differs from the standard SUM8-of-destuffed-payload that all other commands use. differs from the standard SUM8-of-destuffed-payload that all other commands use.
Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26 3. **Params region uses partial DLE stuffing (CONFIRMED 2026-05-05).** The device's
BW TX capture. All 10 frames verified. de-stuffing rule for bytes inside the params region is:
- `10 10` → de-stuffs to `10`
- `10 02 / 03 / 04` → kept literal (these are inner-frame markers)
- `10 X` for other X → de-stuffs to just `X` (drops the leading `0x10`)
Therefore any `0x10` byte in the *logical* params that is followed by a byte NOT in
`{0x02, 0x03, 0x04, 0x10}` MUST be doubled on the wire (`10 X``10 10 X`) so the
device's de-stuffer reproduces the original `10 X` pair. This applies most commonly
to counters with `0x10` in the high byte (e.g. counter=`0x1000` produces logical
params bytes `... 10 00 ...`, which BW encodes on the wire as `... 10 10 00 ...`).
Without this stuffing the device interprets counter=`0x1000` as `0x0000` and returns
the probe response (which contains a copy of the file header + STRT record). That
STRT block then gets embedded in the assembled file body at offset `0x1016`, and
Blastware refuses to open the file — see the v0.14.3 entry in `CHANGELOG.md`.
`0x10` bytes in `offset_hi` (body[5]) are still written RAW — only the params region
has this stuffing requirement. The metadata-page params for counter `0x1002` /
`0x1004` survive without stuffing because `10 02` and `10 04` fall in the "kept
literal" carve-out.
Both differences (1) and (2) confirmed by reproducing Blastware's exact wire bytes from
the 1-2-26 BW TX capture (10 frames). Difference (3) confirmed against the 5-1-26
"bwcap3sec" capture (17 frames, all match byte-for-byte after fix).
### SUB 5A — chunk counter formula (REWRITTEN 2026-05-01 — see 5-1-26 captures) ### SUB 5A — chunk counter formula (REWRITTEN 2026-05-01 — see 5-1-26 captures)
@@ -160,13 +184,28 @@ firmware reserved area for the first slot in a freshly-erased buffer. Harmless
#### Event 2+ case — start_key[2:4] != 0x0000 (continuation events) #### Event 2+ case — start_key[2:4] != 0x0000 (continuation events)
``` ```
1. First chunk at counter = start_key[2:4] + 0x0046 (this IS the probe — response 1. First chunk at counter = start_key[2:4] (this IS the probe — response
contains STRT) contains STRT at byte 17)
2. Sample chunks: counter += 0x0200 each, up to but 2. Sample chunks: counter += 0x0200 each, up to but
not including end_offset not including end_offset
3. TERM frame 3. TERM frame
``` ```
**`start_key` here is the off=0x46 WAVEHDR record key returned by 1F** (e.g. `01112238`),
NOT the off=0x2C boundary key that immediately precedes it. An earlier draft of this
doc described event-N as "probe at start + 0x46" — that formula came from naming the
boundary key as `start_key`. In the iteration walk, `cur_key` passed to
`read_bulk_waveform_stream` is always the off=0x46 key (the partial-record skip path in
`get_events` re-runs 1F to advance past boundary records before invoking 5A), so the
probe counter is just `cur_key[2:4]` with no extra offset. **Adding +0x46 caused the
probe to overshoot, miss the STRT record at byte 17 of the response, fall back to the
`max_chunks=128` cap, and walk ~110 chunks of post-event garbage** — observed in
SFM 5-4-26 capture before the fix.
Confirmed across:
- 5-1-26 "copy 2nd address" BW capture: probe counter=0x2238, key=01112238, STRT@17 end=0x417E.
- 5-4-26 BW 2-sec event capture: probe counter=0x2238, key=01112238, TERM offset_word=0x0146 → end=0x417E.
No metadata pages — those have already been read during event 1 in the same Blastware No metadata pages — those have already been read during event 1 in the same Blastware
session, and BW caches them. Note that the metadata-page reads happen ONCE per session, and BW caches them. Note that the metadata-page reads happen ONCE per
Blastware-session-on-the-device, not once per event, so an SFM session that downloads Blastware-session-on-the-device, not once per event, so an SFM session that downloads
@@ -180,6 +219,12 @@ several events should read 0x1002/0x1004 only once at the start.
- 2026-04-26: `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` (broken — over-read past event end). - 2026-04-26: `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` (broken — over-read past event end).
- 2026-05-01: Increments are 0x0200 not 0x0400; absolute addresses inside event range; bounded - 2026-05-01: Increments are 0x0200 not 0x0400; absolute addresses inside event range; bounded
by STRT end_key, not by `max_chunks` cap or device-side timeout. by STRT end_key, not by `max_chunks` cap or device-side timeout.
- 2026-05-04: Removed spurious `+0x0046` from event-N probe counter. `cur_key` from 1F
is already the off=0x46 WAVEHDR key, so adding +0x46 would have placed the probe one
WAVEHDR past the actual event start. This caused probe responses to lack a STRT
record (no `end_offset` parsed → `0xFFFF` fallback → `max_chunks=128` cap), walking
~110 chunks of post-event circular-buffer garbage. Fixed in protocol.py
`read_bulk_waveform_stream`.
### SUB 5A — STRT record encodes end_offset (NEW 2026-05-01) ### SUB 5A — STRT record encodes end_offset (NEW 2026-05-01)
@@ -254,9 +299,8 @@ Two chunk addresses are GLOBAL device/session metadata, not event-specific:
These are at fixed absolute addresses in the device's flash buffer. They contain the These are at fixed absolute addresses in the device's flash buffer. They contain the
session-start compliance setup (Project/Client/User Name/Seis Loc/Extended Notes ASCII session-start compliance setup (Project/Client/User Name/Seis Loc/Extended Notes ASCII
strings) that A5 frame 7 used to be the source for in the old "0x0400-step" walk. In the strings). Under the v0.14.0+ walk these strings are read directly from the metadata
new walk these strings come from the dedicated metadata pages, not from the sample-chunk pages, not from the sample-chunk stream.
stream.
BW reads them ONCE per Blastware session (during event 1's download) and caches them. BW reads them ONCE per Blastware session (during event 1's download) and caches them.
For SFM, that means: For SFM, that means:
@@ -265,9 +309,10 @@ For SFM, that means:
- Their content does not change when iterating events; only when the user opens - Their content does not change when iterating events; only when the user opens
Compliance Setup → Apply on the device or sends a SUB 71 compliance write. Compliance Setup → Apply on the device or sends a SUB 71 compliance write.
The contents have not been byte-for-byte decoded yet — first task on the implementation The full byte-for-byte layout of the metadata pages has not been mapped — `_decode_a5_metadata_into`
side is to dump 0x1002 + 0x1004 from a fresh capture and verify they include all the locates the ASCII strings via label scans (`Project:`, `Client:`, `User Name:`, `Seis Loc:`,
strings we currently extract from A5[7]. `Extended Notes`) which works correctly across observed captures. Future work could
dump the structural layout if more session-global fields need to be extracted.
### SUB 5A — params are 11 bytes for chunk frames, 10 for termination ### SUB 5A — params are 11 bytes for chunk frames, 10 for termination
@@ -275,16 +320,11 @@ strings we currently extract from A5[7].
confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes. confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes.
Do not swap them. Do not swap them.
### SUB 5A — event-time metadata source (UPDATED 2026-05-01) ### SUB 5A — event-time metadata source (FINALIZED 2026-05-05)
> **Old understanding (deprecated):** the metadata strings live in "A5 frame 7" of the 5A The metadata strings come from the two fixed metadata pages at counter `0x1002` and
> bulk stream. This was a side-effect of the old `0x0400`-step walk: the sample-chunk at `0x1004` (see "SUB 5A — fixed metadata pages 0x1002 and 0x1004" above). These pages
> counter ≈ 0x1400 would happen to include the global 0x1002/0x1004 metadata pages because are GLOBAL session metadata — read once per Blastware/SFM session, not per event.
> the broken counter formula was scanning the wrong region.
>
> **New understanding:** the metadata strings live at fixed counter addresses `0x1002` and
> `0x1004`. See "SUB 5A — fixed metadata pages 0x1002 and 0x1004" above. The 5A
> sample-chunk stream itself does NOT contain these strings any more under the new walk.
``` ```
"Project:" → project description "Project:" → project description
@@ -294,55 +334,71 @@ Do not swap them.
"Extended Notes"→ notes "Extended Notes"→ notes
``` ```
**IMPORTANT — 5A "Project:" is session-start config, NOT per-event (confirmed 2026-04-05):** **IMPORTANT — these strings are session-start config, NOT per-event:**
The "Project:" string in the A5 frame 7 payload reflects the compliance setup from when Project / Client / User Name / Seis Loc reflect the compliance setup from when the
the *monitoring session first started*, not the individual event's project name. The per- *monitoring session first started*, not the individual event's per-event metadata. The
event project name is correctly stored in the 210-byte 0C waveform record and must be authoritative per-event project name is stored in the 210-byte 0C waveform record.
used as the authoritative source. `_decode_a5_metadata_into` therefore only sets `_decode_a5_metadata_into` therefore only sets `project` from the 5A metadata pages
`project` from 5A when 0C didn't already supply one. when 0C didn't already supply one.
"Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C "Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C
record — 5A remains the sole source for those fields and they are set unconditionally. record — the metadata pages are the sole source for those fields and they are set
unconditionally.
> ⚠️ `stop_after_metadata=True` (which scans for `b"Project:"` in the chunk stream and #### Deprecated knobs (do not re-introduce)
> stops one chunk later) is a workaround for the missing end_offset bound — when the new
> STRT-bounded walk lands, this knob becomes obsolete. The proper "stop" condition is
> `next_chunk_counter >= end_offset & 0xFE00`, with the partial tail fetched by the TERM
> frame.
### SUB 5A — end-of-stream — UPDATED 2026-05-01 The `read_bulk_waveform_stream()` function still accepts these legacy kwargs for
backward compatibility, but they are **no-ops** under the v0.14.0+ walk:
> **Previous understanding (now known to be a symptom, not a feature):** "After streaming - `stop_after_metadata=True` — used to scan the chunk stream for `b"Project:"` and stop
> all waveform chunks, the device sends exactly **1 raw byte** then goes silent." This was one chunk later as a workaround for the missing end_offset bound. Obsolete: the loop
> not the device's natural end-of-event signal — it was the device's response when SFM had is now deterministically bounded by `end_offset` parsed from the STRT record at
> walked clean off the end of the addressable buffer region after over-reading by ~5×. data[17] of the probe response, with the partial tail fetched by the TERM frame.
> Under the corrected walk (chunks bounded by `end_offset` from STRT, terminated with the - `extra_chunks_after_metadata` — same era, same reason. No-op.
> proper TERM frame), the stream ends cleanly: TERM request → TERM response (`page=0x0000`,
> sized to the residual `end_offset - next_boundary`). No timeout, no 1-byte teaser.
The `bytes_fed=1 → graceful end` heuristic in `read_bulk_waveform_stream` is still a useful If you find code or docs referencing "A5 frame 7" as the source of metadata strings,
defence-in-depth fallback for malformed events or unexpected device states, but should not that's an old-walk artifact (the broken `0x0400`-step formula occasionally caught the
be the primary loop-exit condition. 0x1002 metadata page at sample-chunk fi=7). Update to reference the dedicated metadata
pages instead.
**Chunk recv timeout must be 10 s, not the default 120 s.** Chunks arrive within ~1 s each. ### SUB 5A — end-of-stream (FINALIZED 2026-05-01)
Using 120 s causes a ~2-minute stall at every end-of-stream detection. The `_recv_one` call
in the chunk loop passes `timeout=10.0` explicitly.
**Typical chunk count under the corrected walk (BE11529, 1024 sps over TCP/cellular):** Under the v0.14.0+ STRT-bounded walk the stream ends cleanly:
A 2-sec event takes 12 sample chunks + 2 metadata pages (event 1) + TERM = ~15 frames.
A 3-sec event takes 16 sample chunks + 2 metadata pages + TERM = ~19 frames.
An 8 KB event 2 (continuation) takes 15 sample chunks + TERM = ~16 frames.
Compare to the old over-read walk: same 2-sec event was producing 37 chunks, with chunks ```
17-37 containing post-event circular-buffer garbage that corrupted the file body. … last full chunk at counter < end_offset
TERM request (offset_word = end_offset - next_boundary,
params address (next_boundary))
TERM response (page_key = 0x0000 or 0x0001, data = the residual
end_offset - next_boundary bytes including the file footer)
```
No timeout-based detection, no "1-byte teaser," no `max_chunks` cap. The chunk loop
exits when `counter + 0x0200 > end_offset`; the TERM frame fetches the tail.
**Chunk recv timeout is 10 s, not the default 120 s.** Chunks arrive within ~1 s each.
Using 120 s would cause a ~2-minute stall on any unexpected timeout. The `_recv_one`
call in the chunk loop passes `timeout=10.0` explicitly.
**Typical chunk count under the v0.14.0+ walk (BE11529, 1024 sps over TCP/cellular):**
| Event duration | Sample chunks | Metadata pages | TERM | Total A5 frames |
|---|---|---|---|---|
| 2-sec (event 1) | ~12 | 2 | 1 | ~15 |
| 3-sec (event 1) | 13 | 2 | 1 | 16 |
| 2-sec (continuation) | 15 | 0 | 1 | 16 |
| 3-sec (continuation) | ~14 | 0 | 1 | ~15 |
For comparison, the deprecated `0x0400`-step walk produced ~37 chunks for a 2-sec
event with chunks 17-37 containing post-event circular-buffer garbage. Do not
re-introduce that walk under any circumstances.
### SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06) ### SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06)
`_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from the `_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from the
9-frame original blast capture where frame 9 was assumed to be a terminator. For current 9-frame original blast capture where frame 9 was assumed to be a terminator. Removed.
35-frame streams, fi==9 is live waveform data (~133 sample-sets were being dropped). TERM detection in the file builder uses `frame.page_key != 0x0010` (sample marker),
Removed. Terminator detection is via `page_key == 0x0000` in `read_bulk_waveform_stream`, not frame index — see `blastware_file.py`.
not frame index.
### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce) ### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)
@@ -985,7 +1041,7 @@ offsets in the raw 1A/E5 payload. Only fields with `✅` have confirmed offsets
**Notes tab:** **Notes tab:**
- Enable User Notes (bool) - Enable User Notes (bool)
- Project, Client, User Name, Seis Loc (ASCII strings) ✅ (sourced from A5 frame 7 via 5A) - Project, Client, User Name, Seis Loc (ASCII strings) ✅ (sourced from 5A metadata pages at counter 0x1002 / 0x1004 — see "SUB 5A — fixed metadata pages" section)
- Enable Extended Notes (bool); Extended Notes text; Extended Notes Title - Enable Extended Notes (bool); Extended Notes text; Extended Notes Title
- Enable Job Number (bool); Job Number (int) - Enable Job Number (bool); Job Number (int)
- Enable Scaled Distance (bool); Distance from Blast (float); Charge Weight (float) — Scaled Distance is derived - Enable Scaled Distance (bool); Distance from Blast (float); Charge Weight (float) — Scaled Distance is derived
@@ -1299,7 +1355,7 @@ body) because writing a dial string may require DLE escaping for embedded contro
- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable - **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
- **Histograms** — decode histogram-mode A5 data (noise floor tracking) - **Histograms** — decode histogram-mode A5 data (noise floor tracking)
- **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed working for Continuous mode events (2026-04-23):** SFM-generated file opens in Blastware, shows correct PPV/waveform/timestamp. File is ~200 bytes shorter than BW (missing last ADC tail slice) — all measurements correct. Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that create spurious STRT markers in the body). Extension mapping: **CONFIRMED FALSE 2026-04-21** extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>` - **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed BYTE-PERFECT against BW reference (v0.14.3, 2026-05-05):** when fed the BW 5-1-26 3-sec capture's A5 frames, the SFM-built file matches BW's saved `M529LKIQ.G10` byte-for-byte (8708 bytes, 0 differences). Live SFM downloads of event 0 (3-sec) and event 1 (3-sec continuation) both open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots. Body assembly is just contiguous concatenation of frame contributions in stream order (probe → meta@0x1002 → meta@0x1004 → samples → TERM); no stripping, no overlay, no special handling. Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that may need different handling — untested under v0.14.x). Extension mapping: extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`
**Serial encoding (CONFIRMED 2026-04-22):** `prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units). **Serial encoding (CONFIRMED 2026-04-22):** `prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units).
@@ -1335,16 +1391,21 @@ body) because writing a dial string may require DLE escaping for embedded contro
| Folder / File | Contents | | Folder / File | Contents |
|---|---| |---|---|
| `1-2-26/` | First SUB 5A BW TX capture — established 5A frame format (raw offset_hi, DLE-aware checksum). 10 frames verified. |
| `3-11-26/raw_bw_20260311_170151.bin` | Full compliance write + event download (SUBs 68→83 confirmed, frames 102112) | | `3-11-26/raw_bw_20260311_170151.bin` | Full compliance write + event download (SUBs 68→83 confirmed, frames 102112) |
| `3-31-26/` | Single-event download (148 BW / 147 S3 frames) — 1E/0A/0C/1F sequence confirmed (single event so token=0xFE appeared to work in either branch) |
| `4-2-26/` | Download-mode BW TX capture — POLL×3 requirement confirmed (frames 68-73 between 1F and first 5A) |
| `4-3-26-multi_event/` | Browse-mode S3 capture with 2+ events — all-zero params for 1F, null sentinel layout, 0A context requirement |
| `4-8-26/` | Monitor status read, start/stop monitoring, SESSION_RESET signal, sensor check |
| `4-11-26 (mitm/ach_mitm_20260411_001912/)` | Full ACH call-home MITM — erase protocol (0xA3/0x06/0xA2), monitor log partial records confirmed |
| `4-20-26/raw_bw_*_recording_mode_*.bin` | Recording mode changes: Continuous→Single Shot, →Histogram, →Histogram+Continuous | | `4-20-26/raw_bw_*_recording_mode_*.bin` | Recording mode changes: Continuous→Single Shot, →Histogram, →Histogram+Continuous |
| `4-20-26/histogram interval/` | Histogram interval changes: 1min, 5min, 15min, 15sec | | `4-20-26/histogram interval/` | Histogram interval changes: 1min, 5min, 15min, 15sec |
| `4-20-26/geo sensitivity/` | Geo sensitivity changes: 1.25 in/s (Sensitive), 10 in/s (Normal) | | `4-20-26/geo sensitivity/` | Geo sensitivity changes: 1.25 in/s (Sensitive), 10 in/s (Normal) |
| `4-20-26/call home settings/` | Call home config read/write captures | | `4-20-26/call home settings/` | Call home config read/write captures |
| `4-8-26/` | Monitor status read, start/stop monitoring, SESSION_RESET signal, sensor check | | `4-27-26/` | BW "open 2sec waveform" + "copy event to disk" + paired SFM "seismo_dl" — first proof of 5× SFM over-read. STRT end_key field located. |
| `4-3-26-multi_event/` | Browse-mode S3 capture with 2+ events (1E/0A/1F iteration confirmed) | | **`5-1-26/comcheck/`** | **Triplet of captures that nailed the v0.14.0 walk:** SFM 3-sec download (`seismo_dl_…`), BW comms-check + 3-sec download (`bwcap3sec/`), BW second-event download + "Download All" (`raw_*_170945` / `_171216`). Confirmed: TERM frame formula across 3 events, metadata pages 0x1002/0x1004 are global session metadata, event-1 vs event-N chunk pattern split, WAVEHDR off=0x46 vs 0x2C disambiguates real events from boundaries. |
| `4-2-26/` | Download-mode BW TX capture (5A bulk stream, POLL×3 requirement confirmed) | | **`5-1-26/comcheck/bwcap3sec/`** | **The byte-perfect reference for v0.14.3.** All 17 BW 5A request frames (probe, 2 metadata, 13 samples, TERM) reproduce byte-for-byte from SFM's framing helpers — including the `10 10 00` DLE-stuffed counter for sample @ 0x1000 that was the long-standing failure mode. |
| `3-31-26/` | Single-event download (148 BW / 147 S3 frames) | | `5-4-26/` | BW MITM captures of "copy 3sec / 2sec / Download All" + paired SFM session (`seismo_dl_20260504_145701`) showing the +0x46 event-N probe bug producing 110-chunk runaway walk. Cross-references against 5-1-26 confirmed device behavior is identical. |
| `mitm/ach_mitm_20260411_001912/` | Full ACH call-home MITM (erase protocol, 0xA3/0x06/0xA2 confirmed) |
To parse BW TX captures: use `bridges/captures/` scripts or adapt the `find_write_frames()` pattern To parse BW TX captures: use `bridges/captures/` scripts or adapt the `find_write_frames()` pattern
in `/tmp/analyze_write_payload.py` — it correctly handles `0x10 0x03` DLE-escaped ETX bytes in `/tmp/analyze_write_payload.py` — it correctly handles `0x10 0x03` DLE-escaped ETX bytes
+26 -14
View File
@@ -1,4 +1,4 @@
# seismo-relay `v0.12.6` # seismo-relay `v0.14.3`
A ground-up replacement for **Blastware** — Instantel's aging Windows-only A ground-up replacement for **Blastware** — Instantel's aging Windows-only
software for managing MiniMate Plus seismographs. software for managing MiniMate Plus seismographs.
@@ -10,6 +10,10 @@ over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).
> pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server > pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server
> handles inbound unit connections, downloads events, and persists everything > handles inbound unit connections, downloads events, and persists everything
> to a SQLite database. SFM REST API exposes device control and DB queries. > to a SQLite database. SFM REST API exposes device control and DB queries.
> **As of v0.14.3 (2026-05-05): SUB 5A bulk waveform protocol is verified
> byte-perfect against Blastware captures across 2-sec, 3-sec, and 10-sec
> events.** Generated `.G10` / `.AB0` files open cleanly in Blastware with
> full Event Reports, frequency analysis, and waveform plots.
> See [CHANGELOG.md](CHANGELOG.md) for full version history. > See [CHANGELOG.md](CHANGELOG.md) for full version history.
--- ---
@@ -194,9 +198,14 @@ with client:
client.delete_all_events() # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2) client.delete_all_events() # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2)
``` ```
`get_events()` runs the full per-event sequence: `1E → 0A → 0C → 5A → 1F`. `get_events()` runs the full per-event sequence:
SUB 5A bulk stream provides `client`, `operator`, and `sensor_location` as they `1E → 0A → 1E(arm token=0xFE) → 0C → 1F(arm) → POLL×3 → 5A → 1F(browse)`.
existed at record time — not backfilled from the current compliance config. SUB 5A bulk stream walks chunks bounded by the `end_offset` extracted from
the STRT record at byte 17 of the probe response — no over-reading, no
chunk-count cap. Project / client / operator / sensor location strings come
from the dedicated metadata pages at counter `0x1002` and `0x1004`,
read once per session (they reflect the compliance setup at session start,
not per individual event).
--- ---
@@ -253,7 +262,7 @@ Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/insta
--- ---
## Compliance Config Features (v0.12.2v0.12.3) ## Compliance Config Features
The REST API and web UI expose full control over device compliance settings: The REST API and web UI expose full control over device compliance settings:
@@ -295,34 +304,36 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
--- ---
## Key Features (v0.10v0.12) ## Key Features
**Device support (v0.12.5):** **Device support:**
- [x] Full read/write/erase pipelines - [x] Full read/write/erase pipelines
- [x] Compliance config (recording mode, sample rate, histogram interval, geo sensitivity, project strings) - [x] Compliance config (recording mode, sample rate, histogram interval, geo sensitivity, project strings)
- [x] Auto Call Home config (read/write ACH settings, dial string, time slots, retries) - [x] Auto Call Home config (read/write ACH settings, dial string, time slots, retries)
- [x] Monitor control (start/stop, status polling, battery/memory) - [x] Monitor control (start/stop, status polling, battery/memory)
- [x] Monitor log entries (continuous monitoring intervals without full waveform download) - [x] Monitor log entries (continuous monitoring intervals without full waveform download)
**Data persistence (v0.11):** **Data persistence:**
- [x] SQLite database (`seismo_relay.db`) with 4 tables: ach_sessions, events, monitor_log, plus false_trigger flag - [x] SQLite database (`seismo_relay.db`) with 4 tables: ach_sessions, events, monitor_log, plus false_trigger flag
- [x] Deduplication by waveform key (handles re-runs and repeat call-homes) - [x] Deduplication by waveform key (handles re-runs and repeat call-homes)
- [x] Post-erase key-reuse detection (tracks high-water mark) - [x] Post-erase key-reuse detection (tracks high-water mark)
- [x] Session state (`ach_state.json`) with downloaded keys and max key - [x] Session state (`ach_state.json`) with downloaded keys and max key
**REST API (v0.12.1):** **REST API:**
- [x] Live device endpoints with in-memory caching (`_LiveCache`) - [x] Live device endpoints with in-memory caching (`_LiveCache`)
- [x] Cache statistics (`/cache/stats`) and manual invalidation (`/cache/device`) - [x] Cache statistics (`/cache/stats`) and manual invalidation (`/cache/device`)
- [x] DB query endpoints (units, events, monitor_log, sessions, false_trigger PATCH) - [x] DB query endpoints (units, events, monitor_log, sessions, false_trigger PATCH)
- [x] Call Home config read/write endpoints - [x] Call Home config read/write endpoints
- [x] Blastware file download endpoint (`/device/event/{index}/blastware_file`) - [x] Blastware file download endpoint (`/device/event/{index}/blastware_file`)
**File output (v0.7+):** **File output (v0.7+, byte-perfect as of v0.14.3):**
- [x] Blastware-compatible `.AB0` file generation (waveform + metadata) - [x] Blastware-compatible `.AB0` / `.G10` file generation (waveform + metadata)
- [x] Multi-channel waveform decode from SUB 5A bulk stream - [x] Multi-channel waveform decode from SUB 5A bulk stream
- [x] Second-resolution timestamp encoding in Blastware filename - [x] Second-resolution timestamp encoding in Blastware filename
- [x] **Byte-perfect against BW reference captures** (verified across 2-sec / 3-sec / 10-sec event durations, both event 0 and event N continuation events)
- [x] STRT-bounded chunk walk + correct event-N probe counter + partial DLE stuffing of `0x10` in 5A params (the four fixes that landed in v0.14.0v0.14.3)
**Capture tools (v0.12.5):** **Capture tools:**
- [x] Serial-to-TCP bridge with raw BW/S3 capture (s3_bridge.py, defaults to auto-capture) - [x] Serial-to-TCP bridge with raw BW/S3 capture (s3_bridge.py, defaults to auto-capture)
- [x] GUI bridge with raw capture checkboxes (gui_bridge.py) - [x] GUI bridge with raw capture checkboxes (gui_bridge.py)
- [x] ACH inbound server with bidirectional capture (ach_server.py saves raw_tx + raw_rx) - [x] ACH inbound server with bidirectional capture (ach_server.py saves raw_tx + raw_rx)
@@ -333,14 +344,15 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
- [x] gui_analyzer.py — standalone analyzer GUI - [x] gui_analyzer.py — standalone analyzer GUI
- [x] frame_db.py — SQLite frame database for capture analysis - [x] frame_db.py — SQLite frame database for capture analysis
**seismo_lab.py GUI (v0.12.5):** **seismo_lab.py GUI:**
- [x] Bridge tab — Serial/TCP mode selector with raw capture options - [x] Bridge tab — Serial/TCP mode selector with raw capture options
- [x] Analyzer tab — BW/S3 capture playback and differencing - [x] Analyzer tab — BW/S3 capture playback and differencing
- [x] Download tab — Live wire-byte capture during event download (new v0.12.5) - [x] Download tab — Live wire-byte capture during event download
- [x] Console tab — Logging and diagnostics - [x] Console tab — Logging and diagnostics
## Roadmap (Future) ## Roadmap (Future)
- [ ] Verify 30-sec event download — body may exceed `0xFFFF` and force the device into a different `end_key` encoding (none of 2/3/10-sec test cases hit this boundary)
- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing - [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first) - [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object - [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
+121 -33
View File
@@ -111,6 +111,9 @@
| 2026-04-20 | §7.6.2, §7.9, Appendix B | **CONFIRMED — Geophone maximum range / sensitivity selector byte location.** Two targeted captures (4-20-26, geo sensitivity folder): one at Normal 10.000 in/s, one at Sensitive 1.250 in/s. E5 read payload diff: exactly 3 bytes differ at channel_label+33 for Tran/Vert/Long. Values: `0x00`=Normal 10.000 in/s, `0x01`=Sensitive 1.250 in/s. Same offset applies to the SUB 71 write payload (which is the same 2126-byte E5-format buffer round-tripped verbatim). **`channel_label+20` reads `0x01` in ALL captures regardless of range setting — it is NOT this field.** Previous hypothesis (uint8 at Tran+20, 0x01=Normal) was WRONG. Stored as `geo_range` in `ComplianceConfig`. Encoded to all three geo channel blocks (Tran/Vert/Long) at label+33. | | 2026-04-20 | §7.6.2, §7.9, Appendix B | **CONFIRMED — Geophone maximum range / sensitivity selector byte location.** Two targeted captures (4-20-26, geo sensitivity folder): one at Normal 10.000 in/s, one at Sensitive 1.250 in/s. E5 read payload diff: exactly 3 bytes differ at channel_label+33 for Tran/Vert/Long. Values: `0x00`=Normal 10.000 in/s, `0x01`=Sensitive 1.250 in/s. Same offset applies to the SUB 71 write payload (which is the same 2126-byte E5-format buffer round-tripped verbatim). **`channel_label+20` reads `0x01` in ALL captures regardless of range setting — it is NOT this field.** Previous hypothesis (uint8 at Tran+20, 0x01=Normal) was WRONG. Stored as `geo_range` in `ComplianceConfig`. Encoded to all three geo channel blocks (Tran/Vert/Long) at label+33. |
| 2026-04-20 | §5.1, §5.3, §7.12 (NEW) | **NEW — Auto Call Home config protocol confirmed from 4-20-26 call home settings captures.** SUB 0x2C (Call Home Config READ, response 0xD3, data offset 0x7C=124) and SUB 0x7E/0x7F (WRITE + CONFIRM, response 0x81/0x80) confirmed. Write payload = read payload (125 bytes) + `\x00\x00` (127 bytes total). **DLE-escaped ETX at raw[117:119]:** the device returns logical value 0x03 (num_retries=3) as `\x10\x03` on the wire — S3FrameParser preserves both bytes as two literals, causing a +1 byte shift for all subsequent fields. Write frame sends these bytes verbatim (device interprets `\x10\x03` as literal value 3). Field map confirmed from 10-frame BW TX diff. See §7.12 for full layout. | | 2026-04-20 | §5.1, §5.3, §7.12 (NEW) | **NEW — Auto Call Home config protocol confirmed from 4-20-26 call home settings captures.** SUB 0x2C (Call Home Config READ, response 0xD3, data offset 0x7C=124) and SUB 0x7E/0x7F (WRITE + CONFIRM, response 0x81/0x80) confirmed. Write payload = read payload (125 bytes) + `\x00\x00` (127 bytes total). **DLE-escaped ETX at raw[117:119]:** the device returns logical value 0x03 (num_retries=3) as `\x10\x03` on the wire — S3FrameParser preserves both bytes as two literals, causing a +1 byte shift for all subsequent fields. Write frame sends these bytes verbatim (device interprets `\x10\x03` as literal value 3). Field map confirmed from 10-frame BW TX diff. See §7.12 for full layout. |
| 2026-05-01 | §7.8.2, §7.8.5 (NEW), §7.8.6 (NEW), §7.8.7 (NEW) | **REWRITTEN — SUB 5A bulk waveform stream protocol.** Five BW MITM captures (4-27-26 "open 2sec waveform" + "copy event to disk", 5-1-26 BW 3-sec + 2nd-event + Download All) prove that the previous chunk-counter formula `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` over-reads 5× past the actual event end. BW reads ~12-16 chunks per event at **0x0200 increments (NOT 0x0400)**, bounded by `end_offset` extracted from the STRT record at `data[23:27]` of the first A5 response. **TERM frame formula corrected:** `offset_word = end_offset - next_boundary`, `params[2:4] = next_boundary BE` where `next_boundary = last_chunk_counter + 0x0200`. Verified across 3 events (offsets 0x1ABE, 0x21F2, 0x417E). **Metadata pages 0x1002 / 0x1004** are global, fixed-address device pages containing Project/Client/User Name/Seis Loc/Extended Notes — read ONCE per Blastware session (not per event). **Event-1 vs event-N split:** events at start_key[2:4]=0 use probe@0x0000 + metadata pages + sample chunks at 0x0600 onward; continuation events skip metadata and start at start_key+0x0046. **WAVEHDR length 0x46 vs 0x2C disambiguates real events from boundary markers** — the "Download All" pattern walks 1E/0A/1F to map all event keys+lengths upfront, then downloads each `0x46`-keyed event in turn. Old `stop_after_metadata=True` knob is a workaround for the missing end_offset bound and becomes obsolete under the new walk. See new §7.8.5 / §7.8.6 / §7.8.7 for full details. | | 2026-05-01 | §7.8.2, §7.8.5 (NEW), §7.8.6 (NEW), §7.8.7 (NEW) | **REWRITTEN — SUB 5A bulk waveform stream protocol.** Five BW MITM captures (4-27-26 "open 2sec waveform" + "copy event to disk", 5-1-26 BW 3-sec + 2nd-event + Download All) prove that the previous chunk-counter formula `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` over-reads 5× past the actual event end. BW reads ~12-16 chunks per event at **0x0200 increments (NOT 0x0400)**, bounded by `end_offset` extracted from the STRT record at `data[23:27]` of the first A5 response. **TERM frame formula corrected:** `offset_word = end_offset - next_boundary`, `params[2:4] = next_boundary BE` where `next_boundary = last_chunk_counter + 0x0200`. Verified across 3 events (offsets 0x1ABE, 0x21F2, 0x417E). **Metadata pages 0x1002 / 0x1004** are global, fixed-address device pages containing Project/Client/User Name/Seis Loc/Extended Notes — read ONCE per Blastware session (not per event). **Event-1 vs event-N split:** events at start_key[2:4]=0 use probe@0x0000 + metadata pages + sample chunks at 0x0600 onward; continuation events skip metadata and start at start_key+0x0046. **WAVEHDR length 0x46 vs 0x2C disambiguates real events from boundary markers** — the "Download All" pattern walks 1E/0A/1F to map all event keys+lengths upfront, then downloads each `0x46`-keyed event in turn. Old `stop_after_metadata=True` knob is a workaround for the missing end_offset bound and becomes obsolete under the new walk. See new §7.8.5 / §7.8.6 / §7.8.7 for full details. |
| 2026-05-04 | §7.8.5, §7.8.8 | **CORRECTED — Event-N probe counter is just `start_offset`, NOT `start_offset + 0x0046`.** The `+0x46` formula in the original §7.8.5 was based on calling the off=0x2C boundary key the "start_key", but in the iteration walk `cur_key` passed into `read_bulk_waveform_stream` is always the off=0x46 WAVEHDR record key from 1F (the partial-record skip path in `get_events` re-runs 1F to advance past 0x2C boundary records). Adding +0x46 placed the probe one WAVEHDR past the actual event start; the response no longer contained STRT at byte 17, `parse_strt_end_offset` returned None, and the chunk loop fell back to the `max_chunks=128` cap, walking ~110 chunks of post-event circular-buffer garbage. Confirmed against both the 5-1-26 "copy 2nd address" capture (probe at counter=0x2238 with key=01112238) and the 5-4-26 BW 2-sec event capture. Fixed in protocol.py `read_bulk_waveform_stream` v0.14.1. |
| 2026-05-05 | §7.8.1 (rule #3 added) | **CONFIRMED — Partial DLE stuffing of `0x10` bytes in 5A params region.** The device's de-stuffing rule for the SUB 5A params region is: `10 10``10`, `10 02/03/04` → kept literal (inner-frame markers), `10 X` for any other X → de-stuffs to just `X` (drops the `0x10`). Therefore any `0x10` byte in the logical params followed by a byte NOT in {0x02, 0x03, 0x04, 0x10} MUST be doubled on the wire. This affects counters with `0x10` in the high byte — most importantly counter=`0x1000`, where logical params bytes `... 10 00 ...` were being sent raw and the device de-stuffed `10 00` to just `00`, returning the response for counter=0x0000 (= the file header + STRT). That STRT block then ended up embedded in the assembled file body at file offset `0x1016` and Blastware refused to open the file. This was the root cause of the long-standing ">1-sec event 0 won't open in BW" pattern (1-sec events worked because their `end_offset < 0x1000`, so no chunk request ever needed counter `0x10__`). All 17 5A request frames in the 5-1-26 bwcap3sec capture (probe + 2 meta + 13 samples + TERM) now match BW byte-for-byte after the fix. Fixed in framing.py `build_5a_frame` v0.14.3. |
| 2026-05-05 | §7.8 / Blastware file format | **CONFIRMED — File body assembly is contiguous concatenation, no de-duplication.** The "duplicate header+STRT strip" hack from v0.13.x was actively destroying valid waveform data — sample chunks at counter `0x1000` and beyond often coincidentally contain the byte sequence `00 12 03 00 STRT` in their delta-encoded ADC stream, and the strip was zeroing 25 bytes per match. Removed in v0.14.2. The correct file body is: probe contribution + meta@0x1002 + meta@0x1004 + sample contributions in stream order + TERM contribution. Verified byte-perfect against BW reference `M529LKIQ.G10` (8708 bytes, 0 differences) when fed the same A5 frames as the BW capture. |
--- ---
@@ -261,7 +264,7 @@ Step 4 — Device sends actual data payload:
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 | | `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 |
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 | | `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 |
| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 | | `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 |
| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum. Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. A5[7] contains event-time metadata (Project:/Client:/User Name:/Seis Loc:). 9+ A5 frames for full waveform; stop_after_metadata=True exits after A5[7]. | ✅ CONFIRMED 2026-04-06 | | `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum, **partial DLE stuffing of 0x10 in params** (`10 X` where X∉{02,03,04,10} must be doubled to `10 10 X` — see §7.8). Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. Walk: probe at counter=`start_offset` (event 1: 0x0000) → metadata pages 0x1002 + 0x1004 (event 1 only) → sample chunks at 0x0600, 0x0800, …, step 0x0200, bounded by `end_offset` parsed from STRT@data[17] of probe response → TERM frame at residual offset_word. Project:/Client:/User Name:/Seis Loc: live in the metadata pages, NOT in the sample-chunk stream. | ✅ CONFIRMED 2026-05-05 (BYTE-PERFECT vs BW capture) |
| `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED | | `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED |
| `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED | | `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED |
| `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED | | `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED |
@@ -837,6 +840,20 @@ MicL: 39 64 1D AA = 0.0000875 psi
### 7.6 Bulk Waveform Stream (SUB A5) — Raw ADC Sample Records ### 7.6 Bulk Waveform Stream (SUB A5) — Raw ADC Sample Records
> ⛔ **§7.6 below describes the deprecated `0x0400`-step walk and is RETAINED FOR HISTORICAL CONTEXT ONLY.**
> The "A5[7] is metadata", "A5[9] is terminator", and chunk-counter frame-index claims in this section
> are all artifacts of the broken walk that was overrunning past event end by ~5×.
>
> **For the corrected protocol (v0.14.0+), use:**
> - **§7.8.5** — chunk addressing (probe at `start_offset`, samples step 0x0200, bounded by STRT `end_offset`)
> - **§7.8.6** — TERM frame formula
> - **§7.8.7** — fixed metadata pages 0x1002 / 0x1004 (this is where Project / Client / User Name / Seis Loc
> strings actually live — NOT in any sample-chunk frame)
> - **§7.8.8** — multi-event "Download All" sequence
>
> The waveform sample encoding (4-channel interleaved s16 LE, 8 bytes per sample-set) described in §7.6.1
> below is still correct. Only the frame-indexing claims and metadata-source claims are wrong.
**Two distinct formats exist depending on recording mode. Both confirmed from captures.** **Two distinct formats exist depending on recording mode. Both confirmed from captures.**
--- ---
@@ -1119,20 +1136,26 @@ Near-ambient: 0x3C75C28F = 0.015 in/s (histogram event, near-zero ambient)
**Project strings** — ASCII label-value pairs (search for label, read null-terminated value): **Project strings** — ASCII label-value pairs (search for label, read null-terminated value):
``` ```
"Project:" → project description (in 0C record ✅) "Project:" → project description (in 0C record ✅, also mirrored in metadata pages)
"Client:" → client name (in SUB 5A / A5 frame 7 ✅ — NOT in 0C) "Client:" → client name (in SUB 5A metadata pages ✅ — NOT in 0C)
"User Name:" → operator / user (in SUB 5A / A5 frame 7 ✅ — NOT in 0C) "User Name:" → operator / user (in SUB 5A metadata pages ✅ — NOT in 0C)
"Seis Loc:" → sensor location (in SUB 5A / A5 frame 7 ✅ — NOT in 0C) "Seis Loc:" → sensor location (in SUB 5A metadata pages ✅ — NOT in 0C)
"Extended Notes"→ notes field (in SUB 5A / A5 frame 7 ✅) "Extended Notes"→ notes field (in SUB 5A metadata pages ✅)
``` ```
> ✅ **2026-04-02 — CONFIRMED:** `Client:`, `User Name:`, and `Seis Loc:` are sourced from > ✅ **UPDATED 2026-05-05:** `Client:`, `User Name:`, and `Seis Loc:` come from the
> **SUB 5A (bulk waveform stream)**, specifically A5 frame 7 of the multi-frame response. > dedicated **SUB 5A metadata pages at counter `0x1002` and `0x1004`** — see §7.8.7.
> They are NOT present in the 210-byte SUB 0C waveform record. The strings reflect the > They are NOT present in the 210-byte SUB 0C waveform record.
> compliance setup that was active when the event was recorded on the device — making SUB 5A >
> the authoritative source for true event-time metadata. The `get_events()` client method > An earlier draft of this doc claimed they came from "A5 frame 7" of the bulk waveform
> now issues a SUB 5A request after each 0C download (`stop_after_metadata=True`) and > stream — that was an artifact of the deprecated `0x0400`-step walk where the broken
> overwrites `event.project_info` with the decoded fields. > chunk counter formula happened to land sample-chunk fi=7 on top of the 0x1002 metadata
> page. Under the corrected v0.14.0+ walk (§7.8.5), sample chunks at `0x1000` / `0x1200`
> contain ordinary waveform data, and the metadata pages are read separately.
>
> The strings reflect the compliance setup that was active when the *monitoring session*
> first started (not per-event). `get_events()` reads the metadata pages once at the start
> of the SFM session and the decoded values are stamped onto every event in that session.
--- ---
@@ -1166,7 +1189,9 @@ return events
### 7.7.7 Updated Download Loop with SUB 5A Metadata ### 7.7.7 Updated Download Loop with SUB 5A Metadata
> **Added 2026-04-02.** Confirmed working on BE11529 over TCP/cellular. > ⛔ **The loop in this subsection is DEPRECATED — it uses the broken `stop_after_metadata=True`
> hack and the wrong sequence ordering.** See §7.8.5–§7.8.8 for the corrected protocol.
> The pseudocode below is preserved as historical record only.
```python ```python
key4, _ = proto.read_event_first() # SUB 1E key4, _ = proto.read_event_first() # SUB 1E
@@ -1201,13 +1226,25 @@ return events
### 7.8 SUB 5A — Bulk Waveform Stream (event-time metadata) ### 7.8 SUB 5A — Bulk Waveform Stream (event-time metadata)
> ✅ **Added 2026-04-02.** Frame format confirmed by reproducing Blastware wire bytes > ✅ **§7.8.1 (frame format) — added 2026-04-02; v0.14.3 partial DLE stuffing finalized 2026-05-05.**
> byte-for-byte from the 1-2-26 BW capture. > Frame format confirmed by reproducing Blastware wire bytes byte-for-byte across the 1-2-26
> capture (10 frames) and the 5-1-26 bwcap3sec capture (17 frames, all match including the
> DLE-stuffed `10 10 00` for counter=0x1000).
SUB 5A initiates a bulk transfer of the raw sample data for a stored event. The response is a SUB 5A initiates a bulk transfer of the raw sample data for a stored event. The response is
sequence of A5 frames. Frame 7 (0-indexed) contains the full compliance setup as it existed a sequence of A5 frames. Project-info ASCII strings (`Project:`, `Client:`, `User Name:`,
when the event was recorded — including `Client:`, `User Name:`, `Seis Loc:`, and `Seis Loc:`, `Extended Notes`) live in the dedicated metadata pages at counter `0x1002`
`Extended Notes` ASCII label-value pairs. and `0x1004` (see §7.8.7), not in the sample-chunk stream.
**For the corrected protocol read in order:**
- §7.8.1 — frame format (raw `offset_hi`, DLE-aware checksum, partial DLE stuffing of params)
- §7.8.5 — chunk addressing (probe → metadata pages → samples → TERM, all bounded by `end_offset`)
- §7.8.6 — TERM frame formula
- §7.8.7 — fixed metadata pages 0x1002 / 0x1004
- §7.8.8 — multi-event "Download All" sequence
§7.8.2–§7.8.4 are retained as historical record of earlier (incorrect) understandings —
do not implement against them.
#### 7.8.1 Frame Format #### 7.8.1 Frame Format
@@ -1218,7 +1255,7 @@ SUB 5A uses a **non-standard frame layout** that differs from all other BW→S3
41 02 10 10 00 5A 00 ^^raw^^ ^^raw^^ ^^stuffed^^ 41 02 10 10 00 5A 00 ^^raw^^ ^^raw^^ ^^stuffed^^
``` ```
Two critical differences from `build_bw_frame`: Three critical differences from `build_bw_frame`:
1. **`offset_hi` is sent raw, not DLE-stuffed.** When `offset_hi = 0x10`, the wire carries 1. **`offset_hi` is sent raw, not DLE-stuffed.** When `offset_hi = 0x10`, the wire carries
a bare `0x10` — NOT the stuffed `10 10` that `build_bw_frame` would produce. The device a bare `0x10` — NOT the stuffed `10 10` that `build_bw_frame` would produce. The device
@@ -1227,6 +1264,31 @@ Two critical differences from `build_bw_frame`:
2. **DLE-aware checksum.** Walking the full frame byte sequence: when a `10 XX` pair is seen, 2. **DLE-aware checksum.** Walking the full frame byte sequence: when a `10 XX` pair is seen,
only `XX` is added to the running sum; lone bytes are added normally. only `XX` is added to the running sum; lone bytes are added normally.
3. **Partial DLE stuffing of `0x10` bytes in the params region** (CONFIRMED 2026-05-05).
The device's de-stuffing rule for the params region is:
- `10 10` → de-stuffs to `10`
- `10 02 / 03 / 04` → kept literal (these are inner-frame markers)
- `10 X` for other X → de-stuffs to just `X` (drops the leading `0x10`)
Therefore any `0x10` byte in the *logical* params that is followed by a byte NOT in
`{0x02, 0x03, 0x04, 0x10}` MUST be doubled on the wire (`10 X``10 10 X`) so the
device's de-stuffer reproduces the original `10 X` pair. This applies most commonly
to counters with `0x10` in the high byte (e.g. counter=`0x1000` produces logical
params bytes `... 10 00 ...`, which BW encodes on the wire as `... 10 10 00 ...`).
Without this stuffing the device interprets counter=`0x1000` as `0x0000` and returns
the probe response (= a copy of the file header + STRT record); that STRT block then
ends up embedded in the assembled file body and Blastware refuses to open the file.
`0x10` bytes in `offset_hi` are still written RAW per (1) above — only the params
region has this stuffing requirement. Metadata-page params for counter `0x1002` /
`0x1004` survive without stuffing because `10 02` / `10 04` fall in the "kept literal"
carve-out.
Verified against BW 5-1-26 bwcap3sec frame 20: params logical bytes
`00 01 11 10 00 00 00 00 00 00 00` (counter=0x1000) are encoded on the wire as
`00 01 11 10 10 00 00 00 00 00 00 00` (12 wire bytes for 11 logical bytes).
#### 7.8.2 Request Sequence — DEPRECATED 2026-05-01 (see §7.8.5–§7.8.7 for the corrected protocol) #### 7.8.2 Request Sequence — DEPRECATED 2026-05-01 (see §7.8.5–§7.8.7 for the corrected protocol)
> ⛔ **The 0x0400-step / max(key4[2:4], 0x0400) formula in this section is WRONG.** Five new > ⛔ **The 0x0400-step / max(key4[2:4], 0x0400) formula in this section is WRONG.** Five new
@@ -1267,11 +1329,20 @@ when the broken 0x0400-step walk passed the global metadata pages at 0x1002/0x10
the corrected walk, those strings come from explicit reads at counter=0x1002 and 0x1004, the corrected walk, those strings come from explicit reads at counter=0x1002 and 0x1004,
not from the sample-chunk stream — see §7.8.7. not from the sample-chunk stream — see §7.8.7.
#### 7.8.3 A5 Frame Layout #### 7.8.3 A5 Frame Layout — DEPRECATED 2026-05-01
Each A5 response frame contains a chunk of raw bulk data. Frame 7 of the stream carries the > ⛔ **The "Frame 7 carries the compliance text block" claim below is WRONG.** It was
compliance text block with all project-info label-value pairs. The `client` layer searches > an artifact of the deprecated `0x0400`-step walk where the broken counter formula
for ASCII labels with a null-terminated value read: > happened to land sample-chunk fi=7 on top of the 0x1002 metadata page in flash.
> Under the corrected v0.14.0+ walk (§7.8.5), Frame 7 of the sample-chunk sequence is
> just sample-chunk #5 (counter=0x1000), and contains either ordinary waveform data or —
> critically when DLE-stuffing of params is wrong (§7.8.1.3) — a duplicate file header +
> STRT block when the device misinterprets counter=0x1000 as 0x0000. See §7.8.7 for the
> actual source of these strings.
Historical claim (NOT TO BE IMPLEMENTED): each A5 response frame contains a chunk of raw
bulk data; Frame 7 of the stream carries the compliance text block with all project-info
label-value pairs:
``` ```
"Project:" → null-terminated project name "Project:" → null-terminated project name
@@ -1281,7 +1352,9 @@ for ASCII labels with a null-terminated value read:
"Extended Notes" → null-terminated notes "Extended Notes" → null-terminated notes
``` ```
All five fields reflect the **setup at event-record time**, not the current device config. All five fields do reflect the **setup at event-record time**, not the current device
config. But the source is the metadata pages (§7.8.7), not "Frame 7" of the sample
stream.
#### 7.8.4 End-of-Stream Behaviour and Chunk Timing — REINTERPRETED 2026-05-01 #### 7.8.4 End-of-Stream Behaviour and Chunk Timing — REINTERPRETED 2026-05-01
@@ -1383,12 +1456,26 @@ the first slot in a freshly-erased buffer. Harmless to skip; BW does the same.
**Event 2+ / start_key[2:4] != 0x0000** (continuation events in a populated buffer): **Event 2+ / start_key[2:4] != 0x0000** (continuation events in a populated buffer):
``` ```
1. First chunk at counter = start_key[2:4] + 0x0046 ← acts as both probe and first 1. First chunk at counter = start_key[2:4] ← acts as both probe and first
sample chunk; response carries STRT sample chunk; response carries STRT at byte 17
2. Walk sample chunks counter += 0x0200 each 2. Walk sample chunks counter += 0x0200 each
3. TERM 3. TERM
``` ```
**`start_key` here is the off=0x46 WAVEHDR record key returned by 1F** (e.g. `01112238`),
NOT the off=0x2C boundary key that immediately precedes it. An earlier draft of this
spec described event-N as "probe at start + 0x46" — that formula was correct only if
"start" meant the boundary key (0x21F2 in the 5-1-26 event 2 case). In the iteration
walk used by SFM and BW, `cur_key` passed into the 5A flow is always the off=0x46 key,
so the probe counter equals `cur_key[2:4]` with no extra offset. Adding +0x46 places
the probe one WAVEHDR past the actual event start, the response no longer contains
STRT at byte 17, and the chunk loop falls back to the `max_chunks` cap.
Confirmed:
- 5-1-26 "copy 2nd address" BW capture: probe counter=0x2238 with key=01112238; A5[0]
has STRT@17 with end_offset=0x417E.
- 5-4-26 BW 2-sec event capture: same probe counter=0x2238, same end_offset=0x417E.
**No metadata-page reads.** Pages 0x1002/0x1004 are session-global and were already read **No metadata-page reads.** Pages 0x1002/0x1004 are session-global and were already read
during event 1 in the same Blastware session. In SFM, treat metadata pages as a once- during event 1 in the same Blastware session. In SFM, treat metadata pages as a once-
per-`MiniMateClient.connect()` (or once-per-call-home) read, not per-event. per-`MiniMateClient.connect()` (or once-per-call-home) read, not per-event.
@@ -1399,7 +1486,8 @@ per-`MiniMateClient.connect()` (or once-per-call-home) read, not per-event.
|---|---|---|---|---|---| |---|---|---|---|---|---|
| 4-27-26 "open 2sec" / "copy event to disk" | `01110000` | `01111ABE` | `0x1ABE` | 6,846 B | 0x0600 (event-1 case) | | 4-27-26 "open 2sec" / "copy event to disk" | `01110000` | `01111ABE` | `0x1ABE` | 6,846 B | 0x0600 (event-1 case) |
| 5-1-26 "copy 3sec" / Download All event 1 | `01110000` | `011121F2` | `0x21F2` | 8,690 B | 0x0600 (event-1 case) | | 5-1-26 "copy 3sec" / Download All event 1 | `01110000` | `011121F2` | `0x21F2` | 8,690 B | 0x0600 (event-1 case) |
| 5-1-26 "copy 2nd address" / DA event 2 | `011121F2` | `0111417E` | event 2 size = 0x1F8C = 8,076 B | 0x2238 (= 0x21F2 + 0x46) | | 5-1-26 "copy 2nd address" / DA event 2 | `01112238` (= 1F result) | `0111417E` | `0x417E`, span 0x1F8C = 8,076 B | 0x2238 (= cur_key[2:4]) |
| 5-4-26 BW 2-sec event | `01112238` | `0111417E` | `0x417E` | 0x2238 (= cur_key[2:4]) |
#### 7.8.6 TERM Frame Formula (NEW 2026-05-01) ✅ #### 7.8.6 TERM Frame Formula (NEW 2026-05-01) ✅
@@ -1545,10 +1633,10 @@ Fields visible in the Blastware "Compliance Setup" dialog. ✅ = byte offset co
| Field | Values / Type | Status | | Field | Values / Type | Status |
|---|---|---| |---|---|---|
| Enable User Notes | bool | ❓ | | Enable User Notes | bool | ❓ |
| Project | ASCII string | ✅ (sourced from A5 frame 7 via SUB 5A) | | Project | ASCII string | ✅ (sourced from SUB 5A metadata pages at counter `0x1002` / `0x1004` — see §7.8.7) |
| Client | ASCII string | ✅ (sourced from A5 frame 7) | | Client | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
| User Name | ASCII string | ✅ (sourced from A5 frame 7) | | User Name | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
| Seis Loc | ASCII string | ✅ (sourced from A5 frame 7) | | Seis Loc | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
| Enable Extended Notes | bool | ❓ | | Enable Extended Notes | bool | ❓ |
| Extended Notes | ASCII text | ❓ | | Extended Notes | ASCII text | ❓ |
| Extended Notes Title | ASCII string | ❓ | | Extended Notes Title | ASCII string | ❓ |
+50 -55
View File
@@ -672,11 +672,10 @@ def write_blastware_file(
# Do NOT use a5_frames[-1] — if _a5_frames contains stray frames from a # Do NOT use a5_frames[-1] — if _a5_frames contains stray frames from a
# subsequent event (a known get_events side-effect), the last frame will # subsequent event (a known get_events side-effect), the last frame will
# not be the terminator and the footer will be mis-identified. # not be the terminator and the footer will be mis-identified.
# TERM detection (v0.14.0): last frame if page_key != 0x0010 (sample marker)
term_idx: Optional[int] = None term_idx: Optional[int] = None
for _i, _f in enumerate(a5_frames): if a5_frames and a5_frames[-1].page_key != 0x0010:
if _f.page_key == 0x0000: term_idx = len(a5_frames) - 1
term_idx = _i
break
if term_idx is not None: if term_idx is not None:
body_frames = a5_frames[:term_idx] body_frames = a5_frames[:term_idx]
@@ -685,64 +684,28 @@ def write_blastware_file(
body_frames = a5_frames body_frames = a5_frames
term_frame = None term_frame = None
# ── Identify first metadata frame and skip "extra chunks" ─────────────── # Frame contribution loop (v0.14.0 BW-exact walk).
# When extra_chunks_after_metadata=1 in read_bulk_waveform_stream(), the # Skip values:
# frame list is: [probe, data..., metadata, extra_chunk, terminator]. # probe (fi=0): probe_skip
# The extra_chunk is downloaded to prime the TCP terminator response — its # meta@0x1002 (fi=1): 13 (6-byte inner header)
# ADC data is NOT part of the Blastware file body. Skip it. # meta@0x1004 (fi=2): 13 (6-byte inner header)
# # sample chunks (fi=3+): 12 (5-byte inner header)
# Rule: any frame at index strictly between first_metadata_fi and last_fi
# (the final frame) is an extra chunk and must be excluded.
#
# If no metadata frame exists (e.g. full_waveform download), first_metadata_fi
# is None and no frames are skipped — all frames contribute normally.
first_metadata_fi: Optional[int] = None
for _fi_scan, _frame_scan in enumerate(body_frames):
if _fi_scan > 0 and any(m in bytes(_frame_scan.data) for m in _METADATA_FRAME_MARKERS):
first_metadata_fi = _fi_scan
break
last_fi = len(body_frames) - 1 last_fi = len(body_frames) - 1
log.warning( log.debug(
"write_blastware_file: %d body_frames first_metadata_fi=%s last_fi=%d", "write_blastware_file: %d body_frames last_fi=%d",
len(body_frames), len(body_frames), last_fi,
str(first_metadata_fi) if first_metadata_fi is not None else "None",
last_fi,
) )
all_bytes = bytearray() all_bytes = bytearray()
for fi, frame in enumerate(body_frames): for fi, frame in enumerate(body_frames):
# Skip "extra chunk" frames: frames after the first metadata frame but
# before the last frame (terminator). These prime the TCP terminator but
# their ADC data must NOT appear in the Blastware file body.
if (first_metadata_fi is not None
and fi > first_metadata_fi
and fi < last_fi):
log.warning(
"write_blastware_file: fi=%d SKIP (extra chunk after metadata fi=%d last_fi=%d)",
fi, first_metadata_fi, last_fi,
)
continue
if fi == 0: if fi == 0:
# Probe frame: always process regardless of classification.
# It holds the STRT record; probe_skip positions us past it.
skip = probe_skip skip = probe_skip
elif fi in (1, 2):
skip = 13 # metadata pages
else: else:
# ALL subsequent frames are included unconditionally — no filtering on skip = 12 # sample chunks
# frame type. In the A5 stream, frame 0 is always the probe response;
# frames 1+ are always data (waveform chunks, compliance config, or
# compliance continuation). Classification is for logging only.
#
# DO NOT gate on classify_frame() here:
# - "probe_or_strt" at fi>0 is always a false positive — ADC binary
# data can coincidentally contain b"STRT\xff\xfe" (confirmed from
# live capture: frames 1 and 5 matched on event key=01110000).
# - "metadata" frames must be included (compliance config body).
# - The compliance block spans 2 frames; skipping either produces a
# truncated file that Blastware rejects.
skip = 13 if fi == 1 else 12
contribution = _frame_body_bytes(frame, skip) contribution = _frame_body_bytes(frame, skip)
log.warning("write_blastware_file: fi=%d skip=%d raw_data=%d contribution=%d", log.warning("write_blastware_file: fi=%d skip=%d raw_data=%d contribution=%d",
@@ -769,11 +732,43 @@ def write_blastware_file(
bytes(all_bytes[-28:]).hex() if len(all_bytes) >= 28 else bytes(all_bytes).hex(), bytes(all_bytes[-28:]).hex() if len(all_bytes) >= 28 else bytes(all_bytes).hex(),
) )
if len(all_bytes) >= 26: # NOTE: The "duplicate header+STRT strip" logic from v0.13.x has been
# REMOVED in v0.14.2. Under the v0.14.0 BW-exact 5A walk, body assembly
# is just contiguous concatenation of frame contributions in stream order
# (probe → meta@0x1002 → meta@0x1004 → samples → TERM), exactly as BW
# writes its files. The previous strip was matching the `00 12 03 00 STRT`
# byte sequence in legitimate waveform data — sample chunks at counter
# 0x1000 and beyond often contain those bytes coincidentally — and
# zeroing 25 bytes of valid samples per match. Compared to a known-good
# BW reference for the same 3-sec event 0, the strip introduced 26 bytes
# of zeros that BW did not have, then propagated alignment differences
# through the rest of the body. See decode_test/5-1-26/bw vs SFM diff
# at file[0x1012..0x102B] (2026-05-04 analysis).
# Find the first valid 0e 08 footer marker (v0.14.0).
footer_pos = -1
pos = 0
while True:
pos = bytes(all_bytes).find(b"\x0e\x08", pos)
if pos < 0 or pos + 26 > len(all_bytes):
break
yr = (all_bytes[pos + 4] << 8) | all_bytes[pos + 5]
if 2015 <= yr <= 2050:
footer_pos = pos
break
pos += 1
if footer_pos >= 0:
body = bytes(all_bytes[:footer_pos])
footer = bytes(all_bytes[footer_pos:footer_pos + 26])
log.warning(
"write_blastware_file: real 0e 08 footer at all_bytes[%d]; "
"truncating %d post-footer bytes",
footer_pos, len(all_bytes) - footer_pos - 26,
)
elif len(all_bytes) >= 26:
body = bytes(all_bytes[:-26]) body = bytes(all_bytes[:-26])
footer = bytes(all_bytes[-26:]) footer = bytes(all_bytes[-26:])
else: else:
# Fallback: no terminator or very short stream → build footer from event metadata
body = bytes(all_bytes) body = bytes(all_bytes)
start_dt = _ts_from_model(event.timestamp) start_dt = _ts_from_model(event.timestamp)
stop_dt: Optional[datetime.datetime] = None stop_dt: Optional[datetime.datetime] = None
@@ -784,7 +779,7 @@ def write_blastware_file(
+ _encode_ts_be(start_dt) + _encode_ts_be(start_dt)
+ _encode_ts_be(stop_dt) + _encode_ts_be(stop_dt)
+ b"\x00\x01\x00\x02\x00\x00" + b"\x00\x01\x00\x02\x00\x00"
+ b"\x00\x00" # CRC placeholder + b"\x00\x00"
) )
# ── Write file ─────────────────────────────────────────────────────────── # ── Write file ───────────────────────────────────────────────────────────
+66 -22
View File
@@ -1345,6 +1345,11 @@ def _decode_waveform_record_into(data: bytes, event: Event) -> None:
event.timestamp = Timestamp.from_continuous_record(data) event.timestamp = Timestamp.from_continuous_record(data)
except Exception as exc: except Exception as exc:
log.warning("continuous record timestamp decode failed: %s", exc) log.warning("continuous record timestamp decode failed: %s", exc)
elif event.record_type == "Waveform (Short)":
try:
event.timestamp = Timestamp.from_short_record(data)
except Exception as exc:
log.warning("short record timestamp decode failed: %s", exc)
# ── Peak values (per-channel PPV + Peak Vector Sum) ─────────────────────── # ── Peak values (per-channel PPV + Peak Vector Sum) ───────────────────────
try: try:
@@ -1636,34 +1641,73 @@ def _decode_a5_waveform(
} }
def _detect_record_format(data: bytes) -> Optional[str]:
"""
Detect which timestamp-header format a 210-byte 0C waveform record uses.
THREE formats observed on BE11529 firmware S338.17:
"single_shot" 9-byte header:
[day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
sub_code=0x10 at byte [1]. Year at [3:5].
"continuous" 10-byte header:
[0x10] [day] [0x10] [month] [year_BE:2] [unknown] [hour] [min] [sec]
marker 0x10 at byte [0] AND byte [2]. Year at [4:6].
"short" 8-byte header (NEW 2026-05-01):
[day] [month] [year_BE:2] [unknown] [hour] [min] [sec]
No marker bytes. Year at [2:4].
Each format has the year (uint16 BE) at a UNIQUE byte position, so we can
disambiguate by scanning each candidate position and picking the one
where the year falls in a sane range (2015..2050).
Returns "single_shot" / "continuous" / "short" or None if no format matches.
"""
if len(data) < 8:
return None
def _sane_year(hi: int, lo: int) -> bool:
y = (hi << 8) | lo
return 2015 <= y <= 2050
# Order matters: prefer formats with stronger marker-byte evidence first.
if data[1] == 0x10 and len(data) >= 9 and _sane_year(data[3], data[4]):
return "single_shot"
if (data[0] == 0x10 and data[2] == 0x10
and len(data) >= 10 and _sane_year(data[4], data[5])):
return "continuous"
if _sane_year(data[2], data[3]):
return "short"
return None
def _extract_record_type(data: bytes) -> Optional[str]: def _extract_record_type(data: bytes) -> Optional[str]:
""" """
Decode the recording mode from byte[1] of the 210-byte waveform record. Return a human-readable name for the waveform record format detected
in the first bytes of a 210-byte 0C record.
Byte[1] is the sub-record code that immediately follows the day byte in the Maps to the format codes returned by _detect_record_format():
9-byte timestamp header at the start of each waveform record: "single_shot" "Waveform"
[day:1] [sub_code:1] [month:1] [year:2 BE] ... "continuous" "Waveform (Continuous)"
"short" "Waveform (Short)"
Confirmed codes ( 2026-04-01): None "Unknown(XX.YY.ZZ)"
0x10 "Waveform" (continuous / single-shot mode)
Histogram mode code is not yet confirmed a histogram event must be
captured with debug=true to identify it. Returns None for unknown codes.
""" """
if len(data) < 2: fmt = _detect_record_format(data)
return None if fmt == "single_shot":
code = data[1]
if code == 0x10:
return "Waveform" return "Waveform"
if code == 0x03: if fmt == "continuous":
# Continuous mode waveform record (confirmed by user — NOT a monitor log).
# The byte layout differs from 0x10 single-shot records: the timestamp
# fields decode as garbage under the 0x10 waveform layout.
# TODO: confirm correct timestamp layout for 0x03 records from a known-time event.
return "Waveform (Continuous)" return "Waveform (Continuous)"
log.warning("_extract_record_type: unknown sub_code=0x%02X", code) if fmt == "short":
return f"Unknown(0x{code:02X})" return "Waveform (Short)"
if len(data) >= 3:
log.warning(
"_extract_record_type: unrecognized header: data[0:3]=%02X %02X %02X",
data[0], data[1], data[2],
)
return f"Unknown({data[0]:02X}.{data[1]:02X}.{data[2]:02X})"
return None
def _extract_peak_floats(data: bytes) -> Optional[PeakValues]: def _extract_peak_floats(data: bytes) -> Optional[PeakValues]:
""" """
+163 -19
View File
@@ -123,8 +123,11 @@ def build_5a_frame(offset_word: int, raw_params: bytes) -> bytes:
Returns: Returns:
Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX] Complete frame bytes: [ACK][STX][stuffed_section][chk][ETX]
""" """
if len(raw_params) not in (10, 11): if len(raw_params) not in (10, 11, 12):
raise ValueError(f"raw_params must be 10 or 11 bytes, got {len(raw_params)}") # 10 = termination params; 11 = regular probe / chunk params;
# 12 = metadata-page params (extra trailing 0x00 — BW byte-perfect quirk
# for the two fixed metadata reads at counter=0x1002 and 0x1004).
raise ValueError(f"raw_params must be 10/11/12 bytes, got {len(raw_params)}")
# Build stuffed section between STX and checksum # Build stuffed section between STX and checksum
s = bytearray() s = bytearray()
@@ -134,8 +137,40 @@ def build_5a_frame(offset_word: int, raw_params: bytes) -> bytes:
s += b"\x00" # field3 s += b"\x00" # field3
s += bytes([(offset_word >> 8) & 0xFF, # offset_hi — raw, NOT stuffed s += bytes([(offset_word >> 8) & 0xFF, # offset_hi — raw, NOT stuffed
offset_word & 0xFF]) # offset_lo offset_word & 0xFF]) # offset_lo
for b in raw_params: # params — NOT DLE-stuffed (raw bytes, match BW wire format) # Params — partial DLE stuffing of 0x10 bytes (CONFIRMED 2026-05-05).
#
# The device's de-stuffing rule for params is:
# • `10 10` → de-stuffs to `10`
# • `10 02/03/04` → kept literal (these are inner-frame markers)
# • `10 X` other → de-stuffs to just `X` (drops the 0x10)
#
# So for any 0x10 byte in the *logical* params that is followed by a
# byte NOT in {0x02, 0x03, 0x04, 0x10}, we must double the 0x10 on the
# wire (`10 X` → `10 10 X`) so the device's de-stuffer reproduces the
# original `10 X` pair. Without this, counter values with `0x10` in
# the high byte (e.g. counter=0x1000 has params bytes `10 00`) are
# silently corrupted to `0x__00` on the device side, and the device
# responds for the wrong address — for counter=0x1000 it returns the
# probe response (counter=0x0000), which contains the file header +
# STRT. That STRT block then lands in the assembled file body and
# Blastware rejects the file as malformed.
#
# Confirmed against BW capture 5-1-26 / bwcap3sec frame 20: params
# logical bytes `00 01 11 10 00 00 00 00 00 00 00` (counter=0x1000)
# are encoded on the wire as `00 01 11 10 10 00 00 00 00 00 00 00`.
# BW frames 13/14 (meta @ 0x1002 / 0x1004) leave `10 02` and `10 04`
# raw — the device handles those literal pairs correctly.
i = 0
while i < len(raw_params):
b = raw_params[i]
s.append(b) s.append(b)
if (
b == 0x10
and i + 1 < len(raw_params)
and raw_params[i + 1] not in (0x02, 0x03, 0x04, 0x10)
):
s.append(0x10) # double the 0x10 so it survives device de-stuffing
i += 1
# DLE-aware checksum: for 0x10 XX pairs count XX; for lone bytes count them # DLE-aware checksum: for 0x10 XX pairs count XX; for lone bytes count them
chk, i = 0, 0 chk, i = 0, 0
@@ -398,28 +433,21 @@ def bulk_waveform_params(key4: bytes, counter: int, *, is_probe: bool = False) -
def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes: def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
""" """
Build the 10-byte params block for the SUB 5A termination request. DEPRECATED 2026-05-01 see bulk_waveform_term_v2().
The termination request uses offset=0x005A and a DIFFERENT params layout Build the 10-byte params block for the SUB 5A termination request, OLD layout
the leading 0x00 byte is dropped, key4[0:2] shifts to params[0:2], and the (used in conjunction with the fixed offset_word=0x005A). Kept for backward
counter high byte is at params[2]: compatibility produces a tiny ~100-byte device-side terminator response
rather than the proper partial-last-chunk + footer payload that BW gets.
params[0] = key4[0] params[0] = key4[0]
params[1] = key4[1] params[1] = key4[1]
params[2] = (counter >> 8) & 0xFF params[2] = (counter >> 8) & 0xFF
params[3:] = zeros params[3:] = zeros
Counter for the termination request = last_regular_counter + 0x0400. Use bulk_waveform_term_v2() for new code it computes the verified
offset_word + params from end_offset (extracted from STRT) and the last
Confirmed from 1-2-26 BW TX capture: final request (frame 83) uses chunk counter.
offset=0x005A, params[0:3] = key4[0:2] + term_counter_hi.
Args:
key4: 4-byte waveform key.
counter: Termination counter (= last regular counter + 0x0400).
Returns:
10-byte params block.
""" """
if len(key4) != 4: if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}") raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
@@ -430,6 +458,123 @@ def bulk_waveform_term_params(key4: bytes, counter: int) -> bytes:
return bytes(p) return bytes(p)
def bulk_waveform_term_v2(
key4: bytes,
end_offset: int,
last_chunk_counter: int,
) -> tuple[int, bytes]:
"""
Compute the SUB 5A TERM frame's offset_word and 10-byte params block.
Confirmed across 3 events (4-27-26 + 5-1-26 captures):
next_boundary = last_chunk_counter + 0x0200
offset_word = end_offset - next_boundary (residual byte count)
params[0] = key4[0] (= 0x01 on every observed device)
params[1] = key4[1] (= 0x11)
params[2] = (next_boundary >> 8) & 0xFF
params[3] = next_boundary & 0xFF
params[4:10] = zeros
Verification:
| end_offset | last_chunk | next_boundary | offset_word | params[2:4] |
| 0x1ABE | 0x1800 | 0x1A00 | 0x00BE | 1A 00 |
| 0x21F2 | 0x1E00 | 0x2000 | 0x01F2 | 20 00 |
| 0x417E | 0x3E38 | 0x4038 | 0x0146 | 40 38 |
The device receives `requested_address = (params[2] << 8) | offset_word`
and replies with `(end_offset - next_boundary)` bytes of waveform tail
starting at `next_boundary` including the 26-byte file footer.
Args:
key4: 4-byte waveform key for this event.
end_offset: Event-end pointer (= `(end_key[2] << 8) | end_key[3]`
from the STRT record at data[23:27] of A5[0]).
last_chunk_counter: Counter of the last full 0x0200-byte chunk fetched
(the chunk that covers [last_chunk_counter,
last_chunk_counter + 0x0200)).
Returns:
(offset_word, params10) tuple. Pass as
`build_5a_frame(offset_word, params)`.
Raises:
ValueError: on inconsistent inputs.
"""
if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
next_boundary = last_chunk_counter + 0x0200
if next_boundary > 0xFFFF:
raise ValueError(
f"next_boundary 0x{next_boundary:04X} exceeds uint16; check inputs"
)
if end_offset <= last_chunk_counter:
raise ValueError(
f"end_offset 0x{end_offset:04X} must be > "
f"last_chunk_counter 0x{last_chunk_counter:04X}"
)
offset_word = end_offset - next_boundary
if offset_word < 0:
# Last chunk overshot end_offset; caller should have stopped one chunk
# earlier. Treat as zero residual.
offset_word = 0
if offset_word > 0xFFFF:
raise ValueError(
f"offset_word 0x{offset_word:04X} exceeds uint16"
)
p = bytearray(10)
p[0] = key4[0]
p[1] = key4[1]
p[2] = (next_boundary >> 8) & 0xFF
p[3] = next_boundary & 0xFF
return offset_word, bytes(p)
# ── End-offset extraction from STRT record ────────────────────────────────────
STRT_MARKER = b"STRT"
def parse_strt_end_offset(a5_data: bytes) -> Optional[int]:
"""
Extract the event-end offset from the STRT record in an A5 response payload.
The first A5 response (the probe response, or the first chunk for events
with non-zero start_key[2:4]) contains a STRT record at byte offset 17 of
`data`. Layout:
data[17:21] "STRT"
data[21:23] ff fe sentinel
data[23:27] end_key 4-byte key of where this event ENDS
data[27:31] start_key
...
Returns `(end_key[2] << 8) | end_key[3]` the absolute device-buffer
address where the event ends. Use this to bound the chunk loop and to
compute the TERM frame.
Verified end_offset values:
| event start_key | end_key | end_offset |
| 01110000 | 01111ABE | 0x1ABE |
| 01110000 | 011121F2 | 0x21F2 |
| 011121F2 | 0111417E | 0x417E |
Args:
a5_data: The `data` field of an A5 response frame (frame.data).
Returns:
The end_offset (uint16) if STRT is found, else None.
"""
pos = a5_data.find(STRT_MARKER)
if pos < 0 or pos + 10 > len(a5_data):
return None
# data[pos+4:pos+6] is "ff fe"; data[pos+6:pos+10] is end_key.
end_key = a5_data[pos + 6 : pos + 10]
if len(end_key) < 4:
return None
return (end_key[2] << 8) | end_key[3]
# ── Pre-built POLL frames ───────────────────────────────────────────────────── # ── Pre-built POLL frames ─────────────────────────────────────────────────────
# #
# POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the # POLL (SUB 0x5B) uses the same two-step pattern as all other reads — the
@@ -470,7 +615,6 @@ class S3Frame:
# ── Streaming S3 frame parser ───────────────────────────────────────────────── # ── Streaming S3 frame parser ─────────────────────────────────────────────────
class S3FrameParser: class S3FrameParser:
""" """
Incremental byte-stream parser for S3BW response frames. Incremental byte-stream parser for S3BW response frames.
+52
View File
@@ -201,6 +201,58 @@ class Timestamp:
second=second, second=second,
) )
@classmethod
def from_short_record(cls, data: bytes) -> "Timestamp":
"""
Decode an 8-byte timestamp header from a 210-byte waveform record.
Wire layout ( CONFIRMED 2026-05-01 against live SFM run on BE11529 in
Continuous mode, day-of-month = 1 May, raw: 01 05 07 ea 00 0d 15 25):
byte[0]: day (uint8)
byte[1]: month (uint8)
bytes[2-3]: year (big-endian uint16)
byte[4]: unknown (0x00 in observed sample)
byte[5]: hour (uint8)
byte[6]: minute (uint8)
byte[7]: second (uint8)
This is a third format observed in the wild distinct from the 9-byte
(single-shot, sub_code=0x10 at [1]) and 10-byte (continuous, 0x10 at
[0] AND [2]) layouts. No marker bytes; disambiguated by where the
year lands when scanned at byte 2/3/4.
Args:
data: at least 8 bytes; only the first 8 are consumed.
Returns:
Decoded Timestamp.
Raises:
ValueError: if data is fewer than 8 bytes.
"""
if len(data) < 8:
raise ValueError(
f"Short record timestamp requires at least 8 bytes, got {len(data)}"
)
day = data[0]
month = data[1]
year = struct.unpack_from(">H", data, 2)[0]
unknown_byte = data[4]
hour = data[5]
minute = data[6]
second = data[7]
return cls(
raw=bytes(data[:8]),
flag=0,
year=year,
unknown_byte=unknown_byte,
month=month,
day=day,
hour=hour,
minute=minute,
second=second,
)
@property @property
def clock_set(self) -> bool: def clock_set(self) -> bool:
"""False when year == 1995 (factory default / battery-lost state).""" """False when year == 1995 (factory default / battery-lost state)."""
+225 -150
View File
@@ -35,6 +35,8 @@ from .framing import (
token_params, token_params,
bulk_waveform_params, bulk_waveform_params,
bulk_waveform_term_params, bulk_waveform_term_params,
bulk_waveform_term_v2,
parse_strt_end_offset,
POLL_PROBE, POLL_PROBE,
POLL_DATA, POLL_DATA,
SESSION_RESET, SESSION_RESET,
@@ -122,16 +124,22 @@ DATA_LENGTHS: dict[int, int] = {
} }
# SUB 5A (BULK_WAVEFORM_STREAM) protocol constants. # SUB 5A (BULK_WAVEFORM_STREAM) protocol constants.
# Confirmed from 1-2-26 BW TX capture analysis (2026-04-02). #
_BULK_CHUNK_OFFSET = 0x1004 # offset field for probe + all regular chunk requests ✅ # 2026-05-01 minimal-fix: the chunk-counter walk is now bounded by the event's
_BULK_TERM_OFFSET = 0x005A # offset field for termination request ✅ # `end_offset` extracted from the STRT record at data[23:27] of the probe
_BULK_COUNTER_STEP = 0x0400 # chunk counter increment per chunk ✅ # response. Without this bound the loop kept asking for chunks past the event
# Chunk counter formula: key4[2:4] + (chunk_num - 1) * 0x0400 # end and the device responded with post-event circular-buffer garbage,
# where key4[2:4] is the event's circular-buffer base offset ((key4[2]<<8)|key4[3]). # corrupting reconstructed Blastware files for events ≥ 2 sec.
# Earlier captures showed 0x1004 for chunk 1 of key 01110000 — that was a Blastware #
# artifact. For keys where key4[2:4] != 0x0000 (e.g. key 01111884) the old # We keep the OLD 0x0400 chunk step here (BW actually uses 0x0200 — see §7.8.5
# "n * 0x0400" formula sends counters from the wrong buffer region and the device # of the protocol reference for the corrected understanding) because the
# returns data from a different event. Confirmed correct 2026-04-24. # existing blastware_file.py builder relies on the 0x0400-step frame structure
# to produce valid files. Switching to BW's 0x0200 step is a separate task
# that also requires updating the file builder.
# BW-exact protocol values (v0.14.0). Verified against 4-27-26 + 5-1-26 captures.
_BULK_CHUNK_OFFSET = 0x1002 # offset_word for probe + all chunk requests
_BULK_TERM_OFFSET = 0x005A # offset_word for the legacy terminator (fallback only)
_BULK_COUNTER_STEP = 0x0200 # chunk counter increment (matches chunk payload size)
# Default timeout values (seconds). # Default timeout values (seconds).
# MiniMate Plus is a slow device — keep these generous. # MiniMate Plus is a slow device — keep these generous.
@@ -526,203 +534,270 @@ class MiniMateProtocol:
self, self,
key4: bytes, key4: bytes,
*, *,
stop_after_metadata: bool = True, stop_after_metadata: bool = True, # DEPRECATED — no-op under BW-exact walk
max_chunks: int = 32, max_chunks: int = 256, # safety cap only; loop is bounded by end_offset
include_terminator: bool = False, include_terminator: bool = False,
extra_chunks_after_metadata: int = 1, extra_chunks_after_metadata: int = 1, # DEPRECATED — no-op
) -> list[S3Frame]: ) -> list[S3Frame]:
""" """
Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event. Download the SUB 5A (BULK_WAVEFORM_STREAM) A5 frames for one event using
Blastware's exact protocol. REWRITTEN 2026-05-02 (v0.14.0).
The bulk waveform stream carries both raw ADC samples (large) and Algorithm (matches BW captures across 2-sec / 3-sec / event-2):
event-time metadata strings ("Project:", "Client:", "User Name:",
"Seis Loc:", "Extended Notes") embedded in one of the middle frames
(confirmed: A5[7] of 9 for 1-2-26 capture).
Protocol is request-per-chunk, NOT a continuous stream: 1. Probe
1. Probe (offset=_BULK_CHUNK_OFFSET, is_probe=True, counter=0x0000) - For events at start_key[2:4] = 0x0000 (first event after erase
2. Chunks (offset=_BULK_CHUNK_OFFSET, is_probe=False, counter+=0x0400) / wrap): probe at counter=0x0000 with full key in params.
3. Loop until metadata found (stop_after_metadata=True) or max_chunks - For continuation events (start_key[2:4] != 0): first chunk at
4. Termination (offset=_BULK_TERM_OFFSET, counter=last+_BULK_COUNTER_STEP) counter = start_key[2:4] + 0x0046; acts as both probe and
Device responds with a final A5 frame (page_key=0x0000). first sample chunk; response carries STRT.
By default the termination frame (page_key=0x0000) is NOT included in the 2. Parse end_offset from STRT record at data[23:27] of the probe response.
returned list. Pass include_terminator=True to append it; the blastware_file
writer needs the terminator frame's body to reconstruct the waveform file footer.
Args: 3. Read two fixed metadata pages at counter=0x1002 and counter=0x1004
key4: 4-byte waveform key from EVENT_HEADER (1E). global session metadata (Project / Client / User Name / Seis Loc
stop_after_metadata: If True (default), send termination as soon as / Extended Notes ASCII strings). Event 1 only; continuation
b"Project:" is found in a frame's data — avoids events skip these (BW caches them across the session).
downloading the full ADC waveform payload (several
hundred KB). Set False to download everything. 4. Walk sample chunks at 0x0200 increments, starting from 0x0600 for
max_chunks: Safety cap on the number of chunk requests sent event 1 or `start + 0x0046 + 0x0200` for continuation events.
(default 32; a typical event uses 9 large frames). Stop when `next_chunk + 0x0200 > end_offset`.
include_terminator: If True, append the terminator A5 frame
(page_key=0x0000) to the returned list. The 5. Send TERM frame with offset_word and params computed by
terminator carries the waveform file footer bytes. `bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)`.
Default False preserves existing caller behaviour. The TERM response contains the partial last chunk (residual =
end_offset - next_boundary) including the 26-byte 0e 08 file
footer.
Returns: Returns:
List of S3Frame objects from each A5 response frame. Frame indices List of S3Frame objects from each A5 response (probe, metadata
match the request sequence: index 0 = probe response, index 1 = first pages, sample chunks, optional TERM response). Caller passes
chunk, etc. If include_terminator=True, the last element is the `include_terminator=True` (e.g. write_blastware_file) to keep the
terminator frame (page_key=0x0000). TERM response in the list it's required to reconstruct the
file footer.
Deprecated kwargs:
stop_after_metadata: legacy "Project:"-string-based stop condition.
No-op under the BW-exact walk; the loop is
deterministically bounded by end_offset from
STRT. Accepted for backward compat.
extra_chunks_after_metadata: same.
Raises: Raises:
ProtocolError: on timeout, bad checksum, or unexpected SUB. ProtocolError: on timeout / bad checksum / unexpected SUB.
Confirmed from 1-2-26 BW TX/RX captures (2026-04-02):
- probe + 8 regular chunks + 1 termination = 10 TX frames
- 9 large A5 responses + 1 terminator A5 = 10 RX frames
- page_key=0x0010 on large frames; page_key=0x0000 on terminator
- "Project:" metadata at A5[7].data[626]
""" """
if len(key4) != 4: if len(key4) != 4:
raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}") raise ValueError(f"waveform key must be 4 bytes, got {len(key4)}")
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xFF - 0x5A = 0xA5 # Quietly accept and warn on deprecated kwargs.
if not stop_after_metadata:
log.debug("5A: stop_after_metadata=False is no-op under BW-exact walk")
if extra_chunks_after_metadata not in (0, 1):
log.debug("5A: extra_chunks_after_metadata=%d is no-op under BW-exact walk",
extra_chunks_after_metadata)
rsp_sub = _expected_rsp_sub(SUB_BULK_WAVEFORM) # 0xA5
frames_data: list[S3Frame] = [] frames_data: list[S3Frame] = []
counter = 0
# BW counter formula (confirmed from 4-3-26 capture for key 0111245a, start_offset = (key4[2] << 8) | key4[3]
# and empirical live-device test 2026-04-06 for key 01110000): is_event_1 = (start_offset == 0)
# counter for chunk n = max(key4[2:4], 0x0400) + (n - 1) * 0x0400
# key4[2:4] is the event's circular-buffer base offset. The max() guard
# ensures chunk 1 never uses counter=0x0000 (which equals the probe address
# and causes the device to re-return STRT record data for the first chunk).
_key4_offset = (key4[2] << 8) | key4[3]
# ── Step 1: probe ──────────────────────────────────────────────────── # ── Step 1: probe / first chunk ──────────────────────────────────────
log.debug("5A probe key=%s key4_offset=0x%04X", key4.hex(), _key4_offset) if is_event_1:
params = bulk_waveform_params(key4, 0, is_probe=True) probe_counter = 0
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params)) probe_params = bulk_waveform_params(key4, 0, is_probe=True)
self._parser.reset() # reset bytes_fed counter before probe recv log.debug("5A probe (event-1) key=%s counter=0x0000", key4.hex())
else:
# Continuation events: first 5A request lands at counter = key[2:4]
# (i.e. the address of the off=0x46 WAVEHDR record returned by 1F).
# The probe response carries STRT at byte 17 with end_offset.
#
# Confirmed 2026-05-04 from 5-1-26 "copy 2nd address" capture
# (BW probes counter=0x2238 with key=01112238, STRT@17 end=0x417E)
# and 5-4-26 BW captures (2-sec event probes counter=0x2238).
#
# The earlier "+0x46" formula in the doc came from calling
# start_key the BOUNDARY (off=0x2C) key, but the iteration walk
# uses 1F's off=0x46 key as cur_key, which already incorporates
# the +0x46 offset relative to the boundary. Adding it again
# caused the probe to overshoot, miss STRT, and run uncapped.
probe_counter = start_offset
probe_params = bulk_waveform_params(key4, probe_counter)
log.debug(
"5A probe (event-N) key=%s counter=0x%04X",
key4.hex(), probe_counter,
)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, probe_params))
self._parser.reset()
try: try:
rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False) rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False)
except TimeoutError: except TimeoutError:
log.warning( log.warning(
"5A probe TIMED OUT for key=%s" "5A probe TIMED OUT for key=%s%d raw bytes received",
"%d raw bytes received (no complete A5 frame assembled)",
key4.hex(), self._parser.bytes_fed, key4.hex(), self._parser.bytes_fed,
) )
raise raise
frames_data.append(rsp)
log.debug("5A A5[0] page_key=0x%04X %d bytes", rsp.page_key, len(rsp.data))
# ── Step 2: chunk loop ─────────────────────────────────────────────── frames_data.append(rsp)
# Counter formula: _chunk_base + (chunk_num - 1) * 0x0400 log.debug("5A A5[0] (probe) page_key=0x%04X %d bytes",
# where _chunk_base = max(key4[2:4], 0x0400). rsp.page_key, len(rsp.data))
#
# For events with key4[2:4] != 0 (e.g. key 0111245a, offset 0x245a): # ── Step 2: parse STRT end_offset from probe response ────────────────
# _chunk_base = 0x245a → chunk 1=0x245a, chunk 2=0x285a, ... end_offset = parse_strt_end_offset(rsp.data)
# Confirmed from 4-3-26 capture. if end_offset is None:
# log.warning(
# For events with key4[2:4] == 0 (e.g. key 01110000): "5A probe response did not contain a STRT record; "
# _chunk_base = max(0, 0x0400) = 0x0400 "cannot bound chunk loop — falling back to max_chunks=%d cap",
# → chunk 1=0x0400, chunk 2=0x0800, ... (= old chunk_num*0x0400) max_chunks,
# CRITICAL: counter=0x0000 (same as the probe) causes the device to )
# re-return the STRT record data for chunk 1, making frame 1 look like end_offset = 0xFFFF # impossible value loop runs to max_chunks
# a second probe response (confirmed from server log: frame 1 len=1097, else:
# contains STRT\xff\xfe, contributes zero body bytes after DLE-strip). log.info(
# counter=0x0400 for chunk 1 confirmed working (empirical test 2026-04-06). "5A STRT start_offset=0x%04X end_offset=0x%04X size=0x%04X",
_chunk_base = max(_key4_offset, _BULK_COUNTER_STEP) start_offset, end_offset, end_offset - start_offset,
for chunk_num in range(1, max_chunks + 1): )
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
params = bulk_waveform_params(key4, counter) # ── Step 3: metadata pages 0x1002 + 0x1004 (event 1 only) ────────────
log.debug("5A chunk %d counter=0x%04X", chunk_num, counter) # Confirmed from BW captures: BW reads these two fixed device-buffer
# pages immediately after the probe for events at start_key[2:4]=0.
# Continuation events skip them (BW caches across the session).
# Their content is global compliance-setup metadata: Project, Client,
# User Name, Seis Loc, Extended Notes.
if is_event_1:
for meta_counter in (0x1002, 0x1004):
# Metadata page params have an extra trailing 0x00 byte
# (12-byte params instead of 11) — empirical from BW captures.
# Checksum-neutral but matches BW byte-for-byte.
meta_params = bytes([
0x00,
key4[0], key4[1],
(meta_counter >> 8) & 0xFF,
meta_counter & 0xFF,
0, 0, 0, 0, 0, 0, 0,
])
log.debug("5A metadata page counter=0x%04X", meta_counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, meta_params))
self._parser.reset()
try:
meta_rsp = self._recv_one(
expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
)
except TimeoutError:
log.warning(
"5A metadata page 0x%04X TIMED OUT — continuing",
meta_counter,
)
continue
frames_data.append(meta_rsp)
log.debug(
"5A meta@0x%04X page_key=0x%04X %d bytes",
meta_counter, meta_rsp.page_key, len(meta_rsp.data),
)
# ── Step 4: sample chunk loop, bounded by end_offset ─────────────────
# Sample chunks start at:
# event 1: counter = 0x0600
# event N (>0): counter = probe_counter + 0x0200
# (probe was the first sample chunk)
if is_event_1:
counter = 0x0600
else:
counter = probe_counter + _BULK_COUNTER_STEP
last_chunk_counter: Optional[int] = (
probe_counter if not is_event_1 else None
)
chunks_fetched = 0
while chunks_fetched < max_chunks:
# Stop when next chunk would straddle the event end.
if counter + _BULK_COUNTER_STEP > end_offset:
log.debug(
"5A chunk loop done at counter=0x%04X (end=0x%04X); "
"%d chunks fetched",
counter, end_offset, chunks_fetched,
)
break
params = bulk_waveform_params(key4, counter)
log.debug("5A chunk #%d counter=0x%04X", chunks_fetched + 1, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params)) self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
self._parser.reset() # reset bytes_fed for accurate per-chunk count self._parser.reset()
try: try:
rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False, timeout=10.0) rsp = self._recv_one(
expected_sub=rsp_sub, reset_parser=False, timeout=10.0,
)
except TimeoutError: except TimeoutError:
raw = self._parser.bytes_fed raw = self._parser.bytes_fed
log.warning( log.warning(
"5A TIMEOUT chunk=%d counter=0x%04X raw_bytes=%d", "5A TIMEOUT chunk=%d counter=0x%04X raw_bytes=%d",
chunk_num, counter, raw, chunks_fetched + 1, counter, raw,
) )
if raw > 0 and frames_data: if raw > 0 and frames_data:
# Device sent a partial byte (likely a bare DLE/ETX end-of-stream
# signal) but never completed a full frame. Treat as graceful
# stream end and fall through to the termination step.
log.warning( log.warning(
"5A end-of-stream detected at chunk=%d (raw_bytes=%d, " "5A unexpected end-of-stream — proceeding to TERM",
"frames_collected=%d) — proceeding to termination",
chunk_num, raw, len(frames_data),
) )
break break
raise raise
log.warning( log.debug(
"5A RX chunk=%d page_key=0x%04X data_len=%d contains_Project=%s", "5A RX chunk=%d page_key=0x%04X data_len=%d",
chunk_num, rsp.page_key, len(rsp.data), b"Project:" in rsp.data, chunks_fetched + 1, rsp.page_key, len(rsp.data),
) )
if rsp.page_key == 0x0000: if rsp.page_key == 0x0000:
# Device unexpectedly terminated mid-stream (no termination needed). # Device terminated mid-stream unexpectedly.
log.debug("5A A5[%d] page_key=0x0000 — device terminated early", chunk_num) log.warning(
"5A unexpected page_key=0x0000 mid-stream at counter=0x%04X",
counter,
)
if include_terminator: if include_terminator:
frames_data.append(rsp) frames_data.append(rsp)
return frames_data return frames_data
frames_data.append(rsp) frames_data.append(rsp)
last_chunk_counter = counter
if stop_after_metadata and b"Project:" in rsp.data: counter += _BULK_COUNTER_STEP
# Download exactly one more chunk after finding metadata — this is chunks_fetched += 1
# what Blastware does. The extra chunk contains the tail ADC data
# and primes the device to return a valid footer in the termination
# response. Without it, termination returns an empty ack with no
# footer bytes (confirmed 2026-04-23 from HxD comparison).
# Download extra_chunks_after_metadata more chunks past the
# metadata. The caller calculates this from record_time and
# sample_rate so we download exactly the right amount of ADC
# data — no more, no less — before terminating.
# The device returns the footer in the termination response only
# after the right amount of data has been consumed.
log.debug("5A A5[%d] metadata found — fetching %d more chunk(s)",
chunk_num, extra_chunks_after_metadata)
for _extra_n in range(extra_chunks_after_metadata):
chunk_num += 1
counter = _chunk_base + (chunk_num - 1) * _BULK_COUNTER_STEP
params = bulk_waveform_params(key4, counter)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
try:
extra = self._recv_one(expected_sub=rsp_sub, timeout=10.0)
log.debug("5A A5[%d] extra chunk page_key=0x%04X data_len=%d",
chunk_num, extra.page_key, len(extra.data))
if extra.page_key == 0x0000:
if include_terminator:
frames_data.append(extra)
return frames_data
frames_data.append(extra)
except TimeoutError:
log.debug("5A extra chunk %d timed out — end of stream", _extra_n + 1)
break
break
else: else:
log.warning( log.warning(
"5A reached max_chunks=%d without end-of-stream; sending termination", "5A reached max_chunks=%d at counter=0x%04X (end=0x%04X)",
max_chunks, max_chunks, counter, end_offset,
) )
# ── Step 3: termination ────────────────────────────────────────────── # ── Step 5: TERM with proper end_offset-derived formula ──────────────
term_counter = counter + _BULK_COUNTER_STEP if last_chunk_counter is None or end_offset == 0xFFFF:
term_params = bulk_waveform_term_params(key4, term_counter) # No STRT or no chunks fetched — fall back to legacy TERM.
log.debug( log.warning(
"5A termination term_counter=0x%04X offset=0x%04X", "5A using legacy TERM (offset_word=0x005A); "
term_counter, _BULK_TERM_OFFSET, "end_offset unavailable or no chunks fetched",
) )
self._send(build_5a_frame(_BULK_TERM_OFFSET, term_params)) legacy_counter = (last_chunk_counter or probe_counter) + _BULK_COUNTER_STEP
try: term_offset_word = _BULK_TERM_OFFSET # 0x005A
term_rsp = self._recv_one(expected_sub=rsp_sub) term_params = bulk_waveform_term_params(key4, legacy_counter)
else:
term_offset_word, term_params = bulk_waveform_term_v2(
key4, end_offset, last_chunk_counter,
)
log.debug( log.debug(
"5A termination response page_key=0x%04X %d bytes", "5A TERM offset_word=0x%04X params[2:4]=%s end=0x%04X "
"last_chunk=0x%04X",
term_offset_word, term_params[2:4].hex(),
end_offset, last_chunk_counter,
)
self._send(build_5a_frame(term_offset_word, term_params))
try:
term_rsp = self._recv_one(expected_sub=rsp_sub, timeout=10.0)
log.info(
"5A TERM response page_key=0x%04X %d bytes",
term_rsp.page_key, len(term_rsp.data), term_rsp.page_key, len(term_rsp.data),
) )
if include_terminator: if include_terminator:
frames_data.append(term_rsp) frames_data.append(term_rsp)
except TimeoutError: except TimeoutError:
log.debug("5A no termination response — device may have already closed") log.warning("5A no TERM response (timeout)")
return frames_data return frames_data
+19 -21
View File
@@ -37,6 +37,7 @@ from __future__ import annotations
import datetime import datetime
import logging import logging
import sys import sys
import tempfile
import threading import threading
import time import time
from pathlib import Path from pathlib import Path
@@ -863,8 +864,8 @@ def device_event_blastware_file(
Supply either *port* (serial) or *host* (TCP/modem). Supply either *port* (serial) or *host* (TCP/modem).
The file is written to /tmp and streamed back as a binary download. The file is written to the OS temp directory and streamed back as a binary
Blastware can open it directly filename encodes serial + timestamp. download. Blastware can open it directly filename encodes serial + timestamp.
Filename format: <prefix><serial3><stem><AB>0<W|H> Filename format: <prefix><serial3><stem><AB>0<W|H>
- prefix letter = chr(ord('B') + floor(serial_numeric / 1000)) - prefix letter = chr(ord('B') + floor(serial_numeric / 1000))
@@ -885,26 +886,13 @@ def device_event_blastware_file(
def _do(): def _do():
with _build_client(port, baud, host, tcp_port, timeout=120.0) as client: with _build_client(port, baud, host, tcp_port, timeout=120.0) as client:
info = client.connect() info = client.connect()
# Use stop_after_metadata=True (full_waveform=False) with 1 extra # Under v0.14.0 BW-exact 5A walk, the chunk loop is bounded by
# chunk after "Project:". The extra chunk is required to prime the # the event end_offset extracted from STRT. No more
# device over TCP: termination at term_counter=metadata_counter+0x0400 # stop_after_metadata / extra_chunks gymnastics — these
# returns only ~90 bytes (no useful footer) over TCP/cellular, but # kwargs are now no-ops.
# termination at metadata_counter+0x0800 (one chunk later) returns
# the full 737-byte frame containing the footer.
#
# Confirmed from 4-26-26 BW RS-232 capture: BW terminates at 0x1800
# without an extra chunk (works on RS-232 but not TCP).
# write_blastware_file() automatically skips the extra chunk's
# contribution — only the probe+ADC+metadata+terminator bytes appear
# in the output file.
#
# full_waveform=True (natural end-of-stream) downloads ALL chunks
# including post-event silence (35+ chunks for a 9-sec event at
# 1024 sps) — this produces 24KB+ files that Blastware rejects.
events = client.get_events( events = client.get_events(
full_waveform=False, full_waveform=False,
stop_after_index=index, stop_after_index=index,
extra_chunks_after_metadata=1,
) )
matching = [ev for ev in events if ev.index == index] matching = [ev for ev in events if ev.index == index]
return matching[0] if matching else None, info return matching[0] if matching else None, info
@@ -940,8 +928,18 @@ def device_event_blastware_file(
# Build filename using the same algorithm Blastware uses # Build filename using the same algorithm Blastware uses
filename = blastware_filename(ev, serial) filename = blastware_filename(ev, serial)
# Write to /tmp so FastAPI can stream it back # Write to OS temp dir (cross-platform: /tmp on Linux/macOS,
out_path = Path("/tmp") / filename # %TEMP% on Windows) so FastAPI can stream it back via FileResponse.
out_path = Path(tempfile.gettempdir()) / filename
# Delete any stale file at this path before writing. On Windows we have
# observed the new (smaller) file getting trailing zero-bytes from the
# previous (larger) file when filesystem semantics around open(...,"wb")
# don't truncate cleanly (e.g. through a synced folder). Explicit unlink
# eliminates that ambiguity.
try:
out_path.unlink()
except FileNotFoundError:
pass
write_blastware_file(ev, a5_frames, out_path) write_blastware_file(ev, a5_frames, out_path)
log.info( log.info(
"blastware_file: wrote %s (%d A5 frames, serial=%s)", "blastware_file: wrote %s (%d A5 frames, serial=%s)",