From 8316a1bbd8a3ad90da7e5789a5719a230d9e8d40 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 20 May 2026 15:41:42 +0000 Subject: [PATCH] docs(protocol): accuracy sweep across the protocol reference MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three-pass audit of docs/instantel_protocol_reference.md against CLAUDE.md and the minimateplus/ implementation. Closes long-standing discrepancies that had accumulated as the protocol understanding evolved month over month. Major corrections: - §2/§3: S3 frames terminate on bare ETX, not DLE+ETX; payload byte[1] is flags / byte[2] is SUB (was wrongly DLE/ADDR). - §4.2: probe responses do not carry data length; DATA_LENGTH is a per-SUB hardcoded constant. - §5.1: dropped stale duplicate "SUB 1C = TRIGGER CONFIG READ" row; SUB 0A lengths corrected from 0x30/0x26 to 0x46/0x2C. - §5.3: added the missing write-frame mechanics (BW_CMD-only doubling, DLE-aware checksum, offset = data[1]+2, ack format, SUB 71 chunk parameters). - §7.6.x: switched compliance-anchor convention from the unstable 10-byte form to the canonical 6-byte `\xbe\x80\x00\x00\x00\x00`; recording_mode confirmed at anchor−8 in both read and write (the prior anchor−3/−4 split caused anchor drift on write). Sample_rate at anchor−6, histogram_interval at anchor−4 (now ✅), record_time at anchor+6. Geo_range row added at channel_label+33. - §7.5b/§8: added the 10-byte sub_code=0x03 continuous-mode timestamp variant; peak vector sum location corrected from fixed offset 87 to label-relative tran_pos−12. - §7.7.2: SUB 1E/1F token byte at params[7], not params[6]. - §7.7.3: SUB 0A length disambiguation rewritten. - §7.8.4/§7.8.7: fi==9 skip marked FIXED; metadata-page TODO replaced with current decoder state. - §11: POLL example wire bytes corrected; SUB 5A row added to checksum table. - §13/§14: device-under-test updated to BE11529/S338.17; TCP Idle Timeout consistency fix (0→2 min); Data Forwarding Timeout units clarified. - §15 (renumbered from second §14): open-question entries already resolved in CLAUDE.md closed out. - Appendix D: extension taxonomy rewritten — extensions encode a timestamp (AB0T scheme), not recording mode. Navigation note added to §7 acknowledging the organic-growth duplicate section numbers (§7.5/§7.5b, §7.6, §7.7, §7.8, §7.9) and pointing readers to the canonical sections for each topic. https://claude.ai/code/session_019tWZybD94YUsBaEGhnM5A2 --- docs/instantel_protocol_reference.md | 728 +++++++++++++++++++-------- 1 file changed, 523 insertions(+), 205 deletions(-) diff --git a/docs/instantel_protocol_reference.md b/docs/instantel_protocol_reference.md index 79d1be7..8eb7685 100644 --- a/docs/instantel_protocol_reference.md +++ b/docs/instantel_protocol_reference.md @@ -11,6 +11,7 @@ | Date | Section | Change | |---|---|---| +| 2026-05-20 | §2, §3, §4.2, §5.1, §5.3, §6, §7.5b, §7.6.1, §7.6.3, §7.6.4, §7.7.2, §7.7.3, §7.7.5, §7.8.4, §7.8.7, §7.9, §8, §11, §12, §13, §14, §15, Appendix D | **DOC AUDIT PASS — accuracy sweep against `CLAUDE.md` + `minimateplus/` code.** Fixed: (1) S3 frames terminate on bare ETX, not DLE+ETX — §2/§3 rewritten. (2) §3 payload layout corrected — byte[1]=flags, byte[2]=SUB (was wrongly labelled DLE/ADDR). (3) §4.2 — probe responses do NOT carry data length; lengths are hardcoded `DATA_LENGTHS` constants. (4) §5.1 — removed stale duplicate "SUB 1C = TRIGGER CONFIG READ" row; SUB 0A lengths corrected from `0x30/0x26` to `0x46/0x2C` (real event / boundary marker). (5) §5.3 — added missing write-frame format (BW_CMD-only doubling, DLE-aware checksum, offset formula, ack format, SUB 71 chunk parameters). (6) §6 — fixed "SUB 06 → channel config read" → event storage range. (7) §7.5b / §8 — added the 10-byte `sub_code=0x03` continuous-mode timestamp variant alongside the 9-byte single-shot layout; peak vector sum location corrected from "fixed offset 87" to `tran_pos − 12` (label-relative). (8) §7.6 / §7.6.1 / §7.6.3 / §7.6.4 — switched compliance-anchor convention from the 10-byte form to the canonical 6-byte `\xbe\x80\x00\x00\x00\x00`; recording_mode confirmed at anchor−8 in BOTH read and write (was wrongly listed as anchor−3 write / anchor−4 read); sample_rate at anchor−6, histogram_interval at anchor−4, record_time at anchor+6; geo_range row added at channel_label+33. (9) §7.7.2 — token byte position corrected from `params[6]` to `params[7]`. (10) §7.8.4 — fi==9 skip marked FIXED (already removed from code); chunk-count totals updated. (11) §7.8.7 — TODO replaced with current state of `_decode_a5_metadata_into`. (12) §7.9 — Histogram Interval upgraded ❓ → ✅. (13) §11 — POLL example wire bytes corrected; SUB 5A row added to checksum table. (14) §13 — device-under-test updated for current primary unit (BE11529 / S338.17). (15) §14 — TCP Idle Timeout fixed (0→2 min); Data Forwarding Timeout units clarified. (16) §15 (renumbered from second §14) — open-question items already resolved in CLAUDE.md closed out. (17) Appendix D — extension taxonomy rewritten to reflect the AB0T timestamp encoding (D.5.2/D.5.3); EXTENSION REFUTED warning replaced with the resolved encoding. | | 2026-05-08 | §7.6.1 (RETRACTION) | **❌ RETRACTED — "raw int16 LE 8 bytes/sample-set" body codec was never validated.** The original 4-2-26 confirmation was based on misreading broken-decoder output (full-scale ±32K noise) as evidence the signal had saturated. BW's own 0C peaks for that capture (Tran=0.420 / Vert=3.870 / Long=0.495 in/s) prove the signal was NOT saturated — none of those exceed 13K ADC counts. No event in the project's archive has ever come close to saturation, yet the decoder consistently produces ±32K noise on every event. Conclusion: the body codec is not raw int16 LE; the actual encoding is open. Body byte distribution is heavily skewed (24% `0x00`, 10.5% `0x10`, lots of `10 XX` pairs) — likely a delta encoding with `0x10` as escape, but unverified. Retraction box added at top of §7.6.1; "fully-saturating event" claim removed from channel-identification note. The histogram codec in §7.6.2 IS verified and decoded correctly (different recording mode, 32-byte blocks); use it as a structural hint when reverse-engineering the waveform codec. | | 2026-02-26 | Initial | Document created from first hex dump analysis | | 2026-02-26 | §2 Frame Structure | **CORRECTED:** Frame uses DLE-STX (`0x10 0x02`) and DLE-ETX (`0x10 0x03`), not bare `0x02`/`0x03`. `0x41` confirmed as ACK not STX. DLE stuffing rule added. | @@ -134,22 +135,31 @@ ## 2. Frame Structure > ⚠️ **2026-02-26 — CORRECTED:** Previous version incorrectly identified `0x41` as STX and `0x02`/`0x03` as bare frame delimiters. The protocol uses proper **DLE framing**. See below. -Every message follows this structure: +Frame structure is asymmetric — BW→S3 (requests we send) and S3→BW (device replies) +use slightly different framing markers, both terminated by a **bare** `0x03` ETX: ``` -[ACK] [DLE+STX] [PAYLOAD...] [CHECKSUM] [DLE+ETX] - 0x41 0x10 0x02 N bytes 1 byte 0x10 0x03 +BW→S3 (our requests): [ACK=0x41] [STX=0x02] [stuffed payload + chk] [ETX=0x03] +S3→BW (device replies): [DLE=0x10] [STX=0x02] [stuffed payload + chk] [ETX=0x03] ``` +**Critical:** The outer frame terminator is a **bare ETX (`0x03`), not `DLE+ETX`.** S3 +responses contain inner sub-frames (notably inside SUB A4 POLL_RESPONSE and SUB E5 +compliance reads) that use `DLE+ETX` (`0x10 0x03`) as *inner-frame* terminators — the +outer parser preserves both bytes as literal payload data when it sees `DLE+ETX` and +only ends the frame on a bare ETX. See §10 (parser state machine) and the +`S3FrameParser` docstring in `minimateplus/framing.py`. + ### Special Byte Definitions | Token | Raw Bytes | Meaning | Certainty | |---|---|---|---| -| ACK | `0x41` (ASCII `'A'`) | Acknowledgment / ready token. Standalone single byte. Sent before every frame by both sides. | ✅ CONFIRMED | +| ACK | `0x41` (ASCII `'A'`) | Acknowledgment / ready token. Standalone single byte; often precedes frames. | ✅ CONFIRMED | | DLE | `0x10` | Data Link Escape. Prefixes the next byte to give it special meaning. | ✅ CONFIRMED — 2026-02-26 | -| STX | `0x10 0x02` | DLE+STX = Start of frame (two-byte sequence) | ✅ CONFIRMED — 2026-02-26 | -| ETX | `0x10 0x03` | DLE+ETX = End of frame (two-byte sequence) | ✅ CONFIRMED — 2026-02-26 | -| CHECKSUM | 1 byte | 8-bit sum of de-stuffed payload bytes, modulo 256. Sits between payload and DLE+ETX. | ✅ CONFIRMED | +| STX (BW→S3) | `0x02` (bare) | BW-side start marker — preceded by ACK, not by DLE | ✅ CONFIRMED | +| STX (S3→BW) | `0x10 0x02` | DLE+STX device-side start-of-frame | ✅ CONFIRMED | +| ETX | `0x03` (bare) | End-of-frame terminator in BOTH directions; `DLE+ETX` inside a frame body is an inner sub-frame terminator preserved as literal payload | ✅ CONFIRMED | +| CHECKSUM | 1 byte | Sits between payload and the bare ETX. Two variants — see §11 (standard SUM8 for reads, DLE-aware large-frame for writes / SUB 5A). | ✅ CONFIRMED | ### DLE Byte Stuffing Rule > ✅ CONFIRMED — 2026-02-26 @@ -168,49 +178,62 @@ Any `0x10` byte appearing **naturally in the payload data** is escaped by doubli ### Frame Parser Notes -- The `0x41` ACK **always arrives in a separate `read()` call** before the frame body due to RS-232 inter-byte timing at 38400 baud. This is normal. -- Your parser must be **stateful and buffered** — read byte by byte, accumulate between DLE+STX and DLE+ETX. Never assume one `read()` = one frame. -- Checksum is computed on the **de-stuffed** payload, not the raw wire bytes. -- The ACK and DLE+STX are **not** included in the checksum. +- The `0x41` ACK **may arrive in a separate `read()` call** before the frame body due to RS-232 inter-byte timing or TCP segmentation. The S3 parser treats stray ACK bytes as no-ops while idle. +- Your parser must be **stateful and buffered** — read byte by byte, accumulate after STX, terminate on a bare ETX. Never assume one `read()` = one frame. +- `DLE+ETX` inside a frame body is treated as inner-frame data and appended literally; only a bare ETX ends the outer frame. +- Checksum is computed on the **de-stuffed** payload for standard reads. SUB 5A and write frames use a DLE-aware variant (see §11). +- ACK and the STX bytes are **not** included in the checksum. ### Checksum Verification Example -Raw frame on wire (with ACK and DLE framing): +Raw S3→BW POLL probe response wire bytes: ``` -41 10 02 | 10 10 00 5B 00 00 00 00 00 00 00 00 00 00 00 00 00 | 6B | 10 03 -^ACK^^STX^ ^---------- stuffed payload (0x10→0x10 0x10) ------^ ^chk^ ^ETX^ +10 02 | 10 00 A4 00 00 00 ... [stuffed payload + chk] ... | 03 +^STX^ ^ETX^ ``` -After de-stuffing (`0x10 0x10` → `0x10`): -``` -De-stuffed: 10 00 5B 00 00 00 00 00 00 00 00 00 00 00 00 00 -Checksum: 10+00+5B+00+... = 0x6B ✅ -``` +After de-stuffing (`0x10 0x10` → `0x10`), the payload is checksum-validated using +the formula from §11. The bare ETX terminates the frame. --- ## 3. Payload Structure -The payload (bytes between DLE+STX and CHECKSUM, after de-stuffing) has consistent internal structure: +> ✅ **CONFIRMED 2026-03-30** — field positions verified bit-for-bit against +> `bridges/captures/3-11-26/raw_bw_*.bin`. The earlier "[CMD][DLE][ADDR]…" layout +> that called bytes [1] and [2] both `0x10` was wrong — byte [1] is `flags` +> (`0x00` for BW requests, `0x10` for S3 responses) and byte [2] is the SUB byte. + +The payload (between STX and the checksum, after de-stuffing) is a 16-byte fixed +header for read commands. BW→S3 and S3→BW use the **same field layout** but +swap the values in bytes [0] and [1]: ``` -[CMD] [DLE] [ADDR] [FLAGS] [SUB_CMD] [OFFSET_HI] [OFFSET_LO] [PARAMS × N] - xx 0x10 0x10 0x00 xx xx xx +BW→S3 (request): S3→BW (response): + [0] BW_CMD 0x10 [0] CMD 0x00 + [1] flags 0x00 [1] flags 0x10 + [2] SUB command [2] SUB 0xFF − request_SUB + [3] 0x00 always [3] PAGE_HI (probe responses: 0x00) + [4] 0x00 always [4] PAGE_LO (probe responses: 0x00) + [5] OFFSET probe:0x00; data:DATA_LENGTH + [5+] data response body + [6:16] params 10-byte field ``` | Field | Position | Notes | Certainty | |---|---|---|---| -| CMD | byte 0 | Command or response code | ✅ CONFIRMED | -| DLE | byte 1 | Always `0x10`. Part of address/routing scheme. On wire this is stuffed as `0x10 0x10`. | ✅ CONFIRMED — 2026-02-26 | -| ADDR | byte 2 | Always observed as `0x10`. Also stuffed on wire. Purpose unknown — may not be an address. | ❓ SPECULATIVE | -| FLAGS | byte 3 | Usually `0x00`. Non-zero values seen in event-keyed requests. | 🔶 INFERRED | -| SUB_CMD | byte 4 | The actual operation being requested. | ✅ CONFIRMED | -| OFFSET_HI | byte 5 | High byte of data offset for paged reads. | ✅ CONFIRMED | -| OFFSET_LO | byte 6 | Low byte of data offset. | ✅ CONFIRMED | +| BW_CMD / CMD | byte 0 | `0x10` in BW→S3 requests; `0x00` in S3→BW responses. The `0x10` is the only byte DLE-stuffed in write frames. | ✅ CONFIRMED | +| flags | byte 1 | `0x00` in BW→S3; `0x10` in S3→BW. The byte pattern that earlier docs read as "ADDR" was actually flags. | ✅ CONFIRMED — 2026-03-30 | +| SUB | byte 2 | Command/response selector. Response SUB = `0xFF − request SUB`. | ✅ CONFIRMED | +| OFFSET (requests) | byte 5 | `0x00` for the probe step; `DATA_LENGTH` (hardcoded constant — see §4.2 and §5.1) for the data fetch step. Bytes [3] and [4] are always `0x00` in requests. | ✅ CONFIRMED — 2026-03-30 | +| PAGE_HI / PAGE_LO (responses) | bytes 3–4 | Observed as `0x00 0x00` in probe responses. For chunked data reads (SUB E5, SUB A5) carries the page key / chunk address. | ✅ CONFIRMED | +| params (requests) | bytes 6–15 | 10-byte parameter field; per-SUB structure (event keys, tokens, channel selectors, chunk addresses). | ✅ CONFIRMED | +| data (responses) | bytes 5+ | Response body. For two-step data fetches the actual record begins at `data[11]` after a fixed 11-byte echo header — see §7.7.4. | ✅ CONFIRMED | -> ❓ **NOTE on bytes 1–2:** After de-stuffing, bytes 1 and 2 are both `0x10` in every observed frame across all captured sessions and both units. Their semantic meaning is not yet confirmed. No capture has shown either field vary across units, commands, or directions. They may represent routing, bus ID, or fixed header constants — or the field boundaries assumed here may be wrong entirely. - -> 🔶 **NOTE:** Because bytes 1 and 2 are both `0x10`, they appear on the wire as four consecutive `0x10` bytes (`0x10 0x10 0x10 0x10`). This is normal — both are stuffed. Do not mistake them for DLE+STX or DLE+ETX. +> The only `0x10` byte in a BW request payload is `BW_CMD` at `[0]`. Other `0x10` +> bytes may appear in the `OFFSET` field (e.g. offset_hi=0x10 for compliance writes) +> or in `params` (notably SUB 5A chunk counters). DLE-stuffing of these `0x10` bytes +> on the wire varies by command — see §5.3 (write frames) and §7.8.1 (SUB 5A). --- @@ -227,26 +250,29 @@ Side B → 10 02 [payload] [chk] 10 03 (response frame) ### 4.2 Two-Step Paged Read Pattern -All data reads use a two-step length-prefixed pattern. It is not optional. +> ✅ **CORRECTED 2026-03-30** — earlier description claimed the probe response carries +> the data length in `PAGE_HI`/`PAGE_LO`. It does NOT. The S3 probe response has both +> bytes `0x00`. **Data lengths are hardcoded constants per SUB** — see the `DATA_LENGTHS` +> dict in `minimateplus/protocol.py` and the per-SUB column in §5.1. + +All data reads use a two-step pattern. It is not optional — sending the data +fetch without the probe causes the device to ignore the request. ``` -Step 1 — Request with offset=0 ("how much data is there?"): - BW → 0x41 - BW → 10 02 [CMD] 10 10 00 [SUB] 00 00 [00 00 ...] [chk] 10 03 +Step 1 — Probe (BW: offset=0 in payload[5]): + BW → 41 02 10 10 00 [SUB] 00 00 00 00 [params×10] [chk] 03 + S3 → 10 02 00 10 [RSP] 00 00 [zeroed body] [chk] 03 + (probe ack — PAGE_HI=PAGE_LO=0x00, no length info) -Step 2 — Device replies with total data length: - S3 → 0x41 - S3 → 10 02 [RSP] 00 10 10 [SUB] 00 00 00 00 00 00 [LEN_HI] [LEN_LO] [chk] 10 03 - -Step 3 — Re-request using LEN as offset ("now send the data"): - BW → 0x41 - BW → 10 02 [CMD] 10 10 00 [SUB] 00 00 [LEN_HI] [LEN_LO] [00 ...] [chk] 10 03 - -Step 4 — Device sends actual data payload: - S3 → 0x41 - S3 → 10 02 [RSP] 00 10 10 [SUB] 00 00 [LEN_HI] [LEN_LO] [DATA...] [chk] 10 03 +Step 2 — Data fetch (BW: offset=DATA_LENGTH in payload[5]): + BW → 41 02 10 10 00 [SUB] 00 00 [DATA_LENGTH] [params×10] [chk] 03 + S3 → 10 02 00 10 [RSP] 00 00 [DATA…] [chk] 03 ``` +Implementations must use the per-SUB `DATA_LENGTH` constant (see §5.1) — there is +no length negotiation step. The two-step nature serves an arming purpose on the +device side, not length discovery. + --- ## 5. Command Reference Table @@ -260,16 +286,15 @@ Step 4 — Device sends actual data payload: | `01` | **FULL CONFIG READ** | Requests complete device configuration block (~0x98 bytes). Firmware, model, serial, channel config, scaling factors. | ✅ CONFIRMED | | `08` | **EVENT INDEX READ** | Requests the event record index (0x58 bytes). Event count and record pointers. | ✅ CONFIRMED | | `06` | **EVENT STORAGE RANGE READ** | Requests event storage range block (0x24 = 36 bytes). Token=0xFE at params[7]. Last 8 bytes of response: first stored event key (`[-8:-4]`) and last stored event key (`[-4:]`). Both equal `01110000` when device is empty. Used by Blastware as part of the erase-all verification step. Previously labelled "CHANNEL CONFIG READ" — function now confirmed from 4-11-26 MITM capture. | ✅ CONFIRMED 2026-04-11 | -| `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED | | `1E` | **EVENT HEADER READ** | Gets first waveform key. Token byte at params[7] (0x00=browse, 0xFE=download-arm). Key at data[11:15]; trailing offset at data[15:19] (0 = only one event). Two uses: (1) all-zero to get key0; (2) token=0xFE after 0A, before 0C — REQUIRED to arm device for SUB 5A. | ✅ CONFIRMED 2026-04-06 | -| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 | +| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH at `data_rsp.data[5]`: **`0x46` (= 70 bytes) = real triggered event start**; **`0x2C` (= 44 bytes) = boundary marker / monitor-log partial record**. Key at params[4..7]. Required before every 1F call to establish device waveform context. Used by the "Download All" walk (§7.8.8) to map real events vs boundary keys. | ✅ CONFIRMED 2026-05-01 | | `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label−12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 | | `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 | | `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum, **partial DLE stuffing of 0x10 in params** (`10 X` where X∉{02,03,04,10} must be doubled to `10 10 X` — see §7.8). Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. Walk: probe at counter=`start_offset` (event 1: 0x0000) → metadata pages 0x1002 + 0x1004 (event 1 only) → sample chunks at 0x0600, 0x0800, …, step 0x0200, bounded by `end_offset` parsed from STRT@data[17] of probe response → TERM frame at residual offset_word. Project:/Client:/User Name:/Seis Loc: live in the metadata pages, NOT in the sample-chunk stream. | ✅ CONFIRMED 2026-05-05 (BYTE-PERFECT vs BW capture) | | `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED | | `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED | | `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED | -| `1A` | **COMPLIANCE CONFIG READ** | Multi-step sequence (A+B+C+D frames). Response (E5) carries recording_mode (uint8 at anchor−4 in E5 sf1), sample_rate (uint16 BE at anchor−2), record_time (float32 BE at anchor+10), trigger/alarm/max_range floats, and project strings. Anchor: `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00`, search cfg[0:150]. Total ~2126 cfg bytes. See §7.6.4 for recording_mode enum. | ✅ CONFIRMED 2026-04-02; recording_mode added 2026-04-20 | +| `1A` | **COMPLIANCE CONFIG READ** | Multi-step sequence (A+B+C+D frames). Response (E5) carries recording_mode (uint8 at **anchor − 8** — same offset in read AND write, confirmed 2026-04-21), sample_rate (uint16 BE at anchor − 6), histogram_interval_sec (uint16 BE at anchor − 4), record_time (float32 BE at anchor + 6), trigger/alarm/max_range floats, and project strings. **Anchor: 6-byte `\xbe\x80\x00\x00\x00\x00`**, search `cfg[0:150]`. Total ~2126 cfg bytes. See §7.6.4 for recording_mode enum. | ✅ CONFIRMED 2026-04-02; recording_mode offset corrected 2026-04-21 | | `2E` | **UNKNOWN READ B** | Read command, response (`D1`) returns 0x1A (26) bytes. Purpose unknown. | 🔶 INFERRED | | `0E` | **CHANNEL SENSOR DATA** | Real-time sensor reading for one channel. Two-step read, data length 0x0A (10 bytes). Channel selector in params[6:8] (0x0000–0x0007 for 8 channels). Response (F1) carries amplitude, frequency, overswing data for that channel. Used by Blastware "Unit Channel Test" comms check. | ✅ CONFIRMED 2026-04-08 | | `98` | **TRIGGER TEST** | Trigger-test command. Single probe frame; `params[0] = 0xFF`. Response (0x67) is all-zero data. Sent twice per Blastware comms-check cycle. Not a full POLL, no monitor state change. | ✅ CONFIRMED 2026-04-08 | @@ -314,12 +339,91 @@ All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which, ### 5.3 Write Commands (Blastware → Device) -> ✅ **CONFIRMED — 2026-02-26** from compliance setup capture (session `185019`). +> ✅ **CONFIRMED — 2026-02-26** from compliance setup capture (session `185019`); +> framing mechanics confirmed 2026-04-07 from `bridges/captures/3-11-26/raw_bw_*_170151.bin`. -Write commands are initiated by Blastware (`BW->S3`) and use SUB bytes in the `0x60`–`0x83` range. The device acknowledges each write with a short response frame containing no data payload. +Write commands are initiated by Blastware (`BW→S3`) and use SUB bytes in the `0x60`–`0x83` range. The device acknowledges each write with a short response frame containing no data payload (17 bytes total wire including framing). **Pattern:** Write SUB = Read SUB + `0x60` (e.g. `0x08` EVENT INDEX READ → `0x68` EVENT INDEX WRITE). +#### Write frame format — minimal DLE stuffing + +Write frames do **NOT** use the full DLE stuffing that read frames do. Only the +`BW_CMD` byte at payload position `[0]` is doubled on the wire — all other bytes +(flags, SUB, offset, params, data, checksum) are written **raw**, even when they +contain `0x10`. See `build_bw_write_frame()` in `minimateplus/framing.py`. + +``` +Wire layout: + [41] ACK + [02] STX + [10 10] BW_CMD doubled (the ONLY DLE stuffing applied) + [00] flags + [SUB] write command byte (0x68–0x83) + [00] always zero + [hi][lo] offset uint16 BE — RAW (not stuffed even if hi=0x10) + [params] 10 bytes — RAW + [data] variable-length write payload — RAW (0x10 bytes not stuffed) + [chk] checksum — RAW (not stuffed even if 0x10) + [03] ETX +``` + +Total wire length = `21 + len(data)` bytes. + +#### Write frame checksum + +All write frames use the **large-frame DLE-aware checksum**: + +```python +chk = (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF +``` + +This sums payload bytes from `SUB` onward, skipping every `0x10` byte, then adds +`0x10` as a constant. The same formula is used for confirm frames (data=`b""`), +since they contain no embedded `0x10` bytes and degrade to the standard SUM8. + +#### Single-chunk write offset formula + +For write payloads that fit in one frame (SUBs `0x68`, `0x69`, `0x82`, `0x7E`): + +``` +offset = data[1] + 2 +``` + +`data[0:2]` is a 2-byte header `[0x00][length]`. The offset on the wire encodes +that embedded length + 2. Confirmed across all single-chunk writes in the +3-11-26 capture (`0x68`: 0x5A, `0x82`: 0x1C, `0x69`: 0xCA, `0x7E`: 0x7E). + +#### Write ack response format + +Every device ack for a write command is a **17-byte zero-data S3 frame**: + +``` +[DLE=0x10] [STX=0x02] [stuffed header + chk] [bare ETX=0x03] +``` + +Data section is zeros; `RSP_SUB = 0xFF − write_request_SUB`. + +#### SUB 71 — multi-chunk compliance write + +The compliance config payload (~2128 bytes) is split into **exactly 3 chunks**. +Confirmed from 3-11-26 BW TX frames 104–108: + +| Chunk | Size | `offset` | `params` (10 bytes hex) | +|---|---|---|---| +| 1 (first) | 1027 bytes | `0x1004` | `00 00 00 00 00 00 00 00 00 00` | +| 2 (middle) | 1055 bytes | `0x1004` | `00 00 00 10 04 00 00 00 00 00` | +| 3 (last) | remainder | `0x002C` | `00 00 08 00 00 00 00 00 00 00` | + +After all three chunks are acked (`0x8E` each), send SUB `72` confirm → device acks `0x8D`. + +#### Full write sequences + +``` +Compliance setup push: 68 → 73 | 71 ×3 → 72 | 82 → 83 | 69 → 74 → 72 +Auto Call Home push: 7E → 7F +``` + | SUB | Name | Description | Response SUB | Certainty | |---|---|---|---|---| | `68` | **EVENT INDEX WRITE** | Writes event index block (mirrors SUB `08` read). Contains event count and timestamps. | `97` | ✅ CONFIRMED | @@ -359,11 +463,11 @@ Write commands are initiated by Blastware (`BW->S3`) and use SUB bytes in the `0 4. S3 → 0x41 + POLL RESPONSE (SUB A4, reports data length = 0x30) 5. BW → 0x41 + POLL frame (SUB 5B, offset = 0x30) 6. S3 → 0x41 + POLL RESPONSE with data: "Instantel" + "MiniMate Plus" -7. BW → SUB 06 → channel config read -8. BW → SUB 15 → serial number -9. BW → SUB 01 → full config block -10. BW → SUB 1A → compliance config (4-frame sequence: A+B+C+D) -11. BW → SUB 08 → event index +7. BW → SUB 15 → serial number +8. BW → SUB 01 → full config block +9. BW → SUB 1A → compliance config (4-frame sequence: A+B+C+D) +10. BW → SUB 08 → event index +11. BW → SUB 06 → event storage range (used during erase verification; see §7.11) ``` ### 6.1 Event Download Sequence (per-event, confirmed from 4-2-26 + 4-3-26 BW captures) @@ -392,6 +496,15 @@ Do NOT use `data[11:15]` (key) as sentinel — event 0 has key=0x00000000. ## 7. Known Data Payloads +> 📑 **Navigation note (2026-05-20):** §7 grew organically and contains some +> duplicate section numbers — most notably **§7.5 / §7.5b** (waveform record), +> **§7.6 / §7.7 / §7.8 / §7.9** appear twice each. Sections that re-use a +> number are clearly banner-marked as either DEPRECATED (the old `0x0400`-step +> SUB 5A protocol) or as alternate-context views of the same SUB. **Always +> consult the most recent dated subsection when sections appear to conflict.** +> Canonical SUB 5A walk is in §7.8.5–§7.8.8; canonical compliance config layout +> is in §7.6 / §7.6.1 / §7.6.3 / §7.6.4. + ### 7.1 Poll Response (SUB A4) — Device Identity Block / Composite Container > ⚠️ **SUB A4 is a composite container frame.** The large A4 payload (~3600+ bytes) contains multiple embedded inner sub-frames using the same DLE framing as the outer protocol (`10 02` start, `10 03` end, `10 10` stuffing). Inner frames carry WRITE_CONFIRM_RESPONSE and TRIGGER_CONFIG_RESPONSE sub-frames among others. Flat byte-by-byte diffing of A4 is unreliable due to phase shifting — use inner-frame-aware diffing (`_diff_a4_payloads()` in s3_analyzer.py). Confirmed 2026-03-11. @@ -546,7 +659,7 @@ The SUB `1A` read response (`E5`) and SUB `71` write block contain per-channel t | Field | Example bytes | Decoded | Certainty | |---|---|---|---| | `[00 00]` | `00 00` | Separator / padding | 🔶 INFERRED | -| ADC scale factor | `40 C6 97 FD` | **6.206053 (in/s)/V — CONFIRMED 2026-04-17.** This is the inverse sensitivity of the standard Instantel geophone = 1/0.161133. Interface Handbook §4.5: `Range = 1.61133 V × 6.206053 = 10.000 in/s`. Used by firmware: `PPV (in/s) = ADC_voltage × 6.206053`. Hardware constant — do NOT write. | ✅ CONFIRMED | +| ADC scale factor | `40 C6 97 FD` | **6.206053 (in/s)/V — CONFIRMED 2026-04-17.** This is the inverse sensitivity of the standard Instantel geophone = 1/0.161133. Interface Handbook §4.5: `Range = 1.61133 V × 6.206053 = 10.000 in/s`. Used by firmware: `PPV (in/s) = ADC_voltage × 6.206053`. Hardware constant — do NOT write. Offset = **channel_label + 28** (same in E5 read and SUB 71 write). | ✅ CONFIRMED | | `[00 00]` | `00 00` | Separator / padding | 🔶 INFERRED | | **Trigger level** | `3F 19 99 9A` | **0.600 in/s** — IEEE 754 BE float | ✅ CONFIRMED | | Unit string | `69 6E 2E 00` | `"in.\0"` | ✅ CONFIRMED | @@ -554,6 +667,7 @@ The SUB `1A` read response (`E5`) and SUB `71` write block contain per-channel t | Unit string | `2F 73 00 00` | `"/s\0\0"` | ✅ CONFIRMED | | `[00 01]` | `00 01` | Unknown flag / separator | 🔶 INFERRED | | Channel label | e.g. `56 65 72 74` | `"Vert"` — identifies which channel | ✅ CONFIRMED | +| **Geo range (sensitivity selector)** | `00` or `01` | **uint8** — `0x00`=Normal 10.000 in/s, `0x01`=Sensitive 1.250 in/s. Offset = **channel_label + 33** (same in E5 read and SUB 71 write). Applied to all three geo channel blocks (Tran/Vert/Long). **Note: `channel_label + 20` reads `0x01` on ALL captures regardless of range — it is NOT this field.** | ✅ CONFIRMED 2026-04-20 | **State transitions observed across captures:** @@ -568,13 +682,26 @@ Values are stored natively in **imperial units (in/s)** — unit strings `"in."` ### 7.6.1 Record Time -> ✅ **CONFIRMED — 2026-04-01** (BE11529 / firmware S338.17). Updated from 2026-03-09 offset-based confirmation; anchor approach supersedes the `+0x28` absolute offset. +> ✅ **CONFIRMED — 2026-04-01** (BE11529 / firmware S338.17). Anchor convention +> updated 2026-04-21 — search for the 6-byte stable anchor, not the 10-byte form. -Record time is stored as a **32-bit IEEE 754 float, big-endian**, located via an anchor pattern (see §7.6.3 below). +Record time is stored as a **32-bit IEEE 754 float, big-endian**, located via an +anchor pattern (see §7.6.3 / §7.6.4 for the rest of the anchor-relative fields). -**Anchor-relative location:** search for the 10-byte sequence `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in `cfg[0:150]`. Record time float is at **anchor + 10**. +**Anchor:** search for the 6-byte sequence `\xbe\x80\x00\x00\x00\x00` in +`cfg[0:150]`. **Record time is at anchor + 6** (i.e. 6 bytes past the start of +the anchor). -> ✅ **2026-04-02 — CORRECTED:** Search range was `cfg[40:100]`. With the compliance-config orphaned-send bug fixed (§7.6.2), the 44-byte accidental header padding is gone and the anchor now appears at `cfg[11]`. Search range widened to `cfg[0:150]`. +> ⚠️ **The earlier "10-byte anchor" form `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` +> is NOT stable.** Its leading two bytes `\x01\x2c` (= 300) are the +> `histogram_interval_sec` field, which changes when the histogram interval is +> reconfigured (e.g. 15 min = `\x03\x84`). Only the 6-byte suffix is constant. +> Code in `client.py` uses `buf.find(b'\xbe\x80\x00\x00\x00\x00', 0, 150)`. + +> ✅ **2026-04-02 — CORRECTED:** Search range was `cfg[40:100]`. With the +> compliance-config orphaned-send bug fixed (§7.6.2), the 44-byte accidental +> header padding is gone and the anchor now appears at `cfg[11]`. Search range +> widened to `cfg[0:150]`. | Record Time | float32 BE bytes | Decoded | |---|---|---| @@ -620,9 +747,11 @@ SUB 1A (compliance config read) requires **four frames**, not the standard 2-ste ### 7.6.3 Sample Rate and DLE Jitter -> ✅ **CONFIRMED — 2026-04-01** (BE11529 / firmware S338.17). Validated across Normal (1024), Fast (2048), and Faster (4096) modes. +> ✅ **CONFIRMED — 2026-04-01** (BE11529 / firmware S338.17). Validated across +> Normal (1024), Fast (2048), and Faster (4096) modes. -**Location:** `uint16 BE` at **anchor − 2**, where anchor = `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in cfg[40:100]. +**Location:** `uint16 BE` at **anchor − 6**, using the 6-byte stable anchor +`\xbe\x80\x00\x00\x00\x00` searched in `cfg[0:150]`. | Device Mode | uint16 BE value | Sa/s | |---|---|---| @@ -632,19 +761,30 @@ SUB 1A (compliance config read) requires **four frames**, not the standard 2-ste **DLE jitter (critical — explains the ±1 byte cfg length variation):** -The sample rate bytes sit immediately before a `0x10` (DLE) prefix byte in the raw S3 frame. For the "Faster" mode (4096 = `0x1000`), the high byte `0x10` is itself a DLE character and must be escaped in the S3 frame as `10 10`. After DLE unstuffing: `10 10 00` → `10 00` (2 bytes). For Normal/Fast modes (high byte = `0x04`/`0x08`), no escaping needed: payload stays 3 bytes. Result: "Faster" mode produces a cfg that is **1 byte shorter** than Normal/Fast, shifting all subsequent absolute offsets by −1. +The sample rate bytes sit a few bytes before the anchor in the raw S3 frame. +For "Faster" mode (4096 = `0x1000`), the high byte `0x10` is itself a DLE +character and was historically escaped on the wire as `10 10`. After DLE +unstuffing: `10 10 00` → `10 00` (2 bytes). For Normal/Fast modes (high byte = +`0x04` / `0x08`), no escaping needed and the payload stays 3 bytes. Result: +"Faster" mode can produce a cfg that is **1 byte shorter** than Normal/Fast, +shifting any absolute offset by −1. -**Why anchor search is required:** any decoder that uses fixed absolute offsets for record_time or sample_rate will produce garbage values when the device is set to "Faster" mode. The 10-byte anchor `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` (`0x012C` = 300 s max-record-length constant, followed by two alarm-level floats `\xbe\x80\x00\x00\x00\x00`) straddles the boundary and is unaffected by this shift. +**Why anchor search is required:** any decoder that uses fixed absolute offsets +for record_time or sample_rate will produce garbage values when the device is +set to "Faster" mode. The 6-byte anchor (start of the alarm-level floats +`\xbe\x80\x00\x00\x00\x00`) is past the DLE-affected bytes and is unaffected by +this shift. --- ### 7.6.4 Recording Mode -> ✅ **CONFIRMED — 2026-04-20** (BE11529 / firmware S338.17). Three targeted captures in a single Blastware session (4-20-26 directory), changing Recording Mode only between each write. +> ✅ **CONFIRMED — 2026-04-20** (BE11529 / firmware S338.17); offsets corrected +> 2026-04-21 after a write-cycle drift bug was traced to writing to the wrong +> byte. **`recording_mode` is at `anchor − 8` for BOTH read and write.** -Recording mode is stored as a **uint8** with different anchor-relative positions depending on whether you are reading from a device response or constructing a write payload. - -**In the SUB 71 write payload (3-chunk compliance write, `cfg[5]`):** +Recording mode is a **uint8** stored at a fixed anchor-relative position (using +the 6-byte stable anchor `\xbe\x80\x00\x00\x00\x00`): | Enum | Mode | |---|---| @@ -654,34 +794,53 @@ Recording mode is stored as a **uint8** with different anchor-relative positions | `0x03` | Histogram | | `0x04` | Histogram + Continuous (combined mode) | -Anchor-relative position: **anchor − 3** (3 bytes before the 10-byte anchor in the write payload). The write payload layout in the region around the anchor: +**Layout around the anchor (same in E5 read response and SUB 71 write payload):** ``` -cfg[anchor - 3] = recording_mode (uint8) -cfg[anchor - 2] = sample_rate_hi (uint8, MSB of uint16 BE) -cfg[anchor - 1] = sample_rate_lo (uint8, LSB of uint16 BE) -cfg[anchor:anchor+10] = \x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00 ← anchor -cfg[anchor + 10:anchor + 14] = record_time (float32 BE) +buf[anchor − 9] = mode_prefix (uint8; 0x00 for Single Shot/Continuous, + 0x10 for Histogram (DLE prefix in E5 wire + encoding of 0x03) and for Histogram+Continuous + (actual stored byte)) +buf[anchor − 8] = recording_mode (uint8) ← SAME OFFSET IN READ AND WRITE +buf[anchor − 7] = 0x10 (constant; part of the sample-rate field area; + do NOT overwrite — see warning below) +buf[anchor − 6] = sample_rate_hi (uint8, MSB of uint16 BE) +buf[anchor − 5] = sample_rate_lo (uint8, LSB of uint16 BE) +buf[anchor − 4 : anchor − 2] = histogram_interval_sec (uint16 BE, seconds; + mode-gated to Histogram) +buf[anchor − 2 : anchor] = 0x00 0x00 (padding) +buf[anchor : anchor + 6] = \xbe\x80\x00\x00\x00\x00 (anchor) +buf[anchor + 6 : anchor + 10] = record_time (float32 BE) ``` -**In the E5 read response (sub-frame 1, page=`0x0010`, `data[17]`):** +> ⚠️ **Do NOT write to `anchor − 7`.** The byte there is a DLE marker that the +> device firmware re-generates on every read. Writing to it shifts the anchor +> position by +1 on every subsequent read, drifting compliance config offsets +> indefinitely. The CLAUDE.md note "write to anchor-7" was a known mistake — +> see the `_encode_compliance_config` comment block in `client.py:2062–2068`. -The anchor appears at `data[21]` in this sub-frame. Recording mode is at `data[17]` = **anchor − 4** (one position earlier than in the write payload). This is because an extra `0x10` byte is present at `data[18]` in the read format (between recording_mode and sample_rate), which is NOT present in the write payload. The read-format layout: +**`compliance_raw` DLE encoding:** -``` -data[17] = recording_mode (uint8) -data[18] = 0x10 ← extra byte present in E5 read only; absent in SUB 71 write -data[19] = sample_rate_hi (uint8, MSB of uint16 BE) -data[20] = sample_rate_lo (uint8, LSB of uint16 BE) -data[21:31] = anchor (\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00) -data[31:35] = record_time (float32 BE) -``` +`compliance_raw` returned by `read_compliance_config()` is NOT purely logical +bytes — it is the wire-encoded payload where any logical `0x03` byte appears as +the literal pair `\x10\x03` (preserved by `S3FrameParser` as inner-frame +literal data). Consequences: -**Chunk checksum at `cfg[1024]`:** The first of the three SUB 71 write chunks (1027 bytes) contains a running checksum byte at `cfg[1024]` whose delta exactly equals the delta of `cfg[5]` (recording_mode). This byte reflects the cumulative change from `recording_mode` through to its position and should not be mistaken for a second copy of the recording_mode field. +- For `recording_mode = 0x03` (Histogram), `compliance_raw[anc − 9] = 0x10` + (the DLE prefix from the wire encoding of `0x03`) and + `compliance_raw[anc − 8] = 0x03` (the actual value). +- For Histogram + Continuous (`0x04`), `compliance_raw[anc − 9] = 0x10` is an + actual stored config byte (not a DLE prefix). +- The 6-byte anchor search (`buf.find(b'\xbe\x80...', 0, 150)`) locates the + anchor correctly regardless of these mode-dependent shifts. +- When SFM writes `recording_mode` and round-trips the rest verbatim, the byte + at `anc − 9` is preserved from the previous read. Transitioning + Histogram → other modes via SFM may leave a `0x10` at `anc − 9` — the device + stores it as a literal byte; it does not affect recording-mode operation + (which is at `anc − 8`) but is one byte different from what BW writes. -**Decode path (`_decode_compliance_config_into`):** use `data[anchor_pos - 4]` where `anchor_pos` is the index of the first byte of the anchor in the assembled E5 cfg bytes. - -**Encode path (`_encode_compliance_config`):** use `cfg[anchor_pos - 3]` = recording_mode value (write-payload offset; no extra `0x10` byte). +**Decode path (`_decode_compliance_config_into`):** `data[anchor_pos − 8]`. +**Encode path (`_encode_compliance_config`):** `buf[anchor_pos − 8] = recording_mode`. --- @@ -809,11 +968,12 @@ Several settings are **mode-gated**: the device only transmits (reads) or accept --- -### 7.5 Full Waveform Record (SUB F3) — 0xD2 bytes (210 bytes) +### 7.5b Full Waveform Record (SUB F3) — 0xD2 bytes (210 bytes) > ✅ **Updated 2026-03-31** — Full layout confirmed. See §7.7.5 for the > complete record structure including timestamp, record type, PPV float > positions, and project strings. +> Section number changed to `7.5b` 2026-05-20 to disambiguate from §7.5 above. Peak values are found by searching for channel label strings `"Tran"`, `"Vert"`, `"Long"`, `"MicL"` and reading `float32 BE` at `label_offset + 6`. @@ -1051,45 +1211,75 @@ a flash sector offset or circular-buffer pointer internal to the S3 DSP). - Subsequent keys: returned by `1F` at `data[11:15]` - Terminator: `00 00 00 00` signals no more events -Example keys from 3-31-26 capture (one Blastware "event" / 4 histogram bins): +Example keys from 3-31-26 capture (one Blastware "event"): ``` -01 11 00 16 ← first bin (full, 0x30 length) -01 11 11 B6 ← second bin (partial, 0x26 length — skipped by 1F/0xFE) -01 11 11 F6 ← third bin (partial, 0x26 length — skipped) -01 11 12 36 ← fourth bin (full, 0x30 length — returned by 1F/0xFE) +01 11 00 16 ← first key (full record — see §7.7.3 for length disambiguation) +01 11 11 B6 ← intermediate boundary key +01 11 11 F6 ← intermediate boundary key +01 11 12 36 ← next real-event key 00 00 00 00 ← terminator ``` +> ⚠️ The 3-31-26 capture was the only "events present" capture available at the +> time of original analysis and contained a single event. The "0x30 = full / +> 0x26 = partial" interpretation in earlier drafts was an artifact of that +> single-event capture; the canonical lengths are `0x46` (real event) and `0x2C` +> (boundary), confirmed across 5-1-26 multi-event captures. See §7.7.3 / §7.8.8. + --- #### 7.7.2 Token Byte (SUB 1E / 1F) -A token byte at `payload[12]` (= `params[6]` in `build_bw_frame`) controls -the 1F advance behaviour: +> ✅ **CORRECTED 2026-04-06** — token byte is at **`params[7]`** (not `params[6]`). +> Confirmed from raw params `00 00 00 00 00 00 00 FE 00 00` in both the 3-31-26 +> and 4-3-26 BW TX captures. With the wrong position the device silently ignores +> the token. + +A token byte at **`params[7]`** controls the 1E/1F behaviour: | Token | Mode | Behaviour | |---|---|---| -| `0x00` | Browse | Advance one record, including partial histogram bins | -| `0xFE` | Download | Skip partial bins, advance to the next full record | +| `0x00` | Browse | Advance one record across the full event chain (including boundary keys) | +| `0xFE` | Download (arm) | Used in the 5A bulk-stream arming sequence — see §6.1 | -**We always use `0xFE`** — it minimises round trips and avoids needing to -handle partial-bin `0C` calls. +**Browse vs download is context-dependent:** + +- `count_events()` and the "Download All" pre-walk (§7.8.8) use **browse mode + (token=`0x00`)** to enumerate the full key chain and discriminate real events + from boundaries via SUB 0A's echoed length (§7.7.3). +- `get_events()` uses `0xFE` for the **arm step** between 0A and 0C (REQUIRED + to wake the 5A state machine — see §6.1) and after 1F to arm the bulk stream, + but then uses browse-mode 1F to advance to the next event key after a + successful 5A. + +Calling 1F with `0xFE` cold (without the preceding 0A and 1E-arm) returns the +null sentinel regardless of how many events are stored — an earlier observation +that token=0xFE "returns null" was the result of testing without those +prerequisite steps. --- #### 7.7.3 Variable DATA_LENGTH for SUB 0A (WAVEFORM_HEADER) +> ✅ **CORRECTED 2026-05-01** — earlier values `0x30` / `0x26` came from the +> deprecated histogram-only walk. Confirmed values for triggered-event downloads +> are `0x46` / `0x2C` (see §5.1, §7.8.8). + Unlike all other SUBs, `0A` does NOT have a fixed data length. The length -is returned in the probe response at `data[4]`: +is echoed in the probe response at `data[5]` (and is also the value the +data-request step sends back as its offset): | Length | Meaning | |---|---| -| `0x30` | Full histogram bin — has a waveform record to download | -| `0x26` | Partial histogram bin — no waveform record | +| `0x46` (70 bytes) | Full triggered-event waveform header — has a real event to download | +| `0x2C` (44 bytes) | Boundary marker / monitor-log partial record — no triggered event | Both the probe and data-request frames carry the same key in `params[4..7]`. -The `read_waveform_header()` method in `protocol.py` reads `probe.data[4]` -and uses that value as the data-request offset. +`read_waveform_header()` (protocol.py) reads the echoed length from the probe +response and uses it as the data-request offset, then returns +`(raw_data, length)` so callers (`get_events`, `get_monitor_log_entries`, +"Download All" walk) can decide whether to issue a 0C/5A download or just +advance the iterator. --- @@ -1123,11 +1313,19 @@ Actual data lengths: The 210-byte record (`data_rsp.data[11:11+0xD2]`) contains: -**Header / Timestamp** (9 bytes at offsets 0–8, ✅ CONFIRMED 2026-04-01): +**Header / Timestamp** — two layouts exist; **detect via `byte[0]`** of the record header: + +- `byte[0] != 0x10` → **9-byte format** (sub_code=0x10, single-shot waveform) +- `byte[0] == 0x10` → **10-byte format** (sub_code=0x03, continuous waveform; same + layout as partial monitor-log records — see §7.10 / `_decode_0a_partial_header` in client.py) + +The auto-detect logic lives in `_decode_waveform_record_into` (client.py) and in +`_decode_0a_partial_header` for monitor log entries. + +**9-byte format — sub_code = 0x10 (Waveform single-shot) ✅ CONFIRMED 2026-04-01:** ``` byte[0]: day (uint8) -byte[1]: sub_code 0x10 = Waveform (continuous/single-shot) ✅ - histogram code not yet captured ❓ +byte[1]: sub_code 0x10 = Waveform single-shot byte[2]: month (uint8) bytes[3–4]: year (uint16 big-endian) byte[5]: unknown (0x00 in all observed samples ❓) @@ -1136,14 +1334,35 @@ byte[7]: minute (uint8) byte[8]: second (uint8) ``` -Thump event raw bytes (2026-04-01 00:28:12): +Thump event raw bytes (2026-04-01 00:28:12, single-shot): ``` 01 10 04 07 ea 00 00 1c 0c ↑ ↑ ↑ ↑──↑ ↑ ↑ ↑ ↑ d=1 sub m=4 y=2026 ? h=0 m=28 s=12 ``` -Cross-referenced against the `.MLG` file for the same event, which stores an +**10-byte format — sub_code = 0x03 (Waveform continuous) ✅ CONFIRMED 2026-04-03:** + +One byte wider than the single-shot layout. All subsequent fields shift +1. + +``` +byte[0]: 0x10 (marker byte; how the decoder distinguishes the two layouts) +byte[1]: day (uint8) +byte[2]: 0x10 (marker; reflects the wire-encoded sub_code=0x03) +byte[3]: month (uint8) +bytes[4–5]: year (uint16 big-endian) +byte[6]: unknown (0x00 ❓) +byte[7]: hour (uint8) +byte[8]: minute (uint8) +byte[9]: second (uint8) +``` + +Continuous event raw bytes (2026-04-03 15:20:17): +``` +10 03 10 04 07 ea 00 0f 14 11 +``` + +Cross-referenced against the `.MLG` file for a single-shot event, which stores an 8-byte timestamp at two offsets (trigger time and end time): ``` MLG format: [day:1][month:1][year:2 LE][?:1][hour:1][min:1][sec:1] @@ -1151,9 +1370,12 @@ MLG format: [day:1][month:1][year:2 LE][?:1][hour:1][min:1][sec:1] 01 04 ea 07 00 00 1c 0f → end time April 1, 2026 00:28:15 (3.0 s record time ✅) ``` -**Record type** — encoded in `byte[1]` (sub_code), NOT as an ASCII string: -- `0x10` → `"Waveform"` (continuous / single-shot mode) ✅ -- histogram sub_code: not yet confirmed — capture a histogram event with `debug=true` +**Record type** — encoded in `byte[1]` (single-shot) or `byte[0]/byte[2]` markers +(continuous), NOT as an ASCII string: +- `0x10` at `byte[1]` → Waveform single-shot ✅ +- `0x10` at `byte[0]` + `0x10` at `byte[2]` → Waveform continuous ✅ +- histogram sub_code: not yet confirmed in a 0C response (the monitor-log + 10-byte timestamp uses the same continuous-mode marker) **Peak particle velocity floats** (✅ CONFIRMED 2026-03-31, re-confirmed 2026-04-01): @@ -1175,16 +1397,27 @@ Confirmed offsets from thump event (2026-04-01, cross-referenced vs Blastware): Channel labels are separated by inner-frame bytes `10 03` (DLE ETX), preserved as literal data by `S3FrameParser`. -**Peak Vector Sum** (✅ CONFIRMED 2026-04-01): -``` -Offset 87: IEEE 754 big-endian float32 - = √(Tran² + Vert² + Long²) at the sample instant of maximum - combined geo motion - NOT the vector sum of the three per-channel peaks (those may - occur at different sample times) +**Peak Vector Sum** (✅ CONFIRMED 2026-04-01; offset corrected 2026-04-03): -Thump event: 0x4079F6C5 = 3.906 in/s ✅ matches Blastware "Peak Vector Sum: 3.906 in/s" -Near-ambient: 0x3C75C28F = 0.015 in/s (histogram event, near-zero ambient) +``` +Location: tran_pos − 12 (label-relative; IEEE 754 big-endian float32) + = √(Tran² + Vert² + Long²) at the sample instant of maximum + combined geo motion + NOT the vector sum of the three per-channel peaks (those may + occur at different sample times) +``` + +> ⚠️ **Do NOT use fixed offset 87.** That value only happens to be correct for +> `sub_code = 0x10` (single-shot) records, where the 9-byte timestamp leaves the +> peak vector sum at byte 87. For `sub_code = 0x03` (continuous) records, the +> 10-byte timestamp pushes everything one byte later — `tran_pos − 12` is the +> canonical label-relative location and works for both record types. + +``` +Thump event (single-shot, tran_pos=99): + Value at tran_pos − 12 = byte 87 = 0x4079F6C5 = 3.906 in/s + ✅ matches Blastware "Peak Vector Sum: 3.906 in/s" +Near-ambient (single-shot): 0x3C75C28F = 0.015 in/s ``` **Project strings** — ASCII label-value pairs (search for label, read null-terminated value): @@ -1428,15 +1661,16 @@ TimeoutError caught (rare under corrected walk): → genuine transport failure; re-raise ``` -**Chunk timing (BE11529, 1024 sps, TCP/cellular):** +**Chunk timing (BE11529, 1024 sps, TCP/cellular) — total A5 frames per event:** | Metric | Observed value | |---|---| | Chunk response time | ~1 s per chunk | -| Chunks for a 2-sec event (corrected walk) | 14 (12 sample chunks + 2 metadata pages) + TERM | -| Chunks for a 3-sec event (corrected walk) | 18 (16 sample chunks + 2 metadata pages) + TERM | -| Chunks for a continuation event (corrected walk) | ~15 sample chunks + TERM (no metadata reread) | -| Chunks under deprecated walk for 2-3 sec event | 37 (over-reads ~5×) | +| 2-sec event 1 (corrected walk) | ~15 frames (probe + 2 metadata pages + ~12 sample chunks + TERM) | +| 3-sec event 1 (corrected walk) | 16 frames (probe + 2 metadata pages + 13 sample chunks + TERM) | +| 2-sec continuation event (corrected walk) | 16 frames (~15 sample chunks + TERM; no metadata reread) | +| 3-sec continuation event (corrected walk) | ~15 frames (~14 sample chunks + TERM) | +| Same event under deprecated 0x0400 walk | ~37 frames (over-reads ~5× into post-event garbage) | | Data per chunk (corrected, 0x0200 size) | ~540–575 bytes wire (= 0x0200 payload + framing) | | Data per chunk (deprecated 0x0400 step) | 1,036–1,123 bytes wire (= 0x0400 payload + framing) | | Safe recv timeout per chunk | **10 s** (10× typical) | @@ -1456,9 +1690,15 @@ PPV (in/s) = ADC_voltage (V) × 6.206053 where `geo_range = 1.61133 V × 6.206053 = 10.000 in/s` is the Normal (Gain=1) full-scale range. The earlier ~9× overread was caused by mistakenly using 6.206053 as the range directly — it is actually the scale factor, and the range itself is `ADC_fullscale × scale_factor = 1.61133 × 6.206053 = 10.000 in/s`. Mic channel uses psi units with its own range (still unresolved). -**Known decoder issue — fi==9 hardcoded skip:** +**Decoder note — fi==9 hardcoded skip (FIXED 2026-04-06):** -`_decode_a5_waveform()` contains `elif fi == 9: continue` from an earlier assumption that frame index 9 is always the device terminator. For streams with more than 9 frames, frame 9 is live waveform data. The skip discards ~1,070 bytes (~133 sample-sets) per event. Terminator detection should use `page_key == 0x0000`, not frame index. This skip should be removed. +`_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from +the original 9-frame blast capture where frame 9 was assumed to be the device +terminator. Removed. Under the v0.14.0+ STRT-bounded walk the chunk loop never +encounters a "terminator frame index" anyway — it stops at `end_offset` +calculated from the STRT record (§7.8.5) and the TERM frame is sent explicitly +(§7.8.6). For other decoders, terminator detection should use +`frame.page_key == 0x0000`, not a hardcoded frame index. #### 7.8.5 Chunk addressing and the STRT end_offset (NEW 2026-05-01) ✅ @@ -1610,14 +1850,23 @@ For SFM: - Their content does not change while iterating events. They DO change when the user applies a new compliance setup (SUB 71 write) — invalidate the cache then. -##### TODO — content layout +##### Content layout — string-search decoder in place -The byte-for-byte layout of pages 0x1002 and 0x1004 has not been decoded. First task on -the implementation side: dump both pages from a fresh capture and verify they include all -the strings currently extracted from the deprecated A5 frame 7 of the chunk stream. -Compare to the existing `_decode_a5_metadata_into` parser — same string-search anchors -(`b"Project:"`, `b"Client:"`, `b"User Name:"`, `b"Seis Loc:"`, `b"Extended Notes"`) likely -apply directly. +The full byte-for-byte structural layout of pages 0x1002 and 0x1004 has not been +mapped, but `_decode_a5_metadata_into` (`minimateplus/client.py`) successfully +extracts the user-facing strings via label scans: + +``` +b"Project:" → project description +b"Client:" → client name +b"User Name:" → operator +b"Seis Loc:" → sensor location +b"Extended Notes" → notes +``` + +This works across all observed captures (4-27-26, 5-1-26, 5-4-26). A future +project could dump the structural layout if additional session-global fields +need to be extracted — but the strings already decode correctly. #### 7.8.8 "Download All" Sequence (NEW 2026-05-01) ✅ @@ -1665,11 +1914,11 @@ Fields visible in the Blastware "Compliance Setup" dialog. ✅ = byte offset co | Field | Values / Type | Status | |---|---|---| -| Recording Mode | Single Shot (`0x00`) / Continuous (`0x01`) / Histogram (`0x03`) / Histogram+Continuous (`0x04`) | ✅ `recording_mode` — write: `cfg[anchor−3]`; read E5 sf1: `data[anchor−4]` — confirmed 2026-04-20 | +| Recording Mode | Single Shot (`0x00`) / Continuous (`0x01`) / Histogram (`0x03`) / Histogram+Continuous (`0x04`) | ✅ `recording_mode` — uint8 at **anchor − 8** (SAME offset in E5 read and SUB 71 write, using 6-byte anchor `\xbe\x80\x00\x00\x00\x00`) — confirmed 2026-04-21 | | Record Stop Mode | Fixed Record Time / Auto / Manual Stop | ❓ Hint: `data[40]` in E5 sf1 changed `01 7F` → `00 00` alongside Continuous → Single Shot; may be related but unconfirmed independently | -| Sample Rate | Standard 1024 / Fast 2048 / Faster 4096 sps | ✅ `sample_rate` (anchor−2) | -| Record Time | float, seconds (3, 5, 8, 10, 13…) | ✅ `record_time` (anchor+10) | -| Histogram Interval | 5 / 15 / 30 / 60 min (mode-gated behind Histogram mode) | ❓ | +| Sample Rate | Standard 1024 / Fast 2048 / Faster 4096 sps | ✅ `sample_rate` — uint16 BE at **anchor − 6** | +| Record Time | float, seconds (3, 5, 8, 10, 13…) | ✅ `record_time` — float32 BE at **anchor + 6** | +| Histogram Interval | `2s / 5s / 15s / 1m / 5m / 15m` (uint16 BE seconds; mode-gated to Histogram / Histogram+Continuous) | ✅ `histogram_interval_sec` — uint16 BE at **anchor − 4**, same in read and write — confirmed 2026-04-20 | | Storage Mode | Save All Data / Save Triggered | ❓ | | Geophone Type | Standard Triaxial / 4.5 Hz Geophone | ❓ | | Geophone — Enable all | bool | ❓ | @@ -2009,11 +2258,12 @@ Appears in event index blocks. Time-of-day fields (hour/min/sec) are absent. > ✅ **2026-02-26 — CONFIRMED:** The year 1995 is the **MiniMate Plus factory default RTC date**, which the device reverts to whenever the internal battery is disconnected or the real-time clock loses power. Any event timestamped around 1995 means the clock was not set. This is known device behavior, not an encoding anomaly. -### 8.2 9-byte format (Full Waveform Record / SUB 0C, bytes 0–8) +### 8.2 9-byte format (Full Waveform Record / SUB 0C, sub_code=0x10) > ✅ **CONFIRMED 2026-04-01** — Cross-referenced against Blastware event report > for BE11529 thump event: "00:28:12 April 1, 2026". -Full date + time, including a sub_code byte that encodes the recording mode. +Full date + time, including a sub_code byte. Used for **single-shot waveform** +recordings (`sub_code = 0x10`). **Observed example (thump event, 2026-04-01):** ``` @@ -2023,7 +2273,7 @@ Full date + time, including a sub_code byte that encodes the recording mode. | Byte(s) | Value | Meaning | Certainty | |---|---|---|---| | `01` | 1 | Day | ✅ | -| `10` | 0x10 | Sub_code: `0x10` = Waveform (continuous mode) | ✅ / histogram code ❓ | +| `10` | 0x10 | Sub_code: `0x10` = Waveform single-shot | ✅ | | `04` | 4 | Month (April) | ✅ | | `07 ea` | 2026 | Year — 16-bit big-endian integer | ✅ | | `00` | 0 | Unknown separator | ❓ | @@ -2031,9 +2281,38 @@ Full date + time, including a sub_code byte that encodes the recording mode. | `1c` | 28 | Minute | ✅ | | `0c` | 12 | Second | ✅ | -The `.MLG` file for the same event stores the timestamp in a different binary -representation (little-endian year, no sub_code byte), confirming the waveform -record and the saved file use distinct serialisation formats. +### 8.3 10-byte format (Full Waveform Record / SUB 0C, sub_code=0x03; monitor-log partial records) +> ✅ **CONFIRMED 2026-04-03** — Cross-referenced against Blastware event report +> for BE11529 (15:20:17 April 3, 2026). Same layout is used for partial-record +> (monitor log, record type `0x2C`) entries — see §7.10 / `_decode_0a_partial_header`. + +Used for **continuous waveform** recordings (`sub_code = 0x03`). One byte +wider than the 9-byte layout — a leading `0x10` marker shifts every subsequent +field by +1. + +**Observed example (continuous event, 2026-04-03 15:20:17):** +``` +10 03 10 04 07 ea 00 0f 14 11 +``` + +| Byte | Value | Meaning | +|---|---|---| +| `10` | 0x10 | Marker; how the decoder distinguishes 10-byte from 9-byte layout | +| `03` | 3 | Day | +| `10` | 0x10 | Marker (reflects wire-encoded sub_code=0x03) | +| `04` | 4 | Month | +| `07 ea` | 2026 | Year — uint16 BE | +| `00` | 0 | Unknown separator | +| `0f` | 15 | Hour | +| `14` | 20 | Minute | +| `11` | 17 | Second | + +**Auto-detection rule:** if `record[0] == 0x10`, use the 10-byte layout; +otherwise use the 9-byte layout. + +The `.MLG` file for a single-shot event stores the timestamp in a different +binary representation (little-endian year, no sub_code byte), confirming the +waveform record and the saved file use distinct serialisation formats. --- @@ -2085,7 +2364,8 @@ ESCAPE: |---|---|---|---| | S3→BW | All frames | `sum(payload) & 0xFF` | All de-stuffed payload bytes `[0:-1]` | | BW→S3 | Small frames (POLL, read cmds) | `sum(payload) & 0xFF` | All de-stuffed payload bytes `[0:-1]` | -| BW→S3 | Large write frames (SUB `68`,`69`,`71`,`82`,`1A`+data) | See formula below | De-stuffed payload bytes `[2:-1]`, skipping `0x10` bytes, plus constant | +| BW→S3 | Large write frames (SUB `68`, `69`, `71`, `82`, `7E`, `1A`+data) | DLE-aware large-frame: `(sum(payload[2:] where b != 0x10) + 0x10) & 0xFF` | De-stuffed payload bytes `[2:]`, skipping `0x10` bytes, plus constant | +| BW→S3 | SUB 5A bulk waveform | DLE-aware over the **stuffed** wire bytes | For each `10 XX` pair in the stuffed section, count `XX`; lone bytes count as-is. See `build_5a_frame` in `minimateplus/framing.py`. | ### BW→S3 Large-Frame Checksum Formula @@ -2197,53 +2477,65 @@ def parse_frame(raw: bytes) -> bytes | None: return payload -# ── Example: build a POLL request (SUB 5B) ──────────────────────────────────── +# ── Example: build a POLL probe request (SUB 5B) ───────────────────────────── +# Layout per §3: BW_CMD=0x10, flags=0x00, SUB=0x5B, [0x00, 0x00], OFFSET=0x00, +# then 10 bytes of params — 16 bytes total payload. poll_payload = bytes([ - 0x02, # CMD - 0x10, 0x10, # DLE, ADDR (each stuffed to 0x10 0x10 on wire) - 0x00, # FLAGS - 0x5B, # SUB: POLL - 0x00, 0x00, # OFFSET_HI, OFFSET_LO - 0x00, 0x00, 0x00, 0x00, # padding - 0x00, 0x00, 0x00, 0x00, - 0x00, 0x00, 0x00, + 0x10, # [0] BW_CMD + 0x00, # [1] flags + 0x5B, # [2] SUB: POLL + 0x00, 0x00, # [3:5] always zero + 0x00, # [5] OFFSET (probe step) + 0x00, 0x00, 0x00, 0x00, 0x00, # [6:11] params + 0x00, 0x00, 0x00, 0x00, 0x00, # [11:16] params ]) -frame = build_frame(poll_payload) -# Wire output: 41 10 02 02 10 10 10 10 00 5B 00 00 00 00 00 00 00 00 00 00 6B 10 03 +# checksum = SUM8 over payload[0:-1] for small read frames; here = (0x10 + 0x5B) & 0xFF = 0x6B +# Wire output (BW→S3): the leading 0x10 is DLE-stuffed to 0x10 0x10: +# 41 02 10 10 00 5B 00 00 00 00 00 00 00 00 00 00 00 00 00 6B 03 +# ^ ^ ^^^^^ ^ ^ +# ACK STX (bare 0x02) chk bare ETX +# See `build_bw_frame()` in `minimateplus/framing.py` for the canonical builder. ``` --- ## 12. Recommended Implementation Sequence -Build in this order — each step is independently testable: +Build in this order — each step is independently testable. **Don't try to +implement event download (step 8 onward) without reading §6.1 first** — the +1E-arm / POLL×3 / 5A walk has multiple device-side state-machine prerequisites +that are easy to miss. -1. **DLE frame parser** — stateful byte-by-byte reader implementing the §10 state machine. Handles ACK, stuffing, de-stuffing, checksum validation. -2. **`connect(port, baud=38400)`** — open port, flush buffer, discard ASCII boot strings, send first POLL frame. +1. **DLE frame parser** — stateful byte-by-byte reader implementing the §10 state machine. Handles ACK, stuffing, de-stuffing, bare-ETX termination, checksum validation. +2. **`connect(port, baud=38400)`** — open port, flush buffer, discard ASCII boot strings (§9), send first POLL frame. 3. **`identify()`** — SUB `5B` two-step read → returns `{"manufacturer": "Instantel", "model": "MiniMate Plus"}`. 4. **`get_serial()`** — SUB `15` two-step read → returns serial number string. -5. **`get_config()`** — SUB `01` two-step read → returns full config dict (firmware, channel scales, etc.). -6. **`get_event_count()`** — SUB `08` two-step read → returns number of stored events. -7. **`get_event_header(index)`** — SUB `1E` read → returns timestamp dict. -8. **`get_event_record(timestamp)`** — SUB `0C` paginated read → returns PPV dict per channel. -9. **`download_waveform(timestamp)`** — SUB `5A` bulk stream → returns raw ADC arrays per channel. -10. **`set_*()`** write commands — not yet captured, requires additional sniffing sessions. +5. **`get_config()`** — SUB `01` / SUB `FE` two-step read → returns full config dict (firmware, calibration date, aux trigger). +6. **`get_event_count()`** — SUB `08` event index read. +7. **`get_event_header(key)`** — SUB `1E` (browse mode, token=`0x00` at params[7]) → first event key; iterate with SUB `1F`. +8. **`get_event_record(key)`** — SUB `0C` paginated read → 210-byte waveform record (peaks, timestamp, project). Use the 9-byte / 10-byte timestamp auto-detect from §7.5b / §8. +9. **`download_waveform(key)`** — SUB `5A` bulk stream. **Requires the full arm sequence from §6.1:** 0A → 1E(arm) → 0C → 1F(arm) → POLL × 3 → 5A. Use the STRT-bounded chunk walk from §7.8.5, the TERM frame from §7.8.6, and the metadata pages from §7.8.7. +10. **Write commands** — see §5.3 for the frame format and SUB pairings. Compliance push, call-home setup, and erase-all are all confirmed (§7.11, §7.12). --- -## 13. Device Under Test +## 13. Devices Under Test -| Field | Value | -|---|---| -| Manufacturer | Instantel | -| Model | MiniMate Plus | -| Serial Number | BE18189 | -| Firmware | S337.17 | -| DSP / Secondary FW | 10.72 | -| Channels | Tran, Vert, Long, MicL (4 channels) | -| Sample Rate | ~1024 sps (🔶 INFERRED) | -| Bridge Config | COM5 (Blastware) ↔ COM4 (Device), 38400 baud | -| Capture Tool | s3_bridge v0.4.0 | +Multiple units have contributed captures to this reference. Primary development +target is **BE11529** (firmware S338.17); earlier sessions used BE18189. + +| Field | Primary unit | Secondary unit(s) | +|---|---|---| +| Manufacturer | Instantel | — | +| Model | MiniMate Plus | — | +| Serial Number | **BE11529** | BE18189, BE6907, BE14036, BE17353, BE18003, BE18191, BE18676 (archive only) | +| Firmware | **S338.17** | S337.17 (BE18189); various S338.x for other units | +| DSP / Secondary FW | 10.72 | 10.72 | +| Channels | Tran, Vert, Long, MicL (4 channels) | — | +| Sample Rate | 1024 / 2048 / 4096 sps (selectable) | — | +| Bridge Config (RS-232) | direct or via Sierra Wireless RV50/RV55 modem, 38400,8N1 | — | +| Bridge Config (TCP) | port 12345 (configurable) | — | +| Capture Tool | s3_bridge v0.5.0+ | v0.4.0 used for 1-2-26 / 3-11-26 captures | --- @@ -2306,7 +2598,7 @@ No ENQ byte or other application-layer handshake is added. The Raven modem's TC | Telnet Echo Mode | 0 — No Echo | No echo of received bytes | | Enable ENQ on TCP Connect | 0 — Disable | No ENQ byte on connect | | TCP Connect Response Delay | 0 | No delay before first byte | -| TCP Idle Timeout | 0 | No modem-level idle disconnect | +| TCP Idle Timeout | **2** (minutes) | Prevents premature disconnect; must match §14.3 | --- @@ -2321,10 +2613,12 @@ The modem's RS-232 port (wired to the MiniMate Plus) must be configured as: | Configure Serial Port | **38400,8N1** | | Flow Control | None | | DB9 Serial Echo | OFF | -| Data Forwarding Timeout | **1 second** (S50=1) | +| Data Forwarding Timeout | **`1` (= 0.1 s)** in ACEmanager units (1 tick = 0.1 s; see §14.3 RV50/RV55 table) | | Data Forwarding Character | 0 (disabled) | -The **Data Forwarding Timeout** is the most protocol-critical setting. The modem **accumulates bytes from the RS-232 port for up to 1 second** before forwarding them as a TCP segment. This means: +The **Data Forwarding Timeout** is the most protocol-critical setting. The modem +accumulates bytes from the RS-232 port for up to the configured interval before +forwarding them as a TCP segment. This means: - A large S3 response frame may arrive as multiple TCP segments with up to 1-second gaps between them. - A `read_until_idle` implementation with `idle_gap < 1.0 s` will **incorrectly declare the frame complete mid-stream**. @@ -2430,22 +2724,22 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger --- -## 14. Open Questions / Still Needs Cracking +## 15. Open Questions / Still Needs Cracking | Question | Priority | Added | Notes | |---|---|---|---| | Timestamp 6-byte format byte[3] — purpose of the separator `0x00` byte | LOW | 2026-02-26 | Not blocking; 9-byte waveform record format (§8.2) fully confirmed without this byte. | | `trail[0]` in serial number response — unit-specific byte, derivation unknown. `trail[1]` resolved as firmware minor version. | MEDIUM | 2026-02-26 | | | Full channel ID mapping in SUB `5A` stream (01/02/03/04 → which sensor?) | MEDIUM | 2026-02-26 | | -| Exact byte boundaries of project string fields in SUB `71` write frame — padding rules unconfirmed | MEDIUM | 2026-02-26 | | +| ~~Exact byte boundaries of project string fields in SUB `71` write frame~~ — **RESOLVED 2026-05-05:** project/client/operator/seis-loc/extended-notes come from SUB 5A metadata pages at counter `0x1002` / `0x1004` (§7.8.7), NOT from the SUB 71 write payload. `_decode_a5_metadata_into` locates them via ASCII label scans. | RESOLVED | 2026-02-26 | Resolved 2026-05-05 | | Purpose of SUB `09` / response `F6` — 202-byte read block | MEDIUM | 2026-02-26 | | | Purpose of SUB `2E` / response `D1` — 26-byte read block | MEDIUM | 2026-02-26 | | -| Full field mapping of SUB `1A` / response `E5` — channel scaling / compliance config block. **SUBSTANTIALLY RESOLVED 2026-04-01:** Multi-frame sequence (§7.6.2), page_key chunking, duplicate-page detection, record_time (§7.6.1), sample_rate (§7.6.3), setup_name, project strings all confirmed. Trigger/alarm level floats in the channel block (§7.6) also confirmed. Remaining unknowns: trigger level (geo), alarm level (geo), max range (in CFG header not yet decoded). | MEDIUM | 2026-02-26 | Substantially resolved 2026-04-01 | +| Full field mapping of SUB `1A` / response `E5` — channel scaling / compliance config block. **RESOLVED 2026-04-21:** Multi-frame sequence (§7.6.2), page_key chunking, duplicate-page detection, record_time at anchor+6 (§7.6.1), sample_rate at anchor−6 (§7.6.3), recording_mode at anchor−8 (§7.6.4), histogram_interval at anchor−4, trigger_level_geo and alarm_level_geo in channel block (§7.6), geo_range at channel_label+33 (§7.6), adc_scale_factor at channel_label+28, and project strings (via 5A metadata pages, §7.8.7) all confirmed. | RESOLVED | 2026-02-26 | Resolved 2026-04-21 | | `0x082A` in channel config block — not trigger, alarm, or record time directly. **RESOLVED: fixed E5 payload length (2090 bytes).** Constant regardless of all settings. | RESOLVED | 2026-03-01 | Resolved 2026-03-09 | | **Record time in wire protocol** — float32 BE, anchor-relative in cfg (see §7.6.1/§7.6.3). **RESOLVED.** Previous `+0x28` offset was unreliable due to DLE jitter — superseded by anchor search. Confirmed at 3, 5, 7, 8, 10, 13 seconds. | RESOLVED | 2026-03-09 | Confirmed 2026-03-09; anchor method confirmed 2026-04-01 | | **Sample rate** — uint16 BE at anchor−2 in cfg. **RESOLVED.** Normal=1024, Fast=2048, Faster=4096. Anchor required to handle DLE jitter. See §7.6.3. | RESOLVED | 2026-04-01 | Confirmed 2026-04-01 | | Unknown uint16 fields at channel block +0A (=80), +0C (=15), +0E (=40), +10 (=21) — manual describes "Sensitive (Gain=8) / Normal (Gain=1)" per-channel range; 80/15/40/21 might encode gain, sensitivity, or ADC config. | LOW | 2026-03-01 | | -| Full trigger configuration field mapping (SUB `1C` / write `82`) | LOW | 2026-02-26 | | +| ~~Full trigger configuration field mapping (SUB `1C` / write `82`)~~ — **PARTIALLY RESOLVED:** SUB `1C` now confirmed as MONITOR STATUS READ (§5.1, §7.10) — not trigger config; the old "0x6E response" was misidentification. SUB `82` (trigger config write) layout partially mapped — Trigger Sample Width at `[22]`. Remaining unknowns: full per-channel structure of the 0x82 payload. | LOW | 2026-02-26 | Updated 2026-05-20 | | Whether SUB `24`/`25` are distinct from SUB `5A` or redundant | LOW | 2026-02-26 | | | **Meaning of `0x07 E7` field in config block — RESOLVED:** Calibration year. uint16 BE at destuffed payload offset 0x56–0x57. Confirmed via two-unit comparison: BE18189 (calibrated 2023) = `07 E7`; BE11529 (calibrated 2025) = `07 E9`. Adjacent bytes at 0x53–0x55 encode remaining calibration date (month confirmed as BCD October for both units; full layout 🔶 INFERRED). | RESOLVED | 2026-02-26 | Resolved 2026-03-31 | | **Trigger Sample Width** — **RESOLVED:** BW→S3 write frame SUB `0x82`, destuffed payload offset `[22]`, uint8. Width=4 → `0x04`, Width=3 → `0x03`. Confirmed via BW-side capture diff. Only visible in `raw_bw.bin` write traffic, not in S3-side compliance reads. | RESOLVED | 2026-03-02 | Confirmed 2026-03-09 | @@ -2596,9 +2890,13 @@ Semantic Interpretation <- settings, events, responses > ✅ CONFIRMED 2026-04-21 — all fields verified by binary diff of reconstructed vs reference > files from the 4-3-26-multi_event capture (M529LIY6.N00, BE11529.MLG). > -> ⚠️ EXTENSION MAPPING REFUTED 2026-04-21 — earlier assumption that extension encodes -> recording mode is **FALSE**. A continuous-mode event produced `.EI0`, not `.9T0`. -> Extension encoding algorithm is unknown. Do not use extension to infer recording mode. +> ✅ **EXTENSION ENCODING FULLY DECODED 2026-04-22** — supersedes the earlier +> "extension encodes recording mode" hypothesis (which was wrong) AND the +> "extension encoding algorithm is unknown" note. Extensions encode a +> **timestamp** (seconds within a 21.6-minute window), NOT recording mode. +> See **D.5.2** for the full formula, confirmed against 3,248 files from a +> 10-year production archive. Cross-references throughout this appendix have +> been updated. ### D.1 Common File Header (22 bytes) @@ -2617,9 +2915,9 @@ All Blastware files (regardless of type) share an 18-byte prefix followed by a 4 | Extension | Type tag | Description | |---|---|---| -| `.N00` | `00 12 03 00` | Waveform event (confirmed) | -| `.9T0` | `00 12 03 00` | Waveform event — same type tag as .N00 (assumed; not independently confirmed) | -| `.EI0` | `00 12 03 00` | Waveform event — same type tag (assumed; continuous-mode event observed 2026-04-21) | +| `AB0` / `AB0W` (Waveform) | `00 12 03 00` | Waveform event — extension encodes timestamp, see D.5.2/D.5.3 | +| `AB0H` (Histogram) | `00 12 03 00` | Histogram event — ACH only | +| `.N00` | `00 12 03 00` | Old-firmware placeholder (Waveform); `blastware_filename()` falls back to this for S338 firmware | | `.MLG` | `22 01 0e a0` | Monitor log | **Extension encoding — new firmware (V10.72+) FULLY DECODED (confirmed 2026-04-22):** @@ -2825,20 +3123,40 @@ event_time = 1985-01-01 + 1,274,886,008s = 2025-05-26 15:00:08 local **Note on local time:** The device's onboard clock is set to the local timezone of the deployment site. The epoch and all timestamps are in that same local time — there is no UTC conversion. Files moved between timezones will decode to the original deployment timezone. -#### D.5.3 Extension taxonomy +#### D.5.3 Extension taxonomy — historical rewrite (FULLY DECODED 2026-04-22) -Third character of extension is always `'0'`. File type is identified by extension, not by the type tag in the header (all waveform extensions share type tag `00 12 03 00`). +The extension third character is always `'0'`. The full encoding scheme is now +mapped — see **D.5.2** for the canonical formula. The earlier "recording-mode" +taxonomy in this section was wrong and has been removed. -| Extension | Recording mode | Sample rate | Status | -|---|---|---|---| -| `.N00` | Single Shot (0x00) | 1024 sps | ✅ CONFIRMED | -| `.9T0` | Continuous (0x01) | 1024 sps | ✅ CONFIRMED | -| `.490` | ? | ? | ❓ observed from M529LJ8V.490 | -| `.5K0` | ? | ? | ❓ observed from M529LJDY.5K0 | -| `.980` | ? | ? | ❓ observed from M529LJDY.980 | -| `.ML0` | ? | ? | ❓ observed from M529LJDY.ML0 (167s duration; possibly Histogram) | +**Summary (new firmware V10.72+ via ACH download):** -**Why 5 extensions for "Continuous"?** Binary analysis of all 6 example files shows that `.9T0`, `.490`, `.5K0`, `.980`, `.ML0` are byte-for-byte identical in all metadata regions (compliance anchor block, channel descriptor blocks `Tran/Vert/Long/MicL`). The A5 frame 7 body reflects the **session-start** compliance config, not the per-event capture config. All 5 files show recording_mode=0x01 and sample_rate=1024 in the body. The extension must therefore encode the **capture-time** compliance state — likely a combination of recording mode, sample rate, and possibly mic units or other options. This cannot be determined from file body alone without capture-time compliance data from the 0C record sub_code and the actual waveform sample count. +``` +Extension = AB0T + +AB = base-36 of (total_seconds % 1296) — seconds within the 21.6-min window +0 = literal digit zero (invariant third character) +T = W (Waveform) | H (Histogram) — present only for ACH downloads +``` + +Direct/manual Blastware downloads produce `.AB0` (3 chars, no trailing type +character). `blastware_filename()` in `minimateplus/blastware_file.py` +implements the formula; it falls back to `.N00` as a placeholder for +old-firmware units (S338) where the extension encoding has not been confirmed. + +The previously listed extensions `.9T0` / `.490` / `.5K0` / `.980` / `.ML0` +were the **same continuous-mode unit recording at different times of day**. +Their bytes were byte-for-byte identical except for header fields that change +per event — exactly what D.5.2 predicts: same firmware, same recording mode, +different timestamp → different extensions. No correlation with recording mode. + +| Extension type char (4-char ACH) | Meaning | +|---|---| +| `W` | Full Waveform | +| `H` | Full Histogram | + +**Micromate Series 4** uses a different scheme entirely (`IDFH`, `IDFW` +observed) and does NOT follow the AB0T formula. **DLE-shift offset note for reading recording_mode from N00/9T0 body:**