From 5866ecdb3e321d1aa9112b47af726d1b9c8b8790 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Thu, 16 Apr 2026 18:17:16 -0400 Subject: [PATCH] docs: update protocol doc to reflect unkown status of max_range_geo. --- CLAUDE.md | 1893 +++++++++++++------------- docs/instantel_protocol_reference.md | 20 +- 2 files changed, 957 insertions(+), 956 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 710fa6e..c642602 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,946 +1,947 @@ -# CLAUDE.md — seismo-relay - -Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for -managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem -(Sierra Wireless RV50 / RV55). Current version: **v0.12.0**. - ---- - -## Project layout - -``` -minimateplus/ ← Python client library (primary focus) - transport.py ← SerialTransport, TcpTransport - framing.py ← DLE codec, frame builders, S3FrameParser - protocol.py ← MiniMateProtocol — wire-level read/write methods - client.py ← MiniMateClient — high-level API (connect, get_events, …) - models.py ← DeviceInfo, EventRecord, ComplianceConfig, … - -sfm/server.py ← FastAPI REST server exposing device data over HTTP -seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs) -docs/ - instantel_protocol_reference.md ← reverse-engineered protocol spec ("the Rosetta Stone") -CHANGELOG.md ← version history -``` - ---- - -## Current implementation state (v0.10.0) - -Full read pipeline + write pipeline + erase pipeline + monitor log working end-to-end over TCP/cellular: - -| Step | SUB | Status | -|---|---|---| -| POLL / startup handshake | 5B | ✅ | -| Serial number | 15 | ✅ | -| Full config (firmware, calibration date, etc.) | FE | ✅ | -| Compliance config (record time, sample rate, geo thresholds) | 1A | ✅ | -| Event index | 08 | ✅ | -| Event header / first key | 1E | ✅ | -| Waveform header | 0A | ✅ | -| Waveform record (peaks, timestamp, project) | 0C | ✅ | -| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 | -| Event advance / next key | 1F | ✅ | -| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 | -| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 | -| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ **new v0.10.0** | - -`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F` - -`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72` - -`delete_all_events()` erase sequence: `0xA3 → 0x1C → 0x06 → 0xA2` - ---- - -## Protocol fundamentals - -### DLE framing - -``` -BW→S3 (our requests): [ACK=0x41] [STX=0x02] [stuffed payload+chk] [ETX=0x03] -S3→BW (device replies): [DLE=0x10] [STX=0x02] [stuffed payload+chk] [bare ETX=0x03] -``` - -- **DLE stuffing rule:** any literal `0x10` byte in the payload is doubled on the wire - (`0x10` → `0x10 0x10`). This includes the checksum byte. -- **Inner-frame terminators:** large S3 responses (A4, E5) contain embedded sub-frames - using `DLE+ETX` as inner terminators. The outer parser treats `DLE+ETX` inside a frame - as literal data — the bare ETX is the ONLY real frame terminator. -- **Response SUB rule:** `response_SUB = 0xFF - request_SUB` - (no known exceptions — earlier note claiming `1C` → `6E` was WRONG; `1C` → `0xE3` confirmed across 338 frames in 4-8-26 captures) -- **Two-step read pattern:** every read command is sent twice — probe step (`offset=0x00`, - get length) then data step (`offset=DATA_LENGTH`, get payload). All data lengths are - hardcoded constants, not read from the probe response. - -### De-stuffed payload header - -``` -BW→S3 (request): - [0] CMD 0x10 - [1] flags 0x00 - [2] SUB command byte - [3] 0x00 always zero - [4] 0x00 always zero - [5] OFFSET 0x00 for probe step; DATA_LENGTH for data step - [6-15] params (key, token, etc. — see helpers in framing.py) - -S3→BW (response): - [0] CMD 0x00 - [1] flags 0x10 - [2] SUB response sub byte - [3] PAGE_HI - [4] PAGE_LO - [5+] data -``` - ---- - -## Critical protocol gotchas (hard-won — do not re-derive) - -### SUB 5A — bulk waveform stream — NON-STANDARD frame format - -**Always use `build_5a_frame()` for SUB 5A. Never use `build_bw_frame()` for SUB 5A.** - -`build_bw_frame` produces WRONG output for 5A for two reasons: - -1. **`offset_hi = 0x10` must NOT be DLE-stuffed.** Blastware sends the offset field raw. - `build_bw_frame` would stuff it to `10 10` on the wire — the device silently ignores - the frame. `build_5a_frame` writes it as a bare `10`. - -2. **DLE-aware checksum.** When computing the checksum, `10 XX` pairs in the stuffed - section contribute only `XX` to the running sum; lone bytes contribute normally. This - differs from the standard SUM8-of-destuffed-payload that all other commands use. - -Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26 -BW TX capture. All 10 frames verified. - -### SUB 5A — chunk counter is monotonic (CORRECTED 2026-04-06) - -**Chunk counters are `chunk_num * 0x0400` for ALL chunks including chunk 1.** - -The 4-2-26 BW TX capture showed `counter=0x1004` for chunk 1 of event key `01110000`, which -led to `_CHUNK1_COUNTER = 0x1004` being hardcoded as a special case. This was a Blastware -artifact, not a protocol requirement. Empirical test 2026-04-06: with `counter=0x1004` for -chunk 1 the device times out (120 s); with `counter=0x0400` (= `1 * 0x0400`) it responds -immediately and streams all frames correctly. - -The 4-3-26 capture confirms the pattern for a second event (key `0111245a`): -chunk 1 = `0x245A`, chunk 2 = `0x285A`, chunk 3 = `0x2C5A` (each +0x0400). Blastware's -true formula is `key4[2:4] + n * 0x0400` — but since `key4[2:4]` of the first event is -`0x0000`, `n * 0x0400` produces the right result. The device does not strictly validate the -counter and streams data for any valid 5A request; using `chunk_num * 0x0400` is correct. - -### SUB 5A — params are 11 bytes for chunk frames, 10 for termination - -`bulk_waveform_params()` returns 11 bytes (extra trailing `0x00`). The 11th byte was -confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes. -Do not swap them. - -### SUB 5A — event-time metadata lives in A5 frame 7 - -The bulk stream sends 9+ A5 response frames. Frame 7 (0-indexed) contains the compliance -setup as it existed when the event was recorded: - -``` -"Project:" → project description -"Client:" → client name ← NOT in the 0C record -"User Name:" → operator name ← NOT in the 0C record -"Seis Loc:" → sensor location ← NOT in the 0C record -"Extended Notes"→ notes -``` - -**IMPORTANT — 5A "Project:" is session-start config, NOT per-event (confirmed 2026-04-05):** -The "Project:" string in the A5 frame 7 payload reflects the compliance setup from when -the *monitoring session first started*, not the individual event's project name. The per- -event project name is correctly stored in the 210-byte 0C waveform record and must be -used as the authoritative source. `_decode_a5_metadata_into` therefore only sets -`project` from 5A when 0C didn't already supply one. - -"Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C -record — 5A remains the sole source for those fields and they are set unconditionally. - -`stop_after_metadata=True` (default) stops the 5A loop as soon as `b"Project:"` appears, -then sends the termination frame. - -### SUB 5A — end-of-stream signal (confirmed 2026-04-06) - -After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to -the next chunk request, then goes silent. This is the natural end-of-stream indicator — NOT -a complete A5 frame. `S3FrameParser.bytes_fed` will be 1; no frame is assembled. - -Handling: on `TimeoutError`, if `bytes_fed > 0` AND frames were already collected, treat as -graceful end-of-stream, break the loop, and proceed to the termination frame. If `bytes_fed -== 0` with no prior frames, it is a genuine transport failure — re-raise. - -**Chunk recv timeout must be 10 s, not the default 120 s.** Chunks arrive within ~1 s each. -Using 120 s causes a ~2-minute stall at every end-of-stream detection. The `_recv_one` call -in the chunk loop passes `timeout=10.0` explicitly. - -**Typical chunk count (BE11529, 1024 sps):** A 9,306-sample event produces 35 chunks before -end-of-stream. Chunks with uniform 1,036-byte data are all-zero ADC samples (post-event -silence). Only the initial variable-size chunks contain actual signal. - -### SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06) - -`_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from the -9-frame original blast capture where frame 9 was assumed to be a terminator. For current -35-frame streams, fi==9 is live waveform data (~133 sample-sets were being dropped). -Removed. Terminator detection is via `page_key == 0x0000` in `read_bulk_waveform_stream`, -not frame index. - -### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce) - -**token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and -4-3-26 BW TX captures confirm it belongs at **`params[7]`** (raw: `00 00 00 00 00 00 00 fe 00 00`). -With the wrong position the device ignores the token and 1F returns null immediately. - -**1F token depends on context:** In browse mode (no 5A), use all-zero params (`browse=True`). -In download mode (get_events with 5A), use token=0xFE (`browse=False`) — this is required to -arm the device's 5A bulk stream state machine. The earlier "empirical" test showing token=0xFE -returns null was done WITHOUT the 1E(arm) step; that test is invalid. BW always uses 1F(0xFE) -in download mode. `count_events` uses `browse=True` (no 5A needed). - -**0A context requirement:** `advance_event()` (1F) only returns a valid next-event key -when a preceding `read_waveform_header()` (0A) call has established device waveform -context for the current key. Call 0A before every event in the loop, not just the first. -Calling 1F cold (after only 1E, with no 0A) returns the null sentinel regardless of how -many events are stored. - -**1F response layout:** The next event's key IS at `data_rsp.data[11:15]` (= payload[16:20]). -Confirmed from 4-3-26 browse-mode S3 captures: -``` -1F after 0A(key0=01110000): data[11:15]=0111245a data[15:19]=00001e36 ← valid -1F after 0A(key1=0111245a): data[11:15]=01114290 data[15:19]=00000046 ← valid -1F null sentinel: data[11:15]=00000000 data[15:19]=00000000 ← done -``` - -**Null sentinel:** `data8[4:8] == b"\x00\x00\x00\x00"` (= `data_rsp.data[15:19]`) -works for BOTH 1E trailing (offset to next event key) and 1F response (null key -echo) — in both cases, all zeros means "no more events." - -**1E response layout:** `data_rsp.data[11:15]` = event 0's actual key; `data_rsp.data[15:19]` -= sample-count offset to the next event key (key1 = key0 + this offset). If offset == 0, -there is only one event. - -**Correct iteration pattern (confirmed empirically with live device, 2+ events):** - -`count_events` (browse mode only, no 5A): -``` -1E(all zeros) → key0, trailing0 ← trailing0 non-zero if event 1 exists -0A(key0) ← REQUIRED: establishes device context -1F(all zeros / browse=True) → key1 ← use all-zero params -0A(key1) ← REQUIRED before each advance -1F(all zeros) → null ← done -``` - -`get_events` (download mode, with 5A): -``` -1E(all zeros) → key0, trailing0 ← trailing0 non-zero if event 1 exists -0A(key0) ← REQUIRED: establishes device context -1E(token=0xFE) ← REQUIRED: arms device for 5A; CONFIRMED 4-2-26 + 4-3-26 -0C(key0) ← read waveform record -1F(token=0xFE) → [discard key] ← REQUIRED: arms 5A bulk stream state machine -POLL × 3 ← REQUIRED: 3 full POLL cycles before 5A (BW frames 68-73) -5A(key0) ← bulk stream; key0 used even though 1F already advanced -1F(all zeros / browse=True) → key1 ← USE THIS for loop iteration (browse=True returns correct key) -0A(key1) -1E(token=0xFE) ← re-arm for next event's 5A -0C(key1) -1F(token=0xFE) → [discard key] ← arm 5A -POLL × 3 -5A(key1) -1F(browse=True) → null ← done -``` - -**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):** -`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine. -Its returned key is cached as `arm_key4` in `get_events()`. - -`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**. If 5A timed out or failed, -sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next -event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the -next event's 5A probe to also time out with 0 bytes received). - -In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint: -- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call -- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort - -The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call, -reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received -but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in -the log but have very different root causes. - -**The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):** -The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE -has been issued between 0A and 0C. This step is present in EVERY download cycle in both -the 4-2-26 and 4-3-26 BW TX captures. - -**1F must come BEFORE 5A (FIXED 2026-04-06):** -BW always calls 1F (advance event) before starting the 5A bulk stream. 5A still uses the -pre-advance key — the device streams the waveform for the key that was set up with 0A+1E-arm+0C -even after 1F has moved the internal pointer to the next event. - -**POLL × 3 required before 5A (FIXED 2026-04-06):** -BW sends exactly 3 complete POLL (SUB 5B) probe+data cycles between the last 1F and the -first 5A probe frame. Confirmed from 4-2-26 BW TX capture frames 68-73. Without these -POLLs the device does not respond to the 5A probe. Use `proto.poll()` (not `startup()` — -`startup()` drains the boot string, which is only needed on initial connect). - -`advance_event(browse=True)` sends all-zero params; `advance_event()` default (browse=False) -sends token=0xFE and is NOT used by any caller. -`advance_event()` returns `(key4, event_data8)`. -Callers (`count_events`, `get_events`) loop while `data8[4:8] != b"\x00\x00\x00\x00"`. - -### SUB 1A — compliance config — orphaned send bug (FIXED, do not re-introduce) - -`read_compliance_config()` sends a 4-frame sequence (A, B, C, D) where: -- Frame A is a probe (no `recv_one` needed — device ACKs but returns no data page) -- Frames B, C, D each need a `recv_one` to collect the response - -**There must be NO extra `self._send(...)` call before the B/C/D recv loop without a -matching `recv_one()`.** An orphaned send shifts all receives one step behind, leaving -frame D's channel block (trigger_level_geo, alarm_level_geo, max_range_geo) unread and -producing only ~1071 bytes instead of ~2126. - -### SUB 1A — anchor search range - -`_decode_compliance_config_into()` locates sample_rate and record_time via the anchor -`b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`. Search range is `cfg[0:150]`. - -Do not narrow this to `cfg[40:100]` — the old range was only accidentally correct because -the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from -its real position (cfg[11]) into the 40–100 window. - -### Sample rate and DLE jitter in cfg data - -Sample rate 4096 (`0x1000`) causes DLE jitter: the frame carries `10 10 00` on the wire, -which unstuffs to `10 00` — 2 bytes instead of 3. This makes frame C 1 byte shorter and -shifts all subsequent absolute offsets by −1. The anchor approach is immune to this. -Do NOT use fixed absolute offsets for sample_rate or record_time. - -### TCP / cellular transport - -- Protocol bytes over TCP are bit-for-bit identical to RS-232. No wrapping. -- The modem (RV50/RV55) forwards bytes with up to ~1s buffering. `TcpTransport` uses - `read_until_idle(idle_gap=1.5s)` to drain the buffer completely before parsing. -- Cold-boot: unit sends the 16-byte ASCII string `"Operating System"` before entering - DLE-framed mode. The parser discards it (scans for DLE+STX). -- RV50/RV55 sends `\r\nRING\r\n\r\nCONNECT\r\n` over TCP to the caller even with - Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to - `S3FrameParser`. - -### Required ACEmanager settings (Sierra Wireless RV50/RV55) - -| Setting | Value | Why | -|---|---|---| -| Configure Serial Port | `38400,8N1` | Must match MiniMate baud | -| Flow Control | `None` | Hardware FC blocks TX if pins unconnected | -| **Quiet Mode** | **Enable** | **Critical.** Disabled injects `RING`/`CONNECT` onto serial, corrupting S3 handshake | -| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency | -| TCP Connect Response Delay | `0` | Non-zero silently drops first POLL frame | -| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect | -| DB9 Serial Echo | `Disable` | Echo corrupts the data stream | - ---- - -## Key confirmed field locations - -### SUB FE — Full Config (166 destuffed bytes) - -| Offset | Field | Type | Notes | -|---|---|---|---| -| 0x34 | firmware version string | ASCII | e.g. `"S338.17"` | -| 0x56–0x57 | calibration year | uint16 BE | `0x07E9` = 2025 | -| 0x0109 | aux trigger enabled | uint8 | `0x00` = off, `0x01` = on | - -### SUB 1A — Compliance Config (~2126 bytes total after 4-frame sequence) - -| Field | How to find it | -|---|---| -| sample_rate | uint16 BE at anchor − 2 | -| record_time | float32 BE at anchor + 10 | -| trigger_level_geo | float32 BE, located in channel block | -| alarm_level_geo | float32 BE, adjacent to trigger_level_geo | -| max_range_geo | float32 BE, adjacent to alarm_level_geo | -| setup_name | ASCII, null-padded, in cfg body | -| project / client / operator / sensor_location | ASCII, label-value pairs | - -Anchor: `b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`, search `cfg[0:150]` - -### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2]) - -**sub_code=0x10 (Waveform single-shot) — 9-byte timestamp header:** - -| Offset | Field | Type | -|---|---|---| -| 0 | day | uint8 | -| 1 | sub_code | uint8 (`0x10`) | -| 2 | month | uint8 | -| 3–4 | year | uint16 BE | -| 5 | unknown | uint8 (always 0) | -| 6 | hour | uint8 | -| 7 | minute | uint8 | -| 8 | second | uint8 | - -**sub_code=0x03 (Waveform continuous) — 10-byte timestamp header (1-byte shift):** - -Confirmed 2026-04-03 against Blastware event report (15:20:17 Apr 3 2026). -Raw wire bytes: `10 03 10 04 07 ea 00 0f 14 11` - -| Offset | Field | Type | Notes | -|---|---|---|---| -| 0 | unknown_a | uint8 | `0x10` observed | -| 1 | day | uint8 | doubles as sub_code position in 0x10 layout | -| 2 | unknown_b | uint8 | `0x10` observed | -| 3 | month | uint8 | | -| 4–5 | year | uint16 BE | | -| 6 | unknown | uint8 | | -| 7 | hour | uint8 | | -| 8 | minute | uint8 | | -| 9 | second | uint8 | | - -**Peak values (both record types):** - -| Location | Field | Type | -|---|---|---| -| `tran_pos - 12` | peak_vector_sum | float32 BE — label-relative, NOT fixed offset | -| `label + 6` | PPV per channel | float32 BE (search for `"Tran"`, `"Vert"`, `"Long"`, `"MicL"`) | - -PPV labels are NOT 4-byte aligned. The label-relative approach is the only reliable method. -`peak_vector_sum` is exactly 12 bytes before the `"Tran"` label — confirmed for both -sub_code=0x10 and sub_code=0x03. Do NOT use fixed offset 87 (only incidentally correct -for 0x10 records). - ---- - -## SFM REST API (sfm/server.py) - -### Live device endpoints (connect to device per-request) - -``` -GET /device/info?port=COM5 ← serial -GET /device/info?host=1.2.3.4&tcp_port=9034 ← cellular -GET /device/events?host=1.2.3.4&tcp_port=9034&baud=38400 -GET /device/event?host=1.2.3.4&tcp_port=9034&index=0 -GET /device/monitor/status?host=1.2.3.4&tcp_port=9034 ← battery, memory, mode -POST /device/monitor/start?host=1.2.3.4&tcp_port=9034 ← start recording -POST /device/monitor/stop?host=1.2.3.4&tcp_port=9034 ← stop recording -``` - -Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing). - -### DB read endpoints (query seismo_relay.db written by ach_server.py) - -``` -GET /db/units ← all known serials + summary stats -GET /db/events?serial=BE11529&from_dt=&to_dt=&limit= ← triggered events, newest first -GET /db/monitor_log?serial=BE11529&from_dt=&to_dt= ← monitoring intervals, newest first -GET /db/sessions?serial=BE11529&limit=50 ← ACH call-home sessions, newest first -PATCH /db/events/{id}/false_trigger?value=true ← flag/unflag false triggers -``` - -DB file: `bridges/captures/seismo_relay.db` (default; override with `--db-path` at startup). -All DB endpoints are read-only except `PATCH /db/events/{id}/false_trigger`. - ---- - -## Key wire captures (reference material) - -| Capture | Location | Contents | -|---|---|---| -| 1-2-26 | `bridges/captures/1-2-26/` | SUB 5A BW TX frames — confirmed 5A frame format, 11-byte params, DLE-aware checksum | -| 3-11-26 | `bridges/captures/3-11-26/` | Full compliance setup write, Aux Trigger capture | -| 3-31-26 | `bridges/captures/3-31-26/` | Complete event download cycle (148 BW / 147 S3 frames) — confirmed 1E/0A/0C/1F sequence; only 1 event stored so token=0xFE appeared to work | -| 4-3-26 | `bridges/captures/4-3-26/` | Browse-mode S3 capture with 2+ events — confirmed all-zero params for 1F, 1F response layout, null sentinel, 0A context requirement | - ---- - -## Write commands (SUBs 68–83) — confirmed 2026-04-07 - -All confirmed from 3-11-26 BW TX capture (`raw_bw_20260311_170151.bin`, frames 102–112). - -### Write frame format — CRITICAL: minimal DLE stuffing - -Write frames do NOT use the same DLE stuffing as read frames. **Only the BW_CMD byte -(0x10 at payload position [0]) is doubled on the wire. All other bytes — flags, sub, -offset, params, data, and checksum — are written RAW without stuffing.** - -Confirmed from all 11 write frames in the 3-11-26/170151 BW capture. ✅ 2026-04-07 - -Do NOT use `dle_stuff()` or `build_bw_frame()` for write commands. Use `build_bw_write_frame()`. - -``` -Actual wire layout: - [41] ACK - [02] STX - [10 10] BW_CMD doubled (ONLY DLE stuffing applied) - [00] flags - [sub] write command byte (0x68–0x83) - [00] always zero - [hi][lo] offset uint16 BE — RAW (not stuffed even if hi=0x10) - [params] 10 bytes — RAW - [data] variable-length write payload — RAW (0x10 bytes not stuffed) - [chk] checksum — RAW (not stuffed even if 0x10) - [03] ETX - -Total wire length = 2 (ACK+STX) + 2 (doubled BW_CMD) + 15 (raw header) + len(data) + 1 (chk) + 1 (ETX) - = 21 + len(data) -``` - -De-stuffed payload (logical; used for checksum computation only): -``` - [0] BW_CMD 0x10 - [1] flags 0x00 - [2] SUB write command byte (0x68–0x83) - [3] 0x00 always zero - [4] offset_hi - [5] offset_lo - [6:16] params 10-byte field (see per-SUB notes below) - [16:] data write payload (variable length; absent for confirm frames) - [-1] chk large-frame DLE-aware checksum (see below) -``` - -Write SUBs = Read SUB + 0x60. Response SUB follows the standard 0xFF − Request SUB rule. - -### Write frame checksum - -All write frames (data frames AND confirm frames) use the **large-frame DLE-aware checksum**: - -```python -chk = (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF -``` - -This is identical to the SUB 5A DLE-aware checksum. Confirmed against all 11 write frames in -the 3-11-26/170151 capture. ✅ 2026-04-07 - -Note: confirm frames contain no embedded 0x10 bytes, so both the standard SUM8 and the -DLE-aware formula produce the same result for them — but `build_bw_write_frame` always uses -the DLE-aware formula for consistency. - -### Write ack responses - -All device acks for write commands are **17-byte zero-data S3 frames**: - -``` -[DLE=0x10][STX=0x02][stuffed(header + chk)][bare ETX=0x03] -``` - -The data section carries zeros; RSP_SUB = 0xFF − write_request_SUB. - -### Write SUB constants and sequences - -| Request SUB | Function | Offset | Response SUB | -|---|---|---|---| -| 0x68 | Event index write | `data[1] + 2` | 0x97 | -| 0x73 | Confirm B (follows 68) | 0 | 0x8C | -| 0x71 | Compliance write (×3 chunks) | see below | 0x8E | -| 0x72 | Confirm A (follows 71×3, 69) | 0 | 0x8D | -| 0x82 | Trigger config write | `data[1] + 2` | 0x7D | -| 0x83 | Trigger confirm (follows 82) | 0 | 0x7C | -| 0x69 | Waveform data write | `data[1] + 2` | 0x96 | -| 0x74 | Confirm C (follows 69) | 0 | 0x8B | - -**Offset formula for single-chunk writes (0x68, 0x69, 0x82):** `offset = data[1] + 2` - -The write payload always begins with a 2-byte header `[0x00][length]`, where `data[1]` is -an embedded length field. The offset encodes this inner length + 2 (accounting for the -header bytes). Confirmed from all three single-chunk write frames in the 3-11-26 capture: - -| SUB | data[0:4] (hex) | data[1] | offset | total data len | -|---|---|---|---|---| -| 0x68 | `00 58 09 00` | 0x58=88 | 0x5A=90 | 91 | -| 0x82 | `00 1A D5 00` | 0x1A=26 | 0x1C=28 | 29 | -| 0x69 | `00 C8 08 00` | 0xC8=200 | 0xCA=202 | 204 | - -Full sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72` - -### SUB 71 — compliance write chunk parameters - -The full compliance config payload (~2128 bytes) is split into exactly 3 chunks. -Confirmed from 3-11-26 BW TX capture frames 104–108: - -| Chunk | Size | `offset` | `params` (10 bytes hex) | -|---|---|---|---| -| 1 (first) | 1027 bytes | 0x1004 | `00 00 00 00 00 00 00 00 00 00` | -| 2 (middle) | 1055 bytes | 0x1004 | `00 00 00 10 04 00 00 00 00 00` | -| 3 (last) | remainder | 0x002C | `00 00 08 00 00 00 00 00 00 00` | - -Total: 1027 + 1055 + N = 2082 + N bytes (N ≈ 46 for a standard 2128-byte config). - -After all 3 chunks are acked (SUB 0x8E each), send SUB 72 confirm → device acks 0x8D. - -### `build_bw_write_frame()` — framing.py - -```python -build_bw_write_frame(sub, data, *, offset=0, params=bytes(10)) -> bytes -``` - -Use for all write commands (SUBs 68–83) including confirm frames (data=b""). -**Do NOT use `build_bw_frame` for write commands** — it uses standard SUM8, not the -large-frame DLE-aware checksum required for writes. - -### `push_config_raw()` — client.py - -```python -client.push_config_raw(event_index_data, compliance_data, trigger_data, waveform_data) -``` - -Orchestrates the full write sequence in the confirmed order. All payloads are raw bytes -(no encoding performed at this level). A higher-level encoder that builds payloads from -a `ComplianceConfig` object is a future task. - ---- - -## Monitoring commands (SUBs 0x1C, 0x96, 0x97) — confirmed 2026-04-08 - -All confirmed from 4-8-26/2ndtry BW TX/S3 capture (clean start → 30s monitor → stop cycle). - -### SUB 0x1C — Monitor status read - -Standard two-step read (probe at offset 0x00, data at offset 0x2C). -Response SUB = 0xFF − 0x1C = **0xE3** (standard formula — no exception). - -**Payload length is 46–47 bytes IDLE, 48–49 bytes MONITORING** — not a reliable sole -indicator due to 1-byte jitter overlap at the boundary. - -**Monitoring flag (CONFIRMED 2026-04-09 — byte diff of all 144 data frames, 2ndtry capture):** -- `section[1] == 0x00` → unit is **idle** -- `section[1] == 0x10` → unit is **monitoring** - -This is `data[12]` (= `frame.data[12]`). The flag is 0x00 in all 36 IDLE_BEFORE frames, -0x10 in all 98 MONITORING frames, and 0x00 in all 10 IDLE_AFTER frames — 100% accurate. - -**HISTORY OF THIS FIELD (do not re-derive):** The original implementation used `section[1]`. -A re-analysis in the prior session incorrectly concluded `section[1]` is always 0x00 and -"corrected" the flag to `section[6]`, which has non-binary values (0xea idle, 0x07 monitoring) -and is device-specific. The 2026-04-09 re-analysis confirms `section[1]` was right. - -**IMPORTANT — `frame.data` has checksum already stripped** by `S3FrameParser._finalise()` -(`raw_payload = body[:-1]`; `data = raw_payload[5:]`). There is NO trailing checksum byte in -`section`. All relative-from-end offsets must account for this. - -Battery and memory fields are present in **both** states: - -| Offset (relative to end) | Field | Type | Notes | -|---|---|---|---| -| `section[-10:-8]` | battery voltage × 100 | uint16 BE | `0x02A8` = 680 → 6.80 V | -| `section[-8:-4]` | memory total (bytes) | uint32 BE | e.g. 983026 ≈ 960 KB | -| `section[-4:]` | memory free (bytes) | uint32 BE | decreases as events are stored | - -### SESSION_RESET signal (`41 03`) — required for monitoring units - -Confirmed from 4-8-26 BW TX captures: Blastware sends a 2-byte `41 03` (ACK + ETX, -no STX) immediately before the first POLL probe AND between the probe and data frames. -This signal is **required to wake units that are actively monitoring** — without it -the unit does not respond to POLL over TCP. Harmless for idle units. - -`SESSION_RESET = bytes([0x41, 0x03])` is defined in `framing.py` and sent by -`protocol.startup()` before and between POLL frames. - -### SUB 0x96 — Start monitoring - -Single write frame, **no data payload** (empty body). -Response SUB = 0xFF − 0x96 = **0x69**. - -Wire bytes (confirmed frame 92 of 2ndtry BW capture): -``` -41 02 10 10 00 96 00 00 00 00 00 00 00 00 00 00 00 00 00 a6 03 -``` - -### SUB 0x97 — Stop monitoring - -Single write frame, **no data payload** (empty body). -Response SUB = 0xFF − 0x97 = **0x68**. - -Wire bytes (confirmed frame 305 of 2ndtry BW capture): -``` -41 02 10 10 00 97 00 00 00 00 00 00 00 00 00 00 00 00 00 a7 03 -``` - -Both start and stop acks are standard 17-byte zero-data S3 frames. - -### On-device sensor check behavior (confirmed 2026-04-08) - -Confirmed from 4-8-26/sensor-check BW+S3 capture (Blastware "Unit Channel Test" comms -check issued while unit was performing its on-device sensor check). - -**Unit IS reachable during on-device sensor check** — POLL (SUB 5B) responded normally -throughout. However, the unit partially handled channel-test commands (SUB 0x0E) for -channels 0–4 and then went **silent for ~40 seconds** while the sensor check ran, before -resuming responses for channels 5–7 and the trigger test (SUB 0x98). - -Key findings: -- On-device sensor check duration: approximately **40 seconds** (log gap `18:40:48` → `18:41:28`) -- Unit IS reachable for POLL during the check window — SESSION_RESET + POLL works -- Partial command responses during check are possible (device may buffer some, drop others) -- The Blastware "Unit Channel Test" (remote comms check, SUBs 0x0E + 0x98) is a SEPARATE - operation from the on-device check — it is a passive remote read; the unit's screen does - not change during a remote comms check - -**SFM behavior after `POST /device/monitor/start`:** `_pollMonitorConfirm()` polls -`/device/monitor/status` every 5 s for up to 60 s, updating the badge on each poll. -Status will show MONITORING once `section[1]` flips to `0x10`. - -### SUBs known from sensor-check capture (4-8-26) — NOT YET IMPLEMENTED - -| BW SUB | RSP SUB | Function | Notes | -|---|---|---|---| -| 0x15 | 0xEA | Serial number / short ID | 2-step read; data offset = 0x0A (10 bytes); confirmed serial `"BE11529"` at `data[11+5:]` | -| 0x01 | 0xFE | Device info block | 2-step read; data offset = 0x98 (152 bytes); payload includes serial + firmware + float config fields | -| 0x0E | 0xF1 | Channel sensor data | 2-step read; channel selector in `params[6:8]` (`0x0000`–`0x0007`); data length 0x0A per channel; used by Blastware "Unit Channel Test" — see docs/ for details | -| 0x98 | 0x67 | Trigger test | Single probe frame (`params[0]=0xFF`); sent twice per test cycle; all-zero data response; used after 0x0E channel scan | - -Blastware's "Unit Channel Test" sequence: `POLL×N → 0x15 → 0x01 → 0x08 → 0x01 → 0x0E×8 → 0x98×2 → 0x0E×8` (repeat pass with live ADC readings). - ---- - -## Compliance config field inventory (from Blastware UI, 2026-04-08) - -Fields visible in the Blastware Compliance Setup dialog — most are NOT YET decoded to byte -offsets in the raw 1A/E5 payload. Only fields with `✅` have confirmed offsets in the code. - -**Recording Setup tab:** -- Recording Mode: Continuous / Single Shot / Histogram (enum) -- Record Stop Mode: Fixed Record Time / Auto / Manual Stop (enum) -- Sample Rate: Standard 1024 / Fast 2048 / Faster 4096 sps ✅ (anchor−2) -- Record Time: float, seconds ✅ (anchor+10) -- Histogram Interval: 5 / 15 / 30 / 60 minutes (enum, mode-gated) -- Storage Mode: Save All Data / Save Triggered (enum) -- Geophone Type: Standard Triaxial / 4.5 Hz (bool/enum) -- Geophone Channels: Enable all geophones (bool), Trigger Source (bool) -- Chan 1-3 Trigger Level (float, in/s) ✅ (`trigger_level_geo`) -- Chan 1-3 Maximum Range: Normal 10.000 / 1.25 in/s (enum) ✅ (`max_range_geo`) -- Microphone Channels: Enable all microphones (bool), Trigger Source (bool) -- Chan 4 Trigger Level (dB or psi depending on units) - -**Notes tab:** -- Enable User Notes (bool) -- Project, Client, User Name, Seis Loc (ASCII strings) ✅ (sourced from A5 frame 7 via 5A) -- Enable Extended Notes (bool); Extended Notes text; Extended Notes Title -- Enable Job Number (bool); Job Number (int) -- Enable Scaled Distance (bool); Distance from Blast (float); Charge Weight (float) — Scaled Distance is derived - -**Special Setups tab:** -- Unit Timer: Timer Mode (Off/On), Start Date/Time, Stop Date/Time -- Self Check: Mode (Off/On), Time (HH:MM) -- Sensor Check: **Before monitoring** / After each event / **Disabled** ❓ (byte offset unknown) -- Measurement Units: Imperial / Metric -- Show Mic units in dB (bool) -- Time Format: 24 Hour / 12 Hour (AM/PM) -- Backlight on Time (seconds) ✅ (event index block +75) -- Power Saving Timeout (minutes) ✅ (event index block +83) -- Monitoring LCD Cycle ✅ (event index block +84:86) -- Set unit time with setup (bool) - -The "Sensor Check" dropdown (`Before monitoring` / `After each event` / `Disabled`) has NOT -been located in the raw config bytes. The user's unit always runs with `Before monitoring`. -Full compliance config encoder is a future task. - ---- - ---- - -## Erase-all protocol (SUBs 0xA3/0xA2/0x06) — confirmed 2026-04-11 - -Full sequence confirmed from 4-11-26 MITM capture of a live Blastware ACH session -(`bridges/captures/mitm/ach_mitm_20260411_001912/`). - -### Wire sequence - -``` -BW → device: SUB 0xA3 params=00 00 00 00 00 00 00 FE 00 00 (begin erase) -device → BW: SUB 0x5C (ack) -BW → device: SUB 0x1C probe (offset=0x00) -device → BW: SUB 0xE3 (probe ack) -BW → device: SUB 0x1C data (offset=0x2C) -device → BW: SUB 0xE3 (monitor status response) -BW → device: SUB 0x06 probe (offset=0x00, params same) -device → BW: SUB 0xF9 (probe ack) -BW → device: SUB 0x06 data (offset=0x24) -device → BW: SUB 0xF9 (36-byte storage range response) -BW → device: SUB 0xA2 params=00 00 00 00 00 00 00 FE 00 00 (confirm erase) -device → BW: SUB 0x5D (ack — device memory is now cleared) -``` - -All frames use standard `build_bw_frame` (not write-format). Response SUBs follow the -standard `0xFF - SUB` formula; no exceptions. - -### SUB 0x06 — event storage range response (36 bytes) - -The 36-byte response body ends with two 4-byte event keys: - -| Offset (from end) | Field | Notes | -|---|---|---| -| `[-8:-4]` | first stored event key | `01110000` when empty | -| `[-4:]` | last stored event key | `01110000` when empty | - -Before erase: ends with ` ` (e.g. `0111ea60 0111eaa6`). -After erase: both bytes read `01110000` — device's empty/reset sentinel. - -### Post-erase key counter reset - -After a successful erase, the device resets its event counter. New events start from -key `0x01110000` again — the same key as the very first event ever recorded. This means -key-based deduplication in the ACH server must account for key reuse: - -- After our own erase: `ach_state.json` `downloaded_keys` and `max_downloaded_key` are - cleared so the next session starts fresh. -- After an external erase: the ACH server detects it by comparing `max(device_keys)` to - `max_downloaded_key` from state. If the device max has rolled back below the historical - max, all current device keys are treated as new regardless of `seen_keys`. - -### ACH server state format (v0.9.0) - -`bridges/captures/ach_state.json`: -```json -{ - "BE11529": { - "downloaded_keys": ["01110000", "0111245a"], - "max_downloaded_key": "0111245a", - "last_seen": "2026-04-11T01:04:36", - "serial": "BE11529", - "peer": "63.43.212.232:51920" - } -} -``` - -`max_downloaded_key` is the high-water mark — the largest key ever downloaded from the -unit. It is NOT reset when events are erased from the device (only when our server does -the erase). Used for post-erase detection. - ---- - -## Monitor log entries — SUB 0x0A partial records (confirmed 2026-04-11) - -Confirmed from 4-11-26 MITM capture: 12 partial records (record type `0x2C`) and 7 full -event records (record type `0x46`) across 19 total 0x0A responses. - -### Record type detection - -`read_waveform_header()` returns `(raw_data, length)` where `raw_data = data_rsp.data` -(the full payload including prefix bytes). The record type is at `raw_data[0]`: - -| Value | Type | How to process | -|---|---|---| -| `0x46` | Full triggered event | Normal download: 0C → 5A → 1F | -| `0x2C` | Monitor log entry (partial) | No 0C/5A; decode inline from 0A payload | - -Length heuristic: `length < 0x40` (64) reliably identifies partial records across all -observed captures. Both checks (`raw_data[0] == 0x2C` and `length < 0x40`) are used. - -### SUB 0x0A partial record (0x2C) payload layout - -All offsets are from `raw_data` (the full `data_rsp.data` array including the 11-byte -prefix before the actual header bytes start). - -``` -raw_data[0] = 0x2C ← record type (partial / monitor log) -raw_data[1:11] = prefix bytes (vary; contain key4 copy, flags, length) -raw_data[11:] = timestamp and ASCII metadata payload -``` - -**Timestamp auto-detection** (confirmed from 4-11-26 capture): - -``` -raw_data[11] == 0x10 → 10-byte sub_code=0x03 format (continuous mode) -raw_data[11] != 0x10 → 9-byte sub_code=0x10 format (single-shot mode) -``` - -**9-byte timestamp format (sub_code=0x10):** - -| Byte | Field | -|---|---| -| 0 | day | -| 1 | `0x10` (sub_code marker) | -| 2 | month | -| 3–4 | year (uint16 BE) | -| 5 | unknown (0x00) | -| 6 | hour | -| 7 | minute | -| 8 | second | - -**10-byte timestamp format (sub_code=0x03):** - -| Byte | Field | -|---|---| -| 0 | `0x10` (marker) | -| 1 | day | -| 2 | `0x10` (marker) | -| 3 | month | -| 4–5 | year (uint16 BE) | -| 6 | unknown (0x00) | -| 7 | hour | -| 8 | minute | -| 9 | second | - -**Two timestamps:** Each partial record contains two timestamps — `start_time` and -`stop_time` — stored consecutively: -- `ts1` (start) at `raw_data[ts_offset : ts_offset + ts_size]` where `ts_offset = 11` -- `ts2` (stop) at `raw_data[ts1_end : ts1_end + ts_size]` - -**Edge case — 1-byte gap between timestamps:** Occurs when ts1 and ts2 share the same -minute:second. If `try_ts(raw_data[ts1_end:])` fails, try `try_ts(raw_data[ts1_end+1:])`. -Confirmed in frames 121, 161, 165 of the 4-11-26 MITM capture. Frame 121 still shows 0s -duration (both decode to 16:02:00) — the extra byte appears in all same-second cases. - -**ASCII metadata after timestamps:** -``` - BE\x00Geo: in/s ... -``` - -- Serial: scan for `b"BE"`, read until `b"\x00"` (e.g. `"BE11529"`) -- Geo threshold: scan for `b"Geo: "`, read float until next space (e.g. `0.254` in/s) - -A separator of variable length (4–5 bytes of `\x00` + flags) sits between the two -timestamps and the ASCII region. The `b"BE"` anchor scan is robust to separator length -variation. - -### `_decode_0a_partial_header(raw_data, index, key4)` — client.py - -Returns a `MonitorLogEntry` or `None`. Called by `get_monitor_log_entries()` for each -event key whose 0x0A response has `raw_data[0] == 0x2C` or `length < 0x40`. - -### `MiniMateClient.get_monitor_log_entries(skip_keys=None)` — client.py - -Browse-mode walk: `1E → 0A → check type → decode if partial → 1F`. No 0x0C or 5A reads -performed. Full (0x46) records are skipped without decoding. Returns `list[MonitorLogEntry]`. - -`skip_keys` (optional `set[str]`): keys in this set are still advanced through the walk -(to avoid disrupting the iteration sequence), but no `MonitorLogEntry` is created for them. - -### `MonitorLogEntry` model — models.py - -```python -@dataclass -class MonitorLogEntry: - index: int # 0-based position - key: str # 8-hex event key - start_time: Optional[datetime.datetime] = None - stop_time: Optional[datetime.datetime] = None - serial: Optional[str] = None - geo_threshold_ips: Optional[float] = None - raw_header: Optional[bytes] = field(default=None, repr=False) - - @property - def duration_seconds(self) -> Optional[float]: ... -``` - -### ACH server integration (v0.10.0) - -After `get_events()`, the ACH server calls `get_monitor_log_entries(skip_keys=seen_keys)`. -New entries are saved to `monitor_log.json` in the session directory. Monitor log keys are -included in `current_keys` for state persistence so they are not re-processed on the next -call-home. - ---- - -## What's next - -- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable -- **Histograms** — decode histogram-mode A5 data (noise floor tracking) -- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object -- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring) -- Modem manager — push RV50/RV55 configs via Sierra Wireless API -- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't - resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred) +# CLAUDE.md — seismo-relay + +Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for +managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem +(Sierra Wireless RV50 / RV55). Current version: **v0.12.0**. + +--- + +## Project layout + +``` +minimateplus/ ← Python client library (primary focus) + transport.py ← SerialTransport, TcpTransport + framing.py ← DLE codec, frame builders, S3FrameParser + protocol.py ← MiniMateProtocol — wire-level read/write methods + client.py ← MiniMateClient — high-level API (connect, get_events, …) + models.py ← DeviceInfo, EventRecord, ComplianceConfig, … + +sfm/server.py ← FastAPI REST server exposing device data over HTTP +seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs) +docs/ + instantel_protocol_reference.md ← reverse-engineered protocol spec ("the Rosetta Stone") +CHANGELOG.md ← version history +``` + +--- + +## Current implementation state (v0.10.0) + +Full read pipeline + write pipeline + erase pipeline + monitor log working end-to-end over TCP/cellular: + +| Step | SUB | Status | +|---|---|---| +| POLL / startup handshake | 5B | ✅ | +| Serial number | 15 | ✅ | +| Full config (firmware, calibration date, etc.) | FE | ✅ | +| Compliance config (record time, sample rate, geo thresholds) | 1A | ✅ | +| Event index | 08 | ✅ | +| Event header / first key | 1E | ✅ | +| Waveform header | 0A | ✅ | +| Waveform record (peaks, timestamp, project) | 0C | ✅ | +| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 | +| Event advance / next key | 1F | ✅ | +| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 | +| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 | +| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ **new v0.10.0** | + +`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F` + +`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72` + +`delete_all_events()` erase sequence: `0xA3 → 0x1C → 0x06 → 0xA2` + +--- + +## Protocol fundamentals + +### DLE framing + +``` +BW→S3 (our requests): [ACK=0x41] [STX=0x02] [stuffed payload+chk] [ETX=0x03] +S3→BW (device replies): [DLE=0x10] [STX=0x02] [stuffed payload+chk] [bare ETX=0x03] +``` + +- **DLE stuffing rule:** any literal `0x10` byte in the payload is doubled on the wire + (`0x10` → `0x10 0x10`). This includes the checksum byte. +- **Inner-frame terminators:** large S3 responses (A4, E5) contain embedded sub-frames + using `DLE+ETX` as inner terminators. The outer parser treats `DLE+ETX` inside a frame + as literal data — the bare ETX is the ONLY real frame terminator. +- **Response SUB rule:** `response_SUB = 0xFF - request_SUB` + (no known exceptions — earlier note claiming `1C` → `6E` was WRONG; `1C` → `0xE3` confirmed across 338 frames in 4-8-26 captures) +- **Two-step read pattern:** every read command is sent twice — probe step (`offset=0x00`, + get length) then data step (`offset=DATA_LENGTH`, get payload). All data lengths are + hardcoded constants, not read from the probe response. + +### De-stuffed payload header + +``` +BW→S3 (request): + [0] CMD 0x10 + [1] flags 0x00 + [2] SUB command byte + [3] 0x00 always zero + [4] 0x00 always zero + [5] OFFSET 0x00 for probe step; DATA_LENGTH for data step + [6-15] params (key, token, etc. — see helpers in framing.py) + +S3→BW (response): + [0] CMD 0x00 + [1] flags 0x10 + [2] SUB response sub byte + [3] PAGE_HI + [4] PAGE_LO + [5+] data +``` + +--- + +## Critical protocol gotchas (hard-won — do not re-derive) + +### SUB 5A — bulk waveform stream — NON-STANDARD frame format + +**Always use `build_5a_frame()` for SUB 5A. Never use `build_bw_frame()` for SUB 5A.** + +`build_bw_frame` produces WRONG output for 5A for two reasons: + +1. **`offset_hi = 0x10` must NOT be DLE-stuffed.** Blastware sends the offset field raw. + `build_bw_frame` would stuff it to `10 10` on the wire — the device silently ignores + the frame. `build_5a_frame` writes it as a bare `10`. + +2. **DLE-aware checksum.** When computing the checksum, `10 XX` pairs in the stuffed + section contribute only `XX` to the running sum; lone bytes contribute normally. This + differs from the standard SUM8-of-destuffed-payload that all other commands use. + +Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26 +BW TX capture. All 10 frames verified. + +### SUB 5A — chunk counter is monotonic (CORRECTED 2026-04-06) + +**Chunk counters are `chunk_num * 0x0400` for ALL chunks including chunk 1.** + +The 4-2-26 BW TX capture showed `counter=0x1004` for chunk 1 of event key `01110000`, which +led to `_CHUNK1_COUNTER = 0x1004` being hardcoded as a special case. This was a Blastware +artifact, not a protocol requirement. Empirical test 2026-04-06: with `counter=0x1004` for +chunk 1 the device times out (120 s); with `counter=0x0400` (= `1 * 0x0400`) it responds +immediately and streams all frames correctly. + +The 4-3-26 capture confirms the pattern for a second event (key `0111245a`): +chunk 1 = `0x245A`, chunk 2 = `0x285A`, chunk 3 = `0x2C5A` (each +0x0400). Blastware's +true formula is `key4[2:4] + n * 0x0400` — but since `key4[2:4]` of the first event is +`0x0000`, `n * 0x0400` produces the right result. The device does not strictly validate the +counter and streams data for any valid 5A request; using `chunk_num * 0x0400` is correct. + +### SUB 5A — params are 11 bytes for chunk frames, 10 for termination + +`bulk_waveform_params()` returns 11 bytes (extra trailing `0x00`). The 11th byte was +confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes. +Do not swap them. + +### SUB 5A — event-time metadata lives in A5 frame 7 + +The bulk stream sends 9+ A5 response frames. Frame 7 (0-indexed) contains the compliance +setup as it existed when the event was recorded: + +``` +"Project:" → project description +"Client:" → client name ← NOT in the 0C record +"User Name:" → operator name ← NOT in the 0C record +"Seis Loc:" → sensor location ← NOT in the 0C record +"Extended Notes"→ notes +``` + +**IMPORTANT — 5A "Project:" is session-start config, NOT per-event (confirmed 2026-04-05):** +The "Project:" string in the A5 frame 7 payload reflects the compliance setup from when +the *monitoring session first started*, not the individual event's project name. The per- +event project name is correctly stored in the 210-byte 0C waveform record and must be +used as the authoritative source. `_decode_a5_metadata_into` therefore only sets +`project` from 5A when 0C didn't already supply one. + +"Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C +record — 5A remains the sole source for those fields and they are set unconditionally. + +`stop_after_metadata=True` (default) stops the 5A loop as soon as `b"Project:"` appears, +then sends the termination frame. + +### SUB 5A — end-of-stream signal (confirmed 2026-04-06) + +After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to +the next chunk request, then goes silent. This is the natural end-of-stream indicator — NOT +a complete A5 frame. `S3FrameParser.bytes_fed` will be 1; no frame is assembled. + +Handling: on `TimeoutError`, if `bytes_fed > 0` AND frames were already collected, treat as +graceful end-of-stream, break the loop, and proceed to the termination frame. If `bytes_fed +== 0` with no prior frames, it is a genuine transport failure — re-raise. + +**Chunk recv timeout must be 10 s, not the default 120 s.** Chunks arrive within ~1 s each. +Using 120 s causes a ~2-minute stall at every end-of-stream detection. The `_recv_one` call +in the chunk loop passes `timeout=10.0` explicitly. + +**Typical chunk count (BE11529, 1024 sps):** A 9,306-sample event produces 35 chunks before +end-of-stream. Chunks with uniform 1,036-byte data are all-zero ADC samples (post-event +silence). Only the initial variable-size chunks contain actual signal. + +### SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06) + +`_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from the +9-frame original blast capture where frame 9 was assumed to be a terminator. For current +35-frame streams, fi==9 is live waveform data (~133 sample-sets were being dropped). +Removed. Terminator detection is via `page_key == 0x0000` in `read_bulk_waveform_stream`, +not frame index. + +### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce) + +**token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and +4-3-26 BW TX captures confirm it belongs at **`params[7]`** (raw: `00 00 00 00 00 00 00 fe 00 00`). +With the wrong position the device ignores the token and 1F returns null immediately. + +**1F token depends on context:** In browse mode (no 5A), use all-zero params (`browse=True`). +In download mode (get_events with 5A), use token=0xFE (`browse=False`) — this is required to +arm the device's 5A bulk stream state machine. The earlier "empirical" test showing token=0xFE +returns null was done WITHOUT the 1E(arm) step; that test is invalid. BW always uses 1F(0xFE) +in download mode. `count_events` uses `browse=True` (no 5A needed). + +**0A context requirement:** `advance_event()` (1F) only returns a valid next-event key +when a preceding `read_waveform_header()` (0A) call has established device waveform +context for the current key. Call 0A before every event in the loop, not just the first. +Calling 1F cold (after only 1E, with no 0A) returns the null sentinel regardless of how +many events are stored. + +**1F response layout:** The next event's key IS at `data_rsp.data[11:15]` (= payload[16:20]). +Confirmed from 4-3-26 browse-mode S3 captures: +``` +1F after 0A(key0=01110000): data[11:15]=0111245a data[15:19]=00001e36 ← valid +1F after 0A(key1=0111245a): data[11:15]=01114290 data[15:19]=00000046 ← valid +1F null sentinel: data[11:15]=00000000 data[15:19]=00000000 ← done +``` + +**Null sentinel:** `data8[4:8] == b"\x00\x00\x00\x00"` (= `data_rsp.data[15:19]`) +works for BOTH 1E trailing (offset to next event key) and 1F response (null key +echo) — in both cases, all zeros means "no more events." + +**1E response layout:** `data_rsp.data[11:15]` = event 0's actual key; `data_rsp.data[15:19]` += sample-count offset to the next event key (key1 = key0 + this offset). If offset == 0, +there is only one event. + +**Correct iteration pattern (confirmed empirically with live device, 2+ events):** + +`count_events` (browse mode only, no 5A): +``` +1E(all zeros) → key0, trailing0 ← trailing0 non-zero if event 1 exists +0A(key0) ← REQUIRED: establishes device context +1F(all zeros / browse=True) → key1 ← use all-zero params +0A(key1) ← REQUIRED before each advance +1F(all zeros) → null ← done +``` + +`get_events` (download mode, with 5A): +``` +1E(all zeros) → key0, trailing0 ← trailing0 non-zero if event 1 exists +0A(key0) ← REQUIRED: establishes device context +1E(token=0xFE) ← REQUIRED: arms device for 5A; CONFIRMED 4-2-26 + 4-3-26 +0C(key0) ← read waveform record +1F(token=0xFE) → [discard key] ← REQUIRED: arms 5A bulk stream state machine +POLL × 3 ← REQUIRED: 3 full POLL cycles before 5A (BW frames 68-73) +5A(key0) ← bulk stream; key0 used even though 1F already advanced +1F(all zeros / browse=True) → key1 ← USE THIS for loop iteration (browse=True returns correct key) +0A(key1) +1E(token=0xFE) ← re-arm for next event's 5A +0C(key1) +1F(token=0xFE) → [discard key] ← arm 5A +POLL × 3 +5A(key1) +1F(browse=True) → null ← done +``` + +**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):** +`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine. +Its returned key is cached as `arm_key4` in `get_events()`. + +`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**. If 5A timed out or failed, +sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next +event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the +next event's 5A probe to also time out with 0 bytes received). + +In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint: +- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call +- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort + +The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call, +reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received +but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in +the log but have very different root causes. + +**The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):** +The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE +has been issued between 0A and 0C. This step is present in EVERY download cycle in both +the 4-2-26 and 4-3-26 BW TX captures. + +**1F must come BEFORE 5A (FIXED 2026-04-06):** +BW always calls 1F (advance event) before starting the 5A bulk stream. 5A still uses the +pre-advance key — the device streams the waveform for the key that was set up with 0A+1E-arm+0C +even after 1F has moved the internal pointer to the next event. + +**POLL × 3 required before 5A (FIXED 2026-04-06):** +BW sends exactly 3 complete POLL (SUB 5B) probe+data cycles between the last 1F and the +first 5A probe frame. Confirmed from 4-2-26 BW TX capture frames 68-73. Without these +POLLs the device does not respond to the 5A probe. Use `proto.poll()` (not `startup()` — +`startup()` drains the boot string, which is only needed on initial connect). + +`advance_event(browse=True)` sends all-zero params; `advance_event()` default (browse=False) +sends token=0xFE and is NOT used by any caller. +`advance_event()` returns `(key4, event_data8)`. +Callers (`count_events`, `get_events`) loop while `data8[4:8] != b"\x00\x00\x00\x00"`. + +### SUB 1A — compliance config — orphaned send bug (FIXED, do not re-introduce) + +`read_compliance_config()` sends a 4-frame sequence (A, B, C, D) where: +- Frame A is a probe (no `recv_one` needed — device ACKs but returns no data page) +- Frames B, C, D each need a `recv_one` to collect the response + +**There must be NO extra `self._send(...)` call before the B/C/D recv loop without a +matching `recv_one()`.** An orphaned send shifts all receives one step behind, leaving +frame D's channel block (trigger_level_geo, alarm_level_geo, max_range_geo) unread and +producing only ~1071 bytes instead of ~2126. + +### SUB 1A — anchor search range + +`_decode_compliance_config_into()` locates sample_rate and record_time via the anchor +`b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`. Search range is `cfg[0:150]`. + +Do not narrow this to `cfg[40:100]` — the old range was only accidentally correct because +the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from +its real position (cfg[11]) into the 40–100 window. + +### Sample rate and DLE jitter in cfg data + +Sample rate 4096 (`0x1000`) causes DLE jitter: the frame carries `10 10 00` on the wire, +which unstuffs to `10 00` — 2 bytes instead of 3. This makes frame C 1 byte shorter and +shifts all subsequent absolute offsets by −1. The anchor approach is immune to this. +Do NOT use fixed absolute offsets for sample_rate or record_time. + +### TCP / cellular transport + +- Protocol bytes over TCP are bit-for-bit identical to RS-232. No wrapping. +- The modem (RV50/RV55) forwards bytes with up to ~1s buffering. `TcpTransport` uses + `read_until_idle(idle_gap=1.5s)` to drain the buffer completely before parsing. +- Cold-boot: unit sends the 16-byte ASCII string `"Operating System"` before entering + DLE-framed mode. The parser discards it (scans for DLE+STX). +- RV50/RV55 sends `\r\nRING\r\n\r\nCONNECT\r\n` over TCP to the caller even with + Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to + `S3FrameParser`. + +### Required ACEmanager settings (Sierra Wireless RV50/RV55) + +| Setting | Value | Why | +|---|---|---| +| Configure Serial Port | `38400,8N1` | Must match MiniMate baud | +| Flow Control | `None` | Hardware FC blocks TX if pins unconnected | +| **Quiet Mode** | **Enable** | **Critical.** Disabled injects `RING`/`CONNECT` onto serial, corrupting S3 handshake | +| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency | +| TCP Connect Response Delay | `0` | Non-zero silently drops first POLL frame | +| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect | +| DB9 Serial Echo | `Disable` | Echo corrupts the data stream | + +--- + +## Key confirmed field locations + +### SUB FE — Full Config (166 destuffed bytes) + +| Offset | Field | Type | Notes | +|---|---|---|---| +| 0x34 | firmware version string | ASCII | e.g. `"S338.17"` | +| 0x56–0x57 | calibration year | uint16 BE | `0x07E9` = 2025 | +| 0x0109 | aux trigger enabled | uint8 | `0x00` = off, `0x01` = on | + +### SUB 1A — Compliance Config (~2126 bytes total after 4-frame sequence) + +| Field | How to find it | +|---|---| +| sample_rate | uint16 BE at anchor − 2 | +| record_time | float32 BE at anchor + 10 | +| trigger_level_geo | float32 BE, located in channel block | +| alarm_level_geo | float32 BE, adjacent to trigger_level_geo | +| max_range_geo | float32 BE, adjacent to alarm_level_geo — **⚠ value and units UNKNOWN** (reads 6.206053 but doesn't match either UI range option; may not be the ADC full-scale — see GitHub issue) | +| setup_name | ASCII, null-padded, in cfg body | +| project / client / operator / sensor_location | ASCII, label-value pairs | + +Anchor: `b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`, search `cfg[0:150]` + +### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2]) + +**sub_code=0x10 (Waveform single-shot) — 9-byte timestamp header:** + +| Offset | Field | Type | +|---|---|---| +| 0 | day | uint8 | +| 1 | sub_code | uint8 (`0x10`) | +| 2 | month | uint8 | +| 3–4 | year | uint16 BE | +| 5 | unknown | uint8 (always 0) | +| 6 | hour | uint8 | +| 7 | minute | uint8 | +| 8 | second | uint8 | + +**sub_code=0x03 (Waveform continuous) — 10-byte timestamp header (1-byte shift):** + +Confirmed 2026-04-03 against Blastware event report (15:20:17 Apr 3 2026). +Raw wire bytes: `10 03 10 04 07 ea 00 0f 14 11` + +| Offset | Field | Type | Notes | +|---|---|---|---| +| 0 | unknown_a | uint8 | `0x10` observed | +| 1 | day | uint8 | doubles as sub_code position in 0x10 layout | +| 2 | unknown_b | uint8 | `0x10` observed | +| 3 | month | uint8 | | +| 4–5 | year | uint16 BE | | +| 6 | unknown | uint8 | | +| 7 | hour | uint8 | | +| 8 | minute | uint8 | | +| 9 | second | uint8 | | + +**Peak values (both record types):** + +| Location | Field | Type | +|---|---|---| +| `tran_pos - 12` | peak_vector_sum | float32 BE — label-relative, NOT fixed offset | +| `label + 6` | PPV per channel | float32 BE (search for `"Tran"`, `"Vert"`, `"Long"`, `"MicL"`) | + +PPV labels are NOT 4-byte aligned. The label-relative approach is the only reliable method. +`peak_vector_sum` is exactly 12 bytes before the `"Tran"` label — confirmed for both +sub_code=0x10 and sub_code=0x03. Do NOT use fixed offset 87 (only incidentally correct +for 0x10 records). + +--- + +## SFM REST API (sfm/server.py) + +### Live device endpoints (connect to device per-request) + +``` +GET /device/info?port=COM5 ← serial +GET /device/info?host=1.2.3.4&tcp_port=9034 ← cellular +GET /device/events?host=1.2.3.4&tcp_port=9034&baud=38400 +GET /device/event?host=1.2.3.4&tcp_port=9034&index=0 +GET /device/monitor/status?host=1.2.3.4&tcp_port=9034 ← battery, memory, mode +POST /device/monitor/start?host=1.2.3.4&tcp_port=9034 ← start recording +POST /device/monitor/stop?host=1.2.3.4&tcp_port=9034 ← stop recording +``` + +Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing). + +### DB read endpoints (query seismo_relay.db written by ach_server.py) + +``` +GET /db/units ← all known serials + summary stats +GET /db/events?serial=BE11529&from_dt=&to_dt=&limit= ← triggered events, newest first +GET /db/monitor_log?serial=BE11529&from_dt=&to_dt= ← monitoring intervals, newest first +GET /db/sessions?serial=BE11529&limit=50 ← ACH call-home sessions, newest first +PATCH /db/events/{id}/false_trigger?value=true ← flag/unflag false triggers +``` + +DB file: `bridges/captures/seismo_relay.db` (default; override with `--db-path` at startup). +All DB endpoints are read-only except `PATCH /db/events/{id}/false_trigger`. + +--- + +## Key wire captures (reference material) + +| Capture | Location | Contents | +|---|---|---| +| 1-2-26 | `bridges/captures/1-2-26/` | SUB 5A BW TX frames — confirmed 5A frame format, 11-byte params, DLE-aware checksum | +| 3-11-26 | `bridges/captures/3-11-26/` | Full compliance setup write, Aux Trigger capture | +| 3-31-26 | `bridges/captures/3-31-26/` | Complete event download cycle (148 BW / 147 S3 frames) — confirmed 1E/0A/0C/1F sequence; only 1 event stored so token=0xFE appeared to work | +| 4-3-26 | `bridges/captures/4-3-26/` | Browse-mode S3 capture with 2+ events — confirmed all-zero params for 1F, 1F response layout, null sentinel, 0A context requirement | + +--- + +## Write commands (SUBs 68–83) — confirmed 2026-04-07 + +All confirmed from 3-11-26 BW TX capture (`raw_bw_20260311_170151.bin`, frames 102–112). + +### Write frame format — CRITICAL: minimal DLE stuffing + +Write frames do NOT use the same DLE stuffing as read frames. **Only the BW_CMD byte +(0x10 at payload position [0]) is doubled on the wire. All other bytes — flags, sub, +offset, params, data, and checksum — are written RAW without stuffing.** + +Confirmed from all 11 write frames in the 3-11-26/170151 BW capture. ✅ 2026-04-07 + +Do NOT use `dle_stuff()` or `build_bw_frame()` for write commands. Use `build_bw_write_frame()`. + +``` +Actual wire layout: + [41] ACK + [02] STX + [10 10] BW_CMD doubled (ONLY DLE stuffing applied) + [00] flags + [sub] write command byte (0x68–0x83) + [00] always zero + [hi][lo] offset uint16 BE — RAW (not stuffed even if hi=0x10) + [params] 10 bytes — RAW + [data] variable-length write payload — RAW (0x10 bytes not stuffed) + [chk] checksum — RAW (not stuffed even if 0x10) + [03] ETX + +Total wire length = 2 (ACK+STX) + 2 (doubled BW_CMD) + 15 (raw header) + len(data) + 1 (chk) + 1 (ETX) + = 21 + len(data) +``` + +De-stuffed payload (logical; used for checksum computation only): +``` + [0] BW_CMD 0x10 + [1] flags 0x00 + [2] SUB write command byte (0x68–0x83) + [3] 0x00 always zero + [4] offset_hi + [5] offset_lo + [6:16] params 10-byte field (see per-SUB notes below) + [16:] data write payload (variable length; absent for confirm frames) + [-1] chk large-frame DLE-aware checksum (see below) +``` + +Write SUBs = Read SUB + 0x60. Response SUB follows the standard 0xFF − Request SUB rule. + +### Write frame checksum + +All write frames (data frames AND confirm frames) use the **large-frame DLE-aware checksum**: + +```python +chk = (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF +``` + +This is identical to the SUB 5A DLE-aware checksum. Confirmed against all 11 write frames in +the 3-11-26/170151 capture. ✅ 2026-04-07 + +Note: confirm frames contain no embedded 0x10 bytes, so both the standard SUM8 and the +DLE-aware formula produce the same result for them — but `build_bw_write_frame` always uses +the DLE-aware formula for consistency. + +### Write ack responses + +All device acks for write commands are **17-byte zero-data S3 frames**: + +``` +[DLE=0x10][STX=0x02][stuffed(header + chk)][bare ETX=0x03] +``` + +The data section carries zeros; RSP_SUB = 0xFF − write_request_SUB. + +### Write SUB constants and sequences + +| Request SUB | Function | Offset | Response SUB | +|---|---|---|---| +| 0x68 | Event index write | `data[1] + 2` | 0x97 | +| 0x73 | Confirm B (follows 68) | 0 | 0x8C | +| 0x71 | Compliance write (×3 chunks) | see below | 0x8E | +| 0x72 | Confirm A (follows 71×3, 69) | 0 | 0x8D | +| 0x82 | Trigger config write | `data[1] + 2` | 0x7D | +| 0x83 | Trigger confirm (follows 82) | 0 | 0x7C | +| 0x69 | Waveform data write | `data[1] + 2` | 0x96 | +| 0x74 | Confirm C (follows 69) | 0 | 0x8B | + +**Offset formula for single-chunk writes (0x68, 0x69, 0x82):** `offset = data[1] + 2` + +The write payload always begins with a 2-byte header `[0x00][length]`, where `data[1]` is +an embedded length field. The offset encodes this inner length + 2 (accounting for the +header bytes). Confirmed from all three single-chunk write frames in the 3-11-26 capture: + +| SUB | data[0:4] (hex) | data[1] | offset | total data len | +|---|---|---|---|---| +| 0x68 | `00 58 09 00` | 0x58=88 | 0x5A=90 | 91 | +| 0x82 | `00 1A D5 00` | 0x1A=26 | 0x1C=28 | 29 | +| 0x69 | `00 C8 08 00` | 0xC8=200 | 0xCA=202 | 204 | + +Full sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72` + +### SUB 71 — compliance write chunk parameters + +The full compliance config payload (~2128 bytes) is split into exactly 3 chunks. +Confirmed from 3-11-26 BW TX capture frames 104–108: + +| Chunk | Size | `offset` | `params` (10 bytes hex) | +|---|---|---|---| +| 1 (first) | 1027 bytes | 0x1004 | `00 00 00 00 00 00 00 00 00 00` | +| 2 (middle) | 1055 bytes | 0x1004 | `00 00 00 10 04 00 00 00 00 00` | +| 3 (last) | remainder | 0x002C | `00 00 08 00 00 00 00 00 00 00` | + +Total: 1027 + 1055 + N = 2082 + N bytes (N ≈ 46 for a standard 2128-byte config). + +After all 3 chunks are acked (SUB 0x8E each), send SUB 72 confirm → device acks 0x8D. + +### `build_bw_write_frame()` — framing.py + +```python +build_bw_write_frame(sub, data, *, offset=0, params=bytes(10)) -> bytes +``` + +Use for all write commands (SUBs 68–83) including confirm frames (data=b""). +**Do NOT use `build_bw_frame` for write commands** — it uses standard SUM8, not the +large-frame DLE-aware checksum required for writes. + +### `push_config_raw()` — client.py + +```python +client.push_config_raw(event_index_data, compliance_data, trigger_data, waveform_data) +``` + +Orchestrates the full write sequence in the confirmed order. All payloads are raw bytes +(no encoding performed at this level). A higher-level encoder that builds payloads from +a `ComplianceConfig` object is a future task. + +--- + +## Monitoring commands (SUBs 0x1C, 0x96, 0x97) — confirmed 2026-04-08 + +All confirmed from 4-8-26/2ndtry BW TX/S3 capture (clean start → 30s monitor → stop cycle). + +### SUB 0x1C — Monitor status read + +Standard two-step read (probe at offset 0x00, data at offset 0x2C). +Response SUB = 0xFF − 0x1C = **0xE3** (standard formula — no exception). + +**Payload length is 46–47 bytes IDLE, 48–49 bytes MONITORING** — not a reliable sole +indicator due to 1-byte jitter overlap at the boundary. + +**Monitoring flag (CONFIRMED 2026-04-09 — byte diff of all 144 data frames, 2ndtry capture):** +- `section[1] == 0x00` → unit is **idle** +- `section[1] == 0x10` → unit is **monitoring** + +This is `data[12]` (= `frame.data[12]`). The flag is 0x00 in all 36 IDLE_BEFORE frames, +0x10 in all 98 MONITORING frames, and 0x00 in all 10 IDLE_AFTER frames — 100% accurate. + +**HISTORY OF THIS FIELD (do not re-derive):** The original implementation used `section[1]`. +A re-analysis in the prior session incorrectly concluded `section[1]` is always 0x00 and +"corrected" the flag to `section[6]`, which has non-binary values (0xea idle, 0x07 monitoring) +and is device-specific. The 2026-04-09 re-analysis confirms `section[1]` was right. + +**IMPORTANT — `frame.data` has checksum already stripped** by `S3FrameParser._finalise()` +(`raw_payload = body[:-1]`; `data = raw_payload[5:]`). There is NO trailing checksum byte in +`section`. All relative-from-end offsets must account for this. + +Battery and memory fields are present in **both** states: + +| Offset (relative to end) | Field | Type | Notes | +|---|---|---|---| +| `section[-10:-8]` | battery voltage × 100 | uint16 BE | `0x02A8` = 680 → 6.80 V | +| `section[-8:-4]` | memory total (bytes) | uint32 BE | e.g. 983026 ≈ 960 KB | +| `section[-4:]` | memory free (bytes) | uint32 BE | decreases as events are stored | + +### SESSION_RESET signal (`41 03`) — required for monitoring units + +Confirmed from 4-8-26 BW TX captures: Blastware sends a 2-byte `41 03` (ACK + ETX, +no STX) immediately before the first POLL probe AND between the probe and data frames. +This signal is **required to wake units that are actively monitoring** — without it +the unit does not respond to POLL over TCP. Harmless for idle units. + +`SESSION_RESET = bytes([0x41, 0x03])` is defined in `framing.py` and sent by +`protocol.startup()` before and between POLL frames. + +### SUB 0x96 — Start monitoring + +Single write frame, **no data payload** (empty body). +Response SUB = 0xFF − 0x96 = **0x69**. + +Wire bytes (confirmed frame 92 of 2ndtry BW capture): +``` +41 02 10 10 00 96 00 00 00 00 00 00 00 00 00 00 00 00 00 a6 03 +``` + +### SUB 0x97 — Stop monitoring + +Single write frame, **no data payload** (empty body). +Response SUB = 0xFF − 0x97 = **0x68**. + +Wire bytes (confirmed frame 305 of 2ndtry BW capture): +``` +41 02 10 10 00 97 00 00 00 00 00 00 00 00 00 00 00 00 00 a7 03 +``` + +Both start and stop acks are standard 17-byte zero-data S3 frames. + +### On-device sensor check behavior (confirmed 2026-04-08) + +Confirmed from 4-8-26/sensor-check BW+S3 capture (Blastware "Unit Channel Test" comms +check issued while unit was performing its on-device sensor check). + +**Unit IS reachable during on-device sensor check** — POLL (SUB 5B) responded normally +throughout. However, the unit partially handled channel-test commands (SUB 0x0E) for +channels 0–4 and then went **silent for ~40 seconds** while the sensor check ran, before +resuming responses for channels 5–7 and the trigger test (SUB 0x98). + +Key findings: +- On-device sensor check duration: approximately **40 seconds** (log gap `18:40:48` → `18:41:28`) +- Unit IS reachable for POLL during the check window — SESSION_RESET + POLL works +- Partial command responses during check are possible (device may buffer some, drop others) +- The Blastware "Unit Channel Test" (remote comms check, SUBs 0x0E + 0x98) is a SEPARATE + operation from the on-device check — it is a passive remote read; the unit's screen does + not change during a remote comms check + +**SFM behavior after `POST /device/monitor/start`:** `_pollMonitorConfirm()` polls +`/device/monitor/status` every 5 s for up to 60 s, updating the badge on each poll. +Status will show MONITORING once `section[1]` flips to `0x10`. + +### SUBs known from sensor-check capture (4-8-26) — NOT YET IMPLEMENTED + +| BW SUB | RSP SUB | Function | Notes | +|---|---|---|---| +| 0x15 | 0xEA | Serial number / short ID | 2-step read; data offset = 0x0A (10 bytes); confirmed serial `"BE11529"` at `data[11+5:]` | +| 0x01 | 0xFE | Device info block | 2-step read; data offset = 0x98 (152 bytes); payload includes serial + firmware + float config fields | +| 0x0E | 0xF1 | Channel sensor data | 2-step read; channel selector in `params[6:8]` (`0x0000`–`0x0007`); data length 0x0A per channel; used by Blastware "Unit Channel Test" — see docs/ for details | +| 0x98 | 0x67 | Trigger test | Single probe frame (`params[0]=0xFF`); sent twice per test cycle; all-zero data response; used after 0x0E channel scan | + +Blastware's "Unit Channel Test" sequence: `POLL×N → 0x15 → 0x01 → 0x08 → 0x01 → 0x0E×8 → 0x98×2 → 0x0E×8` (repeat pass with live ADC readings). + +--- + +## Compliance config field inventory (from Blastware UI, 2026-04-08) + +Fields visible in the Blastware Compliance Setup dialog — most are NOT YET decoded to byte +offsets in the raw 1A/E5 payload. Only fields with `✅` have confirmed offsets in the code. + +**Recording Setup tab:** +- Recording Mode: Continuous / Single Shot / Histogram (enum) +- Record Stop Mode: Fixed Record Time / Auto / Manual Stop (enum) +- Sample Rate: Standard 1024 / Fast 2048 / Faster 4096 sps ✅ (anchor−2) +- Record Time: float, seconds ✅ (anchor+10) +- Histogram Interval: 5 / 15 / 30 / 60 minutes (enum, mode-gated) +- Storage Mode: Save All Data / Save Triggered (enum) +- Geophone Type: Standard Triaxial / 4.5 Hz (bool/enum) +- Geophone Channels: Enable all geophones (bool), Trigger Source (bool) +- Chan 1-3 Trigger Level (float, in/s) ✅ (`trigger_level_geo`) +- Chan 1-3 Maximum Range: Normal 10.000 / 1.25 in/s (enum) ❓ (`max_range_geo` — offset found, reads 6.206053 which matches neither UI value; units and meaning unknown — do NOT use as ADC full-scale) +- Microphone Channels: Enable all microphones (bool), Trigger Source (bool) +- Chan 4 Trigger Level (dB or psi depending on units) + +**Notes tab:** +- Enable User Notes (bool) +- Project, Client, User Name, Seis Loc (ASCII strings) ✅ (sourced from A5 frame 7 via 5A) +- Enable Extended Notes (bool); Extended Notes text; Extended Notes Title +- Enable Job Number (bool); Job Number (int) +- Enable Scaled Distance (bool); Distance from Blast (float); Charge Weight (float) — Scaled Distance is derived + +**Special Setups tab:** +- Unit Timer: Timer Mode (Off/On), Start Date/Time, Stop Date/Time +- Self Check: Mode (Off/On), Time (HH:MM) +- Sensor Check: **Before monitoring** / After each event / **Disabled** ❓ (byte offset unknown) +- Measurement Units: Imperial / Metric +- Show Mic units in dB (bool) +- Time Format: 24 Hour / 12 Hour (AM/PM) +- Backlight on Time (seconds) ✅ (event index block +75) +- Power Saving Timeout (minutes) ✅ (event index block +83) +- Monitoring LCD Cycle ✅ (event index block +84:86) +- Set unit time with setup (bool) + +The "Sensor Check" dropdown (`Before monitoring` / `After each event` / `Disabled`) has NOT +been located in the raw config bytes. The user's unit always runs with `Before monitoring`. +Full compliance config encoder is a future task. + +--- + +--- + +## Erase-all protocol (SUBs 0xA3/0xA2/0x06) — confirmed 2026-04-11 + +Full sequence confirmed from 4-11-26 MITM capture of a live Blastware ACH session +(`bridges/captures/mitm/ach_mitm_20260411_001912/`). + +### Wire sequence + +``` +BW → device: SUB 0xA3 params=00 00 00 00 00 00 00 FE 00 00 (begin erase) +device → BW: SUB 0x5C (ack) +BW → device: SUB 0x1C probe (offset=0x00) +device → BW: SUB 0xE3 (probe ack) +BW → device: SUB 0x1C data (offset=0x2C) +device → BW: SUB 0xE3 (monitor status response) +BW → device: SUB 0x06 probe (offset=0x00, params same) +device → BW: SUB 0xF9 (probe ack) +BW → device: SUB 0x06 data (offset=0x24) +device → BW: SUB 0xF9 (36-byte storage range response) +BW → device: SUB 0xA2 params=00 00 00 00 00 00 00 FE 00 00 (confirm erase) +device → BW: SUB 0x5D (ack — device memory is now cleared) +``` + +All frames use standard `build_bw_frame` (not write-format). Response SUBs follow the +standard `0xFF - SUB` formula; no exceptions. + +### SUB 0x06 — event storage range response (36 bytes) + +The 36-byte response body ends with two 4-byte event keys: + +| Offset (from end) | Field | Notes | +|---|---|---| +| `[-8:-4]` | first stored event key | `01110000` when empty | +| `[-4:]` | last stored event key | `01110000` when empty | + +Before erase: ends with ` ` (e.g. `0111ea60 0111eaa6`). +After erase: both bytes read `01110000` — device's empty/reset sentinel. + +### Post-erase key counter reset + +After a successful erase, the device resets its event counter. New events start from +key `0x01110000` again — the same key as the very first event ever recorded. This means +key-based deduplication in the ACH server must account for key reuse: + +- After our own erase: `ach_state.json` `downloaded_keys` and `max_downloaded_key` are + cleared so the next session starts fresh. +- After an external erase: the ACH server detects it by comparing `max(device_keys)` to + `max_downloaded_key` from state. If the device max has rolled back below the historical + max, all current device keys are treated as new regardless of `seen_keys`. + +### ACH server state format (v0.9.0) + +`bridges/captures/ach_state.json`: +```json +{ + "BE11529": { + "downloaded_keys": ["01110000", "0111245a"], + "max_downloaded_key": "0111245a", + "last_seen": "2026-04-11T01:04:36", + "serial": "BE11529", + "peer": "63.43.212.232:51920" + } +} +``` + +`max_downloaded_key` is the high-water mark — the largest key ever downloaded from the +unit. It is NOT reset when events are erased from the device (only when our server does +the erase). Used for post-erase detection. + +--- + +## Monitor log entries — SUB 0x0A partial records (confirmed 2026-04-11) + +Confirmed from 4-11-26 MITM capture: 12 partial records (record type `0x2C`) and 7 full +event records (record type `0x46`) across 19 total 0x0A responses. + +### Record type detection + +`read_waveform_header()` returns `(raw_data, length)` where `raw_data = data_rsp.data` +(the full payload including prefix bytes). The record type is at `raw_data[0]`: + +| Value | Type | How to process | +|---|---|---| +| `0x46` | Full triggered event | Normal download: 0C → 5A → 1F | +| `0x2C` | Monitor log entry (partial) | No 0C/5A; decode inline from 0A payload | + +Length heuristic: `length < 0x40` (64) reliably identifies partial records across all +observed captures. Both checks (`raw_data[0] == 0x2C` and `length < 0x40`) are used. + +### SUB 0x0A partial record (0x2C) payload layout + +All offsets are from `raw_data` (the full `data_rsp.data` array including the 11-byte +prefix before the actual header bytes start). + +``` +raw_data[0] = 0x2C ← record type (partial / monitor log) +raw_data[1:11] = prefix bytes (vary; contain key4 copy, flags, length) +raw_data[11:] = timestamp and ASCII metadata payload +``` + +**Timestamp auto-detection** (confirmed from 4-11-26 capture): + +``` +raw_data[11] == 0x10 → 10-byte sub_code=0x03 format (continuous mode) +raw_data[11] != 0x10 → 9-byte sub_code=0x10 format (single-shot mode) +``` + +**9-byte timestamp format (sub_code=0x10):** + +| Byte | Field | +|---|---| +| 0 | day | +| 1 | `0x10` (sub_code marker) | +| 2 | month | +| 3–4 | year (uint16 BE) | +| 5 | unknown (0x00) | +| 6 | hour | +| 7 | minute | +| 8 | second | + +**10-byte timestamp format (sub_code=0x03):** + +| Byte | Field | +|---|---| +| 0 | `0x10` (marker) | +| 1 | day | +| 2 | `0x10` (marker) | +| 3 | month | +| 4–5 | year (uint16 BE) | +| 6 | unknown (0x00) | +| 7 | hour | +| 8 | minute | +| 9 | second | + +**Two timestamps:** Each partial record contains two timestamps — `start_time` and +`stop_time` — stored consecutively: +- `ts1` (start) at `raw_data[ts_offset : ts_offset + ts_size]` where `ts_offset = 11` +- `ts2` (stop) at `raw_data[ts1_end : ts1_end + ts_size]` + +**Edge case — 1-byte gap between timestamps:** Occurs when ts1 and ts2 share the same +minute:second. If `try_ts(raw_data[ts1_end:])` fails, try `try_ts(raw_data[ts1_end+1:])`. +Confirmed in frames 121, 161, 165 of the 4-11-26 MITM capture. Frame 121 still shows 0s +duration (both decode to 16:02:00) — the extra byte appears in all same-second cases. + +**ASCII metadata after timestamps:** +``` + BE\x00Geo: in/s ... +``` + +- Serial: scan for `b"BE"`, read until `b"\x00"` (e.g. `"BE11529"`) +- Geo threshold: scan for `b"Geo: "`, read float until next space (e.g. `0.254` in/s) + +A separator of variable length (4–5 bytes of `\x00` + flags) sits between the two +timestamps and the ASCII region. The `b"BE"` anchor scan is robust to separator length +variation. + +### `_decode_0a_partial_header(raw_data, index, key4)` — client.py + +Returns a `MonitorLogEntry` or `None`. Called by `get_monitor_log_entries()` for each +event key whose 0x0A response has `raw_data[0] == 0x2C` or `length < 0x40`. + +### `MiniMateClient.get_monitor_log_entries(skip_keys=None)` — client.py + +Browse-mode walk: `1E → 0A → check type → decode if partial → 1F`. No 0x0C or 5A reads +performed. Full (0x46) records are skipped without decoding. Returns `list[MonitorLogEntry]`. + +`skip_keys` (optional `set[str]`): keys in this set are still advanced through the walk +(to avoid disrupting the iteration sequence), but no `MonitorLogEntry` is created for them. + +### `MonitorLogEntry` model — models.py + +```python +@dataclass +class MonitorLogEntry: + index: int # 0-based position + key: str # 8-hex event key + start_time: Optional[datetime.datetime] = None + stop_time: Optional[datetime.datetime] = None + serial: Optional[str] = None + geo_threshold_ips: Optional[float] = None + raw_header: Optional[bytes] = field(default=None, repr=False) + + @property + def duration_seconds(self) -> Optional[float]: ... +``` + +### ACH server integration (v0.10.0) + +After `get_events()`, the ACH server calls `get_monitor_log_entries(skip_keys=seen_keys)`. +New entries are saved to `monitor_log.json` in the session directory. Monitor log keys are +included in `current_keys` for state persistence so they are not re-processed on the next +call-home. + +--- + +## What's next + +- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable +- **Histograms** — decode histogram-mode A5 data (noise floor tracking) +- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object +- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring) +- Modem manager — push RV50/RV55 configs via Sierra Wireless API +- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't + resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred) + \ No newline at end of file diff --git a/docs/instantel_protocol_reference.md b/docs/instantel_protocol_reference.md index 356cb07..0692402 100644 --- a/docs/instantel_protocol_reference.md +++ b/docs/instantel_protocol_reference.md @@ -36,7 +36,7 @@ | 2026-03-02 | §7.4 Event Index Block | **NEW:** `Monitoring LCD Cycle` identified at offsets +84/+85 as uint16 BE. Default value = 65500 (0xFFDC) = effectively disabled / maximum. Confirmed from operator manual §3.13.1g. | | 2026-03-02 | §7.4 Event Index Block | **UPDATED:** Backlight confirmed as uint8 range 0–255 seconds per operator manual §3.13.1e ("adjustable timer, 0 to 255 seconds"). Power save unit confirmed as minutes per operator manual §3.13.1f. | | 2026-03-02 | Global | **NEW SOURCE:** Operator manual (716U0101 Rev 15) added as reference. Cross-referencing settings definitions, ranges, and units. Header updated. | -| 2026-03-02 | §14 Open Questions | Float 6.2061 in/s mystery: manual confirms only two geo ranges (1.25 in/s and 10.0 in/s). 6.2061 is NOT a user-selectable range → likely internal ADC full-scale calibration constant or hardware range ceiling. Downgraded to LOW priority. | +| 2026-03-02 | §14 Open Questions | Float 6.2061 in/s mystery: manual confirms only two geo ranges (1.25 in/s and 10.0 in/s). 6.2061 is NOT a user-selectable range → originally speculated as internal ADC full-scale constant, but this is NOT confirmed. Using it as ADC full-scale produces ~9× PPV overread. Meaning unknown. Downgraded to LOW 2026-03-02, re-escalated to HIGH 2026-04-16. | | 2026-03-02 | §14 Open Questions | `0x082A` hypothesis refined: 2090 decimal. At 1024 sps, 2 sec record = 2048 samples. Possible that 0x082A = total samples including 0.25s pre-trigger (256 samples) at some adjusted rate. Needs capture with different record time. | | 2026-03-02 | §14 Open Questions | **NEW items added:** Trigger sample width (default=2), Auto Window (1-9 sec), Aux Trigger (enabled/disabled) — all confirmed settings from operator manual not yet mapped in protocol. | | 2026-03-02 | §14 Open Questions | Monitoring LCD Cycle resolved — removed from open questions. | @@ -92,7 +92,7 @@ | 2026-04-06 | §7.8.4 | **NEW — 5A end-of-stream signalling confirmed.** After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to the next chunk request, then goes silent for the full recv timeout. This byte is NOT a complete DLE-framed A5 response — the frame parser accumulates it as `bytes_fed=1` and never assembles a frame. This is the device's natural end-of-stream signal. Handling: on TimeoutError, if `bytes_fed > 0` AND prior chunks were received, treat as graceful end and proceed to the termination frame. A `bytes_fed=0` timeout with no prior chunks is a genuine transport failure and must still raise. | | 2026-04-06 | §7.8.4 | **NEW — 5A chunk timing and count (empirical, BE11529 at 1024 sps).** Each chunk response arrives within ~1 second over TCP/cellular. A 9,306-sample event (≈9.1 s at 1024 sps) produces **35 chunks** before end-of-stream. Chunks 1–16 have varying data lengths (1036–1123 bytes); chunks 17–35 are uniformly 1036 bytes each (post-event silence, all-zero ADC samples). Safe recv timeout for chunk loop: **10 s** (10× typical response time). Default transport timeout (120 s) results in a ~2-minute stall per event at end-of-stream. | | 2026-04-06 | §7.8.3 | **KNOWN ISSUE — `_decode_a5_waveform` hardcoded fi==9 skip.** The decoder contains `elif fi == 9: continue` which was written for the 9-frame original blast capture where frame 9 was a device terminator. For streams with >9 frames (current device produces 35+), frame index 9 is live waveform data — this skip discards ~1,070 bytes (~133 sample-sets) per event. The terminator is now detected via `page_key == 0x0000`, not by frame index. The fi==9 skip should be removed. | -| 2026-04-06 | §7.8 | **CONFIRMED — ADC count-to-physical-unit conversion.** Raw waveform samples are signed 16-bit integers (counts). Conversion: `value = counts × (range / 32767)`. For geo channels: range = 10.000 in/s (from the device's compliance config geo range field). For the mic channel: range is in psi (device-specific). Near-full-scale counts (≈32,700) on all four channels simultaneously indicate ADC saturation (clipping) from a high-amplitude event. | +| 2026-04-06 | §7.8 | **⚠ PARTIALLY INVALIDATED — ADC count-to-physical-unit conversion.** Raw waveform samples are signed 16-bit integers (counts). Conversion formula `value = counts × (range / 32767)` is believed correct, but the `range` value is UNKNOWN. The compliance config field labeled `max_range_geo` reads 6.206053 (bytes `40 C6 97 FD`), which does NOT match either user-selectable range shown in Blastware UI (1.25 or 10.000 in/s). The meaning and units of the 6.206053 value are unresolved — it may not be the ADC full-scale at all. See open question in §14. | | 2026-04-08 | §5.1, §7.10, §12 | **NEW — Monitoring commands confirmed.** SUB 0x1C (monitor status), 0x96 (start monitoring), 0x97 (stop monitoring) all confirmed from 4-8-26/2ndtry capture. SESSION_RESET (`41 03`) required before POLL to wake a monitoring unit. | | 2026-04-09 | §7.10 | **CORRECTED — monitoring flag and battery/memory offsets.** `section[1] == 0x10` is the monitoring flag (100% accurate across 144 data frames in 2ndtry capture). Previous note claiming `section[6]` was wrong — section[6] has device-specific non-binary values (0xea/0x07). Battery/memory offsets corrected: `section[-10:-8]` (battery×100), `section[-8:-4]` (memory_total), `section[-4:]` (memory_free). NOTE: `frame.data` has checksum stripped by parser — earlier offsets of `[-11:-9]`/`[-9:-5]`/`[-5:-1]` were wrong because they assumed a trailing checksum byte that isn't there. | | 2026-04-08 | §7.10 | **NEW — SUBs 0x0E (channel sensor data) and 0x98 (trigger test) observed** in 4-8-26/sensor-check capture (Blastware "Unit Channel Test" comms check). SUB 0x0E: 2-step read with channel selector in `params[6:8]`, data length 0x0A per channel, RSP SUB = 0xF1. SUB 0x98: single probe frame with `params[0] = 0xFF`, RSP SUB = 0x67; sent twice per test cycle. Not yet implemented in SFM. | @@ -528,7 +528,7 @@ The SUB `1A` read response (`E5`) and SUB `71` write block contain per-channel t | Field | Example bytes | Decoded | Certainty | |---|---|---|---| | `[00 00]` | `00 00` | Separator / padding | 🔶 INFERRED | -| Max range float | `40 C6 97 FD` | 6.206 — full-scale range in in/s | 🔶 INFERRED | +| Max range float | `40 C6 97 FD` | 6.206 — **value confirmed, meaning and units UNKNOWN** (does NOT match UI range options 1.25/10.000 in/s; not confirmed as ADC full-scale) | ❓ UNKNOWN | | `[00 00]` | `00 00` | Separator / padding | 🔶 INFERRED | | **Trigger level** | `3F 19 99 9A` | **0.600 in/s** — IEEE 754 BE float | ✅ CONFIRMED | | Unit string | `69 6E 2E 00` | `"in.\0"` | ✅ CONFIRMED | @@ -655,7 +655,7 @@ offset size type value (Tran example) meaning +10 2 uint16 0x0015 = 21 unknown +12 4 bytes 03 02 04 01 flags (recording mode etc.) +16 4 uint32 0x00000003 record time in seconds ✅ CONFIRMED -+1A 4 float32 6.2061 max range (in/s for geo, psi for mic) ++1A 4 float32 6.2061 ❓ UNKNOWN field — value 6.2061 confirmed; meaning/units unresolved (NOT confirmed as max range or ADC full-scale) +1E 2 00 00 padding +20 4 float32 0.6000 trigger level ✅ CONFIRMED +24 4 char[4] "in.\0" / "psi\0" unit string (geo vs mic) @@ -1235,13 +1235,13 @@ TimeoutError caught: Chunks with uniform 1,036-byte payload (chunks 17–35 in the observed event) contain all-zero ADC samples — the device continues recording silence until the configured record time expires before terminating the stream. -**ADC count-to-physical conversion:** +**ADC count-to-physical conversion — ⚠ SCALING UNKNOWN:** -Raw samples are signed 16-bit integers (−32,768 to +32,767). To convert to physical units: +Raw samples are signed 16-bit integers (−32,768 to +32,767). The conversion formula is believed to be: ``` value_in_s (in/s) = counts × (geo_range / 32767) ``` -where `geo_range` is from the compliance config (typically 10.000 in/s). Mic channel uses psi units with its own range. Near-full-scale values on all channels simultaneously indicate ADC saturation (clipping). +However, the correct value of `geo_range` is **unknown**. The compliance config field `max_range_geo` reads 6.206053 (`40 C6 97 FD`) which does NOT match either user-selectable range (1.25 or 10.000 in/s) and produces ~9× too large PPV values compared to the on-device 0C record. Do not use 6.206053 or 10.000 as the scale factor until this is resolved. See §14 open question. Mic channel uses psi units with its own range (also unresolved). **Known decoder issue — fi==9 hardcoded skip:** @@ -1267,7 +1267,7 @@ Fields visible in the Blastware "Compliance Setup" dialog. ✅ = byte offset co | Geophone — Enable all | bool | ❓ | | Geophone — Trigger Source | bool | ❓ | | Chan 1-3 Trigger Level | float, in/s | ✅ `trigger_level_geo` | -| Chan 1-3 Maximum Range | Normal 10.000 / 1.25 in/s | ✅ `max_range_geo` | +| Chan 1-3 Maximum Range | Normal 10.000 / 1.25 in/s | ❓ `max_range_geo` offset found, value=6.206053 — does NOT match UI values; meaning unknown | | Microphone — Enable all | bool | ❓ | | Microphone — Trigger Source | bool | ❓ | | Chan 4 Trigger Level | float, dB or psi | ❓ | @@ -1933,7 +1933,7 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger | **Auxiliary Trigger read location** — **RESOLVED:** SUB `FE` offset `0x0109`, uint8, `0x00`=disabled, `0x01`=enabled. Confirmed 2026-03-11 via controlled toggle capture. | RESOLVED | 2026-03-02 | Resolved 2026-03-11 | | **Auxiliary Trigger write path** — Write command not yet captured in a clean session. Inner frame handshake visible in A4 (multiple WRITE_CONFIRM_RESPONSE SUBs appear, TRIGGER_CONFIG_RESPONSE removed), but the BW→S3 write command itself was in a partial session. Likely SUB `15` or similar. Deferred for clean capture. | LOW | 2026-03-11 | NEW | | ~~**SUB `6E` response to SUB `1C`**~~ — ~~RESOLVED 2026-04-08: This was a misidentification.~~ The `1C → 6E` "exception" was misread — likely an inner A4 sub-frame. Confirmed from 4-8-26 capture (338 frames): SUB 0x1C always → 0xE3. No exceptions to the `0xFF − SUB` rule are known. | RESOLVED | 2026-04-08 | CLOSED | -| **Max Geo Range float 6.2061 in/s** — NOT a user-selectable range (manual only shows 1.25 and 10.0 in/s). Likely internal ADC full-scale constant or hardware range ceiling. Not worth capturing. | LOW | 2026-02-26 | Downgraded 2026-03-02 | +| **Max Geo Range float 6.2061** — offset confirmed in channel block (`+1A`, `40 C6 97 FD`). Meaning and units are UNKNOWN. Value does NOT match either user-selectable range (1.25 / 10.0 in/s). Using it as ADC full-scale produces ~9× PPV overread vs on-device 0C values. Not simply metric vs imperial (25.4 factor doesn't reconcile). Needs investigation: examine surrounding channel block bytes, compare with a Blastware waveform CSV export to back-calculate the correct scale. Upgraded to HIGH priority. | HIGH | 2026-02-26 | Upgraded 2026-04-16 | | MicL channel units — **RESOLVED: psi**, confirmed from `.set` file unit string `"psi\0"` | RESOLVED | 2026-03-01 | | | Backlight offset — **RESOLVED: +4B in event index data**, uint8, seconds | RESOLVED | 2026-03-02 | | | Power save offset — **RESOLVED: +53 in event index data**, uint8, minutes | RESOLVED | 2026-03-02 | | @@ -1962,7 +1962,7 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger | Trigger Level (Mic) | §3.8.6 | Channel block, float | float32 BE | 100–148 dB in 1 dB steps | | Alarm Level (Mic) | §3.9.10 | Channel block, float | float32 BE | higher than mic trigger | | Record Time | §3.8.9 | cfg anchor+10, float32 BE (wire); `.set` +16, uint32 LE (file) | float32 BE (wire) | 1–105 s; confirmed 3→`40400000`, 5→`40A00000`, 8→`41000000`, 13→`41500000`. Use anchor §7.6.1/§7.6.3 — NOT fixed offset. | -| Max Geo Range | §3.8.4 | Channel block, float | float32 BE | 1.25 or 10.0 in/s (user); 6.2061 in protocol = internal constant | +| Max Geo Range | §3.8.4 | Channel block, float | float32 BE | ❓ UNKNOWN — value 6.2061 confirmed at offset, but meaning/units unresolved. Does NOT equal 1.25 or 10.0 in/s. Do NOT use as ADC full-scale. | | Microphone Units | §3.9.7 | Inline unit string | char[4] | `"psi\0"`, `"pa.\0"`, `"dB\0\0"` | | Sample Rate | §3.8.2 | cfg anchor−2, uint16 BE — anchor=`\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in cfg[40:100] | uint16 BE | Normal=1024, Fast=2048, Faster=4096 ✅ CONFIRMED 2026-04-01 (BE11529 S338.17). Anchor required — see §7.6.3 DLE jitter. | | Record Mode | §3.8.1 | Unknown | — | Single Shot, Continuous, Manual, Histogram, Histogram Combo |