Files
seismo-relay/CLAUDE.md
T

25 KiB
Raw Blame History

CLAUDE.md — seismo-relay

Ground-up Python replacement for Blastware, Instantel's Windows-only software for managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55). Current version: v0.8.0.


Project layout

minimateplus/         ← Python client library (primary focus)
  transport.py        ←   SerialTransport, TcpTransport
  framing.py          ←   DLE codec, frame builders, S3FrameParser
  protocol.py         ←   MiniMateProtocol — wire-level read/write methods
  client.py           ←   MiniMateClient — high-level API (connect, get_events, …)
  models.py           ←   DeviceInfo, EventRecord, ComplianceConfig, …

sfm/server.py         ← FastAPI REST server exposing device data over HTTP
seismo_lab.py         ← Tkinter GUI (Bridge + Analyzer + Console tabs)
docs/
  instantel_protocol_reference.md  ← reverse-engineered protocol spec ("the Rosetta Stone")
CHANGELOG.md          ← version history

Current implementation state (v0.8.0)

Full read pipeline + write pipeline working end-to-end over TCP/cellular:

Step SUB Status
POLL / startup handshake 5B
Serial number 15
Full config (firmware, calibration date, etc.) FE
Compliance config (record time, sample rate, geo thresholds) 1A
Event index 08
Event header / first key 1E
Waveform header 0A
Waveform record (peaks, timestamp, project) 0C
Bulk waveform stream (event-time metadata) 5A new v0.6.0
Event advance / next key 1F
Write commands (push config to device) 6883 new v0.8.0

get_events() sequence per event: 1E → 0A → 0C → 5A → 1F

push_config_raw() write sequence: 68→73 | 71×3→72 | 82→83 | 69→74→72


Protocol fundamentals

DLE framing

BW→S3 (our requests):   [ACK=0x41] [STX=0x02] [stuffed payload+chk] [ETX=0x03]
S3→BW (device replies): [DLE=0x10] [STX=0x02] [stuffed payload+chk] [bare ETX=0x03]
  • DLE stuffing rule: any literal 0x10 byte in the payload is doubled on the wire (0x100x10 0x10). This includes the checksum byte.
  • Inner-frame terminators: large S3 responses (A4, E5) contain embedded sub-frames using DLE+ETX as inner terminators. The outer parser treats DLE+ETX inside a frame as literal data — the bare ETX is the ONLY real frame terminator.
  • Response SUB rule: response_SUB = 0xFF - request_SUB (one known exception: SUB 1C → response 6E, not 0xE3)
  • Two-step read pattern: every read command is sent twice — probe step (offset=0x00, get length) then data step (offset=DATA_LENGTH, get payload). All data lengths are hardcoded constants, not read from the probe response.

De-stuffed payload header

BW→S3 (request):
  [0] CMD    0x10
  [1] flags  0x00
  [2] SUB    command byte
  [3] 0x00   always zero
  [4] 0x00   always zero
  [5] OFFSET 0x00 for probe step; DATA_LENGTH for data step
  [6-15]     params (key, token, etc. — see helpers in framing.py)

S3→BW (response):
  [0] CMD    0x00
  [1] flags  0x10
  [2] SUB    response sub byte
  [3] PAGE_HI
  [4] PAGE_LO
  [5+] data

Critical protocol gotchas (hard-won — do not re-derive)

SUB 5A — bulk waveform stream — NON-STANDARD frame format

Always use build_5a_frame() for SUB 5A. Never use build_bw_frame() for SUB 5A.

build_bw_frame produces WRONG output for 5A for two reasons:

  1. offset_hi = 0x10 must NOT be DLE-stuffed. Blastware sends the offset field raw. build_bw_frame would stuff it to 10 10 on the wire — the device silently ignores the frame. build_5a_frame writes it as a bare 10.

  2. DLE-aware checksum. When computing the checksum, 10 XX pairs in the stuffed section contribute only XX to the running sum; lone bytes contribute normally. This differs from the standard SUM8-of-destuffed-payload that all other commands use.

Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26 BW TX capture. All 10 frames verified.

SUB 5A — chunk counter is monotonic (CORRECTED 2026-04-06)

Chunk counters are chunk_num * 0x0400 for ALL chunks including chunk 1.

The 4-2-26 BW TX capture showed counter=0x1004 for chunk 1 of event key 01110000, which led to _CHUNK1_COUNTER = 0x1004 being hardcoded as a special case. This was a Blastware artifact, not a protocol requirement. Empirical test 2026-04-06: with counter=0x1004 for chunk 1 the device times out (120 s); with counter=0x0400 (= 1 * 0x0400) it responds immediately and streams all frames correctly.

The 4-3-26 capture confirms the pattern for a second event (key 0111245a): chunk 1 = 0x245A, chunk 2 = 0x285A, chunk 3 = 0x2C5A (each +0x0400). Blastware's true formula is key4[2:4] + n * 0x0400 — but since key4[2:4] of the first event is 0x0000, n * 0x0400 produces the right result. The device does not strictly validate the counter and streams data for any valid 5A request; using chunk_num * 0x0400 is correct.

SUB 5A — params are 11 bytes for chunk frames, 10 for termination

bulk_waveform_params() returns 11 bytes (extra trailing 0x00). The 11th byte was confirmed from the BW wire capture. bulk_waveform_term_params() returns 10 bytes. Do not swap them.

SUB 5A — event-time metadata lives in A5 frame 7

The bulk stream sends 9+ A5 response frames. Frame 7 (0-indexed) contains the compliance setup as it existed when the event was recorded:

"Project:"      → project description
"Client:"       → client name        ← NOT in the 0C record
"User Name:"    → operator name      ← NOT in the 0C record
"Seis Loc:"     → sensor location    ← NOT in the 0C record
"Extended Notes"→ notes

IMPORTANT — 5A "Project:" is session-start config, NOT per-event (confirmed 2026-04-05): The "Project:" string in the A5 frame 7 payload reflects the compliance setup from when the monitoring session first started, not the individual event's project name. The per- event project name is correctly stored in the 210-byte 0C waveform record and must be used as the authoritative source. _decode_a5_metadata_into therefore only sets project from 5A when 0C didn't already supply one.

"Client:", "User Name:", "Seis Loc:", and "Extended Notes" are NOT present in the 0C record — 5A remains the sole source for those fields and they are set unconditionally.

stop_after_metadata=True (default) stops the 5A loop as soon as b"Project:" appears, then sends the termination frame.

SUB 5A — end-of-stream signal (confirmed 2026-04-06)

After streaming all waveform chunks, the device sends exactly 1 raw byte in response to the next chunk request, then goes silent. This is the natural end-of-stream indicator — NOT a complete A5 frame. S3FrameParser.bytes_fed will be 1; no frame is assembled.

Handling: on TimeoutError, if bytes_fed > 0 AND frames were already collected, treat as graceful end-of-stream, break the loop, and proceed to the termination frame. If bytes_fed == 0 with no prior frames, it is a genuine transport failure — re-raise.

Chunk recv timeout must be 10 s, not the default 120 s. Chunks arrive within ~1 s each. Using 120 s causes a ~2-minute stall at every end-of-stream detection. The _recv_one call in the chunk loop passes timeout=10.0 explicitly.

Typical chunk count (BE11529, 1024 sps): A 9,306-sample event produces 35 chunks before end-of-stream. Chunks with uniform 1,036-byte data are all-zero ADC samples (post-event silence). Only the initial variable-size chunks contain actual signal.

SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06)

_decode_a5_waveform() previously had elif fi == 9: continue — a leftover from the 9-frame original blast capture where frame 9 was assumed to be a terminator. For current 35-frame streams, fi==9 is live waveform data (~133 sample-sets were being dropped). Removed. Terminator detection is via page_key == 0x0000 in read_bulk_waveform_stream, not frame index.

SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)

token_params bug (FIXED): The token byte was at params[6] (wrong). Both 3-31-26 and 4-3-26 BW TX captures confirm it belongs at params[7] (raw: 00 00 00 00 00 00 00 fe 00 00). With the wrong position the device ignores the token and 1F returns null immediately.

1F token depends on context: In browse mode (no 5A), use all-zero params (browse=True). In download mode (get_events with 5A), use token=0xFE (browse=False) — this is required to arm the device's 5A bulk stream state machine. The earlier "empirical" test showing token=0xFE returns null was done WITHOUT the 1E(arm) step; that test is invalid. BW always uses 1F(0xFE) in download mode. count_events uses browse=True (no 5A needed).

0A context requirement: advance_event() (1F) only returns a valid next-event key when a preceding read_waveform_header() (0A) call has established device waveform context for the current key. Call 0A before every event in the loop, not just the first. Calling 1F cold (after only 1E, with no 0A) returns the null sentinel regardless of how many events are stored.

1F response layout: The next event's key IS at data_rsp.data[11:15] (= payload[16:20]). Confirmed from 4-3-26 browse-mode S3 captures:

1F after 0A(key0=01110000):  data[11:15]=0111245a  data[15:19]=00001e36  ← valid
1F after 0A(key1=0111245a):  data[11:15]=01114290  data[15:19]=00000046  ← valid
1F null sentinel:            data[11:15]=00000000  data[15:19]=00000000  ← done

Null sentinel: data8[4:8] == b"\x00\x00\x00\x00" (= data_rsp.data[15:19]) works for BOTH 1E trailing (offset to next event key) and 1F response (null key echo) — in both cases, all zeros means "no more events."

1E response layout: data_rsp.data[11:15] = event 0's actual key; data_rsp.data[15:19] = sample-count offset to the next event key (key1 = key0 + this offset). If offset == 0, there is only one event.

Correct iteration pattern (confirmed empirically with live device, 2+ events):

count_events (browse mode only, no 5A):

1E(all zeros) → key0, trailing0         ← trailing0 non-zero if event 1 exists
0A(key0)                                ← REQUIRED: establishes device context
1F(all zeros / browse=True) → key1      ← use all-zero params
0A(key1)                                ← REQUIRED before each advance
1F(all zeros) → null                    ← done

get_events (download mode, with 5A):

1E(all zeros) → key0, trailing0         ← trailing0 non-zero if event 1 exists
0A(key0)                                ← REQUIRED: establishes device context
1E(token=0xFE)                          ← REQUIRED: arms device for 5A; CONFIRMED 4-2-26 + 4-3-26
0C(key0)                                ← read waveform record
1F(token=0xFE) → [discard key]          ← REQUIRED: arms 5A bulk stream state machine
POLL × 3                                ← REQUIRED: 3 full POLL cycles before 5A (BW frames 68-73)
5A(key0)                                ← bulk stream; key0 used even though 1F already advanced
1F(all zeros / browse=True) → key1      ← USE THIS for loop iteration (browse=True returns correct key)
0A(key1)
1E(token=0xFE)                          ← re-arm for next event's 5A
0C(key1)
1F(token=0xFE) → [discard key]          ← arm 5A
POLL × 3
5A(key1)
1F(browse=True) → null                  ← done

IMPORTANT — conditional browse 1F (UPDATED 2026-04-06): 1F(token=0xFE) (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine. Its returned key is cached as arm_key4 in get_events().

1F(browse=True) AFTER 5A is ONLY sent when 5A succeeded. If 5A timed out or failed, sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the next event's 5A probe to also time out with 0 bytes received).

In the failure path, arm_key4 from 1F(download) is used as a best-effort next-key hint:

  • If arm_key4 != cur_key: use it to advance the loop without any 1F call
  • If arm_key4 == cur_key (device stuck, typical for second+ events when 5A fails): abort

The diagnostic bytes_fed counter on S3FrameParser (incremented in every feed() call, reset by reset()) makes it possible to distinguish "no bytes at all" from "bytes received but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in the log but have very different root causes.

The 1E(token=0xFE) arm step is required (FIXED 2026-04-06): The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE has been issued between 0A and 0C. This step is present in EVERY download cycle in both the 4-2-26 and 4-3-26 BW TX captures.

1F must come BEFORE 5A (FIXED 2026-04-06): BW always calls 1F (advance event) before starting the 5A bulk stream. 5A still uses the pre-advance key — the device streams the waveform for the key that was set up with 0A+1E-arm+0C even after 1F has moved the internal pointer to the next event.

POLL × 3 required before 5A (FIXED 2026-04-06): BW sends exactly 3 complete POLL (SUB 5B) probe+data cycles between the last 1F and the first 5A probe frame. Confirmed from 4-2-26 BW TX capture frames 68-73. Without these POLLs the device does not respond to the 5A probe. Use proto.poll() (not startup()startup() drains the boot string, which is only needed on initial connect).

advance_event(browse=True) sends all-zero params; advance_event() default (browse=False) sends token=0xFE and is NOT used by any caller. advance_event() returns (key4, event_data8). Callers (count_events, get_events) loop while data8[4:8] != b"\x00\x00\x00\x00".

SUB 1A — compliance config — orphaned send bug (FIXED, do not re-introduce)

read_compliance_config() sends a 4-frame sequence (A, B, C, D) where:

  • Frame A is a probe (no recv_one needed — device ACKs but returns no data page)
  • Frames B, C, D each need a recv_one to collect the response

There must be NO extra self._send(...) call before the B/C/D recv loop without a matching recv_one(). An orphaned send shifts all receives one step behind, leaving frame D's channel block (trigger_level_geo, alarm_level_geo, max_range_geo) unread and producing only ~1071 bytes instead of ~2126.

SUB 1A — anchor search range

_decode_compliance_config_into() locates sample_rate and record_time via the anchor b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'. Search range is cfg[0:150].

Do not narrow this to cfg[40:100] — the old range was only accidentally correct because the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from its real position (cfg[11]) into the 40100 window.

Sample rate and DLE jitter in cfg data

Sample rate 4096 (0x1000) causes DLE jitter: the frame carries 10 10 00 on the wire, which unstuffs to 10 00 — 2 bytes instead of 3. This makes frame C 1 byte shorter and shifts all subsequent absolute offsets by 1. The anchor approach is immune to this. Do NOT use fixed absolute offsets for sample_rate or record_time.

TCP / cellular transport

  • Protocol bytes over TCP are bit-for-bit identical to RS-232. No wrapping.
  • The modem (RV50/RV55) forwards bytes with up to ~1s buffering. TcpTransport uses read_until_idle(idle_gap=1.5s) to drain the buffer completely before parsing.
  • Cold-boot: unit sends the 16-byte ASCII string "Operating System" before entering DLE-framed mode. The parser discards it (scans for DLE+STX).
  • RV50/RV55 sends \r\nRING\r\n\r\nCONNECT\r\n over TCP to the caller even with Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to S3FrameParser.

Required ACEmanager settings (Sierra Wireless RV50/RV55)

Setting Value Why
Configure Serial Port 38400,8N1 Must match MiniMate baud
Flow Control None Hardware FC blocks TX if pins unconnected
Quiet Mode Enable Critical. Disabled injects RING/CONNECT onto serial, corrupting S3 handshake
Data Forwarding Timeout 1 (= 0.1 s) Lower latency
TCP Connect Response Delay 0 Non-zero silently drops first POLL frame
TCP Idle Timeout 2 (minutes) Prevents premature disconnect
DB9 Serial Echo Disable Echo corrupts the data stream

Key confirmed field locations

SUB FE — Full Config (166 destuffed bytes)

Offset Field Type Notes
0x34 firmware version string ASCII e.g. "S338.17"
0x560x57 calibration year uint16 BE 0x07E9 = 2025
0x0109 aux trigger enabled uint8 0x00 = off, 0x01 = on

SUB 1A — Compliance Config (~2126 bytes total after 4-frame sequence)

Field How to find it
sample_rate uint16 BE at anchor 2
record_time float32 BE at anchor + 10
trigger_level_geo float32 BE, located in channel block
alarm_level_geo float32 BE, adjacent to trigger_level_geo
max_range_geo float32 BE, adjacent to alarm_level_geo
setup_name ASCII, null-padded, in cfg body
project / client / operator / sensor_location ASCII, label-value pairs

Anchor: b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00', search cfg[0:150]

SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2])

sub_code=0x10 (Waveform single-shot) — 9-byte timestamp header:

Offset Field Type
0 day uint8
1 sub_code uint8 (0x10)
2 month uint8
34 year uint16 BE
5 unknown uint8 (always 0)
6 hour uint8
7 minute uint8
8 second uint8

sub_code=0x03 (Waveform continuous) — 10-byte timestamp header (1-byte shift):

Confirmed 2026-04-03 against Blastware event report (15:20:17 Apr 3 2026). Raw wire bytes: 10 03 10 04 07 ea 00 0f 14 11

Offset Field Type Notes
0 unknown_a uint8 0x10 observed
1 day uint8 doubles as sub_code position in 0x10 layout
2 unknown_b uint8 0x10 observed
3 month uint8
45 year uint16 BE
6 unknown uint8
7 hour uint8
8 minute uint8
9 second uint8

Peak values (both record types):

Location Field Type
tran_pos - 12 peak_vector_sum float32 BE — label-relative, NOT fixed offset
label + 6 PPV per channel float32 BE (search for "Tran", "Vert", "Long", "MicL")

PPV labels are NOT 4-byte aligned. The label-relative approach is the only reliable method. peak_vector_sum is exactly 12 bytes before the "Tran" label — confirmed for both sub_code=0x10 and sub_code=0x03. Do NOT use fixed offset 87 (only incidentally correct for 0x10 records).


SFM REST API (sfm/server.py)

GET /device/info?port=COM5                              ← serial
GET /device/info?host=1.2.3.4&tcp_port=9034            ← cellular
GET /device/events?host=1.2.3.4&tcp_port=9034&baud=38400
GET /device/event?host=1.2.3.4&tcp_port=9034&index=0

Server retries once on ProtocolError for TCP connections (handles cold-boot timing).


Key wire captures (reference material)

Capture Location Contents
1-2-26 bridges/captures/1-2-26/ SUB 5A BW TX frames — confirmed 5A frame format, 11-byte params, DLE-aware checksum
3-11-26 bridges/captures/3-11-26/ Full compliance setup write, Aux Trigger capture
3-31-26 bridges/captures/3-31-26/ Complete event download cycle (148 BW / 147 S3 frames) — confirmed 1E/0A/0C/1F sequence; only 1 event stored so token=0xFE appeared to work
4-3-26 bridges/captures/4-3-26/ Browse-mode S3 capture with 2+ events — confirmed all-zero params for 1F, 1F response layout, null sentinel, 0A context requirement

Write commands (SUBs 6883) — confirmed 2026-04-07

All confirmed from 3-11-26 BW TX capture (raw_bw_20260311_170151.bin, frames 102112).

Write frame format — CRITICAL: minimal DLE stuffing

Write frames do NOT use the same DLE stuffing as read frames. Only the BW_CMD byte (0x10 at payload position [0]) is doubled on the wire. All other bytes — flags, sub, offset, params, data, and checksum — are written RAW without stuffing.

Confirmed from all 11 write frames in the 3-11-26/170151 BW capture. 2026-04-07

Do NOT use dle_stuff() or build_bw_frame() for write commands. Use build_bw_write_frame().

Actual wire layout:
  [41]       ACK
  [02]       STX
  [10 10]    BW_CMD doubled (ONLY DLE stuffing applied)
  [00]       flags
  [sub]      write command byte (0x680x83)
  [00]       always zero
  [hi][lo]   offset uint16 BE — RAW (not stuffed even if hi=0x10)
  [params]   10 bytes — RAW
  [data]     variable-length write payload — RAW (0x10 bytes not stuffed)
  [chk]      checksum — RAW (not stuffed even if 0x10)
  [03]       ETX

Total wire length = 2 (ACK+STX) + 2 (doubled BW_CMD) + 15 (raw header) + len(data) + 1 (chk) + 1 (ETX)
                 = 21 + len(data)

De-stuffed payload (logical; used for checksum computation only):

  [0]     BW_CMD  0x10
  [1]     flags   0x00
  [2]     SUB     write command byte (0x680x83)
  [3]     0x00    always zero
  [4]     offset_hi
  [5]     offset_lo
  [6:16]  params  10-byte field (see per-SUB notes below)
  [16:]   data    write payload (variable length; absent for confirm frames)
  [-1]    chk     large-frame DLE-aware checksum (see below)

Write SUBs = Read SUB + 0x60. Response SUB follows the standard 0xFF Request SUB rule.

Write frame checksum

All write frames (data frames AND confirm frames) use the large-frame DLE-aware checksum:

chk = (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF

This is identical to the SUB 5A DLE-aware checksum. Confirmed against all 11 write frames in the 3-11-26/170151 capture. 2026-04-07

Note: confirm frames contain no embedded 0x10 bytes, so both the standard SUM8 and the DLE-aware formula produce the same result for them — but build_bw_write_frame always uses the DLE-aware formula for consistency.

Write ack responses

All device acks for write commands are 17-byte zero-data S3 frames:

[DLE=0x10][STX=0x02][stuffed(header + chk)][bare ETX=0x03]

The data section carries zeros; RSP_SUB = 0xFF write_request_SUB.

Write SUB constants and sequences

Request SUB Function Offset Response SUB
0x68 Event index write data[1] + 2 0x97
0x73 Confirm B (follows 68) 0 0x8C
0x71 Compliance write (×3 chunks) see below 0x8E
0x72 Confirm A (follows 71×3, 69) 0 0x8D
0x82 Trigger config write data[1] + 2 0x7D
0x83 Trigger confirm (follows 82) 0 0x7C
0x69 Waveform data write data[1] + 2 0x96
0x74 Confirm C (follows 69) 0 0x8B

Offset formula for single-chunk writes (0x68, 0x69, 0x82): offset = data[1] + 2

The write payload always begins with a 2-byte header [0x00][length], where data[1] is an embedded length field. The offset encodes this inner length + 2 (accounting for the header bytes). Confirmed from all three single-chunk write frames in the 3-11-26 capture:

SUB data[0:4] (hex) data[1] offset total data len
0x68 00 58 09 00 0x58=88 0x5A=90 91
0x82 00 1A D5 00 0x1A=26 0x1C=28 29
0x69 00 C8 08 00 0xC8=200 0xCA=202 204

Full sequence: 68→73 | 71×3→72 | 82→83 | 69→74→72

SUB 71 — compliance write chunk parameters

The full compliance config payload (~2128 bytes) is split into exactly 3 chunks. Confirmed from 3-11-26 BW TX capture frames 104108:

Chunk Size offset params (10 bytes hex)
1 (first) 1027 bytes 0x1004 00 00 00 00 00 00 00 00 00 00
2 (middle) 1055 bytes 0x1004 00 00 00 10 04 00 00 00 00 00
3 (last) remainder 0x002C 00 00 08 00 00 00 00 00 00 00

Total: 1027 + 1055 + N = 2082 + N bytes (N ≈ 46 for a standard 2128-byte config).

After all 3 chunks are acked (SUB 0x8E each), send SUB 72 confirm → device acks 0x8D.

build_bw_write_frame() — framing.py

build_bw_write_frame(sub, data, *, offset=0, params=bytes(10)) -> bytes

Use for all write commands (SUBs 6883) including confirm frames (data=b""). Do NOT use build_bw_frame for write commands — it uses standard SUM8, not the large-frame DLE-aware checksum required for writes.

push_config_raw() — client.py

client.push_config_raw(event_index_data, compliance_data, trigger_data, waveform_data)

Orchestrates the full write sequence in the confirmed order. All payloads are raw bytes (no encoding performed at this level). A higher-level encoder that builds payloads from a ComplianceConfig object is a future task.


What's next

  • Compliance config encoder — build raw write payloads from a ComplianceConfig object
  • ACH inbound server — accept call-home connections from field units
  • Modem manager — push RV50/RV55 configs via Sierra Wireless API