serversdown/seismo-relay
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

fix: update MiniMateClient and protocol to ensure correct handling of 1F calls and improve event download sequence

Browse Source
This commit is contained in:
Brian Harrison
2026-04-06 15:58:03 -04:00
parent 227c481022
commit ad1c9e48b0
5 changed files with 130 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+17 -6
View File
@@ -205,12 +205,23 @@ POLL × 3
1F(browse=True) → null ← done
```
**IMPORTANT — two separate 1F calls per event (CONFIRMED 2026-04-06):**
`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine
but returns inconsistent iteration keys after the first event — do NOT use its returned key
for loop control. `1F(browse=True)` AFTER 5A returns the correct next event key for
iteration. The reason is device-side: `1F(0xFE)` appears to arm 5A and peek the next key
without reliably advancing the internal iteration pointer; `1F(all-zero)` advances it.
**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):**
`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine.
Its returned key is cached as `arm_key4` in `get_events()`.
`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**. If 5A timed out or failed,
sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next
event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the
next event's 5A probe to also time out with 0 bytes received).
In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint:
- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call
- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort
The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call,
reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received
but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in
the log but have very different root causes.
**The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):**
The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE
docs/instantel_protocol_reference.md
+40 -17
View File
@@ -80,6 +80,14 @@
| 2026-04-03 | §7.6 | **CONFIRMED — Blast waveform format (4-2-26 capture).** Blast/waveform-mode SUB 5A stream uses 4-channel interleaved signed int16 LE, 8 bytes per sample-set [T,V,L,M]. NOT the 32-byte block format (which is noise/histogram mode only). Frame sizes are NOT multiples of 8 — cross-frame alignment correction required (track global byte offset mod 8; skip `(8-align)%8` bytes at each frame start). A5[0] STRT record confirmed: 21 bytes at db[7:]+11; waveform starts at strt_pos+27 (after 2-byte null pad + 4-byte 0xFF sentinel). Frame index 7 = metadata only, no ADC data. Full §7.6 rewritten. |
| 2026-04-03 | §7.6 | **CONFIRMED — Noise block format details.** 32-byte blocks: LE uint16 type + LE uint16 ctr + 9×int16 LE samples + 10B metadata. Samples are little-endian (previous doc said big-endian — WRONG). Type: 0x0016=sync (appears at start of each A5 frame), 0x0000=data. Noise floor ≈ 9–11 counts. Metadata fixed pattern `00 01 43 [2B var] 00 [pretrig] [rectime] 00 00` confirmed. |
| 2026-04-03 | client.py | **NEW — `_decode_a5_waveform()` and `download_waveform()` implemented.** `_decode_a5_waveform(frames_data, event)` decodes full A5 waveform stream into `event.raw_samples = {"Tran":[…], "Vert":[…], "Long":[…], "Mic":[…]}`. Populates `event.total_samples`, `event.pretrig_samples`, `event.rectime_seconds` from STRT record. Handles cross-frame alignment. `MiniMateClient.download_waveform(event)` calls `read_bulk_waveform_stream(stop_after_metadata=False)` then invokes the decoder. Waveform key stored on Event as `_waveform_key` during `get_events()`. |
| 2026-04-03 | §7.7.5 | **CONFIRMED — sub_code=0x03 (Waveform continuous) uses 10-byte timestamp header** — one byte wider than sub_code=0x10 layout. Cross-referenced against Blastware event report for BE11529 (15:20:17 Apr 3 2026). Raw header: `10 03 10 04 07 ea 00 0f 14 11` = [unknown_a][day][unknown_b][month][year:2 BE][unknown][hour][min][sec]. Peak Vector Sum is at `tran_label − 12` (label-relative, NOT fixed offset 87 — fixed offset only incidentally correct for sub_code=0x10). |
| 2026-04-05 | §7.8 | **CONFIRMED — 5A "Project:" string is session-start config, NOT per-event.** The "Project:" value in A5 frame 7 reflects the compliance setup active when the monitoring session started, not when the individual event was recorded. `_decode_a5_metadata_into()` only sets `project` from 5A when 0C did not already supply one. "Client:", "User Name:", "Seis Loc:", and "Extended Notes" are NOT in the 0C record and are set unconditionally from 5A. |
| 2026-04-06 | §5.1 | **CORRECTED — SUB 1F token position is params[7], NOT params[6].** Both 3-31-26 and 4-3-26 BW TX captures confirm: raw params `00 00 00 00 00 00 00 FE 00 00`, token byte at index 7. Previous doc (params[6]) was wrong — with wrong position the device ignores the token and 1F returns null immediately. |
| 2026-04-06 | §6.1 | **NEW — Full event download sequence documented** (§6.1). Sequence confirmed from 4-2-26 and 4-3-26 BW TX captures: `1E(all-zero) → [per event: 0A → 1E(arm/0xFE) → 0C → 1F(arm/0xFE) → POLL×3 → 5A → 1F(browse)]`. Each step documented with confirmed requirements. |
| 2026-04-06 | §6.1 | **CONFIRMED — 1E(token=0xFE) arm step required.** Device silently ignores all 5A probe frames unless a second 1E with token=0xFE is sent between 0A and 0C. Present in every download cycle in the 4-2-26 and 4-3-26 captures. |
| 2026-04-06 | §6.1 | **CONFIRMED — SUB 1F(token=0xFE) must precede POLL×3 before 5A.** BW always sends 1F(0xFE) before the 3 POLL cycles before 5A. 5A still uses the pre-advance key (set by 0A+1E-arm+0C); 1F only arms the device's 5A state machine. |
| 2026-04-06 | §6.1 | **CONFIRMED — browse 1F must be conditional.** Calling 1F(browse=True/all-zero) after a FAILED 5A disrupts device state and causes the next event's 5A probe to time out with 0 bytes received. Browse 1F is only called after a SUCCESSFUL 5A. Failure fallback: use the key returned by the prior 1F(arm/0xFE) call. |
| 2026-04-06 | §7.8 | **ADDED — `bytes_fed` diagnostic counter on S3FrameParser.** Counts raw bytes fed to the parser since last `reset()`. Logged at WARNING when 5A probe times out — distinguishes "device sent no bytes at all" from "device responded but frame was malformed or had wrong SUB". |
---
@@ -226,15 +234,15 @@ Step 4 — Device sends actual data payload:
| `08` | **EVENT INDEX READ** | Requests the event record index (0x58 bytes). Event count and record pointers. | ✅ CONFIRMED |
| `06` | **CHANNEL CONFIG READ** | Requests channel configuration block (0x24 bytes). | ✅ CONFIRMED |
| `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED |
| `1E` | **EVENT HEADER READ** | Gets the first waveform key (4-byte opaque record address). All-zero params; key returned at data[11:15]. | ✅ CONFIRMED 2026-03-31 |
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. | ✅ CONFIRMED 2026-03-31 |
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. 9-byte timestamp at bytes[0–8]; record sub_code at byte[1] (0x10=Waveform); PPV floats at channel label+6; Peak Vector Sum float at offset 87; project strings. Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-01 |
| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[6]: 0x00=browse (one step), 0xFE=download (skip partial bins). Returns next key at data[11:15]; zeros = no more events. | ✅ CONFIRMED 2026-03-31 |
| `5A` | **BULK WAVEFORM STREAM** | Initiates bulk download of raw ADC sample data, keyed by waveform key. Large multi-page transfer. | ✅ CONFIRMED |
| `1E` | **EVENT HEADER READ** | Gets first waveform key. Token byte at params[7] (0x00=browse, 0xFE=download-arm). Key at data[11:15]; trailing offset at data[15:19] (0 = only one event). Two uses: (1) all-zero to get key0; (2) token=0xFE after 0A, before 0C — REQUIRED to arm device for SUB 5A. | ✅ CONFIRMED 2026-04-06 |
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 |
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label−12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 |
| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 |
| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum. Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. A5[7] contains event-time metadata (Project:/Client:/User Name:/Seis Loc:). 9+ A5 frames for full waveform; stop_after_metadata=True exits after A5[7]. | ✅ CONFIRMED 2026-04-06 |
| `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED |
| `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED |
| `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED |
| `1A` | **CHANNEL SCALING / COMPLIANCE CONFIG READ** | Read command, response (`E5`) returns large block containing IEEE 754 floats including trigger level, alarm level, max range, and unit strings. Contains `0x082A` — purpose unknown, possibly alarm threshold or record config. | 🔶 INFERRED |
| `1A` | **COMPLIANCE CONFIG READ** | Multi-step sequence (A+B+C+D frames). Response (E5) carries sample_rate (uint16 BE at anchor−2), record_time (float32 BE at anchor+10), trigger/alarm/max_range floats, and project strings. Anchor: `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00`, search cfg[0:150]. Total ~2126 cfg bytes. | ✅ CONFIRMED 2026-04-02 |
| `2E` | **UNKNOWN READ B** | Read command, response (`D1`) returns 0x1A (26) bytes. Purpose unknown. | 🔶 INFERRED |
All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which, after de-stuffing, is just the DLE+CMD combination — see §3).
@@ -305,20 +313,35 @@ Write commands are initiated by Blastware (`BW->S3`) and use SUB bytes in the `0
4. S3 → 0x41 + POLL RESPONSE (SUB A4, reports data length = 0x30)
5. BW → 0x41 + POLL frame (SUB 5B, offset = 0x30)
6. S3 → 0x41 + POLL RESPONSE with data: "Instantel" + "MiniMate Plus"
7. [Poll loop repeats 3–5× during initialization]
8. BW → SUB 06 → channel config read
9. BW → SUB 15 → serial number
10. BW → SUB 01 → full config block
7. BW → SUB 06 → channel config read
8. BW → SUB 15 → serial number
9. BW → SUB 01 → full config block
10. BW → SUB 1A → compliance config (4-frame sequence: A+B+C+D)
11. BW → SUB 08 → event index
12. BW → SUB 1E → first event header
13. BW → SUB 0A → waveform header (timestamp-keyed)
14. BW → SUB 0C → full waveform record download (2 pages)
15. BW → SUB 1F → advance / close event
16. [Repeat steps 12–15 for each stored event]
17. BW → SUB 5A → bulk raw waveform stream
18. Poll loop resumes (SUB 5B keepalive every ~80ms)
```
### 6.1 Event Download Sequence (per-event, confirmed from 4-2-26 + 4-3-26 BW captures)
```
# Once per session:
BW → SUB 1E (all-zero params) → key0, trailing0
# Per event (repeat until null sentinel data[15:19]=0x00000000):
BW → SUB 0A (key) ← REQUIRED before every 1F; establishes device context
BW → SUB 1E (token=0xFE) ← REQUIRED arm step; must be BETWEEN 0A and 0C
BW → SUB 0C (key) ← read waveform record (peaks, timestamp, project)
BW → SUB 1F (token=0xFE) ← arm 5A state machine; do NOT use returned key
BW → SUB 5B × 3 ← REQUIRED: 3 full POLL probe+data cycles before 5A
BW → SUB 5A (key) ← bulk waveform stream (A5 frames)
# IF 5A succeeded:
BW → SUB 1F (all-zero) → next_key ← use this for loop iteration
# IF 5A failed/timed out:
# Do NOT send 1F(browse) — use cached key from 1F(token=0xFE) above instead
```
**Null sentinel:** `data[15:19] == b"\x00\x00\x00\x00"` in both 1E and 1F responses.
Do NOT use `data[11:15]` (key) as sentinel — event 0 has key=0x00000000.
---
## 7. Known Data Payloads
minimateplus/client.py
+57 -19
View File
@@ -326,14 +326,13 @@ class MiniMateClient:
)
# SUB 1F (download-arm) — send token=0xFE BEFORE POLL+5A to arm the
# device's bulk stream state machine. The key returned by this call
# is NOT used for loop iteration — 1F(download) returns inconsistent
# keys after the first event (device-side state issue). A second
# browse-mode 1F call after 5A handles actual key advancement.
# device's bulk stream state machine. Cache the returned key as a
# fallback for loop iteration when 5A fails (see iteration block below).
# Confirmed from 4-2-26 capture frames 66-67 (1F before frames 68-73 POLL).
arm_key4: Optional[bytes] = None
try:
proto.advance_event(browse=False) # arm 5A — discard returned key
log.info("get_events: 1F(download) — 5A armed")
arm_key4, _ = proto.advance_event(browse=False) # arm 5A
log.info("get_events: 1F(download) — 5A armed, arm_key=%s", arm_key4.hex())
except ProtocolError as exc:
log.warning("get_events: 1F(download) arm failed: %s", exc)
@@ -349,6 +348,7 @@ class MiniMateClient:
# SUB 5A — bulk waveform stream (uses cur_key, the event set up by 0A+1E+0C).
# By default (full_waveform=False): stop after frame 7 for metadata only.
# When full_waveform=True: fetch all chunks and decode raw ADC samples.
a5_ok = False
try:
if full_waveform:
log.info(
@@ -358,6 +358,7 @@ class MiniMateClient:
cur_key, stop_after_metadata=False, max_chunks=128
)
if a5_frames:
a5_ok = True
_decode_a5_metadata_into(a5_frames, ev)
_decode_a5_waveform(a5_frames, ev)
log.info(
@@ -372,6 +373,7 @@ class MiniMateClient:
cur_key, stop_after_metadata=True
)
if a5_frames:
a5_ok = True
_decode_a5_metadata_into(a5_frames, ev)
log.debug(
"get_events: 5A metadata client=%r operator=%r",
@@ -384,19 +386,55 @@ class MiniMateClient:
cur_key.hex(), exc,
)
# SUB 1F (browse) — advance event pointer for loop iteration.
# browse=True (all-zero params) confirmed correct for multi-event
# iteration from 4-3-26 browse-mode S3 captures. Must come AFTER
# 5A — the download-mode 1F above is for 5A arming only.
try:
key4, data8 = proto.advance_event(browse=True)
log.info(
"get_events: 1F(browse) → key=%s trailing=%s",
key4.hex(), data8[4:8].hex(),
)
except ProtocolError as exc:
log.warning("get_events: 1F(browse) failed: %s — stopping", exc)
key4, data8 = b"\x00\x00\x00\x00", b"\x00\x00\x00\x00\x00\x00\x00\x00"
# SUB 1F — loop iteration.
#
# IMPORTANT: browse 1F (all-zero params) is ONLY called when 5A
# succeeded. If 5A timed out or failed, calling browse 1F disrupts
# the device's internal state and causes the NEXT event's 5A to also
# fail. In the failure path, use the key cached from 1F(download)
# above as a best-effort fallback for iteration.
#
# Confirmed from 4-3-26 browse-mode captures: browse=True params
# are correct for multi-event iteration. Conditional logic added
# 2026-04-06 to avoid post-failure state disruption.
if a5_ok:
# 5A succeeded — use browse 1F for reliable key advancement.
try:
key4, data8 = proto.advance_event(browse=True)
log.info(
"get_events: 1F(browse) → key=%s trailing=%s",
key4.hex(), data8[4:8].hex(),
)
except ProtocolError as exc:
log.warning("get_events: 1F(browse) failed: %s — stopping", exc)
key4 = b"\x00\x00\x00\x00"
data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00"
else:
# 5A failed — skip browse 1F to avoid further state disruption.
# Use the arm_key4 returned by 1F(download) as the next-key hint.
if arm_key4 is None or arm_key4 == cur_key:
# 1F(download) returned no valid next key (or same key = stuck).
# Stop iteration to prevent infinite loop.
log.warning(
"get_events: 5A failed and 1F(download) returned no valid "
"next key (arm_key=%s, cur_key=%s) — stopping iteration",
arm_key4.hex() if arm_key4 else "None",
cur_key.hex(),
)
key4 = b"\x00\x00\x00\x00"
data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00"
else:
# arm_key4 is a valid non-stuck next key — use it.
# Construct a synthetic data8 with non-null trailing so the
# loop continues (the real trailing is unknown but non-null
# since we have a valid arm_key4).
key4 = arm_key4
data8 = arm_key4 + b"\x00\x00\x00\x01"
log.warning(
"get_events: 5A failed — advancing via arm_key=%s "
"(browse 1F skipped to preserve device state)",
key4.hex(),
)
events.append(ev)
idx += 1
minimateplus/framing.py
+6 -3
View File
@@ -386,13 +386,15 @@ class S3FrameParser:
_IN_FRAME_DLE = 3
def __init__(self) -> None:
self._state = self._IDLE
self._body = bytearray() # accumulates de-stuffed frame bytes
self._state = self._IDLE
self._body = bytearray() # accumulates de-stuffed frame bytes
self.frames: list[S3Frame] = []
self.bytes_fed: int = 0 # cumulative raw bytes fed since last reset
def reset(self) -> None:
self._state = self._IDLE
self._state = self._IDLE
self._body.clear()
self.bytes_fed = 0
def feed(self, data: bytes) -> list[S3Frame]:
"""
@@ -401,6 +403,7 @@ class S3FrameParser:
Returns a list of S3Frame objects completed during this call.
All completed frames are also appended to self.frames.
"""
self.bytes_fed += len(data)
completed: list[S3Frame] = []
for b in data:
frame = self._step(b)
minimateplus/protocol.py
+10 -1
View File
@@ -494,7 +494,16 @@ class MiniMateProtocol:
log.debug("5A probe key=%s", key4.hex())
params = bulk_waveform_params(key4, 0, is_probe=True)
self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
rsp = self._recv_one(expected_sub=rsp_sub)
self._parser.reset() # reset bytes_fed counter before probe recv
try:
rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False)
except TimeoutError:
log.warning(
"5A probe TIMED OUT for key=%s — "
"%d raw bytes received (no complete A5 frame assembled)",
key4.hex(), self._parser.bytes_fed,
)
raise
frames_data.append(rsp.data)
log.debug("5A A5[0] page_key=0x%04X %d bytes", rsp.page_key, len(rsp.data))