diff --git a/CLAUDE.md b/CLAUDE.md index af5ff2f..ec524e6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -205,12 +205,23 @@ POLL × 3 1F(browse=True) → null ← done ``` -**IMPORTANT — two separate 1F calls per event (CONFIRMED 2026-04-06):** -`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine -but returns inconsistent iteration keys after the first event — do NOT use its returned key -for loop control. `1F(browse=True)` AFTER 5A returns the correct next event key for -iteration. The reason is device-side: `1F(0xFE)` appears to arm 5A and peek the next key -without reliably advancing the internal iteration pointer; `1F(all-zero)` advances it. +**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):** +`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine. +Its returned key is cached as `arm_key4` in `get_events()`. + +`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**. If 5A timed out or failed, +sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next +event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the +next event's 5A probe to also time out with 0 bytes received). + +In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint: +- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call +- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort + +The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call, +reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received +but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in +the log but have very different root causes. **The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):** The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE diff --git a/docs/instantel_protocol_reference.md b/docs/instantel_protocol_reference.md index fbd4890..719a80e 100644 --- a/docs/instantel_protocol_reference.md +++ b/docs/instantel_protocol_reference.md @@ -80,6 +80,14 @@ | 2026-04-03 | §7.6 | **CONFIRMED — Blast waveform format (4-2-26 capture).** Blast/waveform-mode SUB 5A stream uses 4-channel interleaved signed int16 LE, 8 bytes per sample-set [T,V,L,M]. NOT the 32-byte block format (which is noise/histogram mode only). Frame sizes are NOT multiples of 8 — cross-frame alignment correction required (track global byte offset mod 8; skip `(8-align)%8` bytes at each frame start). A5[0] STRT record confirmed: 21 bytes at db[7:]+11; waveform starts at strt_pos+27 (after 2-byte null pad + 4-byte 0xFF sentinel). Frame index 7 = metadata only, no ADC data. Full §7.6 rewritten. | | 2026-04-03 | §7.6 | **CONFIRMED — Noise block format details.** 32-byte blocks: LE uint16 type + LE uint16 ctr + 9×int16 LE samples + 10B metadata. Samples are little-endian (previous doc said big-endian — WRONG). Type: 0x0016=sync (appears at start of each A5 frame), 0x0000=data. Noise floor ≈ 9–11 counts. Metadata fixed pattern `00 01 43 [2B var] 00 [pretrig] [rectime] 00 00` confirmed. | | 2026-04-03 | client.py | **NEW — `_decode_a5_waveform()` and `download_waveform()` implemented.** `_decode_a5_waveform(frames_data, event)` decodes full A5 waveform stream into `event.raw_samples = {"Tran":[…], "Vert":[…], "Long":[…], "Mic":[…]}`. Populates `event.total_samples`, `event.pretrig_samples`, `event.rectime_seconds` from STRT record. Handles cross-frame alignment. `MiniMateClient.download_waveform(event)` calls `read_bulk_waveform_stream(stop_after_metadata=False)` then invokes the decoder. Waveform key stored on Event as `_waveform_key` during `get_events()`. | +| 2026-04-03 | §7.7.5 | **CONFIRMED — sub_code=0x03 (Waveform continuous) uses 10-byte timestamp header** — one byte wider than sub_code=0x10 layout. Cross-referenced against Blastware event report for BE11529 (15:20:17 Apr 3 2026). Raw header: `10 03 10 04 07 ea 00 0f 14 11` = [unknown_a][day][unknown_b][month][year:2 BE][unknown][hour][min][sec]. Peak Vector Sum is at `tran_label − 12` (label-relative, NOT fixed offset 87 — fixed offset only incidentally correct for sub_code=0x10). | +| 2026-04-05 | §7.8 | **CONFIRMED — 5A "Project:" string is session-start config, NOT per-event.** The "Project:" value in A5 frame 7 reflects the compliance setup active when the monitoring session started, not when the individual event was recorded. `_decode_a5_metadata_into()` only sets `project` from 5A when 0C did not already supply one. "Client:", "User Name:", "Seis Loc:", and "Extended Notes" are NOT in the 0C record and are set unconditionally from 5A. | +| 2026-04-06 | §5.1 | **CORRECTED — SUB 1F token position is params[7], NOT params[6].** Both 3-31-26 and 4-3-26 BW TX captures confirm: raw params `00 00 00 00 00 00 00 FE 00 00`, token byte at index 7. Previous doc (params[6]) was wrong — with wrong position the device ignores the token and 1F returns null immediately. | +| 2026-04-06 | §6.1 | **NEW — Full event download sequence documented** (§6.1). Sequence confirmed from 4-2-26 and 4-3-26 BW TX captures: `1E(all-zero) → [per event: 0A → 1E(arm/0xFE) → 0C → 1F(arm/0xFE) → POLL×3 → 5A → 1F(browse)]`. Each step documented with confirmed requirements. | +| 2026-04-06 | §6.1 | **CONFIRMED — 1E(token=0xFE) arm step required.** Device silently ignores all 5A probe frames unless a second 1E with token=0xFE is sent between 0A and 0C. Present in every download cycle in the 4-2-26 and 4-3-26 captures. | +| 2026-04-06 | §6.1 | **CONFIRMED — SUB 1F(token=0xFE) must precede POLL×3 before 5A.** BW always sends 1F(0xFE) before the 3 POLL cycles before 5A. 5A still uses the pre-advance key (set by 0A+1E-arm+0C); 1F only arms the device's 5A state machine. | +| 2026-04-06 | §6.1 | **CONFIRMED — browse 1F must be conditional.** Calling 1F(browse=True/all-zero) after a FAILED 5A disrupts device state and causes the next event's 5A probe to time out with 0 bytes received. Browse 1F is only called after a SUCCESSFUL 5A. Failure fallback: use the key returned by the prior 1F(arm/0xFE) call. | +| 2026-04-06 | §7.8 | **ADDED — `bytes_fed` diagnostic counter on S3FrameParser.** Counts raw bytes fed to the parser since last `reset()`. Logged at WARNING when 5A probe times out — distinguishes "device sent no bytes at all" from "device responded but frame was malformed or had wrong SUB". | --- @@ -226,15 +234,15 @@ Step 4 — Device sends actual data payload: | `08` | **EVENT INDEX READ** | Requests the event record index (0x58 bytes). Event count and record pointers. | ✅ CONFIRMED | | `06` | **CHANNEL CONFIG READ** | Requests channel configuration block (0x24 bytes). | ✅ CONFIRMED | | `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED | -| `1E` | **EVENT HEADER READ** | Gets the first waveform key (4-byte opaque record address). All-zero params; key returned at data[11:15]. | ✅ CONFIRMED 2026-03-31 | -| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. | ✅ CONFIRMED 2026-03-31 | -| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. 9-byte timestamp at bytes[0–8]; record sub_code at byte[1] (0x10=Waveform); PPV floats at channel label+6; Peak Vector Sum float at offset 87; project strings. Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-01 | -| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[6]: 0x00=browse (one step), 0xFE=download (skip partial bins). Returns next key at data[11:15]; zeros = no more events. | ✅ CONFIRMED 2026-03-31 | -| `5A` | **BULK WAVEFORM STREAM** | Initiates bulk download of raw ADC sample data, keyed by waveform key. Large multi-page transfer. | ✅ CONFIRMED | +| `1E` | **EVENT HEADER READ** | Gets first waveform key. Token byte at params[7] (0x00=browse, 0xFE=download-arm). Key at data[11:15]; trailing offset at data[15:19] (0 = only one event). Two uses: (1) all-zero to get key0; (2) token=0xFE after 0A, before 0C — REQUIRED to arm device for SUB 5A. | ✅ CONFIRMED 2026-04-06 | +| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 | +| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label−12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 | +| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 | +| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum. Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. A5[7] contains event-time metadata (Project:/Client:/User Name:/Seis Loc:). 9+ A5 frames for full waveform; stop_after_metadata=True exits after A5[7]. | ✅ CONFIRMED 2026-04-06 | | `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED | | `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED | | `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED | -| `1A` | **CHANNEL SCALING / COMPLIANCE CONFIG READ** | Read command, response (`E5`) returns large block containing IEEE 754 floats including trigger level, alarm level, max range, and unit strings. Contains `0x082A` — purpose unknown, possibly alarm threshold or record config. | 🔶 INFERRED | +| `1A` | **COMPLIANCE CONFIG READ** | Multi-step sequence (A+B+C+D frames). Response (E5) carries sample_rate (uint16 BE at anchor−2), record_time (float32 BE at anchor+10), trigger/alarm/max_range floats, and project strings. Anchor: `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00`, search cfg[0:150]. Total ~2126 cfg bytes. | ✅ CONFIRMED 2026-04-02 | | `2E` | **UNKNOWN READ B** | Read command, response (`D1`) returns 0x1A (26) bytes. Purpose unknown. | 🔶 INFERRED | All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which, after de-stuffing, is just the DLE+CMD combination — see §3). @@ -305,20 +313,35 @@ Write commands are initiated by Blastware (`BW->S3`) and use SUB bytes in the `0 4. S3 → 0x41 + POLL RESPONSE (SUB A4, reports data length = 0x30) 5. BW → 0x41 + POLL frame (SUB 5B, offset = 0x30) 6. S3 → 0x41 + POLL RESPONSE with data: "Instantel" + "MiniMate Plus" -7. [Poll loop repeats 3–5× during initialization] -8. BW → SUB 06 → channel config read -9. BW → SUB 15 → serial number -10. BW → SUB 01 → full config block +7. BW → SUB 06 → channel config read +8. BW → SUB 15 → serial number +9. BW → SUB 01 → full config block +10. BW → SUB 1A → compliance config (4-frame sequence: A+B+C+D) 11. BW → SUB 08 → event index -12. BW → SUB 1E → first event header -13. BW → SUB 0A → waveform header (timestamp-keyed) -14. BW → SUB 0C → full waveform record download (2 pages) -15. BW → SUB 1F → advance / close event -16. [Repeat steps 12–15 for each stored event] -17. BW → SUB 5A → bulk raw waveform stream -18. Poll loop resumes (SUB 5B keepalive every ~80ms) ``` +### 6.1 Event Download Sequence (per-event, confirmed from 4-2-26 + 4-3-26 BW captures) + +``` +# Once per session: +BW → SUB 1E (all-zero params) → key0, trailing0 + +# Per event (repeat until null sentinel data[15:19]=0x00000000): +BW → SUB 0A (key) ← REQUIRED before every 1F; establishes device context +BW → SUB 1E (token=0xFE) ← REQUIRED arm step; must be BETWEEN 0A and 0C +BW → SUB 0C (key) ← read waveform record (peaks, timestamp, project) +BW → SUB 1F (token=0xFE) ← arm 5A state machine; do NOT use returned key +BW → SUB 5B × 3 ← REQUIRED: 3 full POLL probe+data cycles before 5A +BW → SUB 5A (key) ← bulk waveform stream (A5 frames) +# IF 5A succeeded: +BW → SUB 1F (all-zero) → next_key ← use this for loop iteration +# IF 5A failed/timed out: +# Do NOT send 1F(browse) — use cached key from 1F(token=0xFE) above instead +``` + +**Null sentinel:** `data[15:19] == b"\x00\x00\x00\x00"` in both 1E and 1F responses. +Do NOT use `data[11:15]` (key) as sentinel — event 0 has key=0x00000000. + --- ## 7. Known Data Payloads diff --git a/minimateplus/client.py b/minimateplus/client.py index a7cc063..3f5c9d2 100644 --- a/minimateplus/client.py +++ b/minimateplus/client.py @@ -326,14 +326,13 @@ class MiniMateClient: ) # SUB 1F (download-arm) — send token=0xFE BEFORE POLL+5A to arm the - # device's bulk stream state machine. The key returned by this call - # is NOT used for loop iteration — 1F(download) returns inconsistent - # keys after the first event (device-side state issue). A second - # browse-mode 1F call after 5A handles actual key advancement. + # device's bulk stream state machine. Cache the returned key as a + # fallback for loop iteration when 5A fails (see iteration block below). # Confirmed from 4-2-26 capture frames 66-67 (1F before frames 68-73 POLL). + arm_key4: Optional[bytes] = None try: - proto.advance_event(browse=False) # arm 5A — discard returned key - log.info("get_events: 1F(download) — 5A armed") + arm_key4, _ = proto.advance_event(browse=False) # arm 5A + log.info("get_events: 1F(download) — 5A armed, arm_key=%s", arm_key4.hex()) except ProtocolError as exc: log.warning("get_events: 1F(download) arm failed: %s", exc) @@ -349,6 +348,7 @@ class MiniMateClient: # SUB 5A — bulk waveform stream (uses cur_key, the event set up by 0A+1E+0C). # By default (full_waveform=False): stop after frame 7 for metadata only. # When full_waveform=True: fetch all chunks and decode raw ADC samples. + a5_ok = False try: if full_waveform: log.info( @@ -358,6 +358,7 @@ class MiniMateClient: cur_key, stop_after_metadata=False, max_chunks=128 ) if a5_frames: + a5_ok = True _decode_a5_metadata_into(a5_frames, ev) _decode_a5_waveform(a5_frames, ev) log.info( @@ -372,6 +373,7 @@ class MiniMateClient: cur_key, stop_after_metadata=True ) if a5_frames: + a5_ok = True _decode_a5_metadata_into(a5_frames, ev) log.debug( "get_events: 5A metadata client=%r operator=%r", @@ -384,19 +386,55 @@ class MiniMateClient: cur_key.hex(), exc, ) - # SUB 1F (browse) — advance event pointer for loop iteration. - # browse=True (all-zero params) confirmed correct for multi-event - # iteration from 4-3-26 browse-mode S3 captures. Must come AFTER - # 5A — the download-mode 1F above is for 5A arming only. - try: - key4, data8 = proto.advance_event(browse=True) - log.info( - "get_events: 1F(browse) → key=%s trailing=%s", - key4.hex(), data8[4:8].hex(), - ) - except ProtocolError as exc: - log.warning("get_events: 1F(browse) failed: %s — stopping", exc) - key4, data8 = b"\x00\x00\x00\x00", b"\x00\x00\x00\x00\x00\x00\x00\x00" + # SUB 1F — loop iteration. + # + # IMPORTANT: browse 1F (all-zero params) is ONLY called when 5A + # succeeded. If 5A timed out or failed, calling browse 1F disrupts + # the device's internal state and causes the NEXT event's 5A to also + # fail. In the failure path, use the key cached from 1F(download) + # above as a best-effort fallback for iteration. + # + # Confirmed from 4-3-26 browse-mode captures: browse=True params + # are correct for multi-event iteration. Conditional logic added + # 2026-04-06 to avoid post-failure state disruption. + if a5_ok: + # 5A succeeded — use browse 1F for reliable key advancement. + try: + key4, data8 = proto.advance_event(browse=True) + log.info( + "get_events: 1F(browse) → key=%s trailing=%s", + key4.hex(), data8[4:8].hex(), + ) + except ProtocolError as exc: + log.warning("get_events: 1F(browse) failed: %s — stopping", exc) + key4 = b"\x00\x00\x00\x00" + data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00" + else: + # 5A failed — skip browse 1F to avoid further state disruption. + # Use the arm_key4 returned by 1F(download) as the next-key hint. + if arm_key4 is None or arm_key4 == cur_key: + # 1F(download) returned no valid next key (or same key = stuck). + # Stop iteration to prevent infinite loop. + log.warning( + "get_events: 5A failed and 1F(download) returned no valid " + "next key (arm_key=%s, cur_key=%s) — stopping iteration", + arm_key4.hex() if arm_key4 else "None", + cur_key.hex(), + ) + key4 = b"\x00\x00\x00\x00" + data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00" + else: + # arm_key4 is a valid non-stuck next key — use it. + # Construct a synthetic data8 with non-null trailing so the + # loop continues (the real trailing is unknown but non-null + # since we have a valid arm_key4). + key4 = arm_key4 + data8 = arm_key4 + b"\x00\x00\x00\x01" + log.warning( + "get_events: 5A failed — advancing via arm_key=%s " + "(browse 1F skipped to preserve device state)", + key4.hex(), + ) events.append(ev) idx += 1 diff --git a/minimateplus/framing.py b/minimateplus/framing.py index a7ad295..28c4c4f 100644 --- a/minimateplus/framing.py +++ b/minimateplus/framing.py @@ -386,13 +386,15 @@ class S3FrameParser: _IN_FRAME_DLE = 3 def __init__(self) -> None: - self._state = self._IDLE - self._body = bytearray() # accumulates de-stuffed frame bytes + self._state = self._IDLE + self._body = bytearray() # accumulates de-stuffed frame bytes self.frames: list[S3Frame] = [] + self.bytes_fed: int = 0 # cumulative raw bytes fed since last reset def reset(self) -> None: - self._state = self._IDLE + self._state = self._IDLE self._body.clear() + self.bytes_fed = 0 def feed(self, data: bytes) -> list[S3Frame]: """ @@ -401,6 +403,7 @@ class S3FrameParser: Returns a list of S3Frame objects completed during this call. All completed frames are also appended to self.frames. """ + self.bytes_fed += len(data) completed: list[S3Frame] = [] for b in data: frame = self._step(b) diff --git a/minimateplus/protocol.py b/minimateplus/protocol.py index af4cbd2..d2b8208 100644 --- a/minimateplus/protocol.py +++ b/minimateplus/protocol.py @@ -494,7 +494,16 @@ class MiniMateProtocol: log.debug("5A probe key=%s", key4.hex()) params = bulk_waveform_params(key4, 0, is_probe=True) self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params)) - rsp = self._recv_one(expected_sub=rsp_sub) + self._parser.reset() # reset bytes_fed counter before probe recv + try: + rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False) + except TimeoutError: + log.warning( + "5A probe TIMED OUT for key=%s — " + "%d raw bytes received (no complete A5 frame assembled)", + key4.hex(), self._parser.bytes_fed, + ) + raise frames_data.append(rsp.data) log.debug("5A A5[0] page_key=0x%04X %d bytes", rsp.page_key, len(rsp.data))