fix: update MiniMateClient and protocol to ensure correct handling of 1F calls and improve event download sequence

CLAUDE.md

@@ -205,12 +205,23 @@ POLL × 3
 1F(browse=True) → null                  ← done
 ```
 
-**IMPORTANT — two separate 1F calls per event (CONFIRMED 2026-04-06):**
+**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):**
-`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine
+`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine.
-but returns inconsistent iteration keys after the first event — do NOT use its returned key
+Its returned key is cached as `arm_key4` in `get_events()`.
-for loop control.  `1F(browse=True)` AFTER 5A returns the correct next event key for
+
-iteration.  The reason is device-side: `1F(0xFE)` appears to arm 5A and peek the next key
+`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**.  If 5A timed out or failed,
-without reliably advancing the internal iteration pointer; `1F(all-zero)` advances it.
+sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next
+event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the
+next event's 5A probe to also time out with 0 bytes received).
+
+In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint:
+- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call
+- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort
+
+The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call,
+reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received
+but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in
+the log but have very different root causes.
 
 **The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):**
 The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE

docs/instantel_protocol_reference.md

@@ -80,6 +80,14 @@
 | 2026-04-03 | §7.6 | **CONFIRMED — Blast waveform format (4-2-26 capture).** Blast/waveform-mode SUB 5A stream uses 4-channel interleaved signed int16 LE, 8 bytes per sample-set [T,V,L,M]. NOT the 32-byte block format (which is noise/histogram mode only). Frame sizes are NOT multiples of 8 — cross-frame alignment correction required (track global byte offset mod 8; skip `(8-align)%8` bytes at each frame start). A5[0] STRT record confirmed: 21 bytes at db[7:]+11; waveform starts at strt_pos+27 (after 2-byte null pad + 4-byte 0xFF sentinel). Frame index 7 = metadata only, no ADC data. Full §7.6 rewritten. |
 | 2026-04-03 | §7.6 | **CONFIRMED — Noise block format details.** 32-byte blocks: LE uint16 type + LE uint16 ctr + 9×int16 LE samples + 10B metadata. Samples are little-endian (previous doc said big-endian — WRONG). Type: 0x0016=sync (appears at start of each A5 frame), 0x0000=data. Noise floor ≈ 9–11 counts. Metadata fixed pattern `00 01 43 [2B var] 00 [pretrig] [rectime] 00 00` confirmed. |
 | 2026-04-03 | client.py | **NEW — `_decode_a5_waveform()` and `download_waveform()` implemented.** `_decode_a5_waveform(frames_data, event)` decodes full A5 waveform stream into `event.raw_samples = {"Tran":[…], "Vert":[…], "Long":[…], "Mic":[…]}`. Populates `event.total_samples`, `event.pretrig_samples`, `event.rectime_seconds` from STRT record. Handles cross-frame alignment. `MiniMateClient.download_waveform(event)` calls `read_bulk_waveform_stream(stop_after_metadata=False)` then invokes the decoder. Waveform key stored on Event as `_waveform_key` during `get_events()`. |
+| 2026-04-03 | §7.7.5 | **CONFIRMED — sub_code=0x03 (Waveform continuous) uses 10-byte timestamp header** — one byte wider than sub_code=0x10 layout. Cross-referenced against Blastware event report for BE11529 (15:20:17 Apr 3 2026). Raw header: `10 03 10 04 07 ea 00 0f 14 11` = [unknown_a][day][unknown_b][month][year:2 BE][unknown][hour][min][sec]. Peak Vector Sum is at `tran_label − 12` (label-relative, NOT fixed offset 87 — fixed offset only incidentally correct for sub_code=0x10). |
+| 2026-04-05 | §7.8 | **CONFIRMED — 5A "Project:" string is session-start config, NOT per-event.** The "Project:" value in A5 frame 7 reflects the compliance setup active when the monitoring session started, not when the individual event was recorded. `_decode_a5_metadata_into()` only sets `project` from 5A when 0C did not already supply one. "Client:", "User Name:", "Seis Loc:", and "Extended Notes" are NOT in the 0C record and are set unconditionally from 5A. |
+| 2026-04-06 | §5.1 | **CORRECTED — SUB 1F token position is params[7], NOT params[6].** Both 3-31-26 and 4-3-26 BW TX captures confirm: raw params `00 00 00 00 00 00 00 FE 00 00`, token byte at index 7. Previous doc (params[6]) was wrong — with wrong position the device ignores the token and 1F returns null immediately. |
+| 2026-04-06 | §6.1 | **NEW — Full event download sequence documented** (§6.1). Sequence confirmed from 4-2-26 and 4-3-26 BW TX captures: `1E(all-zero) → [per event: 0A → 1E(arm/0xFE) → 0C → 1F(arm/0xFE) → POLL×3 → 5A → 1F(browse)]`. Each step documented with confirmed requirements. |
+| 2026-04-06 | §6.1 | **CONFIRMED — 1E(token=0xFE) arm step required.** Device silently ignores all 5A probe frames unless a second 1E with token=0xFE is sent between 0A and 0C. Present in every download cycle in the 4-2-26 and 4-3-26 captures. |
+| 2026-04-06 | §6.1 | **CONFIRMED — SUB 1F(token=0xFE) must precede POLL×3 before 5A.** BW always sends 1F(0xFE) before the 3 POLL cycles before 5A. 5A still uses the pre-advance key (set by 0A+1E-arm+0C); 1F only arms the device's 5A state machine. |
+| 2026-04-06 | §6.1 | **CONFIRMED — browse 1F must be conditional.** Calling 1F(browse=True/all-zero) after a FAILED 5A disrupts device state and causes the next event's 5A probe to time out with 0 bytes received. Browse 1F is only called after a SUCCESSFUL 5A. Failure fallback: use the key returned by the prior 1F(arm/0xFE) call. |
+| 2026-04-06 | §7.8 | **ADDED — `bytes_fed` diagnostic counter on S3FrameParser.** Counts raw bytes fed to the parser since last `reset()`. Logged at WARNING when 5A probe times out — distinguishes "device sent no bytes at all" from "device responded but frame was malformed or had wrong SUB". |
 
 ---
 
@@ -226,15 +234,15 @@ Step 4 — Device sends actual data payload:
 | `08` | **EVENT INDEX READ** | Requests the event record index (0x58 bytes). Event count and record pointers. | ✅ CONFIRMED |
 | `06` | **CHANNEL CONFIG READ** | Requests channel configuration block (0x24 bytes). | ✅ CONFIRMED |
 | `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED |
-| `1E` | **EVENT HEADER READ** | Gets the first waveform key (4-byte opaque record address). All-zero params; key returned at data[11:15]. | ✅ CONFIRMED 2026-03-31 |
+| `1E` | **EVENT HEADER READ** | Gets first waveform key. Token byte at params[7] (0x00=browse, 0xFE=download-arm). Key at data[11:15]; trailing offset at data[15:19] (0 = only one event). Two uses: (1) all-zero to get key0; (2) token=0xFE after 0A, before 0C — REQUIRED to arm device for SUB 5A. | ✅ CONFIRMED 2026-04-06 |
-| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. | ✅ CONFIRMED 2026-03-31 |
+| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 |
-| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. 9-byte timestamp at bytes[0–8]; record sub_code at byte[1] (0x10=Waveform); PPV floats at channel label+6; Peak Vector Sum float at offset 87; project strings. Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-01 |
+| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label−12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 |
-| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[6]: 0x00=browse (one step), 0xFE=download (skip partial bins). Returns next key at data[11:15]; zeros = no more events. | ✅ CONFIRMED 2026-03-31 |
+| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 |
-| `5A` | **BULK WAVEFORM STREAM** | Initiates bulk download of raw ADC sample data, keyed by waveform key. Large multi-page transfer. | ✅ CONFIRMED |
+| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum. Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. A5[7] contains event-time metadata (Project:/Client:/User Name:/Seis Loc:). 9+ A5 frames for full waveform; stop_after_metadata=True exits after A5[7]. | ✅ CONFIRMED 2026-04-06 |
 | `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED |
 | `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED |
 | `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED |
-| `1A` | **CHANNEL SCALING / COMPLIANCE CONFIG READ** | Read command, response (`E5`) returns large block containing IEEE 754 floats including trigger level, alarm level, max range, and unit strings. Contains `0x082A` — purpose unknown, possibly alarm threshold or record config. | 🔶 INFERRED |
+| `1A` | **COMPLIANCE CONFIG READ** | Multi-step sequence (A+B+C+D frames). Response (E5) carries sample_rate (uint16 BE at anchor−2), record_time (float32 BE at anchor+10), trigger/alarm/max_range floats, and project strings. Anchor: `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00`, search cfg[0:150]. Total ~2126 cfg bytes. | ✅ CONFIRMED 2026-04-02 |
 | `2E` | **UNKNOWN READ B** | Read command, response (`D1`) returns 0x1A (26) bytes. Purpose unknown. | 🔶 INFERRED |
 
 All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which, after de-stuffing, is just the DLE+CMD combination — see §3).
@@ -305,20 +313,35 @@ Write commands are initiated by Blastware (`BW->S3`) and use SUB bytes in the `0
 4.  S3  →  0x41 + POLL RESPONSE (SUB A4, reports data length = 0x30)
 5.  BW  →  0x41 + POLL frame (SUB 5B, offset = 0x30)
 6.  S3  →  0x41 + POLL RESPONSE with data: "Instantel" + "MiniMate Plus"
-7.  [Poll loop repeats 3–5× during initialization]
+7.  BW  →  SUB 06  → channel config read
-8.  BW  →  SUB 06  → channel config read
+8.  BW  →  SUB 15  → serial number
-9.  BW  →  SUB 15  → serial number
+9.  BW  →  SUB 01  → full config block
-10. BW  →  SUB 01  → full config block
+10. BW  →  SUB 1A  → compliance config (4-frame sequence: A+B+C+D)
 11. BW  →  SUB 08  → event index
-12. BW  →  SUB 1E  → first event header
-13. BW  →  SUB 0A  → waveform header (timestamp-keyed)
-14. BW  →  SUB 0C  → full waveform record download (2 pages)
-15. BW  →  SUB 1F  → advance / close event
-16. [Repeat steps 12–15 for each stored event]
-17. BW  →  SUB 5A  → bulk raw waveform stream
-18. Poll loop resumes (SUB 5B keepalive every ~80ms)
 ```
 
+### 6.1 Event Download Sequence (per-event, confirmed from 4-2-26 + 4-3-26 BW captures)
+
+```
+# Once per session:
+BW  →  SUB 1E (all-zero params)    → key0, trailing0
+
+# Per event (repeat until null sentinel data[15:19]=0x00000000):
+BW  →  SUB 0A (key)                ← REQUIRED before every 1F; establishes device context
+BW  →  SUB 1E (token=0xFE)         ← REQUIRED arm step; must be BETWEEN 0A and 0C
+BW  →  SUB 0C (key)                ← read waveform record (peaks, timestamp, project)
+BW  →  SUB 1F (token=0xFE)         ← arm 5A state machine; do NOT use returned key
+BW  →  SUB 5B × 3                  ← REQUIRED: 3 full POLL probe+data cycles before 5A
+BW  →  SUB 5A (key)                ← bulk waveform stream (A5 frames)
+# IF 5A succeeded:
+BW  →  SUB 1F (all-zero)           → next_key  ← use this for loop iteration
+# IF 5A failed/timed out:
+# Do NOT send 1F(browse) — use cached key from 1F(token=0xFE) above instead
+```
+
+**Null sentinel:** `data[15:19] == b"\x00\x00\x00\x00"` in both 1E and 1F responses.
+Do NOT use `data[11:15]` (key) as sentinel — event 0 has key=0x00000000.
+
 ---
 
 ## 7. Known Data Payloads

minimateplus/client.py

@@ -326,14 +326,13 @@ class MiniMateClient:
                     )
 
                 # SUB 1F (download-arm) — send token=0xFE BEFORE POLL+5A to arm the
-                # device's bulk stream state machine.  The key returned by this call
+                # device's bulk stream state machine.  Cache the returned key as a
-                # is NOT used for loop iteration — 1F(download) returns inconsistent
+                # fallback for loop iteration when 5A fails (see iteration block below).
-                # keys after the first event (device-side state issue).  A second
-                # browse-mode 1F call after 5A handles actual key advancement.
                 # Confirmed from 4-2-26 capture frames 66-67 (1F before frames 68-73 POLL).
+                arm_key4: Optional[bytes] = None
                 try:
-                    proto.advance_event(browse=False)   # arm 5A — discard returned key
+                    arm_key4, _ = proto.advance_event(browse=False)   # arm 5A
-                    log.info("get_events: 1F(download) — 5A armed")
+                    log.info("get_events: 1F(download) — 5A armed, arm_key=%s", arm_key4.hex())
                 except ProtocolError as exc:
                     log.warning("get_events: 1F(download) arm failed: %s", exc)
 
@@ -349,6 +348,7 @@ class MiniMateClient:
                 # SUB 5A — bulk waveform stream (uses cur_key, the event set up by 0A+1E+0C).
                 # By default (full_waveform=False): stop after frame 7 for metadata only.
                 # When full_waveform=True: fetch all chunks and decode raw ADC samples.
+                a5_ok = False
                 try:
                     if full_waveform:
                         log.info(
@@ -358,6 +358,7 @@ class MiniMateClient:
                             cur_key, stop_after_metadata=False, max_chunks=128
                         )
                         if a5_frames:
+                            a5_ok = True
                             _decode_a5_metadata_into(a5_frames, ev)
                             _decode_a5_waveform(a5_frames, ev)
                             log.info(
@@ -372,6 +373,7 @@ class MiniMateClient:
                             cur_key, stop_after_metadata=True
                         )
                         if a5_frames:
+                            a5_ok = True
                             _decode_a5_metadata_into(a5_frames, ev)
                             log.debug(
                                 "get_events: 5A metadata  client=%r  operator=%r",
@@ -384,10 +386,19 @@ class MiniMateClient:
                         cur_key.hex(), exc,
                     )
 
-                # SUB 1F (browse) — advance event pointer for loop iteration.
+                # SUB 1F — loop iteration.
-                # browse=True (all-zero params) confirmed correct for multi-event
+                #
-                # iteration from 4-3-26 browse-mode S3 captures.  Must come AFTER
+                # IMPORTANT: browse 1F (all-zero params) is ONLY called when 5A
-                # 5A — the download-mode 1F above is for 5A arming only.
+                # succeeded.  If 5A timed out or failed, calling browse 1F disrupts
+                # the device's internal state and causes the NEXT event's 5A to also
+                # fail.  In the failure path, use the key cached from 1F(download)
+                # above as a best-effort fallback for iteration.
+                #
+                # Confirmed from 4-3-26 browse-mode captures: browse=True params
+                # are correct for multi-event iteration.  Conditional logic added
+                # 2026-04-06 to avoid post-failure state disruption.
+                if a5_ok:
+                    # 5A succeeded — use browse 1F for reliable key advancement.
                     try:
                         key4, data8 = proto.advance_event(browse=True)
                         log.info(
@@ -396,7 +407,34 @@ class MiniMateClient:
                         )
                     except ProtocolError as exc:
                         log.warning("get_events: 1F(browse) failed: %s — stopping", exc)
-                    key4, data8 = b"\x00\x00\x00\x00", b"\x00\x00\x00\x00\x00\x00\x00\x00"
+                        key4 = b"\x00\x00\x00\x00"
+                        data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00"
+                else:
+                    # 5A failed — skip browse 1F to avoid further state disruption.
+                    # Use the arm_key4 returned by 1F(download) as the next-key hint.
+                    if arm_key4 is None or arm_key4 == cur_key:
+                        # 1F(download) returned no valid next key (or same key = stuck).
+                        # Stop iteration to prevent infinite loop.
+                        log.warning(
+                            "get_events: 5A failed and 1F(download) returned no valid "
+                            "next key (arm_key=%s, cur_key=%s) — stopping iteration",
+                            arm_key4.hex() if arm_key4 else "None",
+                            cur_key.hex(),
+                        )
+                        key4 = b"\x00\x00\x00\x00"
+                        data8 = b"\x00\x00\x00\x00\x00\x00\x00\x00"
+                    else:
+                        # arm_key4 is a valid non-stuck next key — use it.
+                        # Construct a synthetic data8 with non-null trailing so the
+                        # loop continues (the real trailing is unknown but non-null
+                        # since we have a valid arm_key4).
+                        key4 = arm_key4
+                        data8 = arm_key4 + b"\x00\x00\x00\x01"
+                        log.warning(
+                            "get_events: 5A failed — advancing via arm_key=%s "
+                            "(browse 1F skipped to preserve device state)",
+                            key4.hex(),
+                        )
 
                 events.append(ev)
                 idx += 1

minimateplus/framing.py

@@ -389,10 +389,12 @@ class S3FrameParser:
         self._state  = self._IDLE
         self._body   = bytearray()   # accumulates de-stuffed frame bytes
         self.frames: list[S3Frame] = []
+        self.bytes_fed: int = 0       # cumulative raw bytes fed since last reset
 
     def reset(self) -> None:
         self._state    = self._IDLE
         self._body.clear()
+        self.bytes_fed = 0
 
     def feed(self, data: bytes) -> list[S3Frame]:
         """
@@ -401,6 +403,7 @@ class S3FrameParser:
         Returns a list of S3Frame objects completed during this call.
         All completed frames are also appended to self.frames.
         """
+        self.bytes_fed += len(data)
         completed: list[S3Frame] = []
         for b in data:
             frame = self._step(b)

minimateplus/protocol.py

@@ -494,7 +494,16 @@ class MiniMateProtocol:
         log.debug("5A probe  key=%s", key4.hex())
         params = bulk_waveform_params(key4, 0, is_probe=True)
         self._send(build_5a_frame(_BULK_CHUNK_OFFSET, params))
-        rsp = self._recv_one(expected_sub=rsp_sub)
+        self._parser.reset()        # reset bytes_fed counter before probe recv
+        try:
+            rsp = self._recv_one(expected_sub=rsp_sub, reset_parser=False)
+        except TimeoutError:
+            log.warning(
+                "5A probe TIMED OUT for key=%s — "
+                "%d raw bytes received (no complete A5 frame assembled)",
+                key4.hex(), self._parser.bytes_fed,
+            )
+            raise
         frames_data.append(rsp.data)
         log.debug("5A A5[0]  page_key=0x%04X  %d bytes", rsp.page_key, len(rsp.data))