doc: update to .0.6.0 with full working event read loop

CHANGELOG.md

@@ -4,6 +4,27 @@ All notable changes to seismo-relay are documented here.
 
 ---
 
+## v0.6.0 — 2026-04-02
+
+### Added
+- **True event-time metadata via SUB 5A bulk waveform stream** — `get_events()` now issues a SUB 5A request after each SUB 0C download, reads the A5 response frames, and extracts the `Client:`, `User Name:`, and `Seis Loc:` fields as they existed at the moment the event was recorded. Previously these fields were backfilled from the current compliance config (SUB 1A), which reflects today's setup, not the setup active when the event triggered.
+  - `build_5a_frame(offset_word, raw_params)` in `framing.py` — reproduces Blastware's exact wire format for SUB 5A requests: raw (non-DLE-stuffed) `offset_hi`, DLE-stuffed params, and a DLE-aware checksum where `10 XX` pairs count only `XX`.
+  - `bulk_waveform_params()` returns 11 bytes (extra trailing `0x00` confirmed from 1-2-26 BW wire capture).
+  - `read_bulk_waveform_stream(key4, *, stop_after_metadata=True, max_chunks=32)` in `protocol.py` — loops sending chunk requests (counter increments `0x0400` per chunk), stops early when `b"Project:"` is found, then sends a termination frame.
+  - `_decode_a5_metadata_into(frames_data, event)` in `client.py` — needle-searches A5 frame data for `Project:`, `Client:`, `User Name:`, `Seis Loc:`, `Extended Notes` and overwrites `event.project_info`.
+- **`get_events()` sequence extended** — now `1E → 0A → 0C → 5A → 1F` per event.
+
+### Fixed
+- **Compliance config (SUB 1A) channel block missing** — orphaned `self._send(build_bw_frame(SUB_COMPLIANCE, 0x2A, _DATA_PARAMS))` before the B/C/D receive loop had no corresponding `recv_one()`, shifting all subsequent receives one step behind and leaving frame D's channel-block data (trigger_level_geo, alarm_level_geo, max_range_geo) unread. Removed the orphaned send. Total config bytes received now correctly ~2126 (was ~1071).
+- **Compliance config anchor search range** — `_decode_compliance_config_into()` searched `cfg[40:100]` for the sample-rate/record-time anchor. With the orphaned-send bug fixed the 44-byte padding it had been adding is gone, and the anchor now appears at `cfg[11]`. Search widened to `cfg[0:150]` to be robust to future layout shifts.
+- Removed byte-content deduplication from `read_compliance_config()` — was masking the real receive-ordering bug.
+
+### Protocol / Documentation
+- **SUB 5A frame format confirmed** — `offset_hi` byte (`0x10`) must be sent raw (not DLE-stuffed); checksum is DLE-aware (only the second byte of a `10 XX` pair is summed). Standard `build_bw_frame` DLE-stuffs `0x10` incorrectly for 5A — a dedicated `build_5a_frame` is required.
+- **Event-time metadata source confirmed** — `Client:`, `User Name:`, and `Seis Loc:` strings are present in A5 frame 7 of the bulk waveform stream (SUB 5A), not in the 210-byte SUB 0C waveform record. They reflect the compliance setup as it was when the event was stored on the device.
+
+---
+
 ## v0.5.0 — 2026-03-31
 
 ### Added

README.md

@@ -1,4 +1,4 @@
-# seismo-relay  `v0.5.0`
+# seismo-relay  `v0.6.0`
 
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing MiniMate Plus seismographs.
@@ -6,8 +6,10 @@ software for managing MiniMate Plus seismographs.
 Built in Python. Runs on Windows. Connects to instruments over direct RS-232
 or cellular modem (Sierra Wireless RV50 / RV55).
 
-> **Status:** Active development. Core read pipeline working (device info,
+> **Status:** Active development. Full read pipeline working end-to-end:
-> config, event index). Event download and write commands in progress.
+> device info, compliance config (with geo thresholds), event download with
+> true event-time metadata (project / client / operator / sensor location
+> sourced from the device at record-time via SUB 5A). Write commands in progress.
 > See [CHANGELOG.md](CHANGELOG.md) for version history.
 
 ---
@@ -148,7 +150,7 @@ Port: 9034          ← Device Port in ACEmanager (call-up mode)
 from minimateplus.transport import TcpTransport
 from minimateplus.client import MiniMateClient
 
-client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034))
+client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
 info = client.connect()
 ```
 
@@ -182,12 +184,17 @@ client = MiniMateClient(port="COM5")
 client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
 
 with client:
-    info    = client.connect()        # DeviceInfo — model, serial, firmware
+    info    = client.connect()        # DeviceInfo — model, serial, firmware, compliance config
     serial  = client.get_serial()     # Serial number string
     config  = client.get_config()     # Full config block (bytes)
-    events  = client.get_events()     # Event index
+    events  = client.get_events()     # List[EventRecord] with true event-time metadata
 ```
 
+`get_events()` runs the full download sequence per event: `1E → 0A → 0C → 5A → 1F`.
+The SUB 5A bulk waveform stream is used to retrieve `client`, `operator`, and
+`sensor_location` as they existed at record time — not backfilled from the current
+compliance config.
+
 ---
 
 ## Protocol quick-reference
@@ -244,7 +251,8 @@ Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate P
 
 ## Roadmap
 
-- [ ] Event download — pull waveform records from the unit (SUBs `1E` → `0A` → `0C` → `5A`)
+- [x] Event download — pull waveform records from the unit (`1E → 0A → 0C → 5A → 1F`)
+- [x] True event-time metadata — project / client / operator / sensor location from SUB 5A
 - [ ] Write commands — push config changes to the unit (compliance setup, channel config, trigger settings)
 - [ ] ACH inbound server — accept call-home connections from field units
 - [ ] Modem manager — push standard configs to RV50/RV55 fleet via Sierra Wireless API

docs/instantel_protocol_reference.md

@@ -72,6 +72,11 @@
 | 2026-04-01 | §7.6.1 | **CORRECTED — Record time offset.** Previous doc (`+0x28` from E5 data page2 start) was correct for single-frame reads but unreliable for BE11529 due to a 1-byte DLE jitter (see §7.6.3). The `minimateplus` library now uses an anchor-based approach: search for `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in cfg[40:100]; record time float32 BE is at anchor+10. Validated at 3.0, 5.0, and 8.0 seconds. |
 | 2026-04-01 | §7.6.3 (NEW) | **NEW — Sample rate confirmed and documented.** Sample rate (Normal=1024 / Fast=2048 / Faster=4096 Sa/s) is stored as uint16 BE at anchor−2, where anchor is the 10-byte sequence `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00`. DLE jitter root cause: 4096 = 0x1000, so in the raw S3 frame the sample-rate bytes are sent as `10 10 00` (DLE-escaped `10`); after DLE unstuffing → `10 00` (2 bytes instead of 3 for 1024/2048), making frame C 1 byte shorter and shifting all subsequent offsets by −1. Anchor search is immune to this shift. All three modes confirmed on BE11529 firmware S338.17. |
 | 2026-04-01 | §5.1 | **CONFIRMED — `_pending_frames` buffer and `reset_parser=False` parameter.** `MiniMateProtocol._recv_one()` now supports `reset_parser=False` to preserve parser state between consecutive reads within a multi-frame sequence. A `_pending_frames: list[S3Frame]` buffer stores extra frames parsed from a single TCP chunk when multiple E5 responses arrive together. Required for reliable SUB 1A frame B/C/D sequence on BE11529. |
+| 2026-04-02 | §7.8 (NEW) | **CONFIRMED — SUB 5A frame format.** `offset_hi` byte (`0x10`) must be sent **raw, not DLE-stuffed** — standard `build_bw_frame` incorrectly stuffs it to `10 10` on the wire; device ignores the frame. BW sends it as bare `10`. Checksum is **DLE-aware**: when walking the byte sequence, `10 XX` pairs contribute only `XX` to the sum; lone bytes contribute normally. `build_5a_frame()` reproduces BW's exact wire format. |
+| 2026-04-02 | §7.8 | **CONFIRMED — SUB 5A params are 11 bytes (not 10) for chunk frames.** Extra trailing `0x00` confirmed from 1-2-26 BW wire capture. Probe frame and termination frame differ — see `bulk_waveform_params()` in `framing.py`. |
+| 2026-04-02 | §7.7.5 | **CONFIRMED — Event-time metadata source.** `Client:`, `User Name:`, and `Seis Loc:` strings are present in **A5 frame 7** of the SUB 5A bulk waveform stream — they are NOT in the 210-byte SUB 0C waveform record. They reflect the compliance setup active when the event was stored on the device (not the current setup). `get_events()` now issues SUB 5A after each 0C download. Sequence: `1E → 0A → 0C → 5A → 1F`. |
+| 2026-04-02 | §7.6.2 | **FIXED — Compliance config orphaned send bug.** An extra `self._send(SUB_COMPLIANCE / 0x2A / DATA_PARAMS)` before the B/C/D receive loop had no corresponding `recv_one()`. Every receive in the loop was consuming the previous send's response, leaving frame D's channel block unread. Bug removed. Total config bytes now ~2126 (was ~1071 due to truncation). `trigger_level_geo`, `alarm_level_geo`, `max_range_geo` are now correctly populated. |
+| 2026-04-02 | §7.6.1 | **CORRECTED — Anchor search range.** Previous doc stated anchor search range `cfg[40:100]`. With the orphaned-send bug fixed, the 44-byte header padding is gone and the anchor now appears at `cfg[11]`. Corrected to `cfg[0:150]`. |
 
 ---
 
@@ -495,7 +500,9 @@ Values are stored natively in **imperial units (in/s)** — unit strings `"in."`
 
 Record time is stored as a **32-bit IEEE 754 float, big-endian**, located via an anchor pattern (see §7.6.3 below).
 
-**Anchor-relative location:** search for the 10-byte sequence `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in cfg[40:100]. Record time float is at **anchor + 10**.
+**Anchor-relative location:** search for the 10-byte sequence `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` in `cfg[0:150]`. Record time float is at **anchor + 10**.
+
+> ✅ **2026-04-02 — CORRECTED:** Search range was `cfg[40:100]`. With the compliance-config orphaned-send bug fixed (§7.6.2), the 44-byte accidental header padding is gone and the anchor now appears at `cfg[11]`. Search range widened to `cfg[0:150]`.
 
 | Record Time | float32 BE bytes | Decoded |
 |---|---|---|
@@ -907,16 +914,20 @@ Near-ambient: 0x3C75C28F  =  0.015 in/s  (histogram event, near-zero ambient)
 
 **Project strings** — ASCII label-value pairs (search for label, read null-terminated value):
 ```
-"Project:"      → project description   (present in 0C record ✅)
+"Project:"      → project description   (in 0C record ✅)
-"Client:"       → client name           (NOT in 0C; comes from compliance config SUB 1A/E5 ❓)
+"Client:"       → client name           (in SUB 5A / A5 frame 7 ✅  — NOT in 0C)
-"User Name:"    → operator / user       (NOT confirmed in 0C ❓)
+"User Name:"    → operator / user       (in SUB 5A / A5 frame 7 ✅  — NOT in 0C)
-"Seis Loc:"     → sensor location       (NOT confirmed in 0C ❓)
+"Seis Loc:"     → sensor location       (in SUB 5A / A5 frame 7 ✅  — NOT in 0C)
-"Extended Notes"→ notes field
+"Extended Notes"→ notes field           (in SUB 5A / A5 frame 7 ✅)
 ```
 
-> ❓ **Clarification needed:** Only "Project:" has been confirmed in the 210-byte 0C record.
+> ✅ **2026-04-02 — CONFIRMED:** `Client:`, `User Name:`, and `Seis Loc:` are sourced from
-> "Client:", "User Name:", and "Seis Loc:" appear in the Blastware event report but their
+> **SUB 5A (bulk waveform stream)**, specifically A5 frame 7 of the multi-frame response.
-> source in the protocol (0C vs SUB 1A/E5 compliance config) is not yet confirmed.
+> They are NOT present in the 210-byte SUB 0C waveform record.  The strings reflect the
+> compliance setup that was active when the event was recorded on the device — making SUB 5A
+> the authoritative source for true event-time metadata.  The `get_events()` client method
+> now issues a SUB 5A request after each 0C download (`stop_after_metadata=True`) and
+> overwrites `event.project_info` with the decoded fields.
 
 ---
 
@@ -948,6 +959,101 @@ return events
 
 ---
 
+### 7.7.7 Updated Download Loop with SUB 5A Metadata
+
+> ✅ **Added 2026-04-02.** Confirmed working on BE11529 over TCP/cellular.
+
+```python
+key4, _ = proto.read_event_first()                       # SUB 1E
+if key4 == b'\x00\x00\x00\x00':
+    return []
+
+events = []
+is_first = True
+
+while key4 != b'\x00\x00\x00\x00':
+    if is_first:
+        _header, rec_len = proto.read_waveform_header(key4)  # SUB 0A
+        is_first = False
+        if rec_len < 0x30:
+            key4 = proto.advance_event()
+            continue
+
+    record = proto.read_waveform_record(key4)            # SUB 0C (0xD2 bytes)
+    event  = decode(record)
+
+    a5_data = proto.read_bulk_waveform_stream(            # SUB 5A → A5 frames
+        key4, stop_after_metadata=True)
+    client._decode_a5_metadata_into(a5_data, event)       # overwrites project_info
+
+    events.append(event)
+    key4 = proto.advance_event()                          # SUB 1F (token=0xFE)
+
+return events
+```
+
+---
+
+### 7.8 SUB 5A — Bulk Waveform Stream (event-time metadata)
+
+> ✅ **Added 2026-04-02.** Frame format confirmed by reproducing Blastware wire bytes
+> byte-for-byte from the 1-2-26 BW capture.
+
+SUB 5A initiates a bulk transfer of the raw sample data for a stored event. The response is a
+sequence of A5 frames. Frame 7 (0-indexed) contains the full compliance setup as it existed
+when the event was recorded — including `Client:`, `User Name:`, `Seis Loc:`, and
+`Extended Notes` ASCII label-value pairs.
+
+#### 7.8.1 Frame Format
+
+SUB 5A uses a **non-standard frame layout** that differs from all other BW→S3 write commands.
+
+```
+[ACK][STX][10][10][00][5A][00][offset_hi][offset_lo][params...][chk][ETX]
+  41  02  10  10  00  5A  00     ^^raw^^    ^^raw^^  ^^stuffed^^
+```
+
+Two critical differences from `build_bw_frame`:
+
+1. **`offset_hi` is sent raw, not DLE-stuffed.** When `offset_hi = 0x10`, the wire carries
+   a bare `0x10` — NOT the stuffed `10 10` that `build_bw_frame` would produce. The device
+   ignores frames where this byte is incorrectly stuffed.
+
+2. **DLE-aware checksum.** Walking the full frame byte sequence: when a `10 XX` pair is seen,
+   only `XX` is added to the running sum; lone bytes are added normally.
+
+#### 7.8.2 Request Sequence
+
+| Frame | offset_word | params | Purpose |
+|---|---|---|---|
+| Probe | `0x1004` | 10 bytes (`bulk_waveform_params(0)`) | Initiate transfer |
+| Chunk 1 | `0x1004` | 11 bytes (`bulk_waveform_params(counter)`) | First data chunk |
+| Chunk 2 | `0x1004` | 11 bytes, counter += `0x0400` | Second chunk |
+| … | … | … | … |
+| Termination | `0x005A` | 11 bytes, term_counter = last+`0x0400` | End transfer |
+
+The `stop_after_metadata=True` flag causes the loop to stop as soon as `b"Project:"` is
+found in the accumulated A5 frame data, typically after 7–9 chunks.  A termination frame
+is always sent before returning.
+
+#### 7.8.3 A5 Frame Layout
+
+Each A5 response frame contains a chunk of raw bulk data. Frame 7 of the stream carries the
+compliance text block with all project-info label-value pairs.  The `client` layer searches
+for ASCII labels with a null-terminated value read:
+
+```
+"Project:"       → null-terminated project name
+"Client:"        → null-terminated client name
+"User Name:"     → null-terminated operator name
+"Seis Loc:"      → null-terminated sensor location
+"Extended Notes" → null-terminated notes
+```
+
+All five fields reflect the **setup at event-record time**, not the current device config.
+
+---
+
 ## 8. Timestamp Format
 
 Two timestamp wire formats are used: