# Spec — a live client for Series IV (Micromate) Drafted 2026-09-26, ahead of implementation. The protocol work is finished; this is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can run. **Read that document first.** Everything here assumes it, and every constant below is sourced from it rather than restated with justification. --- ## Goal and scope `micromate/` is codec-only today — `idf_file.py`, `models.py`, the report writers. There is no way to talk to a unit. This adds the live half, mirroring `minimateplus/`. **In scope, first pass:** - connect over TCP (a field modem) or serial/USB (a bench unit) - identify a unit, read its state, clock, memory and setups - walk the event chain and download events - return `Event` objects the existing codec already understands **Explicitly out of scope, first pass:** - ⚠ **Any write.** Setups, schedules, call-home config, monitoring start/stop, and per-event delete are all mapped, and none of them will be implemented here. **No command has ever been originated against a unit by this project** — every write observed was performed by THOR while we recorded. Keeping that true through the read client is deliberate: it means the first thing we ever send to a customer's instrument is a decision someone made on purpose, not a side effect of a client that happened to grow a method. - the inbound call-home session — still the one protocol unknown --- ## Layout ``` micromate/ framing.py NEW frame building, response parsing, checksum protocol.py NEW one method per wire command, returns raw payloads client.py NEW high-level API, returns models idf_file.py (existing — decodes what 0x5A returns, unchanged) models.py (existing — extend, do not fork) ``` **Transport is reused, not rewritten.** `minimateplus/transport.py` is byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`, `TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55 habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller. Import it. ⚠ Do **not** import `minimateplus.framing`. The two framings differ in ways that look small and are not, and a shared module would accumulate `if series ==` branches until neither case is readable. --- ## `micromate/framing.py` ### Requests Series IV accepts Series III request frames unmodified. The simplest correct implementation re-exports the builder rather than duplicating it: ```python from minimateplus.framing import build_bw_frame # requests are identical ``` ⚠ One open question, flagged in the protocol reference and **not** settled: whether `0x10` bytes inside request *params* need stuffing. No probe we sent carried one. Until it is settled, assert on it rather than guessing: ```python def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes: if 0x10 in params: raise NotImplementedError( "params containing 0x10 — stuffing rule unconfirmed; see " "micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'" ) return build_bw_frame(sub, offset, params) ``` That turns an unknown into a loud failure instead of a corrupt frame. ### Responses — where Series III's parser cannot follow | | Series III | Micromate | |---|---|---| | frame start | `DLE STX` | **bare `STX`** | | `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) | | destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** | The first row is why `S3FrameParser` returns nothing at all on Series IV traffic: it scans for `DLE+STX`, which never appears. The third is a genuine **simplification** — no inner-frame carve-out. Validated by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`: four candidate destuffing rules were tried, and only this one makes all frames validate. ### Checksum ```python def checksum(payload: bytes) -> int: return sum(b for b in payload if b != 0x10) & 0xFF ``` The DLE-aware variant, same as Series III's `5A` and write frames — not the plain SUM8 of ordinary Series III reads. ### ⚠ The SUB byte can be escaped When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`. Reading it positionally without destuffing reports `0x10`. This bit once already — `SUB 0x02` was logged as `SUB_10` for an afternoon. Destuff first, then index. ### Response shape ```python @dataclass class MicromateFrame: sub: int # response SUB; request = 0xFF - sub flags: int # 0xC5 Blastware line, 0x03 Thor line page_hi: int page_lo: int data: bytes # payload[5:], checksum stripped checksum_valid: bool @property def firmware_line(self) -> str: # "blastware" | "thor" | "unknown" @property def declared_length(self) -> int: # uint16 BE at data[3:5] (= payload[8:10]) ``` ⚠ **`declared_length` is a uint16 BE.** Read as a single byte it under-reads `SUB 0x1A` by 47x — 44 against a true 2092. This is the single most expensive mistake available in this protocol and it has already been made once. `MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`, accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is what distinguishes "no bytes at all" from "bytes but no complete frame" on a timeout, and that distinction earned its keep during the Series III work). --- ## `micromate/protocol.py` One method per command, returning raw payload bytes. No interpretation — that belongs in `client.py`. **Reads use `offset = 0xFFFF`** and return the whole block in one response; Series III's two-step probe/data dance is unnecessary. `POLL` is the exception, taking its data length. Per-command offsets, all observed: | command | SUB | rsp | offset | returns | |---|---|---|---|---| | poll | `0x5B` | `0xA4` | `0x0030` | device string, model | | serial | `0x15` | `0xEA` | `0x000A` | `UM12947` | | device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration | | state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring | | monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory | | storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent | | active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` | | first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head | | next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name | | compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block | | call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B | | arm event | `0x93` | `0x6C` | — | before every event | | first event | `0x1E` | `0xE1` | `0xFFFF` | key + size | | next event | `0x1F` | `0xE0` | `0xFFFF` | key + size | | event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks | | event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record | | bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** | ⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`). Parse **forward** from `declared_length`, never backward from the end — Series III reads battery and memory from the end of that block, and doing so on a BD unit yields a battery voltage of **577.92 V**. ⚠ **Test the monitoring flag for non-zero**, never against a constant. It has read both `0x0E` and `0x0C` while monitoring. ### `0x5A` — simpler than Series III, deliberately No arming ritual, no chunk loop, no `STRT` end-offset parsing, no `TERM` frame. One request returns the whole event: ```python offset_word = 0x1000 + 2 * ceil(size / 512) # size from the chain walk ``` The payload **is** the `.IDFW` file, byte for byte — so it feeds `micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged. ⚠ Do not port the Series III `5A` walk. Its address arithmetic caused a 5x over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series III side. None of that applies here. --- ## `micromate/client.py` ```python class MicromateClient: def __init__(self, transport: BaseTransport): ... def open(self) / close(self) / is_open(self) # identity and state def connect(self) -> DeviceInfo # poll → serial → device info → state def get_state(self) -> UnitState # monitoring?, clock, battery, memory def get_active_setup(self) -> str def list_setups(self) -> list[str] # 0x3F → 0x40… until empty # events def list_events(self) -> list[EventRef] # 0x93 → 0x1E → 0x1F… (key + size) def download_event(self, ref) -> bytes # raw .IDFW/.IDFH def get_event(self, ref) -> Event # download + decode via idf_file ``` `connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) — ⚠ but note the reference records that **whether the unit requires it is untested**. Do it because it is known-good, not because it is known-necessary, and say so in the docstring. `list_events()` returns the key *and* the size, because `download_event()` needs the size to compute its offset word. --- ## Tests **Offline, from captured bytes — no hardware.** This is the part worth doing first, because it can be fully verified tonight's-captures-style before any unit is involved. ``` tests/test_micromate_framing.py ``` ⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must not depend on files being present. **Embed the frames as hex constants** — they are 16–68 bytes each, and a handful covers every case: | case | why | |---|---| | POLL probe reply, 19 B | shortest valid frame | | POLL data reply, 68 B | contains a literal `0x10` — only the DLE-aware checksum matches | | `0x1A` response, 2108 B | exercises `declared_length` as a true uint16 (`0x082C`) | | a Thor-line reply, `flags = 0x03` | `0x03` is ETX; proves destuffing before framing | | a frame whose SUB is `0x02` | arrives as `10 02`; proves destuff-then-index | | a truncated frame | parser must return nothing, not a bad frame | | a corrupted checksum | `checksum_valid == False`, frame still returned | Then a round-trip assertion: feed a whole captured session through the parser and assert the frame count and every SUB, against `scratch/mm_frame_parse.py`'s output — which is already known good, having parsed 24, 38 and 40-frame sessions with zero bad checksums. **Live, second:** against the bench unit on mint-mac via `mm_link.py`. `connect()`, `list_setups()` (should return the 23 known names), `list_events()`, then `download_event()` and assert the bytes decode and match a `/db/import/idf_file` ingest of the same event. --- ## Order of work 1. `framing.py` + its tests — offline, verifiable immediately 2. `protocol.py` — reads only, one method per row of the table above 3. `client.py` — `connect()`, `get_state()`, `list_setups()` 4. the event chain and `download_event()` 5. decode end-to-end and compare against a store event Steps 1–2 need no hardware at all. --- ## Open questions to settle while implementing - **Request param stuffing** — raise `NotImplementedError` rather than guess. - **Is THOR's preamble required?** Try one command cold and find out; it is a two-minute test with the bench unit and it removes a ritual if unnecessary. - **`Event` model fit** — Series IV carries fields Series III lacks (setup file name, `LMic`/`SMic` channels). Extend `micromate/models.py`; do not fork the shared `Event`. - **Which `0x0C` fields to trust.** The peak float there runs 2–5% above `max(T,V,L)` and is **not** the vector sum; its offset was inferred, not established. The reference marks it do-not-rely-on — prefer decoded samples.