seismo-relay/docs/micromate_client_spec.md

# Spec — a live client for Series IV (Micromate)

Drafted 2026-09-26, ahead of implementation.  The protocol work is finished; this
is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
run.

**Read that document first.**  Everything here assumes it, and every constant
below is sourced from it rather than restated with justification.

---

## Goal and scope

`micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
writers.  There is no way to talk to a unit.  This adds the live half, mirroring
`minimateplus/`.

**In scope, first pass:**

- connect over TCP (a field modem) or serial/USB (a bench unit)
- identify a unit, read its state, clock, memory and setups
- walk the event chain and download events
- return `Event` objects the existing codec already understands

**Explicitly out of scope, first pass:**

- ⚠ **Any write.**  Setups, schedules, call-home config, monitoring start/stop,
  and per-event delete are all mapped, and none of them will be implemented
  here.  **No command has ever been originated against a unit by this project**
  — every write observed was performed by THOR while we recorded.  Keeping that
  true through the read client is deliberate: it means the first thing we ever
  send to a customer's instrument is a decision someone made on purpose, not a
  side effect of a client that happened to grow a method.
- the inbound call-home session — still the one protocol unknown

---

## Layout

```
micromate/
  framing.py     NEW   frame building, response parsing, checksum
  protocol.py    NEW   one method per wire command, returns raw payloads
  client.py      NEW   high-level API, returns models
  idf_file.py    (existing — decodes what 0x5A returns, unchanged)
  models.py      (existing — extend, do not fork)
```

**Transport is reused, not rewritten.**  `minimateplus/transport.py` is
byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
`TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller.  Import it.

⚠ Do **not** import `minimateplus.framing`.  The two framings differ in ways
that look small and are not, and a shared module would accumulate `if series ==`
branches until neither case is readable.

---

## `micromate/framing.py`

### Requests

Series IV accepts Series III request frames unmodified.  The simplest correct
implementation re-exports the builder rather than duplicating it:

```python
from minimateplus.framing import build_bw_frame   # requests are identical
```

⚠ One open question, flagged in the protocol reference and **not** settled:
whether `0x10` bytes inside request *params* need stuffing.  No probe we sent
carried one.  Until it is settled, assert on it rather than guessing:

```python
def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
    if 0x10 in params:
        raise NotImplementedError(
            "params containing 0x10 — stuffing rule unconfirmed; see "
            "micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'"
        )
    return build_bw_frame(sub, offset, params)
```

That turns an unknown into a loud failure instead of a corrupt frame.

### Responses — where Series III's parser cannot follow

| | Series III | Micromate |
|---|---|---|
| frame start | `DLE STX` | **bare `STX`** |
| `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
| destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |

The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
it scans for `DLE+STX`, which never appears.

The third is a genuine **simplification** — no inner-frame carve-out.  Validated
by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
four candidate destuffing rules were tried, and only this one makes all frames
validate.

### Checksum

```python
def checksum(payload: bytes) -> int:
    return sum(b for b in payload if b != 0x10) & 0xFF
```

The DLE-aware variant, same as Series III's `5A` and write frames — not the plain
SUM8 of ordinary Series III reads.

### ⚠ The SUB byte can be escaped

When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
Reading it positionally without destuffing reports `0x10`.  This bit once
already — `SUB 0x02` was logged as `SUB_10` for an afternoon.  Destuff first,
then index.

### Response shape

```python
@dataclass
class MicromateFrame:
    sub: int              # response SUB;  request = 0xFF - sub
    flags: int            # 0xC5 Blastware line, 0x03 Thor line
    page_hi: int
    page_lo: int
    data: bytes           # payload[5:], checksum stripped
    checksum_valid: bool

    @property
    def firmware_line(self) -> str:      # "blastware" | "thor" | "unknown"
    @property
    def declared_length(self) -> int:    # uint16 BE at data[3:5]  (= payload[8:10])
```

⚠ **`declared_length` is a uint16 BE.**  Read as a single byte it under-reads
`SUB 0x1A` by 47x — 44 against a true 2092.  This is the single most expensive
mistake available in this protocol and it has already been made once.

`MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
what distinguishes "no bytes at all" from "bytes but no complete frame" on a
timeout, and that distinction earned its keep during the Series III work).

---

## `micromate/protocol.py`

One method per command, returning raw payload bytes.  No interpretation — that
belongs in `client.py`.

**Reads use `offset = 0xFFFF`** and return the whole block in one response;
Series III's two-step probe/data dance is unnecessary.  `POLL` is the exception,
taking its data length.  Per-command offsets, all observed:

| command | SUB | rsp | offset | returns |
|---|---|---|---|---|
| poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
| serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
| device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
| state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
| monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
| storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
| active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
| first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
| next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
| compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
| call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
| arm event | `0x93` | `0x6C` | — | before every event |
| first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
| next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
| event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks |
| event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record |
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** |

⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
Parse **forward** from `declared_length`, never backward from the end — Series
III reads battery and memory from the end of that block, and doing so on a BD
unit yields a battery voltage of **577.92 V**.

⚠ **Test the monitoring flag for non-zero**, never against a constant.  It has
read both `0x0E` and `0x0C` while monitoring.

### `0x5A` — simpler than Series III, deliberately

No arming ritual, no chunk loop, no `STRT` end-offset parsing, no `TERM` frame.
One request returns the whole event:

```python
offset_word = 0x1000 + 2 * ceil(size / 512)     # size from the chain walk
```

The payload **is** the `.IDFW` file, byte for byte — so it feeds
`micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.

⚠ Do not port the Series III `5A` walk.  Its address arithmetic caused a 5x
over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
III side.  None of that applies here.

---

## `micromate/client.py`

```python
class MicromateClient:
    def __init__(self, transport: BaseTransport): ...
    def open(self) / close(self) / is_open(self)

    # identity and state
    def connect(self) -> DeviceInfo          # poll → serial → device info → state
    def get_state(self) -> UnitState         # monitoring?, clock, battery, memory
    def get_active_setup(self) -> str
    def list_setups(self) -> list[str]       # 0x3F → 0x40… until empty

    # events
    def list_events(self) -> list[EventRef]  # 0x93 → 0x1E → 0x1F… (key + size)
    def download_event(self, ref) -> bytes   # raw .IDFW/.IDFH
    def get_event(self, ref) -> Event        # download + decode via idf_file
```

`connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
⚠ but note the reference records that **whether the unit requires it is
untested**.  Do it because it is known-good, not because it is known-necessary,
and say so in the docstring.

`list_events()` returns the key *and* the size, because `download_event()` needs
the size to compute its offset word.

---

## Tests

**Offline, from captured bytes — no hardware.**  This is the part worth doing
first, because it can be fully verified tonight's-captures-style before any unit
is involved.

```
tests/test_micromate_framing.py
```

⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
not depend on files being present.  **Embed the frames as hex constants** — they
are 16–68 bytes each, and a handful covers every case:

| case | why |
|---|---|
| POLL probe reply, 19 B | shortest valid frame |
| POLL data reply, 68 B | contains a literal `0x10` — only the DLE-aware checksum matches |
| `0x1A` response, 2108 B | exercises `declared_length` as a true uint16 (`0x082C`) |
| a Thor-line reply, `flags = 0x03` | `0x03` is ETX; proves destuffing before framing |
| a frame whose SUB is `0x02` | arrives as `10 02`; proves destuff-then-index |
| a truncated frame | parser must return nothing, not a bad frame |
| a corrupted checksum | `checksum_valid == False`, frame still returned |

Then a round-trip assertion: feed a whole captured session through the parser and
assert the frame count and every SUB, against `scratch/mm_frame_parse.py`'s
output — which is already known good, having parsed 24, 38 and 40-frame sessions
with zero bad checksums.

**Live, second:** against the bench unit on mint-mac via `mm_link.py`.
`connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
then `download_event()` and assert the bytes decode and match a
`/db/import/idf_file` ingest of the same event.

---

## Order of work

1. `framing.py` + its tests — offline, verifiable immediately
2. `protocol.py` — reads only, one method per row of the table above
3. `client.py` — `connect()`, `get_state()`, `list_setups()`
4. the event chain and `download_event()`
5. decode end-to-end and compare against a store event

Steps 1–2 need no hardware at all.

---

## Open questions to settle while implementing

- **Request param stuffing** — raise `NotImplementedError` rather than guess.
- **Is THOR's preamble required?**  Try one command cold and find out; it is a
  two-minute test with the bench unit and it removes a ritual if unnecessary.
- **`Event` model fit** — Series IV carries fields Series III lacks (setup file
  name, `LMic`/`SMic` channels).  Extend `micromate/models.py`; do not fork the
  shared `Event`.
- **Which `0x0C` fields to trust.**  The peak float there runs 2–5% above
  `max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
  established.  The reference marks it do-not-rely-on — prefer decoded samples.