seismo-relay/docs/micromate_client_spec.md

Spec — a live client for Series IV (Micromate)

Drafted 2026-09-26, ahead of implementation. The protocol work is finished; this is the plan for turning docs/micromate_protocol_reference.md into code SFM can run.

Read that document first. Everything here assumes it, and every constant below is sourced from it rather than restated with justification.

---

Goal and scope

micromate/ is codec-only today — idf_file.py, models.py, the report writers. There is no way to talk to a unit. This adds the live half, mirroring minimateplus/.

In scope, first pass:

• connect over TCP (a field modem) or serial/USB (a bench unit)
• identify a unit, read its state, clock, memory and setups
• walk the event chain and download events
• return Event objects the existing codec already understands

Explicitly out of scope, first pass:

• ⚠ Any write. Setups, schedules, call-home config, monitoring start/stop, and per-event delete are all mapped, and none of them will be implemented here. No command has ever been originated against a unit by this project — every write observed was performed by THOR while we recorded. Keeping that true through the read client is deliberate: it means the first thing we ever send to a customer's instrument is a decision someone made on purpose, not a side effect of a client that happened to grow a method.
• the inbound call-home session — still the one protocol unknown

---

Layout

micromate/
  framing.py     NEW   frame building, response parsing, checksum
  protocol.py    NEW   one method per wire command, returns raw payloads
  client.py      NEW   high-level API, returns models
  idf_file.py    (existing — decodes what 0x5A returns, unchanged)
  models.py      (existing — extend, do not fork)

Transport is reused, not rewritten. minimateplus/transport.py is byte-level and protocol-agnostic — BaseTransport, SerialTransport, TcpTransport, plus read_until_idle() which already handles the RV50/RV55 habit of emitting \r\nRING\r\n\r\nCONNECT\r\n to a caller. Import it.

⚠ Do not import minimateplus.framing. The two framings differ in ways that look small and are not, and a shared module would accumulate if series == branches until neither case is readable.

---

micromate/framing.py

Requests

Series IV accepts Series III request frames unmodified. The simplest correct implementation re-exports the builder rather than duplicating it:

from minimateplus.framing import build_bw_frame   # requests are identical

⚠ One open question, flagged in the protocol reference and not settled: whether 0x10 bytes inside request params need stuffing. No probe we sent carried one. Until it is settled, assert on it rather than guessing:

def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
    if 0x10 in params:
        raise NotImplementedError(
            "params containing 0x10 — stuffing rule unconfirmed; see "
            "micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'"
        )
    return build_bw_frame(sub, offset, params)

That turns an unknown into a loud failure instead of a corrupt frame.

Responses — where Series III's parser cannot follow

	Series III	Micromate
frame start	DLE STX	bare STX
payload[1]	0x10	0xC5 (Blastware fw) / 0x03 (Thor fw)
destuffing	DLE+ETX kept as literal inner-frame data	10 XX → XX, uniformly

The first row is why S3FrameParser returns nothing at all on Series IV traffic: it scans for DLE+STX, which never appears.

The third is a genuine simplification — no inner-frame carve-out. Validated by checksum across every capture in bridges/captures/9-24-26 - micromate2/: four candidate destuffing rules were tried, and only this one makes all frames validate.

Checksum

def checksum(payload: bytes) -> int:
    return sum(b for b in payload if b != 0x10) & 0xFF

The DLE-aware variant, same as Series III's 5A and write frames — not the plain SUM8 of ordinary Series III reads.

⚠ The SUB byte can be escaped

When a SUB's value is 0x02, 0x03, 0x04 or 0x10 it arrives as 10 XX. Reading it positionally without destuffing reports 0x10. This bit once already — SUB 0x02 was logged as SUB_10 for an afternoon. Destuff first, then index.

Response shape

@dataclass
class MicromateFrame:
    sub: int              # response SUB;  request = 0xFF - sub
    flags: int            # 0xC5 Blastware line, 0x03 Thor line
    page_hi: int
    page_lo: int
    data: bytes           # payload[5:], checksum stripped
    checksum_valid: bool

    @property
    def firmware_line(self) -> str:      # "blastware" | "thor" | "unknown"
    @property
    def declared_length(self) -> int:    # uint16 BE at data[3:5]  (= payload[8:10])

⚠ declared_length is a uint16 BE. Read as a single byte it under-reads SUB 0x1A by 47x — 44 against a true 2092. This is the single most expensive mistake available in this protocol and it has already been made once.

MicromateFrameParser mirrors S3FrameParser: feed(bytes) -> list[frame], accumulates in .frames, reset(), and keeps the bytes_fed counter (it is what distinguishes "no bytes at all" from "bytes but no complete frame" on a timeout, and that distinction earned its keep during the Series III work).

---

micromate/protocol.py

One method per command, returning raw payload bytes. No interpretation — that belongs in client.py.

Reads use offset = 0xFFFF and return the whole block in one response; Series III's two-step probe/data dance is unnecessary. POLL is the exception, taking its data length. Per-command offsets, all observed:

command	SUB	rsp	offset	returns
poll	0x5B	0xA4	0x0030	device string, model
serial	0x15	0xEA	0x000A	UM12947
device info	0x01	0xFE	0xFFFF	firmware, calibration
state	0x49	0xB6	0xFFFF	data[11]: non-zero = monitoring
monitor status	0x1C	0xE3	0xFFFF	flag, device clock, battery, memory
storage range	0x06	0xF9	0xFFFF	event storage extent
active setup name	0x41	0xBE	0xFFFF	TEST1.mmb
first setup	0x3F	0xC0	0xFFFF	setup-list walk head
next setup	0x40	0xBF	0xFFFF	…until an empty name
compliance config	0x1A	0xE5	0xFFFF	~2103 B setup block
call-home config	0x2C	0xD3	0xFFFF	137 B
arm event	0x93	0x6C	—	before every event
first event	0x1E	0xE1	0xFFFF	key + size
next event	0x1F	0xE0	0xFFFF	key + size
event record	0x0C	0xF3	0xFFFF	210 B — project, location, peaks
event header	0x0A	0xF5	0xFFFF	30 B list record
bulk download	0x5A	0xA5	computed	the .IDFW verbatim

⚠ SUB 0x1C is 4 bytes longer on the Thor firmware line (0x30 vs 0x2C). Parse forward from declared_length, never backward from the end — Series III reads battery and memory from the end of that block, and doing so on a BD unit yields a battery voltage of 577.92 V.

⚠ Test the monitoring flag for non-zero, never against a constant. It has read both 0x0E and 0x0C while monitoring.

0x5A — simpler than Series III, deliberately

No arming ritual, no chunk loop, no STRT end-offset parsing, no TERM frame. One request returns the whole event:

offset_word = 0x1000 + 2 * ceil(size / 512)     # size from the chain walk

The payload is the .IDFW file, byte for byte — so it feeds micromate.idf_file.read_idf_file() and /db/import/idf_file unchanged.

⚠ Do not port the Series III 5A walk. Its address arithmetic caused a 5x over-read and a > 64 KB page-boundary bug that is still open on the Series III side. None of that applies here.

---

micromate/client.py

class MicromateClient:
    def __init__(self, transport: BaseTransport): ...
    def open(self) / close(self) / is_open(self)

    # identity and state
    def connect(self) -> DeviceInfo          # poll → serial → device info → state
    def get_state(self) -> UnitState         # monitoring?, clock, battery, memory
    def get_active_setup(self) -> str
    def list_setups(self) -> list[str]       # 0x3F → 0x40… until empty

    # events
    def list_events(self) -> list[EventRef]  # 0x93 → 0x1E → 0x1F… (key + size)
    def download_event(self, ref) -> bytes   # raw .IDFW/.IDFH
    def get_event(self, ref) -> Event        # download + decode via idf_file

connect() should mirror THOR's preamble (POLL → SERIAL → 0x49 → POLL) — ⚠ but note the reference records that whether the unit requires it is untested. Do it because it is known-good, not because it is known-necessary, and say so in the docstring.

list_events() returns the key and the size, because download_event() needs the size to compute its offset word.

---

Tests

Offline, from captured bytes — no hardware. This is the part worth doing first, because it can be fully verified tonight's-captures-style before any unit is involved.

tests/test_micromate_framing.py

⚠ bridges/captures/ and tests/fixtures/ are both gitignored, so tests must not depend on files being present. Embed the frames as hex constants — they are 16–68 bytes each, and a handful covers every case:

case	why
POLL probe reply, 19 B	shortest valid frame
POLL data reply, 68 B	contains a literal 0x10 — only the DLE-aware checksum matches
0x1A response, 2108 B	exercises declared_length as a true uint16 (0x082C)
a Thor-line reply, flags = 0x03	0x03 is ETX; proves destuffing before framing
a frame whose SUB is 0x02	arrives as 10 02; proves destuff-then-index
a truncated frame	parser must return nothing, not a bad frame
a corrupted checksum	checksum_valid == False, frame still returned

Then a round-trip assertion: feed a whole captured session through the parser and assert the frame count and every SUB, against scratch/mm_frame_parse.py's output — which is already known good, having parsed 24, 38 and 40-frame sessions with zero bad checksums.

Live, second: against the bench unit on mint-mac via mm_link.py. connect(), list_setups() (should return the 23 known names), list_events(), then download_event() and assert the bytes decode and match a /db/import/idf_file ingest of the same event.

---

Order of work

1. framing.py + its tests — offline, verifiable immediately
2. protocol.py — reads only, one method per row of the table above
3. client.py — connect(), get_state(), list_setups()
4. the event chain and download_event()
5. decode end-to-end and compare against a store event

Steps 1–2 need no hardware at all.

---

Open questions to settle while implementing

• Request param stuffing — raise NotImplementedError rather than guess.
• Is THOR's preamble required? Try one command cold and find out; it is a two-minute test with the bench unit and it removes a ritual if unnecessary.
• Event model fit — Series IV carries fields Series III lacks (setup file name, LMic/SMic channels). Extend micromate/models.py; do not fork the shared Event.
• Which 0x0C fields to trust. The peak float there runs 2–5% above max(T,V,L) and is not the vector sum; its offset was inferred, not established. The reference marks it do-not-rely-on — prefer decoded samples.