fix(micromate): the 0x5A chunk offset is a uint32 -- a 64 KB cap was one event away

UM20147 holds a 72,560-byte event. chunk_params() wrote the offset as a uint16 at params[2:4], because every offset THOR was observed to send fits in two bytes (largest 0x3400 = 13,312), which caps a download at 65,536 B -- so that event could not have been fetched at all. params[0:4] is demonstrably ONE 4-byte field: chunk 0 puts the 4-byte event key there. Writing the offset as a uint32 BE in the same slot is BYTE-IDENTICAL for every offset below 65,536, so nothing verified against THOR's 74 captured frames changes -- the replay test still matches all of them -- and the range extends to 4 GB. Above 64 KB this is inference and the docstring says so: the field's width is established, the device's handling of a non-zero high byte is not. Also newly reachable: at offset 1 MiB params[1] is 0x10, which must go out as `10 10`. Nothing below 64 KB can produce that, so the uint32 change is what first makes the case possible -- and an unescaped 0x10 in 5A params is the exact bug that cost the Series III walk a release. Tested. THIS IS THE SERIES III 64 KB PAGE-BOUNDARY BUG WEARING A DIFFERENT HAT. There, parse_strt_end_offset() discards the key's page byte and the walk crashes once a unit's buffer crosses 64 KB; that one is still open. The transferable lesson: an address field whose high bytes are zero in every capture is not a narrow field, it is an untested one. Same mistake, found twice, in code written years apart. Confirmed in the same run: 0x06 content[0:4] IS the event count -- UM20147 holds 5 events and reads 5, making it three for three across both firmware lines (0->0, 6->6, 5->5). And content[4:8] is a CONSTANT, not a count: it reads 9 on a unit with 6 events and on a unit with 5. list_events() still walks to the sentinel; the count is worth adopting as a pre-check, not a replacement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
tooling(micromate): exercise the event chain, and test the 0x06 count candidate
2026-10-01 00:41:08 -04:00 · 2026-09-30 18:48:58 -04:00 · 2026-09-30 18:29:32 -04:00 · 2026-09-30 14:33:10 -04:00 · 2026-09-30 14:31:23 -04:00 · 2026-09-30 14:27:31 -04:00
18 changed files with 8057 additions and 39 deletions
@@ -22,6 +22,15 @@ All notable changes to seismo-relay are documented here.
  `event_datetime` stays authoritative (unit-clock drift).
  ⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
 - **`scratch/mm_frame_parse.py` accepted either checksum rule, so it could not
  falsify either.**  It tried plain SUM8 *and* a DLE-aware variant and reported
  whichever matched, which is why it never flagged a bad frame and why the
  protocol reference carried the wrong rule for two days — "zero bad checksums
  across three sessions" was true and carried no information.  It now validates
  against SUM8 alone and names the DLE-aware result only as a near-miss, never as
  a pass.  Still zero bad frames across all captures, strictly tighter.
 ### Added
 - **Diagnostics tab in the SFM standalone webapp.**  Surfaces the device
@@ -46,6 +55,78 @@ All notable changes to seismo-relay are documented here.
  500s.  Reproduced on BE12599.  Production is unaffected: it ingests complete
  files via the watcher path and never runs this walk.
 - **The Micromate (Series IV) live wire protocol, reverse-engineered end to
  end** — `docs/micromate_protocol_reference.md`.  Worked out against a bench
  UM12947 over USB and a recording relay, with THOR driving every write so that
  no command has ever been originated against a unit by this project.  **A
  Micromate answers Series III command frames**, with three framing differences:
  responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
  or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
  **uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
  `SUB 0x1A` by 47x.  Read path, event chain, and `SUB 0x5A` streaming the
  `.IDFW` file verbatim are all confirmed.
 - **Series IV setup management, fully mapped.**  `0x41` reads the active setup
  name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
  write it back.  **Setups are read-modify-write** — the written block is the
  read block, 91% byte-identical at a fixed 11-byte shift.  `0xDA` **creates**
  files rather than only overwriting, confirmed on the unit's own screen, and an
  overwrite is protocol-identical to a create: no handshake, no warning, and no
  protection even over the *active* setup of a monitoring unit.
 - **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
  records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
  half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
  name.  Verified entry-for-entry against the operator's own THOR screen.
 - **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
  `0x8D`/`0x8E` write.  This **retracts** an earlier conclusion in the same
  document that no such command existed; that was inferred from absent firmware
  strings and was wrong.
 - **Monitoring control and per-event delete.**  `0x96`/`0x97` start and stop as
  on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
  for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
  constant.  Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
  which is safer than Series III's erase-everything.  `SUB 0x1C` also carries the
  device clock.
 - **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
  identically as "disconnected": refused, connect timeout (the silent-drop
  signature of a trusted-IP whitelist), **connected but no reply** (the modem
  answered and the unit did not), and replied.  Each verdict names what to try
  next.  `--slots N` tests single-session modem behaviour.
 - **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
  decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
  `onewaydev`) driven by a control file.  No pyserial; stdlib `termios` only.
 - **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
  — a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
  all, since it scans for `DLE+STX`), byte-exact capture recovery from a
  `socat -x` relay log, and a serial-port stand-in that answers as a unit.
 - **A live client for Series IV — `micromate/{framing,protocol,client}.py`.**
  `micromate/` was codec-only; it can now talk to a unit.  Connect over TCP (a
  cellular modem) or serial/USB, identify a unit, read its state, clock, battery,
  memory and setups, walk the event chain and download events as `.IDFW`/`.IDFH`
  bytes the existing codec already decodes.  **98 offline tests**, every response
  constant a real captured data section.  `minimateplus/transport.py` is reused
  as-is; `minimateplus/framing.py` deliberately is **not** — see *Changed*.
  ⚠ **Read-only.**  Setups, schedules, call-home config, monitoring start/stop
  and per-event delete are all mapped and none are implemented.  No command has
  ever been originated against a unit by this project; every write in the
  protocol reference was performed by THOR while we recorded.
 - **Verified on real hardware, both transports, both firmware lines.**  Identical
  wire bytes over USB CDC-ACM and an RX55 in PAD mode — 11,580 B in and 761 B out
  to the byte.  Every `11.0BD` inference confirmed on UM20147, including the four
  extra trailing bytes in `SUB 0x1C` that make Series III's from-the-end offsets
  report a battery voltage of **577.92 V**.  A monitoring unit answers reads with
  no `SESSION_RESET`, which Series III requires.  All six bench events decode
  with the existing IDF codec — 4 waveforms at 12,288/12,288/12,288/8,192 samples
  and 2 histograms — so `/db/import/idf_file` ingests a directly downloaded event
  unchanged.
 - **`bridges/mm_client_check.py`** — drives the read client against a unit and
  reports per-command timings, read counts and byte totals, so two transports or
  two firmware lines can be diffed.  `--capture DIR` writes a `raw_bw_*`/`raw_s3_*`
  pair in the layout `scratch/mm_frame_parse.py` reads, turning a field run into
  a test fixture.  No pyserial — stdlib `termios`, because `pip install` is
  refused outright by PEP 668 on the distros the bench hosts run.
 ### Changed
 - **Connecting to a unit no longer walks its event chain.**  `/device/events`
@@ -58,11 +139,83 @@ All notable changes to seismo-relay are documented here.
  toolbar.  Knowing whether a unit's ACH is on no longer requires reading every
  event it has stored.
 - **Recorded what THOR actually does on the wire**, measured rather than assumed.
  A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
  MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
  `MONITOR_STATUS` check at 60 s.  The **status interval is honoured; the
  connection interval is not** — it sets `(status / connection) - 1` checks per
  cycle, so equal values yield *zero* cheap checks and every connection becomes
  the expensive one.
 - **Two THOR defects reproduced with timestamps.**  After a connection drops
  mid-download it retries **once**, stops polling entirely and **never resumes**,
  while displaying `Connected` for as long as it is left alone — and `Idle` for a
  unit that is actively recording.  Separately, THOR's own log shows a
  **subscription leak**: one logical event dispatched to a growing number of
  handlers, **1 to 12 over ten hours** of uptime, consistent with the field
  report that only a restart recovers it.
 - **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
  either firmware line.  A PL2303 cable (Benfei) leaves a unit with no working
  modem port; an FTDI cable (Sabrent) works.  Both are in circulation and
  indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
 - **Six rules in the Series-4 client spec were wrong, and measuring against the
  captures caught all six before any code shipped.**  Each fails *silently* —
  a frame the unit ignores, or a checksum that reads as bad:
  requests escape **four** byte values (`0x02 0x03 0x04 0x10`), not one, so the
  Series III builder reproduces only **161 of THOR's 218** read frames and misses
  *every* `SUB 0x5A` download; the response checksum is **plain SUM8** of the
  de-stuffed payload, not the DLE-aware variant, which disagrees with the wire on
  **55 of 251** frames; `SUB 0x5A` is a **1024-byte chunk loop**, not one request
  per event; `SUB 0x0A` is the **monitor-log walk** (same request repeated,
  device-side cursor), not a keyed event-header read; `1E`/`1F` carry **token
  `0xFE`** at `params[7]`; and `SUB 0x01` has **no THOR precedent at all**.
  Recorded in `docs/micromate_protocol_reference.md` as corrections in place.
 - **`SUB 0x0C`'s peak float is the per-sample peak vector sum** — resolved after
  being marked *do-not-rely-on*.  Exact to **0.000%** on all four bench waveforms
  against a PVS recomputed from decoded samples, so its offset (`Tran` label − 12)
  is established rather than inferred.  The earlier "not the vector sum" reading
  compared against `sqrt(Σpeak²)` from the three *reported* peaks, which is an
  upper bound: the channel maxima do not occur at the same instant.  **This gives
  the decoder a free self-check** — the device computed that number from the same
  samples, independently of our codec, so a mismatch means the decode is wrong.
  Worth having in a codebase whose channel truncations have historically been
  silent.
 - **Cellular costs ~0.65 s per round trip, independent of payload size.**
  Measured on UM12947 over an RX55: `list_setups()` on a unit with 24 setups takes
  **16.0 s**, against 0.46 s over USB, for 24 commands.  A 1,024-byte chunk and a
  16-byte state read cost the same.  **Over cellular, minimise round trips, not
  bytes** — cache the setup list rather than refreshing it on a timer, and note a
  13 KB event is 14 chunks ≈ 8.4 s of latency against ~0.03 s of data.  The modem
  also needs **fewer** reads than USB, not more: it buffers ~1 s then forwards one
  large segment where CDC-ACM delivers many small ones.
 - **Event keys collide across units.**  UM20147's first event and UM12947's first
  event are both `055d4a81` — different sizes, different contents, identical key.
  The counter starts from the same value on every unit, so **a key is meaningless
  without its serial**.  Anything that stores, deduplicates or addresses Series IV
  events must key on `(serial, event_key)`; keyed on the event key alone, one
  unit's event is silently treated as a duplicate of another's and simply never
  ingested.  Series III has the cousin of this — its counter resets after an
  erase, so keys are reused *within* a unit, which is why `ach_state.json` already
  tracks `max_downloaded_key` per serial.
 - **Setup file names are spelled three ways by three commands.**  In one session
  on one unit: `0x41` reported `test2.MMB`, `0x40` reported `test2.mmb`, and `0x0C`
  reported `test2`.  **Compare setup names case-insensitively and without the
  extension** — an exact-string test of "is the active setup one I know about?"
  answers *no*.
 ### Migration
-**None.**  Frontend and documentation only — no codec, waveform-store or DB
+**None.**  No codec change, no waveform-store change, no DB or schema change,
-change, no schema change, and no `TOOL_VERSION` bump.  The webapp is served
+and **no `TOOL_VERSION` bump** — so no `backfill_sidecars.py` /
-from the image, so the change appears after the next `sfm` rebuild.
+`backfill_event_shape.py` run is owed.  The webapp is served from the image, so
 its changes appear after the next `sfm` rebuild.
 The Series-4 live client adds **new** modules under `micromate/`
 (`framing.py`, `protocol.py`, `client.py`) and appends two dataclasses to
 `micromate/models.py`; `micromate/idf_file.py` — the codec — is untouched, and
 nothing under `sfm/` or `minimateplus/` changed.  The new modules are additive
 and nothing imports them yet, so an existing deployment behaves identically.
 ---
@@ -10,10 +10,38 @@ pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
 ---
-## Where things stand (updated 2026-08-28)
+## Where things stand (updated 2026-09-26)
 Read this first when picking the project back up.
 - **The Series-4 LIVE wire protocol is reverse-engineered end to end
  (2026-09-25).** `docs/micromate_protocol_reference.md` is the Series-4
  Rosetta Stone, sibling to `instantel_protocol_reference.md`. **A Micromate
  answers Series III command frames** — three framing differences: responses
  have **no leading `DLE`** (a bare `STX`), `payload[1]` is `0xC5` (Blastware
  firmware) or `0x03` (Thor firmware) rather than `0x10`, and the data length
  is a **uint16 BE at `payload[8:10]`** (as a byte it under-reads `SUB 0x1A`
  by 47x). Read path, event chain, setups, scheduler, monitoring control and
  per-event delete are all mapped; **the inbound call-home session is the only
  protocol unknown left.**
  ⚠ **No command has ever been originated against a unit by this project.**
  Every write was performed by THOR while we recorded. That line is worth
  keeping.
  ⚠ `micromate/` still has **no live client** — it is codec-only. The
  `minimateplus/` stack (transport/framing/protocol/client) has no Series-4
  counterpart yet. `minimateplus.transport` is protocol-agnostic and reusable.
 - **Bench tooling for device diagnosis (2026-09-25).** `bridges/mm_probe.py`
  distinguishes the four faults THOR reports identically as "disconnected"
  (refused / connect timeout / **connected but no reply** / replied) and names
  what to try next. `bridges/mm_link.py` is a stand-in for a cellular modem
  with a decoded log and fault injection. `scratch/mm_frame_parse.py` exists
  because **`S3FrameParser` cannot see Micromate responses at all** — it scans
  for `DLE+STX`, which never appears in Series-4 traffic.
 - **A Micromate's USB-A host port drives FTDI and CDC-ACM only** — no Prolific,
  in either firmware line. TMI buys both Sabrent (FTDI) and Benfei (PL2303)
  cables and they are indistinguishable by eye. A PL2303 cable leaves a unit
  with **no working modem port at all**; identify by `lsusb` VID, `0403` vs
  `067b`. This accounted for a unit that could not be deployed.
 - **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
  archive decodes **14,338 / 14,338** paired files exactly against their
  preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
@@ -92,6 +120,9 @@ Read this first when picking the project back up.
  **v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
  fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
  files are archive-only and were never ingested).
  ✅ **The v0.30.0 Series-4 backfill HAS been run on prod (2026-09-25).** Every
  stored Series-4 geophone value was ~3.3% low until then; that is corrected and
  the job does not need repeating.
 - **The "offset" hardware fault has its own journal** --
  `docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
  **persistent** — it stays until the geophone is serviced. Detect it with
@@ -103,7 +134,18 @@ Read this first when picking the project back up.
  `SUB 0x0E` (unimplemented), which may carry those very numbers.
-When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
+When new information about a protocol is discovered, record it in the matching
 reference **in addition to** this document:
 | series | document |
 |---|---|
 | Series III (MiniMate Plus / BlastMate) | `docs/instantel_protocol_reference.md` |
 | **Series IV (Micromate / THOR)** | **`docs/micromate_protocol_reference.md`** |
 | Thor IDF file format | `docs/idf_protocol_reference.md` |
 Both protocol references carry retractions in place rather than deleting what
 turned out to be wrong — that convention has already saved re-deriving the same
 mistakes twice, so keep it.
 ---
@@ -276,8 +318,15 @@ minimateplus/         ← Python client library (primary focus)
 sfm/server.py         ← FastAPI REST server exposing device data over HTTP
 seismo_lab.py         ← Tkinter GUI (Bridge + Analyzer + Console tabs)
 bridges/
  mm_probe.py         ←   name the fault behind a dead unit (4 verdicts, read-only)
  mm_link.py          ←   bench stand-in for a cellular modem, with fault injection
  ach_mitm.py         ←   TCP relay for recording a Series-3 ACH session
 docs/
-  instantel_protocol_reference.md  ← reverse-engineered protocol spec ("the Rosetta Stone")
+  instantel_protocol_reference.md  ← Series III protocol spec ("the Rosetta Stone")
  micromate_protocol_reference.md  ← Series IV protocol spec + THOR's measured behaviour
  idf_protocol_reference.md        ← Thor IDF file format
 CHANGELOG.md          ← version history
 ```
@@ -0,0 +1,393 @@
 #!/usr/bin/env python3
 """
 mm_client_check.py — exercise the Micromate read client against a real unit.
 **Read-only.**  It sends POLL, SERIAL, state, monitor status, the setup walk and
 (optionally) one event download.  It never writes, never erases, never starts or
 stops monitoring.
 Why it exists
 -------------
 `micromate/{framing,protocol,client}.py` are verified against captures taken
 **over USB**, on **one firmware line** (`11.0CB`).  Two things that cannot be
 verified that way:
  * **the modem path.**  An RX55/RV55 bridges serial to TCP transparently, but
    it buffers up to ~1 s before forwarding, so a single logical response can
    arrive as many small reads.  The client reads to frame completion rather
    than using idle-gap detection, which should be strictly more robust — but
    "should be" is the point of this script.
  * **the other firmware line.**  `11.0BD` reports `flags = 0x03`, a shorter
    model string, and a `SUB 0x1C` block 4 bytes longer.  Everything about that
    is currently inference from one 2026-09-23 sweep whose captures never
    landed in the repo.
 Run it over both paths and diff the two reports.  Anything that differs beyond
 timings is a finding.
 Usage
 -----
    # over the modem
    python3 bridges/mm_client_check.py 63.45.161.30:9034
    # over USB / direct serial
    python3 bridges/mm_client_check.py /dev/ttyACM0 --baud 115200
    # include one event download (still read-only)
    python3 bridges/mm_client_check.py <target> --download
 ⚠ These modems bridge ONE TCP session to serial at a time.  If THOR holds the
 unit, this will connect and then see nothing — that is contention, not a fault.
 `bridges/mm_probe.py` explains that case; disconnect THOR first.
 """
 from __future__ import annotations
 import argparse
 import errno
 import os
 import select
 import sys
 import termios
 import time
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from micromate.client import MicromateClient, _content  # noqa: E402
 from micromate.protocol import ProtocolError  # noqa: E402
 from minimateplus.transport import TcpTransport  # noqa: E402
 class StdlibSerial:
    """Raw serial on stdlib `termios` — no pyserial.
    `minimateplus.SerialTransport` needs pyserial, and a bench host is whatever
    is to hand.  On a PEP 668 distro (Mint 22, Ubuntu 24.04, Debian 12) a plain
    `pip install pyserial` is refused outright, so a diagnostic that depends on
    it is one you cannot run at the moment you need it.  `bridges/mm_link.py`
    and `scratch/fake_unit.py` already take this approach; this is the same
    ~30 lines, and it means the tool runs on a stock Python 3 anywhere.
    Not a general replacement for SerialTransport — no flow control, no
    parity options, Linux/macOS only.  Enough for a Micromate, which is 8N1
    with no handshaking.
    """
    _BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
             57600: termios.B57600, 115200: termios.B115200}
    def __init__(self, path: str, baud: int = 115200) -> None:
        if baud not in self._BAUD:
            raise ValueError(f"unsupported baud {baud}; pick from {sorted(self._BAUD)}")
        self.path, self.baud, self.fd = path, baud, None
    def connect(self) -> None:
        if self.fd is not None:
            return
        self.fd = os.open(self.path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
        a = termios.tcgetattr(self.fd)
        a[0] = a[1] = a[3] = 0                               # raw in/out, non-canonical
        a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL  # 8N1, ignore modem lines
        a[4] = a[5] = self._BAUD[self.baud]
        a[6] = list(a[6])
        a[6][termios.VMIN] = 0
        a[6][termios.VTIME] = 0
        termios.tcsetattr(self.fd, termios.TCSANOW, a)
        termios.tcflush(self.fd, termios.TCIOFLUSH)
    def disconnect(self) -> None:
        if self.fd is not None:
            os.close(self.fd)
            self.fd = None
    def is_connected(self) -> bool:
        return self.fd is not None
    def read(self, n: int) -> bytes:
        if self.fd is None:
            return b""
        r, _, _ = select.select([self.fd], [], [], 0.05)
        if not r:
            return b""
        try:
            return os.read(self.fd, n)
        except OSError as e:
            if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
                return b""
            raise
    def write(self, data: bytes) -> None:
        if self.fd is None:
            raise OSError("port is not open")
        while data:
            data = data[os.write(self.fd, data):]
 class _Timed:
    """Count bytes and time each read, so the two transports can be compared.
    With `capture`, also writes the raw byte streams to a `raw_bw_*` /
    `raw_s3_*` pair in the layout `scratch/mm_frame_parse.py` already reads --
    so a run on an unfamiliar unit can be turned into test fixtures without
    setting up a relay.
    """
    def __init__(self, inner, capture: str | None = None) -> None:
        self._inner = inner
        self.reads = 0
        self.bytes_in = 0
        self.bytes_out = 0
        self._bw = self._s3 = None
        if capture:
            stamp = time.strftime("%Y%m%d_%H%M%S")
            d = Path(capture)
            d.mkdir(parents=True, exist_ok=True)
            self.bw_path = d / f"raw_bw_{stamp}_mm_client_check.bin"
            self.s3_path = d / f"raw_s3_{stamp}_mm_client_check.bin"
            self._bw = open(self.bw_path, "wb")
            self._s3 = open(self.s3_path, "wb")
    def close_capture(self) -> None:
        for f in (self._bw, self._s3):
            if f:
                f.close()
    def connect(self):
        return self._inner.connect()
    def disconnect(self):
        return self._inner.disconnect()
    def is_connected(self):
        return self._inner.is_connected()
    def write(self, data: bytes):
        self.bytes_out += len(data)
        if self._bw:
            self._bw.write(data); self._bw.flush()
        return self._inner.write(data)
    def read(self, n: int) -> bytes:
        chunk = self._inner.read(n)
        if chunk:
            self.reads += 1
            self.bytes_in += len(chunk)
            if self._s3:
                self._s3.write(chunk); self._s3.flush()
        return chunk
 # ⚠ HYPOTHESIS, 6 events.  content[11] of the 0x0C record separated 4 waveforms
 # from 2 histograms cleanly and was constant within each group.  A 4/2 split is
 # thin evidence for a byte that could be anything, so _decode() below does NOT
 # trust it -- it tries the other suffix on failure and says when the guess was
 # wrong.  The protocol reference states no type field is known; this may be it.
 _TYPE_BYTE = 11
 _TYPES = {0x07: ".IDFW", 0x08: ".IDFH"}
 def _event_type(record: bytes) -> str:
    if len(record) <= _TYPE_BYTE:
        return "?"
    b = record[_TYPE_BYTE]
    return {0x07: "waveform", 0x08: "histogram"}.get(b, f"unknown(0x{b:02x})")
 def _decode(blob: bytes, key: bytes, record: bytes) -> None:
    """Decode the downloaded bytes, proving they are a real event file.
    read_idf_file() picks waveform vs histogram from the FILENAME SUFFIX, and a
    wire download has no filename -- so the suffix has to come from somewhere.
    This tries the 0x0C type byte first and the other suffix second; getting a
    decode either way proves the chunk assembly, and which one worked is itself
    the finding.
    """
    import tempfile
    from micromate.idf_file import read_idf_file
    guess = _TYPES.get(record[_TYPE_BYTE] if len(record) > _TYPE_BYTE else -1, ".IDFW")
    order = [guess] + [e for e in (".IDFW", ".IDFH") if e != guess]
    for n, ext in enumerate(order):
        with tempfile.NamedTemporaryFile(suffix=ext, delete=False) as f:
            f.write(blob)
            tmp = f.name
        try:
            res = read_idf_file(tmp)
            samples = sum(len(v) for v in getattr(res, "samples", {}).values())
            note = "" if n == 0 else f"  *** the 0x0C type byte guessed {guess} — WRONG ***"
            print(f"       decoded OK as {ext}: {samples} samples{note}")
            os.unlink(tmp)
            return
        except Exception as e:
            last = f"{ext}: {type(e).__name__}: {e}"
        finally:
            if os.path.exists(tmp):
                os.unlink(tmp)
    print(f"       decode failed BOTH ways — last: {last}")
    out = Path(f"./{key.hex()}.bin")
    out.write_bytes(blob)
    print(f"       saved to {out} for offline analysis")
 def step(label: str, fn):
    """Run one read, report how long it took and what it returned."""
    t0 = time.monotonic()
    try:
        value = fn()
    except Exception as e:
        print(f"  {label:.<26} FAILED  {type(e).__name__}: {e}")
        return None
    ms = 1000 * (time.monotonic() - t0)
    shown = value if isinstance(value, str) else repr(value)
    if isinstance(value, list):
        shown = f"{len(value)} entries"
    print(f"  {label:.<26} {ms:7.0f} ms   {shown}")
    return value
 def main() -> int:
    ap = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    ap.add_argument("target", help="host:port for TCP, or a serial device path")
    ap.add_argument("--baud", type=int, default=115200,
                    help="serial only; the USB-A/FTDI path runs at 115200. "
                         "Ignored by the USB-B 'PC' port, which is CDC-ACM "
                         "and negotiates its own rate.")
    ap.add_argument("--timeout", type=float, default=10.0)
    ap.add_argument("--download", action="store_true",
                    help="also download the first stored event (read-only)")
    ap.add_argument("--capture", metavar="DIR",
                    help="also write a raw_bw_*/raw_s3_*.bin pair to DIR, so "
                         "this run can become a test fixture.  Worth doing on "
                         "any unit whose firmware line is new to us.")
    ap.add_argument("--lenient", action="store_true",
                    help="do not raise on a bad checksum — for diagnosis only")
    a = ap.parse_args()
    if ":" in a.target and not Path(a.target).exists():
        host, _, port = a.target.rpartition(":")
        inner = TcpTransport(host, int(port), connect_timeout=a.timeout)
        path = f"TCP {host}:{port}"
    else:
        inner = StdlibSerial(a.target, baud=a.baud)
        path = f"serial {a.target} @ {a.baud}"
    transport = _Timed(inner, capture=a.capture)
    mm = MicromateClient(transport, recv_timeout=a.timeout,
                         strict_checksums=not a.lenient)
    print(f"\n{path}   (read-only: POLL, SERIAL, state, status, setups)\n")
    t0 = time.monotonic()
    try:
        mm.open()
    except OSError as e:
        print(f"  connect....................  FAILED  {e}")
        return 2
    print(f"  {'connect':.<26} {1000*(time.monotonic()-t0):7.0f} ms")
    try:
        info = step("connect() identity", mm.connect)
        if info:
            print(f"       serial={info.serial}  model={info.model}  "
                  f"fw={info.firmware_line}  monitoring={info.monitoring}")
            print(f"       active setup={info.active_setup!r}")
            if info.firmware_line == "thor":
                print("       *** 11.0BD unit — the FIRST one this code has met. ***")
                print("       *** Check the battery and clock below carefully:  ***")
                print("       *** its 0x1C block is 4 bytes longer.             ***")
        state = step("get_state()", mm.get_state)
        if state:
            print(f"       {state}")
            if state.battery_volts and not 2.5 < state.battery_volts < 9.0:
                print(f"       *** battery {state.battery_volts} V is impossible — "
                      f"this is the from-the-end offset bug. ***")
            if state.device_time is None:
                print("       *** device clock did not decode — dump raw below. ***")
                print(f"       raw 0x1C content: {_content(state.raw).hex(' ')}")
        setups = step("list_setups()", mm.list_setups)
        if setups:
            print(f"       first={setups[0]!r}  last={setups[-1]!r}")
        # ── SUB 0x06: is content[0:4] really the event count? ─────────────
        # Two samples on one unit said yes, and THOR reads it BEFORE the chain
        # walk then stops without ever reading the sentinel.  A third value
        # either confirms it or kills it.
        claimed = None
        raw06 = step("0x06 storage range", mm.protocol.read_storage_range)
        if raw06:
            c = _content(raw06)
            claimed = int.from_bytes(c[0:4], "big")
            print(f"       content[0:4] = {claimed}   <- CANDIDATE: event count")
            print(f"       content[4:8] = {int.from_bytes(c[4:8],'big')}   "
                  f"<- unexplained (read 9 alongside a 6 on UM12947)")
        if a.download:
            print("\n  event chain (read-only), via MicromateClient:")
            t1 = time.monotonic()
            refs = mm.list_events()                 # walks to the sentinel
            walk = time.monotonic() - t1
            print(f"       {len(refs)} events in {walk:.1f} s "
                  f"({walk/max(len(refs),1):.2f} s each, 3 round trips per event)")
            if claimed is not None:
                verdict = ("✓ AGREES" if claimed == len(refs)
                           else f"✗ DISAGREES (0x06 said {claimed})")
                print(f"       0x06 count vs chain length: {verdict}")
            for ref in refs:
                print(f"       {ref}")
                print(f"         would be filed as {ref.filename}")
            if refs:
                # Download via iter_events, which reproduces THOR's interleaved
                # order -- the one with captures behind it.
                print("\n  download (first event, THOR's interleaved order):")
                for ref in mm.iter_events():
                    t1 = time.monotonic()
                    try:
                        result = mm.get_event(ref)       # verify=True
                    except Exception as e:
                        print(f"       {ref.key_hex}: {type(e).__name__}: {e}")
                        break
                    dt = max(time.monotonic() - t1, 1e-6)
                    n = sum(len(v) for v in getattr(result, "samples", {}).values())
                    err = mm.decode_error(ref, result)
                    check = ("PVS agrees to %+.4f%%" % (100 * err)
                             if err is not None else "PVS check n/a (histogram)")
                    print(f"       {ref.key_hex} {ref.record_type:9} {ref.size:6} B "
                          f"in {dt:.1f} s -> {n} samples, {check}")
                    break
    except ProtocolError as e:
        print(f"\n  ABORTED  {type(e).__name__}: {e}")
        return 3
    finally:
        mm.close()
        transport.close_capture()
        if a.capture:
            print(f"\n  capture written:\n    {transport.bw_path}\n    {transport.s3_path}")
            print("  parse it with: python3 scratch/mm_frame_parse.py "
                  f"{transport.bw_path} {transport.s3_path}")
    elapsed = time.monotonic() - t0
    print(f"\n  transport: {transport.reads} reads, "
          f"{transport.bytes_in} B in, {transport.bytes_out} B out, "
          f"{elapsed:.1f} s total")
    print("  Measured 2026-09-29, UM12947, same unit both ways:")
    print("     USB-B (CDC-ACM)   83 reads   list_setups  0.46 s   download 394 KiB/s")
    print("     RX55 (TCP)        36 reads   list_setups 16.05 s   download 1.6 KiB/s")
    print("  The modem needs FEWER reads, not more -- it buffers ~1 s and then")
    print("  forwards one large segment, where CDC-ACM delivers many small ones.")
    print("  Cost is ~0.65 s PER ROUND TRIP regardless of payload size, so what")
    print("  matters over cellular is the number of commands, not the bytes.\n")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,338 @@
 #!/usr/bin/env python3
 """
 mm_link.py — a "perfect modem" between THOR and a Micromate, with a readable
 log and deliberate fault injection.
 Why
 ---
 THOR gives almost no visibility into a connection: a refresh button, two poll
 intervals, and no way to see whether a check succeeded, timed out, or was never
 sent.  When a unit "won't stay connected" there is nothing to look at.
 This sits where the cellular modem would sit and answers the question directly:
  * **What is THOR actually doing?**  Every frame is decoded and timestamped —
    `POLL`, `MONITOR_STATUS`, `SETUP_NAME_READ` — not a hex dump.
  * **Is it even trying?**  Silence is visible: the log shows gaps.
  * **How does it behave when the link misbehaves?**  Faults can be injected on
    demand, which a real cell link will not do on cue.
 Point THOR at this host and port exactly as if it were a modem (Communication:
 TCP, IP: <this host>, Port: <--listen>).
 Fault injection
 ---------------
 Write a mode into the control file (default `mm_link.ctl`) and it takes effect
 on the next byte:
    echo pass      > mm_link.ctl    # normal relay
    echo blackhole > mm_link.ctl    # TCP stays up, bytes are swallowed
    echo drop      > mm_link.ctl    # close the connection abruptly (RST-ish)
    echo delay:2.0 > mm_link.ctl    # forward, but 2 s late in both directions
    echo onewaydev > mm_link.ctl    # THOR->unit passes, unit->THOR is swallowed
 **`blackhole` is the one that matters.**  It reproduces the classic cellular
 failure: the socket is still open as far as both ends are concerned, but nothing
 crosses.  A client that relies on TCP to tell it the peer is gone will sit there
 until the OS keepalive fires — which by default is about two hours.
 Usage
 -----
    python3 bridges/mm_link.py --serial /dev/ttyACM0 --baud 115200 \\
        --listen 12345 --logdir ~/mm-captures
 Writes, per session:
    <logdir>/mmlink_<ts>/session.log   decoded, timestamped, human-readable
    <logdir>/mmlink_<ts>/raw_bw.bin    THOR -> unit, raw
    <logdir>/mmlink_<ts>/raw_s3.bin    unit -> THOR, raw
 The raw pair loads straight into `scratch/mm_frame_parse.py`.
 """
 from __future__ import annotations
 import argparse
 import datetime
 import os
 import socket
 import sys
 import threading
 import time
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scratch"))
 try:
    from mm_frame_parse import SUBNAME, destuff  # noqa: F401
 except Exception:                                 # pragma: no cover
    SUBNAME = {}
 import errno
 import select
 import termios
 DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
 _BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
         57600: termios.B57600, 115200: termios.B115200}
 class SerialPort:
    """Minimal raw serial port on stdlib termios — no pyserial dependency.
    The bench hosts are whatever is to hand; requiring a pip install on someone
    else's machine is a poor trade for the ~30 lines this saves.
    """
    def __init__(self, path: str, baud: int):
        if baud not in _BAUD:
            raise ValueError(f"unsupported baud {baud}; pick one of {sorted(_BAUD)}")
        self.fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
        a = termios.tcgetattr(self.fd)
        a[0] = 0                                             # iflag: no translation
        a[1] = 0                                             # oflag: raw
        a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL  # cflag: 8N1, ignore modem lines
        a[3] = 0                                             # lflag: non-canonical, no echo
        a[4] = a[5] = _BAUD[baud]
        a[6] = list(a[6])
        a[6][termios.VMIN] = 0
        a[6][termios.VTIME] = 0
        termios.tcsetattr(self.fd, termios.TCSANOW, a)
        termios.tcflush(self.fd, termios.TCIOFLUSH)
    def read(self, n: int) -> bytes:
        r, _, _ = select.select([self.fd], [], [], 0.2)
        if not r:
            return b""
        try:
            return os.read(self.fd, n)
        except OSError as e:
            if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
                return b""
            raise
    def write(self, data: bytes) -> None:
        while data:
            try:
                data = data[os.write(self.fd, data):]
            except OSError as e:
                if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
                    select.select([], [self.fd], [], 0.2)
                    continue
                raise
    def close(self) -> None:
        try:
            os.close(self.fd)
        except OSError:
            pass
 def name_of(sub: int, is_request: bool) -> str:
    if is_request:
        return SUBNAME.get(sub, f"SUB_{sub:02X}")
    return "rsp " + SUBNAME.get(0xFF - sub, f"SUB_{0xFF - sub:02X}")
 class FrameSniffer:
    """Accumulate bytes and report complete frames, without altering the stream."""
    def __init__(self, is_request: bool):
        self.is_request = is_request
        self.buf = bytearray()
    def feed(self, data: bytes):
        """Yield (sub, payload_len) for each complete frame seen."""
        self.buf.extend(data)
        while True:
            start = -1
            for i, b in enumerate(self.buf):
                if self.is_request and b == ACK and i + 1 < len(self.buf) and self.buf[i + 1] == STX:
                    start = i
                    break
                if not self.is_request and b == STX:
                    start = i
                    break
            if start < 0:
                if len(self.buf) > 8192:
                    del self.buf[:-16]
                return
            j = start + (2 if self.is_request else 1)
            end = -1
            while j < len(self.buf):
                if self.buf[j] == DLE and j + 1 < len(self.buf):
                    j += 2
                    continue
                if self.buf[j] == ETX:
                    end = j
                    break
                j += 1
            if end < 0:
                return                      # wait for more bytes
            body = self.buf[start:end + 1]
            del self.buf[:end + 1]
            # SUB sits at a fixed spot past the leading framing -- but it is
            # DLE-escaped when its own value is 0x02/0x03/0x04/0x10, so a raw
            # read reports 0x10 for those.  SUB 0x02 was being logged as
            # "SUB_10" until this was handled.
            off = 5 if self.is_request else 3
            if len(body) > off:
                sub = body[off]
                if sub == DLE and len(body) > off + 1:
                    sub = body[off + 1]
                yield sub, len(body)
 class Link:
    def __init__(self, args):
        self.args = args
        self.mode = "pass"
        self.delay = 0.0
        self.ctl = Path(args.control)
        self.session: Path | None = None
        self.log_fh = None
        self.raw = {}
        self.t0 = time.time()
        self.counts = {}
    # ── logging ────────────────────────────────────────────────────────────
    def open_session(self):
        ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
        self.session = Path(self.args.logdir) / f"mmlink_{ts}"
        self.session.mkdir(parents=True, exist_ok=True)
        self.log_fh = open(self.session / "session.log", "a", buffering=1)
        self.raw = {
            "bw": open(self.session / "raw_bw.bin", "ab"),
            "s3": open(self.session / "raw_s3.bin", "ab"),
        }
        self.say(f"=== session {ts} — serial {self.args.serial} @ {self.args.baud} ===")
    def say(self, text: str):
        line = f"{datetime.datetime.now().strftime('%H:%M:%S.%f')[:-3]}  {text}"
        print(line, flush=True)
        if self.log_fh:
            self.log_fh.write(line + "\n")
    # ── control file ───────────────────────────────────────────────────────
    def poll_control(self):
        while True:
            try:
                if self.ctl.exists():
                    want = self.ctl.read_text().strip().lower()
                    if want.startswith("delay:"):
                        d = float(want.split(":", 1)[1])
                        if ("delay", d) != (self.mode, self.delay):
                            self.mode, self.delay = "delay", d
                            self.say(f"*** MODE -> delay {d}s ***")
                    elif want and want != self.mode:
                        self.mode, self.delay = want, 0.0
                        self.say(f"*** MODE -> {want} ***")
            except Exception:
                pass
            time.sleep(0.25)
    # ── the relay ──────────────────────────────────────────────────────────
    def pump(self, src, dst, tag: str, is_request: bool, stop: threading.Event):
        sniff = FrameSniffer(is_request)
        arrow = "THOR->unit" if is_request else "unit->THOR"
        last = time.time()
        while not stop.is_set():
            timed_out = False
            try:
                data = src.recv(4096) if isinstance(src, socket.socket) else src.read(4096)
            except TimeoutError:
                timed_out = True
                # socket.timeout subclasses OSError, so it MUST be caught first.
                # Treating it as a dead socket closes the connection after 200 ms
                # of quiet -- which is exactly what `blackhole` produces, so the
                # relay killed the link it was supposed to be faking a fault on.
                data = b""
            except OSError:
                break
            if isinstance(src, socket.socket) and data == b"" and not timed_out:
                self.say(f"{arrow}: peer closed the connection")
                break
            if not data:
                if time.time() - last > self.args.quiet_after and self.counts:
                    self.say(f"--- {self.args.quiet_after:.0f}s with no traffic ---")
                    last = time.time()
                continue
            last = time.time()
            self.raw[tag].write(data)
            self.raw[tag].flush()
            for sub, ln in sniff.feed(data):
                label = name_of(sub, is_request)
                self.counts[label] = self.counts.get(label, 0) + 1
                self.say(f"{arrow}  {label:<20} ({ln} B)"
                         + ("" if self.mode == "pass" else f"   [mode={self.mode}]"))
            mode = self.mode
            if mode == "drop":
                self.say(f"{arrow}: DROPPING the connection (fault injection)")
                stop.set()
                break
            if mode == "blackhole":
                continue                      # swallow, keep the socket open
            if mode == "onewaydev" and not is_request:
                continue                      # unit's replies never reach THOR
            if mode == "delay" and self.delay:
                time.sleep(self.delay)
            try:
                if isinstance(dst, socket.socket):
                    dst.sendall(data)
                else:
                    dst.write(data)
            except OSError:
                break
        stop.set()
    def serve(self):
        srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        srv.bind(("0.0.0.0", self.args.listen))
        srv.listen(5)
        self.open_session()
        self.say(f"listening on 0.0.0.0:{self.args.listen}   control file: {self.ctl}")
        self.say("point THOR at this host/port as Communication=TCP")
        threading.Thread(target=self.poll_control, daemon=True).start()
        while True:
            conn, addr = srv.accept()
            conn.settimeout(0.2)
            self.say(f"+++ THOR connected from {addr[0]}:{addr[1]} +++")
            try:
                ser = SerialPort(self.args.serial, self.args.baud)
            except OSError as e:
                self.say(f"!!! cannot open {self.args.serial}: {e}")
                conn.close()
                continue
            stop = threading.Event()
            ts = [
                threading.Thread(target=self.pump, args=(conn, ser, "bw", True, stop), daemon=True),
                threading.Thread(target=self.pump, args=(ser, conn, "s3", False, stop), daemon=True),
            ]
            for t in ts:
                t.start()
            for t in ts:
                t.join()
            conn.close()
            ser.close()
            summary = ", ".join(f"{k}x{v}" for k, v in sorted(self.counts.items()))
            self.say(f"--- connection closed.  frames this session: {summary or 'none'} ---")
 def main():
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--serial", default="/dev/ttyACM0")
    ap.add_argument("--baud", type=int, default=115200)
    ap.add_argument("--listen", type=int, default=12345)
    ap.add_argument("--logdir", default=os.path.expanduser("~/mm-captures"))
    ap.add_argument("--control", default="mm_link.ctl")
    ap.add_argument("--quiet-after", type=float, default=30.0,
                    help="log a marker after this many seconds of silence")
    Link(ap.parse_args()).serve()
 if __name__ == "__main__":
    main()
@@ -0,0 +1,239 @@
 #!/usr/bin/env python3
 """
 mm_probe.py — answer "why can't we reach this unit?" in one command.
 THOR reports a failed connection as "disconnected" and nothing else.  That single
 word covers at least four completely different faults with four different fixes,
 and telling them apart is the difference between a modem reboot and a site visit:
  * **connection refused**   something answered and said no — wrong port, or the
                             modem is refusing a further session
  * **connect timed out**    nothing answered at all — trusted-IP whitelist,
                             firewall, or the modem is off the network
  * **connected, no reply**  the MODEM answered but the unit did not.  The TCP
                             path is fine; the modem is not forwarding to serial.
                             This is the signature of a wedged transparent-TCP
                             session, and it is the one THOR cannot distinguish
                             from any of the others
  * **replied**              the unit is alive; the problem is upstream software
 Read-only.  It sends `POLL`, then optionally `SERIAL` and the state read — the
 same three commands THOR's own connection check uses — and never writes.
 Usage
 -----
    python3 bridges/mm_probe.py 63.45.161.30:9034
    python3 bridges/mm_probe.py 10.0.0.8:12345 --timeout 5
    python3 bridges/mm_probe.py <host:port> --slots 3
 `--slots N` opens N connections at once and reports how many the far end accepts.
 A transparent-TCP modem typically serves **one** session; if the first succeeds
 and the rest are refused or hang, that confirms the single-slot behaviour and
 explains why a leaked session takes a unit offline until the slot frees.
 Works for both series: a Series III reply opens `DLE STX`, a Micromate reply
 opens with a bare `STX`, so the probe also tells you which one answered.
 """
 from __future__ import annotations
 import argparse
 import socket
 import sys
 import time
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.framing import build_bw_frame  # noqa: E402
 DLE, STX, ETX = 0x10, 0x02, 0x03
 def destuff(raw: bytes) -> bytes:
    """Strip framing and DLE escapes; return the payload without its checksum."""
    i = 1 if raw and raw[0] == STX else (2 if len(raw) > 1 and raw[1] == STX else 0)
    out = bytearray()
    while i < len(raw):
        b = raw[i]
        if b == DLE and i + 1 < len(raw):
            out.append(raw[i + 1])
            i += 2
            continue
        if b == ETX:
            break
        out.append(b)
        i += 1
    return bytes(out[:-1]) if len(out) > 1 else b""
 # Reads are two-step on Series III: a probe at offset 0, then a data read at the
 # block's length.  THOR sends these offsets, and they also work on a Micromate.
 OFFSETS = {0x5B: 0x0030, 0x15: 0x000A, 0x49: 0xFFFF}
 def exchange(sock: socket.socket, sub: int, timeout: float) -> tuple[bytes, float]:
    sock.sendall(build_bw_frame(sub, OFFSETS.get(sub, 0)))
    t0 = time.time()
    buf, deadline = b"", t0 + timeout
    sock.settimeout(0.3)
    while time.time() < deadline:
        try:
            chunk = sock.recv(4096)
            if not chunk:
                break
            buf += chunk
            if buf.endswith(bytes([ETX])) and len(buf) > 8:
                break
        except TimeoutError:
            continue
        except OSError:
            break
    return buf, time.time() - t0
 def step(n: int, label: str, result: str) -> None:
    print(f"  [{n}] {label:.<28} {result}")
 def probe(host: str, port: int, timeout: float) -> int:
    print(f"\ntarget {host}:{port}   (read-only: POLL, SERIAL, state)\n")
    # ── 1. TCP ────────────────────────────────────────────────────────────
    t0 = time.time()
    try:
        sock = socket.create_connection((host, port), timeout=timeout)
    except ConnectionRefusedError:
        step(1, "TCP connect", f"REFUSED after {1000*(time.time()-t0):.0f} ms")
        print("\nverdict: something answered and actively refused.")
        print("  Not a silent firewall drop — the host is reachable.")
        print("  Wrong port, the service is down, or the modem is refusing")
        print("  an additional session because its one slot is in use.")
        return 2
    except (TimeoutError, socket.timeout):
        step(1, "TCP connect", f"TIMED OUT after {time.time()-t0:.1f} s")
        print("\nverdict: nothing answered at all.")
        print("  A silent drop, which is what a trusted-IP whitelist looks")
        print("  like — it discards rather than refuses.  Check the modem's")
        print("  Trusted IPs (and note a VPN changes the IP you arrive from),")
        print("  the firewall, and whether the modem is on the network.")
        return 3
    except OSError as e:
        step(1, "TCP connect", f"FAILED: {e}")
        return 4
    step(1, "TCP connect", f"ok ({1000*(time.time()-t0):.0f} ms)")
    # ── 2. POLL ───────────────────────────────────────────────────────────
    try:
        raw, dt = exchange(sock, 0x5B, timeout)
    except OSError as e:
        step(2, "POLL", f"send failed: {e}")
        sock.close()
        return 4
    if not raw:
        step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
        print("\nverdict: the MODEM answered but the unit did not.")
        print("  TCP is fine end to end — something accepted the connection.")
        print("  What is missing is the serial side.  Two quite different")
        print("  causes produce this, and they are NOT distinguishable from")
        print("  here:")
        print("\n  1. SOMEONE ELSE HOLDS THE SESSION.  These modems bridge ONE")
        print("     TCP session to serial at a time.  A second connection is")
        print("     accepted and then simply not forwarded.  Confirmed 2026-09-26:")
        print("     with THOR connected this probe saw exactly this; the moment")
        print("     THOR disconnected the same probe returned the serial number.")
        print("     ** Check whether THOR (or anything else) has the unit first. **")
        print("\n  2. The serial path is genuinely broken — a stale session the")
        print("     modem never released, a cable the unit cannot enumerate, or")
        print("     a unit that is off.")
        print("\n  Try, in order:")
        print("    1. Disconnect any other client and re-probe.  If it answers,")
        print("       it was contention, not a fault.")
        print("    2. The cable's chipset.  A Micromate drives FTDI and CDC-ACM")
        print("       only — a Prolific PL2303 gives it no serial port at all.")
        print("       lsusb: FTDI is 0403, Prolific 067b.")
        print("    3. Power-cycle the UNIT with the cable attached (hold power")
        print("       5 s, through the two-stage prompt).  Its USB host rescans")
        print("       on cold boot; it may not on hot-swap.")
        print("    4. AirLink OS -> TCP Idle Timeout.  If 0/disabled, a stale")
        print("       session holds the slot indefinitely.  2 minutes is the")
        print("       value this project standardised on.")
        sock.close()
        return 5
    series = "Series III (DLE STX)" if raw[0] == DLE else "Micromate (bare STX)"
    step(2, "POLL", f"reply {len(raw)} B in {1000*dt:.0f} ms")
    p = destuff(raw)
    ok = len(p) > 3 and p[2] == 0xFF - 0x5B
    step(3, "frame", f"{'valid' if ok else 'MALFORMED'}, {series}")
    if not ok:
        print("\nverdict: something replied, but not a seismograph.")
        print("  Another service is on this port, or the modem is in a mode")
        print("  that injects its own text (check Quiet Mode / AT echo).")
        print(f"  first bytes: {raw[:16].hex(' ')}")
        sock.close()
        return 6
    # ── 3. identity + state ───────────────────────────────────────────────
    for n, (sub, label) in enumerate(((0x15, "serial"), (0x49, "state")), start=4):
        try:
            r, dt = exchange(sock, sub, timeout)
            d = destuff(r)[5:]
            if sub == 0x15:
                # serial is a null-terminated run; a further field follows it
                serial = bytes(d[11:]).split(b"\x00")[0]
                step(n, label, serial.decode("ascii", "replace") or "(empty)")
            else:
                step(n, label, "MONITORING" if len(d) > 11 and d[11] else "idle")
        except OSError:
            step(n, label, "no reply")
    sock.close()
    print("\nverdict: the unit is alive and answering.")
    print("  If THOR still shows it disconnected, the fault is in THOR, not")
    print("  the network or the device.")
    return 0
 def slots(host: str, port: int, n: int, timeout: float) -> None:
    print(f"\nopening {n} simultaneous connections to {host}:{port}\n")
    held = []
    for i in range(n):
        try:
            s = socket.create_connection((host, port), timeout=timeout)
            held.append(s)
            step(i + 1, f"connection {i+1}", "accepted")
        except ConnectionRefusedError:
            step(i + 1, f"connection {i+1}", "REFUSED")
        except (TimeoutError, socket.timeout):
            step(i + 1, f"connection {i+1}", "timed out")
        except OSError as e:
            step(i + 1, f"connection {i+1}", f"failed: {e}")
    print(f"\n{len(held)} of {n} accepted.")
    if len(held) == 1:
        print("  Single-slot behaviour confirmed — this far end serves ONE")
        print("  session at a time.  A connection that is never closed takes")
        print("  the unit offline until the idle timeout frees the slot.")
    for s in held:
        s.close()
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("target", help="host:port, e.g. 63.45.161.30:9034")
    ap.add_argument("--timeout", type=float, default=10.0)
    ap.add_argument("--slots", type=int, metavar="N",
                    help="open N simultaneous connections to test single-slot behaviour")
    a = ap.parse_args()
    host, _, port = a.target.rpartition(":")
    if not host:
        ap.error("target must be host:port")
    if a.slots:
        slots(host, int(port), a.slots, a.timeout)
        return 0
    return probe(host, int(port), a.timeout)
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,408 @@
 # Spec — a live client for Series IV (Micromate)
 Drafted 2026-09-26, ahead of implementation.  The protocol work is finished; this
 is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
 run.
 **Read that document first.**  Everything here assumes it, and every constant
 below is sourced from it rather than restated with justification.
 ---
 ## Goal and scope
 `micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
 writers.  There is no way to talk to a unit.  This adds the live half, mirroring
 `minimateplus/`.
 **In scope, first pass:**
 - connect over TCP (a field modem) or serial/USB (a bench unit)
 - identify a unit, read its state, clock, memory and setups
 - walk the event chain and download events
 - return `Event` objects the existing codec already understands
 **Explicitly out of scope, first pass:**
 - ⚠ **Any write.**  Setups, schedules, call-home config, monitoring start/stop,
  and per-event delete are all mapped, and none of them will be implemented
  here.  **No command has ever been originated against a unit by this project**
  — every write observed was performed by THOR while we recorded.  Keeping that
  true through the read client is deliberate: it means the first thing we ever
  send to a customer's instrument is a decision someone made on purpose, not a
  side effect of a client that happened to grow a method.
 - the inbound call-home session — still the one protocol unknown
 ---
 ## Layout
 ```
 micromate/
  framing.py     NEW   frame building, response parsing, checksum
  protocol.py    NEW   one method per wire command, returns raw payloads
  client.py      NEW   high-level API, returns models
  idf_file.py    (existing — decodes what 0x5A returns, unchanged)
  models.py      (existing — extend, do not fork)
 ```
 **Transport is reused, not rewritten.**  `minimateplus/transport.py` is
 byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
 `TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
 habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller.  Import it.
 ⚠ Do **not** import `minimateplus.framing`.  The two framings differ in ways
 that look small and are not, and a shared module would accumulate `if series ==`
 branches until neither case is readable.
 ---
 ## `micromate/framing.py` — ✅ BUILT 2026-09-27
 Implemented, with `tests/test_micromate_framing.py` (31 tests, all passing).
 **Two things in this section as originally drafted were wrong**, and both were
 caught by measuring against the captures before writing code rather than after.
 They are left in place below, struck through, because both are the kind of
 mistake that would be made again.
 ### Requests
 ~~Series IV accepts Series III request frames unmodified.  The simplest correct
 implementation re-exports the builder rather than duplicating it:~~
 ```python
 from minimateplus.framing import build_bw_frame   # ✗ WRONG — 161/218
 ```
 ⚠ **`build_bw_frame` reproduces only 161 of Thor's 218 captured read frames.**
 The payload layout is identical; the stuffing is not.  A Micromate escapes
 **four** byte values — `0x02`, `0x03`, `0x04`, `0x10` — where Series III
 escapes one.  The frames this breaks are **every `SUB 0x5A` download**
 (`offset = 0x0400` → a literal `0x04` in `offset_hi`) and the scheduler enable.
 An unescaped `0x03`/`0x04` terminates the frame early, so the unit just does not
 answer.
 `build_request()` with the correct escape set reproduces **218/218**.
 ✅ **Settled: `0x10` inside params needs no special handling.**  The planned
 `NotImplementedError` guard is unnecessary — Thor sends
 `params = 00 00 10 00 …` on `SUB 0x5A` in five captured frames and the wire
 carries an ordinary doubled `10 10`.  One rule covers the whole payload.
 ### Responses — where Series III's parser cannot follow
 | | Series III | Micromate |
 |---|---|---|
 | frame start | `DLE STX` | **bare `STX`** |
 | `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
 | destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |
 The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
 it scans for `DLE+STX`, which never appears.
 The third is a genuine **simplification** — no inner-frame carve-out.  Validated
 by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
 four candidate destuffing rules were tried, and only this one makes all frames
 validate.
 ### Checksum
 ```python
 def checksum(payload: bytes) -> int:
    return sum(payload) & 0xFF          # payload already de-stuffed
 ```
 ~~The DLE-aware variant, same as Series III's `5A` and write frames.~~
 ⚠ **Plain SUM8, not the DLE-aware variant** — 251/251 both directions.  The
 DLE-aware form is the right answer paired with *Series III* de-stuffing, which
 leaves an escaped byte in the payload as two bytes.  De-stuffing `10 XX → XX`
 already removes the `0x10`, so excluding it again subtracts the correction
 twice, and the result disagrees with the wire on **55 of 251** captured
 responses — every frame holding a literal `0x10`.
 `scratch/mm_frame_parse.py` shipped with exactly that pairing.  It looked clean
 only because it accepts a frame matching *either* rule, so it labelled those 55
 `SUM8` and never flagged one bad.  "Zero bad checksums" was true and carried no
 information.  Fixed there too.
 ### ⚠ The SUB byte can be escaped
 When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
 Reading it positionally without destuffing reports `0x10`.  This bit once
 already — `SUB 0x02` was logged as `SUB_10` for an afternoon.  Destuff first,
 then index.
 ### Response shape
 ```python
@dataclass
 class MicromateFrame:
    sub: int              # response SUB;  request = 0xFF - sub
    flags: int            # 0xC5 Blastware line, 0x03 Thor line
    page_hi: int
    page_lo: int
    data: bytes           # payload[5:], checksum stripped
    checksum_valid: bool
    @property
    def request_sub(self) -> int:        # 0xFF - sub
    @property
    def page_key(self) -> int:           # uint16 BE at payload[3:5]
    @property
    def firmware_line(self) -> str:      # "blastware" | "thor" | "unknown"
    @property
    def probe_length(self) -> int | None: # uint16 BE at data[3:5] (= payload[8:10])
 ```
 ⚠ **`probe_length` is a uint16 BE.**  Read as a single byte it under-reads
 `SUB 0x1A` by 47x — 44 against a true 2092.  This is the single most expensive
 mistake available in this protocol and it has already been made once.
 Renamed from `declared_length`, because it is **only meaningful in the reply to
 an `offset = 0` probe** — and Thor never probes.  Across all 251 captured
 responses the field reads 0 or a page count, never a length, precisely because
 that session is single-step reads throughout.  `page_key` is the field that
 carries meaning there.  The only genuine probe reply we hold is the POLL one
 preserved in `scratch/fake_unit.py`.
 `MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
 accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
 what distinguishes "no bytes at all" from "bytes but no complete frame" on a
 timeout, and that distinction earned its keep during the Series III work).
 ---
 ## `micromate/protocol.py` — ✅ BUILT 2026-09-27
 Implemented, with `tests/test_micromate_protocol.py` (35 tests).  Reads only;
 nothing here writes, erases or changes monitoring state.
 The tests replay Thor's captured responses through a scripted transport and
 assert **the bytes we emit are the bytes Thor emits** — including a full replay
 of the six-event download session, all 56 `0x5A` frames byte-for-byte.  That is
 a stronger guarantee than "our parser understands the device": a passing test
 means a real unit has already answered exactly that frame.
 One method per command, returning raw payload bytes.  No interpretation — that
 belongs in `client.py`.
 **Reads use `offset = 0xFFFF`** and return the whole block in one response;
 Series III's two-step probe/data dance is unnecessary.  `POLL` is the exception,
 taking its data length.  Per-command offsets, all observed:
 | command | SUB | rsp | offset | returns |
 |---|---|---|---|---|
 | poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
 | serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
 | device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
 | state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
 | monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
 | storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
 | active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
 | first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
 | next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
 | compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
 | call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
 | arm event | `0x93` | `0x6C` | — | before every event |
 | first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
 | next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
 | event record | `0x0C` | `0xF3` | `0xFFFF` | 221 B — project, location, peaks |
 | ~~event header~~ **monitor log** | `0x0A` | `0xF5` | `0xFFFF` | ⚠ 297 B, a **walk** — see below |
 | bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim**, 1024 B at a time |
 ⚠ **Corrected 2026-09-27, from Thor's frames.** Three rows of the table above
 were wrong or incomplete, and the last one is a different command than labelled:
 - **`0x0A` is the monitor-log walk**, not a keyed "30 B list record" read.  The
  *same request repeated* returns successive 297-byte records — serial, mode,
  thresholds — until an 11-byte ack ends the list.  The device holds the cursor;
  nothing in the request selects a record.  Series III reaches this data through
  a record-type discriminator on its event chain; here it has its own cursor and
  the event chain never sees it.
 - **`0x1E`/`0x1F` carry token `0xFE` at `params[7]`.**  The reference documents
  all-zero params (our own probing, which also worked).  Thor's form is the one
  with mileage.
 - **`0x01` has no Thor frame behind it** — it is never read in any captured
  session.  Its `0xFFFF` comes from our probes.
 And one useful negative: **no `SESSION_RESET` (`41 03`)**.  Series III needs that
 2-byte signal or a monitoring unit will not answer `POLL` over TCP.  Thor never
 sends it — zero occurrences across 8 sessions, including 40 frames exchanged
 with a unit that *was* monitoring.
 ⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
 Parse **forward** from `declared_length`, never backward from the end — Series
 III reads battery and memory from the end of that block, and doing so on a BD
 unit yields a battery voltage of **577.92 V**.
 ⚠ **Test the monitoring flag for non-zero**, never against a constant.  It has
 read both `0x0E` and `0x0C` while monitoring.
 ### `0x5A` — a bounded chunk loop, and much simpler than Series III
 ⚠ **Corrected 2026-09-27.**  This section said "no chunk loop — one request
 returns the whole event", with `offset_word = 0x1000 + 2 * ceil(size / 512)`.
 That describes our own 2026-09-23 probes, which set `offset_hi = 0x10`.  **Thor
 chunks**, and Thor's form is the one verified from bytes on disk:
 ```python
 n = ceil(size / 1024)                       # size from the chain walk
 for i in range(n):
    offset = min(1024, size - 1024 * i)     # a BYTE COUNT
    params = key4 + bytes(6) if i == 0 else bytes(2) + pack(">H", 1024*i) + bytes(6)
    file_bytes += response.data[11:]        # response data is exactly offset + 11
 ```
 Verified on all six bench events (4,076 → 13,424 B): `sum(offsets) == size`
 exactly, with the predicted chunk count and final offset every time.
 Still no arming ritual for `0x5A` itself, no `STRT` end-offset parsing and no
 `TERM` frame — the simplification the original claim celebrated is real, it just
 is not single-shot.  (`SUB 0x93` arms the *chain walk*, before `1E`/`1F`, not
 the download.)
 The concatenated payload **is** the `.IDFW` file, byte for byte — so it feeds
 `micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.
 ⚠ Do not port the Series III `5A` walk.  Its address arithmetic caused a 5x
 over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
 III side.  None of that applies here: the chunk index is a byte offset into the
 file, bounded by a size the device told us, and it cannot run past the event.
 ⚠ **`assert sum(len(chunk) - 11 for chunk in chunks) == size`.**  A silently
 short event is the failure mode this project has been bitten by repeatedly on
 the Series III side, and here the check is free because the size is known up
 front.
 ---
 ## `micromate/client.py` — ✅ BUILT (read half) 2026-09-27
 `connect()`, `get_state()`, `get_active_setup()`, `list_setups()` plus
 `MicromateDeviceInfo` / `MicromateState` in `models.py`.  26 tests, every
 response constant a real captured data section.
 ⚠ **`connect()` is deliberately narrower than this spec asked for.**  The spec
 said to mirror Thor's `POLL → SERIAL → 0x49 → POLL` "because it is known-good".
 Measurement showed the four-command form is Thor's *connection check*, present
 in 3 of 8 sessions, and its fourth frame repeats its first — so `connect()`
 sends the three reads that gather something.  `0x01` is not read at all: Thor
 never reads it, its layout is unmapped, and `firmware_line` comes free from any
 response's flags byte.
 Event-chain methods (`list_events`, `download_event`, `get_event`) are step 4
 and not yet written; `MicromateProtocol.read_event_file()` already does the
 download.
 ```python
 class MicromateClient:
    def __init__(self, transport: BaseTransport): ...
    def open(self) / close(self) / is_open(self)
    # identity and state
    def connect(self) -> DeviceInfo          # poll → serial → device info → state
    def get_state(self) -> UnitState         # monitoring?, clock, battery, memory
    def get_active_setup(self) -> str
    def list_setups(self) -> list[str]       # 0x3F → 0x40… until empty
    # events
    def list_events(self) -> list[EventRef]  # 0x93 → 0x1E → 0x1F… (key + size)
    def download_event(self, ref) -> bytes   # raw .IDFW/.IDFH
    def get_event(self, ref) -> Event        # download + decode via idf_file
 ```
 `connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
 ⚠ but note the reference records that **whether the unit requires it is
 untested**.  Do it because it is known-good, not because it is known-necessary,
 and say so in the docstring.
 `list_events()` returns the key *and* the size, because `download_event()` needs
 the size to compute its offset word.
 ---
 ## Tests
 **Offline, from captured bytes — no hardware.**  This is the part worth doing
 first, because it can be fully verified tonight's-captures-style before any unit
 is involved.
 ```
 tests/test_micromate_framing.py
 ```
 ⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
 not depend on files being present.  **Embed the frames as hex constants** — they
 are 19–138 bytes each.  ✅ Done; what actually landed:
 | case | source | why |
 |---|---|---|
 | POLL probe reply, 19 B | captured (via `fake_unit.py`) | shortest valid frame; the only real probe reply we hold |
 | `0x5A` chunk, 138 B | captured | holds literal `0x10` **and** literal `0x41` — the checksum case, and proves ACK is not escaped |
 | `0x49` state reply, 25 B | captured | a literal `0x02`, escaped |
 | `0x48` file reply, 24 B | captured | escaped `0x04` in `data[0]` — one byte late without destuffing |
 | 8 Thor request frames | captured | byte-for-byte against `build_request()`, incl. both `0x5A` forms and the scheduler enable |
 | `probe_length = 0x082C` | **synthesised** | no probe reply for `0x1A` exists on disk — the 9-24-26 session never probes |
 | Thor-line reply, `flags = 0x03` | **synthesised** | no 11.0BD capture is in the repo; built by flipping one byte of the real POLL reply |
 | escaped checksum byte | **synthesised** | the shortest real one is 1,070 B, too long to embed for one assertion |
 | truncated / corrupt / split-across-feeds | derived | parser must return nothing, flag rather than swallow, and survive any split point |
 ⚠ Synthesised frames are marked `SYNTH_`-style in the test and each says what it
 stands in for and why no capture was available.  Do not let that set grow
 quietly: the `flags = 0x03` case in particular is the **only** coverage of half
 the fleet, and it deserves a real 11.0BD capture the next time UM20147 is on a
 bench.
 Two corpus-backed tests run when the captures happen to be on the box and skip
 cleanly otherwise: **251 response frames parse with zero bad checksums**, and
 **`build_request()` reproduces 218/218 read frames**.  The second is the test
 that would have caught the escape-set error, so it is worth the skip marker.
 ⚠ Do **not** assert against `scratch/mm_frame_parse.py`'s output as the original
 plan proposed.  That script accepts either checksum rule and is wrong about
 which one is right — using it as an oracle would have pinned the bug.
 **Live, second:** against the bench unit on mint-mac via `mm_link.py`.
 `connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
 then `download_event()` and assert the bytes decode and match a
 `/db/import/idf_file` ingest of the same event.
 ---
 ## Order of work
 1. ✅ `framing.py` + its tests — **done 2026-09-27**, 31 tests, offline
 2. ✅ `protocol.py` + its tests — **done 2026-09-27**, 35 tests, offline
 3. ✅ `client.py` + its tests — **done 2026-09-27**, 26 tests, offline
 4. the event chain and `download_event()`
 5. decode end-to-end and compare against a store event
 Steps 1–2 need no hardware at all.
 **Worth carrying forward.** Both steps began by measuring against the captures
 rather than trusting this document, and both found errors in it — three in the
 framing rules (the escape set, 26% of frames; the checksum, 22%; the `0x5A`
 chunk model) and three more in the command table (`0x0A`'s meaning, the
 `1E`/`1F` token, `0x01`'s provenance). All six fail quietly. The captures are on
 disk and a measure-then-write loop costs about two minutes per rule, so keep
 doing it for `client.py`'s field offsets — and treat this spec as a plan, not a
 source.
 ---
 ## Open questions to settle while implementing
 - ~~**Request param stuffing**~~ — ✅ settled 2026-09-27; no special handling.
 - **Is the single-request `0x5A` form real?**  Our 2026-09-23 probes set
  `offset_hi = 0x10` and appeared to get a whole 11 KB event back, where Thor
  chunks at 1024 B.  Plausibly a distinct streaming mode that returns several
  frames.  One bench test settles it; implement Thor's form regardless.
 - **Is THOR's preamble required?**  Try one command cold and find out; it is a
  two-minute test with the bench unit and it removes a ritual if unnecessary.
 - **`Event` model fit** — Series IV carries fields Series III lacks (setup file
  name, `LMic`/`SMic` channels).  Extend `micromate/models.py`; do not fork the
  shared `Event`.
 - **Which `0x0C` fields to trust.**  The peak float there runs 2–5% above
  `max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
  established.  The reference marks it do-not-rely-on — prefer decoded samples.
@@ -0,0 +1,568 @@
 """
 client.py — high-level API for a live Micromate (Series IV).
 Owns the transport, turns raw payloads into models.  Read-only, like the layer
 below it: nothing here writes, erases, or changes monitoring state.
    with MicromateClient(TcpTransport("63.45.161.30", 9034)) as mm:
        info = mm.connect()
        print(info)                    # UM12947  MM/ISEE/S/IO  blastware fw  idle
        print(mm.get_state())          # idle  2026-09-25 01:14:05  3.80 V  memory 0.4% used
        for name in mm.list_setups():
            print(name)
 The response layout, measured rather than assumed
 -------------------------------------------------
 **Every response carries an 11-byte prefix, and the content starts at
 ``data[11]``.**  That one rule covers every command.
 ⚠ **``data[0]`` looks like the content length and is only its low byte.**  A
 2,092-byte setup block (`SUB 0x1A`) reports 44, and a 1,024-byte download chunk
 reports 0.  It happens to be right for every response shorter than 256 bytes,
 which is most of them — so it reads as a working length field right up until it
 silently loses 2,048 bytes.  There is no high byte anywhere in the prefix; it is
 ``length & 0xFF`` and nothing more.
 This is the same trap as ``MicromateFrame.probe_length``, in a different place,
 and it is now the third time a length in this protocol has been read too narrow.
 **Take the content as ``data[11:]`` and let the frame's own length bound it.**
 """
 from __future__ import annotations
 import datetime
 import logging
 import math
 import os
 import struct
 from typing import Optional
 from minimateplus.transport import BaseTransport
 from .models import MicromateDeviceInfo, MicromateEventRef, MicromateState
 from .protocol import MicromateProtocol, ProtocolError
 log = logging.getLogger(__name__)
 # The content of every response begins here; the first 11 bytes are a prefix
 # whose only decoded field is an unreliable low-byte length (see module docstring).
 CONTENT = 11
 # Field offsets, relative to the start of content.  Sources are named because
 # two of them disagree with docs/micromate_protocol_reference.md.
 _STATE_FLAG = 0             # 0x49: 0x00 idle, 0x02 monitoring
 _MS_FLAG = 1                # 0x1C: monitoring flag — test NON-ZERO
 _MS_DAY, _MS_MONTH, _MS_YEAR = 2, 3, slice(4, 6)
 _MS_UNKNOWN_6 = 6           # ⚠ NOT the hour — see read note below
 _MS_HOUR, _MS_MIN, _MS_SEC = 7, 8, 9
 _MS_BATTERY = slice(34, 36)     # uint16 BE, volts × 100
 _MS_MEM_TOTAL = slice(36, 40)   # uint32 BE
 _MS_MEM_FREE = slice(40, 44)    # uint32 BE
 # `SUB 0x0C` record, relative to content.  Established against 7 events across
 # both firmware lines.
 _REC_DAY, _REC_MONTH, _REC_YEAR = 0, 1, slice(2, 4)
 _REC_UNKNOWN_4 = 4              # ⚠ same shape as 0x1C's content[6]; undecoded
 _REC_HOUR, _REC_MIN, _REC_SEC = 5, 6, 7
 _REC_TYPE = 11                  # 0x07 waveform, 0x08 histogram
 _REC_LOCATION = 12
 _REC_SETUP = 34
 _REC_SERIAL = 76
 _RECORD_TYPES = {0x07: "waveform", 0x08: "histogram"}
 # The channel labels the record carries, in the order they appear.  The peak
 # float sits `label + 6`; the peak vector sum sits 12 bytes BEFORE "Tran".
 _REC_CHANNELS = (b"Tran", b"Vert", b"Long", b"Mic")
 _REC_PVS_BACK = 12
 # A chain walk terminates on an all-zero key.  The ceilings below are guards
 # against a device cursor that never advances, not fleet limits.
 _NULL_KEY = bytes(4)
 _MAX_EVENTS = 4096
 # A setup-list walk that does not terminate is a bug, not a big fleet.  The
 # bench unit holds 22 setups; this is a generous ceiling, not a limit.
 _MAX_SETUPS = 512
 def _content(data: bytes) -> bytes:
    """Strip the 11-byte response prefix."""
    return data[CONTENT:] if len(data) > CONTENT else b""
 def _cstring(buf: bytes, offset: int = 0) -> str:
    """A null-terminated ASCII run, stripped."""
    return buf[offset:].split(b"\x00")[0].decode("ascii", "replace").strip()
 class DecodeMismatch(ProtocolError):
    """Our decoded peak disagrees with the one the device computed itself."""
 class MicromateClient:
    """High-level read-only client for one Micromate.
    Owns the transport, unlike ``MicromateProtocol``, which borrows it.
    """
    def __init__(
        self,
        transport: BaseTransport,
        recv_timeout: float = 10.0,
        strict_checksums: bool = True,
    ) -> None:
        self._transport = transport
        self._proto = MicromateProtocol(
            transport, recv_timeout=recv_timeout, strict_checksums=strict_checksums
        )
        self._firmware_line: Optional[str] = None
        self._serial: Optional[str] = None
    # ── Lifecycle ─────────────────────────────────────────────────────────────
    def open(self) -> None:
        self._transport.connect()
    def close(self) -> None:
        self._transport.disconnect()
    def is_open(self) -> bool:
        return self._transport.is_connected()
    def __enter__(self) -> "MicromateClient":
        self.open()
        return self
    def __exit__(self, *_) -> None:
        self.close()
    @property
    def protocol(self) -> MicromateProtocol:
        """The wire layer, for anything this class does not wrap yet."""
        return self._proto
    # ── Identity ──────────────────────────────────────────────────────────────
    def connect(self, *, with_active_setup: bool = True) -> MicromateDeviceInfo:
        """`POLL → SERIAL → state`, plus the active setup name.
        ⚠ **This is deliberately not Thor's full preamble.**  Thor sends
        `POLL → SERIAL → 0x49 → POLL` and the client spec said to copy it
        verbatim on the grounds that it is known-good.  Measuring all 8 captured
        sessions showed the only invariant is that a session **opens with
        POLL** — the four-command form appears in 3 of 8 and is Thor's
        *connection check*, run where it wants to refresh what it displays.  The
        trailing POLL is a repeat of the first.
        So this sends the three reads that actually gather something.  Dropping
        the fourth is a judgement call on measured evidence, not a proof that
        nothing depends on it; if a unit ever refuses the next command after a
        cold connect, put it back and say so in the protocol reference.
        `SUB 0x01` (device info) is **not** read.  Thor never reads it in any
        captured session, its field layout is unmapped beyond eight `1.0f`
        floats, and `firmware_line` — the one thing we would want from it — comes
        free from the flags byte of any response.
        """
        poll = self._proto.poll()
        self._firmware_line = poll.firmware_line
        manufacturer, model = self._parse_poll(poll.data)
        serial = _cstring(_content(self._proto.read_serial()))
        self._serial = serial
        monitoring = self._parse_state(self._proto.read_state())
        info = MicromateDeviceInfo(
            serial=serial,
            manufacturer=manufacturer,
            model=model,
            firmware_line=poll.firmware_line,
            monitoring=monitoring,
        )
        if with_active_setup:
            try:
                info.active_setup = self.get_active_setup()
            except ProtocolError as e:
                # Not worth failing a connect over: a unit with no setup loaded
                # is a real state, and the caller can still read everything else.
                log.warning("active setup unreadable: %s", e)
        log.info("connected: %s", info)
        return info
    @staticmethod
    def _parse_poll(data: bytes) -> tuple[Optional[str], Optional[str]]:
        """Manufacturer and model out of the POLL block.
        `Instantel` sits at content[4] and the model at content[26], with 13
        binary bytes between them.
        ⚠ A generic "find the printable runs" scan does **not** work here, which
        cost a test failure before it cost anything worse.  content[3] is `0x50`
        — printable as `P` — sitting immediately before `Instantel`, so a run
        scan returns `PInstantel`.  Nothing distinguishes a length or tag byte
        from text by inspection.
        So: the manufacturer comes from a fixed offset, and the model is found by
        searching for `MM/`.  That anchor is structural rather than positional,
        which matters because the model string **differs by firmware line** —
        `MM/ISEE/S/IO` on the Blastware build, `MM/ISEE/S` on the Thor build —
        and only its tail changes.
        """
        c = _content(data)
        manufacturer = _cstring(c, 4) or None
        idx = c.find(b"MM/")
        model = _cstring(c, idx) if idx >= 0 else None
        return manufacturer, model
    @staticmethod
    def _parse_state(data: bytes) -> Optional[bool]:
        """`SUB 0x49` content[0]: 0x00 idle, 0x02 monitoring.
        ⚠ Tested for non-zero, never against `0x02`.  The sibling flag in
        `SUB 0x1C` has read both `0x0E` and `0x0C` while monitoring, so this
        family of flags is not a stable enum.
        """
        c = _content(data)
        return bool(c[_STATE_FLAG]) if c else None
    # ── State ─────────────────────────────────────────────────────────────────
    def get_state(self) -> MicromateState:
        """`SUB 0x1C` — monitoring, device clock, battery, memory.
        ⚠ Every offset here is **forward from the start of content**, never
        backward from the end.  Series III reads battery and memory from the end
        of this block, and this block is **4 bytes longer on the Thor firmware
        line** — applying from-the-end offsets to a `11.0BD` unit yields a
        battery voltage of 577.92 V.  The four extra bytes are trailing, so
        from-the-start offsets hold for both lines.
        ✅ **Confirmed on `11.0BD` 2026-09-30.**  UM20147 read back 3.55 V and
        a clock correct to the second over USB, so the from-the-start offsets do
        survive the four extra trailing bytes.  Had they not, the battery would
        have read 577.92 V — which is what makes this cheap to check.
        """
        data = self._proto.read_monitor_status()
        c = _content(data)
        if len(c) < 44:
            raise ProtocolError(
                f"monitor status content is {len(c)} B, need at least 44"
            )
        battery = int.from_bytes(c[_MS_BATTERY], "big") / 100.0
        return MicromateState(
            monitoring=bool(c[_MS_FLAG]),
            device_time=self._parse_clock(c),
            battery_volts=battery,
            memory_total_bytes=int.from_bytes(c[_MS_MEM_TOTAL], "big"),
            memory_free_bytes=int.from_bytes(c[_MS_MEM_FREE], "big"),
            raw=data,
        )
    @staticmethod
    def _parse_clock(c: bytes) -> Optional[datetime.datetime]:
        """The unit's own clock, in its own local time.
        ⚠ **content[6] is not part of the time.**  The layout is day, month,
        year, *one unidentified byte*, then h/m/s — so the hour is at content[7].
        The protocol reference's `SUB 0x1C` section has this right and names
        `data[17]` as unidentified; its one-line summary in the divergences list
        ("day/month/year/h/m/s at `data[13:21]`") reads as six contiguous fields
        and is the version worth not trusting.
        Re-measured here across three captures: content[6] read 32, 100 and 116,
        none a valid hour, while content[7:10] gave 19:12:25, 19:13:34 and
        01:14:05 against capture filenames stamped 19:12:14, 19:12:14 and
        01:14:03 — each seconds to a minute after its session opened, which is
        what a device clock should do.
        content[6] is undecoded and deliberately not exposed.
        """
        try:
            return datetime.datetime(
                year=int.from_bytes(c[_MS_YEAR], "big"),
                month=c[_MS_MONTH],
                day=c[_MS_DAY],
                hour=c[_MS_HOUR],
                minute=c[_MS_MIN],
                second=c[_MS_SEC],
            )
        except ValueError as e:
            # A unit with a dead clock battery reports an impossible date.  That
            # is information, not a reason to fail the whole state read.
            log.warning("device clock unreadable (%s): %s", e, c[2:10].hex(" "))
            return None
    # ── Setups ────────────────────────────────────────────────────────────────
    def get_active_setup(self) -> str:
        """`SUB 0x41` — the loaded `.MMB` file name, e.g. `TEST1.mmb`."""
        return _cstring(_content(self._proto.read_active_setup_name()))
    def list_setups(self) -> list[str]:
        """`0x3F` then `0x40`… — every setup file stored on the unit.
        A cursor walk: the device holds the position, so the same `0x40` request
        returns the next name.  **An empty name terminates the list** — it is
        not an error and not a real setup.
        Measured on the bench unit: 23 responses, 22 names then the empty one,
        `factory.MMB` first through `TEST1.mmb` last.
        """
        names: list[str] = []
        raw = self._proto.read_first_setup()
        for _ in range(_MAX_SETUPS):
            name = _cstring(_content(raw))
            if not name:
                return names
            names.append(name)
            raw = self._proto.read_next_setup()
        raise ProtocolError(
            f"setup list did not terminate after {_MAX_SETUPS} entries — the "
            f"device cursor is not advancing"
        )
    # ── Events ────────────────────────────────────────────────────────────────
    def serial(self) -> str:
        """The unit's serial, cached from `connect()` or read on demand.
        Needed by anything that handles an event, because an event key is
        ambiguous without it — see `MicromateEventRef`.
        """
        if self._serial is None:
            self._serial = _cstring(_content(self._proto.read_serial()))
        return self._serial
    def list_events(self, *, with_records: bool = True) -> list[MicromateEventRef]:
        """Walk the event chain.  `0x93 → 1E`, then `0x93 → 1F` until the sentinel.
        THOR sends `0x93` before **every** chain read, and this mirrors that.
        The chain ends on an all-zero key.
        ⚠ `with_records=True` costs **one extra round trip per event** for the
        `0x0C` read, and over cellular a round trip is ~0.65 s regardless of
        size.  On a unit with 40 events that is the difference between ~52 s and
        ~78 s.  Pass False when you only need "what is here and how big" — but
        note the **record is where the type and timestamp live**, so without it
        `ref.filename` is None and `get_event()` cannot pick a suffix.
        ⚠ **To download, prefer `iter_events()`.**  This walks the whole chain
        first; THOR interleaves, downloading each event before advancing.
        `0x5A` addresses an event by key, so downloading afterwards *should*
        work — but "should" is doing real work in that sentence, and the
        interleaved order is the one with captures behind it.  See
        `iter_events()`.
        """
        serial = self.serial()
        refs: list[MicromateEventRef] = []
        for i in range(_MAX_EVENTS):
            self._proto.arm_event()
            raw = (self._proto.read_event_first() if i == 0
                   else self._proto.read_event_next())
            c = _content(raw)
            if len(c) < 8:
                raise ProtocolError(
                    f"chain entry {i}: {len(c)} B of content, need 8 (key + size)"
                )
            key, size = c[0:4], int.from_bytes(c[4:8], "big")
            if key == _NULL_KEY:
                return refs                       # the sentinel, not an error
            ref = MicromateEventRef(index=i, key=key, size=size, serial=serial)
            if with_records:
                self._read_record_into(ref)
            refs.append(ref)
        raise ProtocolError(
            f"event chain did not terminate after {_MAX_EVENTS} entries — the "
            f"device cursor is not advancing"
        )
    def iter_events(self, *, with_records: bool = True):
        """Walk the chain, yielding each event **at the cursor position THOR uses.**
            for ref in mm.iter_events():
                if ref.is_histogram:
                    continue
                data = mm.download_event(ref)      # ← safe here
        Why this exists alongside `list_events()`: THOR's captured order is
            0x93 → 1E → 0C → 5A×n → 0x93 → 1F → 0C → 5A×n → …
        — it downloads each event *before* advancing the chain.  `list_events()`
        walks to the end first, which is fine for browsing (the browse walk is
        separately attested) but means a later download happens with the device
        cursor parked past the event.  `0x5A` is key-addressed, so it very
        probably does not care; nothing observed says it does, and nothing
        observed says it does not.
        Downloading inside this loop reproduces THOR's sequence exactly, so it
        is the path to use when it matters.  ⚠ Do not advance the generator
        before finishing with the event it yielded.
        """
        serial = self.serial()
        for i in range(_MAX_EVENTS):
            self._proto.arm_event()
            raw = (self._proto.read_event_first() if i == 0
                   else self._proto.read_event_next())
            c = _content(raw)
            if len(c) < 8:
                raise ProtocolError(
                    f"chain entry {i}: {len(c)} B of content, need 8 (key + size)"
                )
            key, size = c[0:4], int.from_bytes(c[4:8], "big")
            if key == _NULL_KEY:
                return
            ref = MicromateEventRef(index=i, key=key, size=size, serial=serial)
            if with_records:
                self._read_record_into(ref)
            yield ref
        raise ProtocolError(
            f"event chain did not terminate after {_MAX_EVENTS} entries — the "
            f"device cursor is not advancing"
        )
    def _read_record_into(self, ref: MicromateEventRef) -> None:
        """`SUB 0x0C` — 210 B of content: timestamp, type, names, peaks."""
        raw = self._proto.read_event_record(ref.key)
        c = _content(raw)
        if len(c) <= _REC_SERIAL:
            raise ProtocolError(f"event record is {len(c)} B, too short to decode")
        ref.raw_record = raw
        ref.record_type = _RECORD_TYPES.get(c[_REC_TYPE])
        if ref.record_type is None:
            # Worth saying out loud rather than filing the event as a waveform:
            # the suffix decides which codec runs.
            log.warning("event %s: unknown record type 0x%02x at content[%d]",
                        ref.key_hex, c[_REC_TYPE], _REC_TYPE)
        try:
            ref.timestamp = datetime.datetime(
                year=int.from_bytes(c[_REC_YEAR], "big"),
                month=c[_REC_MONTH], day=c[_REC_DAY],
                hour=c[_REC_HOUR], minute=c[_REC_MIN], second=c[_REC_SEC],
            )
        except ValueError as e:
            log.warning("event %s: bad timestamp (%s): %s",
                        ref.key_hex, e, c[:8].hex(" "))
        ref.sensor_location = _cstring(c, _REC_LOCATION) or None
        ref.setup = _cstring(c, _REC_SETUP) or None
        # Prefer the record's own serial over the cached one — they have always
        # agreed, but the record is the event's own account of where it came from.
        if rec_serial := _cstring(c, _REC_SERIAL):
            ref.serial = rec_serial
        peaks: dict[str, float] = {}
        for label in _REC_CHANNELS:
            i = c.find(label)
            if i < 0 or i + len(label) + 10 > len(c):
                continue
            peaks[label.decode()] = struct.unpack(
                ">f", c[i + len(label) + 2: i + len(label) + 6])[0]
        ref.peaks_ips = peaks or None
        tran = c.find(b"Tran")
        if tran >= _REC_PVS_BACK:
            ref.peak_vector_sum_ips = struct.unpack(
                ">f", c[tran - _REC_PVS_BACK: tran - _REC_PVS_BACK + 4])[0]
    def download_event(self, ref: MicromateEventRef) -> bytes:
        """The raw `.IDFW`/`.IDFH` bytes, exactly as THOR would have stored them.
        Feeds `micromate.idf_file.read_idf_file()` and `/db/import/idf_file`
        unchanged — no new codec work is needed for a directly downloaded event.
        """
        return self._proto.read_event_file(ref.key, ref.size)
    def get_event(self, ref: MicromateEventRef, *, verify: bool = True,
                  tolerance: float = 0.01):
        """Download and decode one event.
        Returns the codec's `IdfReadResult`.  Needs `ref.record_type`, since
        `read_idf_file()` dispatches on the filename suffix and there is no
        filename on the wire — so call `list_events(with_records=True)` first.
        ⚠ `verify=True` re-computes the **peak vector sum** from the decoded
        samples and compares it against the float the *device* put in the `0x0C`
        record.  Those are two independent computations over the same samples —
        the device's from its own firmware, ours from our codec — so a
        disagreement means our decode is wrong.  Measured agreement on the bench
        events is **0.000%**.
        This is cheap insurance in a codebase whose decode failures have
        historically been *silent*: unhandled block tags shorten a channel and
        nothing raises.  ⚠ It is a decode-correctness check, **not** a
        truncation detector — a channel cut after its peak still yields the
        right PVS.
        Histograms are not verified: `samples` is empty for them.
        """
        if not ref.record_type:
            raise ValueError(
                f"event {ref.key_hex}: record_type is unknown, so the codec "
                f"cannot be dispatched.  Use list_events(with_records=True)."
            )
        blob = self.download_event(ref)
        import tempfile
        from .idf_file import read_idf_file
        # read_idf_file dispatches on the suffix, so the bytes need a name.
        with tempfile.NamedTemporaryFile(suffix=ref.suffix, delete=False) as f:
            f.write(blob)
            tmp = f.name
        try:
            result = read_idf_file(tmp)
        finally:
            os.unlink(tmp)
        if verify and not ref.is_histogram:
            err = self.decode_error(ref, result)
            if err is not None and abs(err) > tolerance:
                raise DecodeMismatch(
                    f"event {ref.uid}: decoded peak vector sum differs from the "
                    f"device's own by {100 * err:+.3f}% (tolerance "
                    f"{100 * tolerance:.1f}%) — the decode is suspect, not the "
                    f"device.  Stored {ref.peak_vector_sum_ips:.5f} in/s."
                )
            if err is not None:
                log.debug("event %s: PVS agrees to %+.4f%%", ref.uid, 100 * err)
        return result
    @staticmethod
    def decode_error(ref: MicromateEventRef, result) -> Optional[float]:
        """Relative error between our decoded PVS and the device's stored one.
        None when either side is unavailable.  Positive means the device's
        figure is higher than ours.
        """
        from .idf_file import geo_count_to_ips
        stored = ref.peak_vector_sum_ips
        if not stored or not getattr(result, "samples", None):
            return None
        ch = {k.lower(): v for k, v in result.samples.items()}
        try:
            t, v, l = ch["tran"], ch["vert"], ch["long"]
        except KeyError:
            return None
        n = min(len(t), len(v), len(l))
        if not n:
            return None
        pvs = max(
            math.sqrt(geo_count_to_ips(t[i]) ** 2
                      + geo_count_to_ips(v[i]) ** 2
                      + geo_count_to_ips(l[i]) ** 2)
            for i in range(n)
        )
        return (stored - pvs) / pvs if pvs else None
@@ -0,0 +1,304 @@
 """
 framing.py — frame codec for the Instantel Micromate (Series IV) wire protocol.
 A Micromate answers Series III *command* frames, so the request side looks
 familiar.  The framing underneath is not the same, and the differences are all
 of the kind that produce a silently-ignored frame rather than an error:
    Series III response:  [DLE 0x10] [STX 0x02] … [chk] [ETX 0x03]
    Micromate response:              [STX 0x02] … [chk] [ETX 0x03]
                                     ^ no leading DLE
 That missing byte is why `minimateplus.framing.S3FrameParser` returns *nothing*
 on Micromate traffic — it locates frames by scanning for `DLE STX`, which never
 occurs.  A capture holding 12 acknowledged writes reads as 12 unanswered
 requests.
 De-stuffed payload layout (both directions):
    request                        response
    [0]  CMD    0x10               [0]  CMD    0x00
    [1]  flags  0x00               [1]  flags  0xC5 / 0x03   ← firmware line
    [2]  SUB                       [2]  SUB    0xFF − request_SUB
    [3]  0x00                      [3]  PAGE_HI
    [4]  offset_hi                 [4]  PAGE_LO
    [5]  offset_lo                 [5+] data
    [6:16] params (10 bytes)
 Everything below was established against the 251 request and 251 response
 frames in `bridges/captures/9-24-26 - micromate2/` (UM12947, firmware 11.0CB).
 Where a rule is asserted, the number of frames it was checked on is given — the
 two rules that look like small details cost 26% and 22% of frames respectively
 when guessed wrong, so the counts are the point.
 """
 from __future__ import annotations
 from dataclasses import dataclass
 from typing import Optional
 # ── Protocol byte constants ───────────────────────────────────────────────────
 DLE = 0x10   # Data Link Escape
 STX = 0x02   # Start of text — begins a frame
 ETX = 0x03   # End of text   — ends a frame
 ACK = 0x41   # Frame-start marker on the request side
 MM_CMD      = 0x10   # payload[0] in a request
 MM_RSP_CMD  = 0x00   # payload[0] in a response
 # payload[1] of a response identifies the firmware line it came from.
 # ⚠ Two units, one of each — a strong hypothesis, not a proven encoding.
 FLAGS_BLASTWARE = 0xC5   # the 11.0CB line  (UM12947)
 FLAGS_THOR      = 0x03   # the 11.0BD line  (UM20147)
 # ⚠ THE ESCAPE SET.  A Micromate escapes exactly these four byte values,
 # prefixing each with a DLE — and nothing else.  Established by re-stuffing
 # every captured frame and comparing to the wire: 251/251 responses and 251/251
 # requests reproduce byte-for-byte with this set, and no other candidate set
 # reproduces even 200 of either.
 #
 # The two near-misses are worth naming, because both look plausible:
 #   * `{0x10}` alone — the Series III rule — reproduces 130/251 responses and
 #     177/251 requests.
 #   * adding ACK (0x41) reproduces only 196/251 responses: a literal 0x41 in
 #     the data is NOT escaped.
 _ESCAPED = frozenset({STX, ETX, 0x04, DLE})
 # A response header is 5 bytes; a frame must also carry its checksum.
 _MIN_PAYLOAD = 5
 _REQUEST_PAYLOAD_SIZE = 16
 # ── Stuffing ──────────────────────────────────────────────────────────────────
 def stuff(data: bytes) -> bytes:
    """Escape every byte the Micromate escapes: `XX` → `10 XX` for the four."""
    out = bytearray()
    for b in data:
        if b in _ESCAPED:
            out.append(DLE)
        out.append(b)
    return bytes(out)
 def unstuff(data: bytes) -> bytes:
    """Reverse `stuff()`: `10 XX` → `XX`, for any XX.
    Uniform, with no inner-frame carve-out — which is a real simplification
    over Series III, where `DLE+ETX` inside a frame is literal data that must
    survive de-stuffing.  Since only four byte values are ever escaped, taking
    *any* `10 XX` as `XX` is exact rather than merely convenient.
    """
    out = bytearray()
    i = 0
    while i < len(data):
        if data[i] == DLE and i + 1 < len(data):
            out.append(data[i + 1])
            i += 2
        else:
            out.append(data[i])
            i += 1
    return bytes(out)
 # ── Checksum ──────────────────────────────────────────────────────────────────
 def checksum(payload: bytes) -> int:
    """SUM8 of the **de-stuffed** payload, mod 256.  251/251 both directions.
    ⚠ Do NOT exclude `0x10` bytes from this sum.  The DLE-aware checksum that
    Series III uses for its `5A` and write frames is the right answer to a
    *different* question: it pairs with Series III de-stuffing, which leaves
    escaped bytes in the payload as two bytes.  De-stuffing uniformly already
    removes the DLE, so excluding `0x10` as well subtracts the correction
    twice.
    That combination — uniform de-stuffing *and* an exclusive sum — is what
    `scratch/mm_frame_parse.py` shipped with.  It disagrees with the wire on
    **55 of 251** captured response frames, all of them frames whose payload
    holds a literal `0x10`.  The script only ever looked correct because it
    accepts a frame that matches *either* rule, so it reported those 55 as
    plain SUM8 and never flagged one bad.
    """
    return sum(payload) & 0xFF
 # ── Request builder ───────────────────────────────────────────────────────────
 def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
    """Build a host→unit command frame.
    ⚠ Do **not** substitute `minimateplus.framing.build_bw_frame()` here, even
    though the payload layout is identical.  That builder escapes only `0x10`,
    so it reproduces just **161 of Thor's 218** captured read frames.  The 57 it
    gets wrong are not edge cases:
      * every `SUB 0x5A` bulk download — `offset = 0x0400` puts a literal
        `0x04` in `offset_hi`, which must go out as `10 04`
      * `SUB 0x47` (scheduler enable), whose params carry a `0x03`
    An unescaped `0x03` or `0x04` reads as a frame terminator, so the unit sees
    a truncated frame and simply does not answer.  That is indistinguishable
    from a dead unit, and event download would have hit it on the first try.
    With the correct escape set this builder reproduces **218/218**.
    Args:
        sub:    command SUB byte.
        offset: uint16 at payload[4:5].  Micromate reads are single-step —
                Thor asks for `0xFFFF` and gets the whole block — so this is
                usually `0xFFFF`, not Series III's probe-then-data pair.
        params: exactly 10 bytes at payload[6:16].
    A `0x10` inside `params` is fine and needs no special handling: Thor sends
    `SUB 0x5A` with `params = 00 00 10 00 …` and the wire carries `10 10`.
    (This was the spec's one open question; five captured frames settle it.)
    """
    if len(params) != 10:
        raise ValueError(f"params must be exactly 10 bytes, got {len(params)}")
    if not 0 <= offset <= 0xFFFF:
        raise ValueError(f"offset must fit in uint16, got {offset:#x}")
    if not 0 <= sub <= 0xFF:
        raise ValueError(f"sub must be a single byte, got {sub:#x}")
    payload = bytes([MM_CMD, 0x00, sub, 0x00, (offset >> 8) & 0xFF, offset & 0xFF]) + params
    body = payload + bytes([checksum(payload)])
    return bytes([ACK, STX]) + stuff(body) + bytes([ETX])
 # ── Response frame ────────────────────────────────────────────────────────────
@dataclass
 class MicromateFrame:
    """A parsed, de-stuffed unit→host response frame."""
    sub: int               # response SUB;  the request was 0xFF − this
    flags: int             # payload[1] — 0xC5 Blastware line, 0x03 Thor line
    page_hi: int
    page_lo: int
    data: bytes            # payload[5:], checksum stripped
    checksum_valid: bool
    chk_byte: int = 0      # the checksum byte as received
    @property
    def request_sub(self) -> int:
        """The SUB this is answering.  No known exception to `0xFF − SUB`."""
        return 0xFF - self.sub
    @property
    def page_key(self) -> int:
        """payload[3:5] as a uint16 BE — a page/address on `0x5A` responses."""
        return (self.page_hi << 8) | self.page_lo
    @property
    def firmware_line(self) -> str:
        return {FLAGS_BLASTWARE: "blastware", FLAGS_THOR: "thor"}.get(self.flags, "unknown")
    @property
    def probe_length(self) -> Optional[int]:
        """Data length declared by a **probe** response: uint16 BE at data[3:5].
        ⚠ Only meaningful in the reply to an `offset = 0` probe.  Series III
        hardcodes a `DATA_LENGTHS` table; a Micromate will tell you instead,
        which already caught one divergence (call-home config is `0x7E`, where
        Series III has `0x7C`).
        ⚠ It is a **uint16 BE**, not a byte.  Read as `data[3]` alone it is
        right only while the high byte is zero, and wrong by 47x for
        `SUB 0x1A`: a true `0x082C` (2092) reads as 44.
        Returns None on a frame too short to hold the field.  Note this reads
        as 0 on the single-step reads Thor actually uses — those are not probes,
        and `page_key` is the meaningful field there.
        """
        if len(self.data) < 5:
            return None
        return (self.data[3] << 8) | self.data[4]
 # ── Streaming parser ──────────────────────────────────────────────────────────
 class MicromateFrameParser:
    """Incremental parser for unit→host frames.  Mirrors `S3FrameParser`.
    Feed bytes with `feed()`; completed frames are returned and also collected
    in `.frames`.
        IDLE       — scanning for a bare STX
        IN_FRAME   — collecting; bare ETX terminates
        AFTER_DLE  — the next byte is literal, whatever it is
    Request frames are rejected rather than parsed: a frame whose `payload[0]`
    is not `0x00` is dropped, so feeding a bidirectional capture yields only
    the responses.
    """
    _IDLE, _IN_FRAME, _AFTER_DLE = 0, 1, 2
    def __init__(self) -> None:
        self._state = self._IDLE
        self._body = bytearray()
        self.frames: list[MicromateFrame] = []
        # Distinguishes "no bytes at all" from "bytes but no complete frame" on
        # a timeout.  That distinction earned its keep during the Series III
        # work and costs one integer here.
        self.bytes_fed: int = 0
    def reset(self) -> None:
        self._state = self._IDLE
        self._body.clear()
        self.bytes_fed = 0
    def feed(self, data: bytes) -> list[MicromateFrame]:
        self.bytes_fed += len(data)
        completed: list[MicromateFrame] = []
        for b in data:
            frame = self._step(b)
            if frame is not None:
                completed.append(frame)
                self.frames.append(frame)
        return completed
    def _step(self, b: int) -> Optional[MicromateFrame]:
        if self._state == self._IDLE:
            if b == STX:
                self._body.clear()
                self._state = self._IN_FRAME
            # Boot strings, modem RING/CONNECT chatter and stray ACKs land here
            # and are discarded.
        elif self._state == self._IN_FRAME:
            if b == DLE:
                self._state = self._AFTER_DLE
            elif b == ETX:
                self._state = self._IDLE
                return self._finalise()
            else:
                self._body.append(b)
        elif self._state == self._AFTER_DLE:
            # Uniform rule: the escaped byte is itself, including 0x03.
            self._body.append(b)
            self._state = self._IN_FRAME
        return None
    def _finalise(self) -> Optional[MicromateFrame]:
        body = bytes(self._body)
        if len(body) < _MIN_PAYLOAD + 1:
            return None
        payload, chk_received = body[:-1], body[-1]
        if payload[0] != MM_RSP_CMD:
            return None   # a request frame, or garbage that framed by accident
        return MicromateFrame(
            sub            = payload[2],
            flags          = payload[1],
            page_hi        = payload[3],
            page_lo        = payload[4],
            data           = payload[5:],
            checksum_valid = (chk_received == checksum(payload)),
            chk_byte       = chk_received,
        )
@@ -396,3 +396,158 @@ class IdfEvent:
        )
        ev._waveform_key = waveform_key
        return ev
 # ── Live-device models (2026-09-27) ───────────────────────────────────────────
 #
 # These describe what a unit reports over the wire, not what Thor wrote to a
 # file.  Everything above this line came out of Thor's exports; everything below
 # came out of Thor's *traffic*.  Field offsets are recorded in
 # ``micromate/client.py`` next to the code that reads them.
@dataclass
 class MicromateDeviceInfo:
    """Identity gathered by ``MicromateClient.connect()``.
    Sourced from three reads:
      ``0x5B`` POLL   → manufacturer, model
      ``0x15`` SERIAL → serial
      ``0x49`` STATE  → monitoring
    plus ``firmware_line``, which comes free from the flags byte of any
    response and needs no read of its own.
    """
    serial: str
    manufacturer: Optional[str] = None      # "Instantel"
    model: Optional[str] = None             # "MM/ISEE/S/IO" (CB) / "MM/ISEE/S" (BD)
    firmware_line: Optional[str] = None     # "blastware" | "thor" | "unknown"
    monitoring: Optional[bool] = None
    active_setup: Optional[str] = None      # e.g. "TEST1.mmb"
    def __str__(self) -> str:
        bits = [self.serial]
        if self.model:
            bits.append(self.model)
        if self.firmware_line:
            bits.append(f"{self.firmware_line} fw")
        if self.monitoring is not None:
            bits.append("MONITORING" if self.monitoring else "idle")
        if self.active_setup:
            bits.append(f"setup={self.active_setup}")
        return "  ".join(bits)
@dataclass
 class MicromateState:
    """A unit's live state, from ``SUB 0x1C``.
    ``device_time`` is the unit's own clock, in its own local timezone — it is
    NOT converted.  Nothing else this protocol exposes reports the unit's time,
    which makes it the only way to detect a drifted clock before it lands in
    event timestamps.
    """
    monitoring: bool
    device_time: Optional[datetime.datetime] = None
    battery_volts: Optional[float] = None
    memory_total_bytes: Optional[int] = None
    memory_free_bytes: Optional[int] = None
    raw: Optional[bytes] = field(default=None, repr=False)
    @property
    def memory_used_bytes(self) -> Optional[int]:
        if self.memory_total_bytes is None or self.memory_free_bytes is None:
            return None
        return self.memory_total_bytes - self.memory_free_bytes
    @property
    def memory_used_fraction(self) -> Optional[float]:
        used = self.memory_used_bytes
        if used is None or not self.memory_total_bytes:
            return None
        return used / self.memory_total_bytes
    def __str__(self) -> str:
        bits = ["MONITORING" if self.monitoring else "idle"]
        if self.device_time:
            bits.append(self.device_time.strftime("%Y-%m-%d %H:%M:%S"))
        if self.battery_volts is not None:
            bits.append(f"{self.battery_volts:.2f} V")
        frac = self.memory_used_fraction
        if frac is not None:
            bits.append(f"memory {frac * 100:.1f}% used")
        return "  ".join(bits)
@dataclass
 class MicromateEventRef:
    """One entry in a unit's event chain, from `1E`/`1F` and optionally `0x0C`.
    ⚠ **`key` is NOT unique across units.**  The event counter starts from the
    same value on every Micromate — UM12947 and UM20147 both have an event
    `055d4a81`, with different sizes and different contents.  Use `uid`, or key
    on `(serial, key_hex)`, for anything that stores or deduplicates.  A store
    keyed on the event key alone silently treats one unit's event as a duplicate
    of another's, and nothing raises.
    """
    index: int
    key: bytes                     # 4-byte event key from the chain walk
    size: int                      # bytes the device will send for this event
    serial: Optional[str] = None   # the unit, because `key` alone is ambiguous
    # From `SUB 0x0C` — one extra round trip per event, so optional.
    record_type: Optional[str] = None            # "waveform" | "histogram"
    timestamp: Optional[datetime.datetime] = None
    setup: Optional[str] = None                  # setup file name, no extension
    sensor_location: Optional[str] = None
    peak_vector_sum_ips: Optional[float] = None  # per-sample PVS, device-computed
    peaks_ips: Optional[Dict[str, float]] = None # {"Tran": …, "Vert": …, …}
    raw_record: Optional[bytes] = field(default=None, repr=False)
    @property
    def key_hex(self) -> str:
        return self.key.hex()
    @property
    def uid(self) -> str:
        """`SERIAL:key` — safe to use as a primary key.  See the class note."""
        return f"{self.serial or '?'}:{self.key_hex}"
    @property
    def is_histogram(self) -> Optional[bool]:
        if self.record_type is None:
            return None
        return self.record_type == "histogram"
    @property
    def suffix(self) -> Optional[str]:
        return {"waveform": ".IDFW", "histogram": ".IDFH"}.get(self.record_type or "")
    @property
    def filename(self) -> Optional[str]:
        """The name THOR would have given this event.
        `<serial>_<YYYYMMDDHHMMSS>.IDF{W,H}` — e.g.
        `UM12947_20260923163319.IDFW`.  Verified against the production store
        for all five bench events.
        ⚠ The type comes from the **protocol**, not the payload, so it has to be
        carried here from the `0x0C` read.  Returns None without it: guessing
        the suffix would file a histogram as a waveform, and `read_idf_file()`
        dispatches on exactly that.
        """
        if not (self.serial and self.timestamp and self.suffix):
            return None
        return f"{self.serial}_{self.timestamp:%Y%m%d%H%M%S}{self.suffix}"
    def __str__(self) -> str:
        bits = [self.uid, f"{self.size} B"]
        if self.record_type:
            bits.append(self.record_type)
        if self.timestamp:
            bits.append(self.timestamp.strftime("%Y-%m-%d %H:%M:%S"))
        if self.peak_vector_sum_ips is not None:
            bits.append(f"PVS {self.peak_vector_sum_ips:.4f} in/s")
        return "  ".join(bits)
@@ -0,0 +1,521 @@
 """
 protocol.py — one method per Micromate (Series IV) wire command.
 Returns raw payload bytes.  Interpretation belongs in ``client.py``; this layer
 knows frames, offsets and sequencing, and nothing about what a field means.
 Scope: **reads only.**  Nothing here writes, erases, or changes monitoring
 state.  That is deliberate and worth keeping — no command has ever been
 originated against a unit by this project; every write in
 ``docs/micromate_protocol_reference.md`` was performed by THOR while we
 recorded.  The first thing this codebase ever sends to a customer's instrument
 should be a decision someone made on purpose, not a side effect of a client
 that grew a method.
 Every offset and params layout below was read off THOR's own frames in
 ``bridges/captures/9-24-26 - micromate2/`` rather than taken from the spec
 table, because the same exercise during the framing work found three of that
 table's rules wrong.  It found three more here:
  * ``0x0A`` is the **monitor-log walk** — the same request repeated, the
    device advancing its own cursor, terminated by a short response — not the
    keyed "event header, 30 B list record" the spec describes.
  * ``0x1E``/``0x1F`` carry **token 0xFE** at ``params[7]``.  The protocol
    reference documents all-zero params for the browse walk; that was our own
    probing, and THOR does not do it that way.
  * There is **no fixed preamble**.  Sessions open with ``POLL`` and go
    straight to the operation.  ``POLL → SERIAL → 0x49 → POLL`` appears in 3 of
    8 captured sessions and is THOR's *connection check*, not a handshake.
 And one useful negative: **no ``SESSION_RESET`` (``41 03``).**  Series III
 needs that 2-byte signal to wake a monitoring unit or it will not answer POLL
 over TCP.  THOR never sends it — 0 occurrences across all 8 sessions, including
 40 frames exchanged with a unit that *was* monitoring.
 """
 from __future__ import annotations
 import logging
 import math
 import struct
 import time
 from typing import Optional
 from minimateplus.transport import BaseTransport
 from .framing import MicromateFrame, MicromateFrameParser, build_request
 log = logging.getLogger(__name__)
 DEFAULT_RECV_TIMEOUT = 10.0
 # An acknowledgement carries an 11-byte data section and nothing else.  It is
 # also how the monitor-log walk says "no more records".
 ACK_DATA_LEN = 11
 # ── Command SUBs ──────────────────────────────────────────────────────────────
 SUB_DEVICE_INFO       = 0x01
 SUB_STORAGE_RANGE     = 0x06
 SUB_EVENT_INDEX       = 0x08
 SUB_MONITOR_LOG       = 0x0A
 SUB_EVENT_RECORD      = 0x0C
 SUB_SERIAL            = 0x15
 SUB_COMPLIANCE_CONFIG = 0x1A
 SUB_MONITOR_STATUS    = 0x1C
 SUB_EVENT_FIRST       = 0x1E
 SUB_EVENT_NEXT        = 0x1F
 SUB_CALL_HOME_CONFIG  = 0x2C
 SUB_TRIGGER_CONFIG    = 0x2E
 SUB_SETUP_FIRST       = 0x3F
 SUB_SETUP_NEXT        = 0x40
 SUB_SETUP_ACTIVE      = 0x41
 SUB_STATE             = 0x49
 SUB_BULK_DOWNLOAD     = 0x5A
 SUB_POLL              = 0x5B
 SUB_ARM_EVENT         = 0x93
 # ⚠ Reads are SINGLE-STEP.  Series III probes at offset 0 to learn the length,
 # then reads again at that length; a Micromate returns the whole block when
 # asked for 0xFFFF.  THOR never probes, which is why `MicromateFrame.probe_length`
 # reads 0 on live traffic.
 READ_ALL = 0xFFFF
 # The two commands that do NOT use READ_ALL, and the data length each returned
 # on UM12947 (firmware 11.0CB).
 _OFFSETS = {
    SUB_POLL:   0x0030,   # 59 B  — the one offset THOR treats as a constant
    SUB_SERIAL: 0x000A,   # 21 B
 }
 # Data-section lengths observed, for orientation only — deliberately NOT
 # asserted.  `SUB 0x1C` is 4 bytes longer on the Thor firmware line (0x30 vs
 # 0x2C declared), so a length check here would fire spuriously on half the
 # fleet.  See the protocol reference, "A/B: Blastware build vs Thor build".
 OBSERVED_DATA_LEN = {
    SUB_STORAGE_RANGE: 47,   SUB_EVENT_INDEX: 101,  SUB_MONITOR_LOG: 297,
    SUB_EVENT_RECORD: 221,   SUB_SERIAL: 21,        SUB_COMPLIANCE_CONFIG: 2103,
    SUB_MONITOR_STATUS: 55,  SUB_EVENT_FIRST: 19,   SUB_EVENT_NEXT: 19,
    SUB_CALL_HOME_CONFIG: 137, SUB_TRIGGER_CONFIG: 39,
    SUB_SETUP_FIRST: 266,    SUB_SETUP_NEXT: 266,   SUB_SETUP_ACTIVE: 266,
    SUB_STATE: 16,           SUB_POLL: 59,          SUB_ARM_EVENT: ACK_DATA_LEN,
 }
 # `1E`/`1F` carry this at params[7].  Series III uses the same value to arm its
 # bulk stream; here THOR sends it on every chain read, browse or download.
 EVENT_TOKEN = 0xFE
 # `SUB 0x5A` chunk size, in bytes of file payload per response.
 CHUNK_SIZE = 1024
 # Every `0x5A` response prefixes the file bytes with 11 bytes of header.
 _CHUNK_PREFIX = 11
 # ── Exceptions ────────────────────────────────────────────────────────────────
 class ProtocolError(Exception):
    """The device violated the expected protocol."""
 class TimeoutError(ProtocolError):
    """No response arrived within the allowed time."""
 class ChecksumError(ProtocolError):
    """A received frame failed its checksum."""
 class UnexpectedResponse(ProtocolError):
    """The response SUB did not match the request."""
 class ShortRead(ProtocolError):
    """A bulk download returned fewer bytes than the device promised."""
 # ── Params builders ───────────────────────────────────────────────────────────
 def token_params(token: int = EVENT_TOKEN) -> bytes:
    """`1E`/`1F`: the token sits at params[7]."""
    return bytes(7) + bytes([token]) + bytes(2)
 def key_params(key4: bytes) -> bytes:
    """`0x0C`: the full 4-byte event key at params[4:8]."""
    if len(key4) != 4:
        raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
    return bytes(4) + key4 + bytes(2)
 def key_lo_params(key4: bytes) -> bytes:
    """`0x0A`: only the key's **low two bytes**, at params[6:8].
    ⚠ Inferred from a single key value.  All nine captured `0x0A` frames carry
    `4a 81`, and the event key in play was `055d4a81` — so this is consistent
    with "the low half of the current key" and equally consistent with "a
    cursor handle that happened to equal it".  Both readings produce the same
    bytes for that key, so one event cannot separate them.
    It does not matter much in practice: the walk works with the same params
    repeated, so whichever it is, passing the current key is right.
    """
    if len(key4) != 4:
        raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
    return bytes(6) + key4[2:4] + bytes(2)
 def chunk_params(key4: bytes, byte_offset: int) -> bytes:
    """`0x5A`: the key opens the file, then a byte offset walks it.
    Chunk 0 carries the event key at params[0:4] — that is what says "from the
    beginning".  Later chunks carry the byte offset **in that same 4-byte slot**,
    as a uint32 BE.
    ⚠ **Written as a uint32 deliberately, and this matters above 64 KB.**  Every
    offset THOR was observed to send fits in two bytes — the largest was `0x3400`
    (13,312) — so `params[0:2]` was always `00 00` and the field looks like a
    uint16 at `params[2:4]`.  Reading it that way caps a download at **65,536
    bytes**, and UM20147 currently holds a **72,560-byte** event, so that cap is
    not hypothetical.
    A uint32 here is **byte-identical for every offset below 65,536**, so it
    changes nothing that was verified against THOR's frames (the replay test
    asserts all 74 of them) and extends the range to 4 GB.  Above 64 KB it is
    **inference**: `params[0:4]` is demonstrably one 4-byte field, because chunk 0
    puts a 4-byte key in it, but no capture exercises a carry into `params[1]`.
    ⚠ This is the **Series III 64 KB page-boundary bug in a new guise** — there,
    `parse_strt_end_offset()` discards the key's page byte and the `5A` walk
    crashes once a unit's buffer crosses 64 KB, which is *still open* on that
    side.  The lesson that transfers: an address field whose high bytes are zero
    in every capture is not a narrow field, it is an untested one.
    """
    if byte_offset == 0:
        if len(key4) != 4:
            raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
        return key4 + bytes(6)
    if not 0 <= byte_offset <= 0xFFFFFFFF:
        raise ValueError(f"byte_offset must fit in uint32, got {byte_offset}")
    return struct.pack(">I", byte_offset) + bytes(6)
 # ── Protocol ──────────────────────────────────────────────────────────────────
 class MicromateProtocol:
    """Wire-level command set for one open connection to a Micromate.
    Does not own the transport; lifetime belongs to the client.
        proto = MicromateProtocol(transport)
        proto.poll()
        serial = proto.read_serial()
    """
    def __init__(
        self,
        transport: BaseTransport,
        recv_timeout: float = DEFAULT_RECV_TIMEOUT,
        strict_checksums: bool = True,
    ) -> None:
        """
        Args:
            strict_checksums: raise on a bad checksum.  **Defaults to True,
                unlike the Series III sibling**, which logs and continues
                because its parser cannot reliably tell an inner-frame
                delimiter from a checksum byte.  That excuse does not apply
                here: the Micromate rule is plain SUM8 over the de-stuffed
                payload and it holds on 251 of 251 captured frames, so a
                mismatch means something real — line noise, a desync, or a rule
                we have wrong — and all three are worth hearing about.
                The lenient Series III default is instructive: it hid the fact
                that the documented checksum rule was wrong for two days.  Set
                False only to get a field diagnosis unstuck.
        """
        self._transport = transport
        self._recv_timeout = recv_timeout
        self._strict = strict_checksums
        self._parser = MicromateFrameParser()
        self._pending: list[MicromateFrame] = []
    # ── Identity and state ────────────────────────────────────────────────────
    def poll(self) -> MicromateFrame:
        """`0x5B` → `0xA4`.  Handshake; carries the ID block and model string.
        Every captured session opens with this and nothing before it.
        """
        return self._exchange(SUB_POLL)
    def read_serial(self) -> bytes:
        """`0x15` → `0xEA`.  ASCII, null-terminated — e.g. `UM12947`."""
        return self._read(SUB_SERIAL)
    def read_device_info(self) -> bytes:
        """`0x01` → `0xFE`.  Firmware, calibration, per-channel float block.
        ⚠ THOR never sends this in any captured session, so the `0xFFFF` offset
        is from our own 2026-09-23 probes rather than from THOR's behaviour.  It
        answered correctly on both firmware lines, but it is the one read here
        with no THOR frame behind it.
        """
        return self._read(SUB_DEVICE_INFO)
    def read_state(self) -> bytes:
        """`0x49` → `0xB6`.  A cheap monitoring check; 16 B.
        ⚠ Test `data[11]` for **non-zero**, never against a constant — it has
        read both `0x0E` and `0x0C` while monitoring.
        """
        return self._read(SUB_STATE)
    def read_monitor_status(self) -> bytes:
        """`0x1C` → `0xE3`.  Flag, **device clock**, battery, memory.
        ⚠ Parse **forward** from the declared length, never backward from the
        end.  This block is 4 bytes longer on the Thor firmware line, and Series
        III's relative-to-end offsets yield a battery voltage of 577.92 V on a
        `11.0BD` unit.
        """
        return self._read(SUB_MONITOR_STATUS)
    def read_storage_range(self) -> bytes:
        """`0x06` → `0xF9`.  Event storage extent; 47 B."""
        return self._read(SUB_STORAGE_RANGE)
    def read_event_index(self) -> bytes:
        """`0x08` → `0xF7`.  101 B.  Contents not yet mapped."""
        return self._read(SUB_EVENT_INDEX)
    def read_trigger_config(self) -> bytes:
        """`0x2E` → `0xD1`.  39 B.  Series IV only; no Series III equivalent."""
        return self._read(SUB_TRIGGER_CONFIG)
    def read_compliance_config(self) -> bytes:
        """`0x1A` → `0xE5`.  The whole active setup — 2103 B on UM12947.
        One response.  Series III needs a 4-frame sequence for the same thing.
        """
        return self._read(SUB_COMPLIANCE_CONFIG)
    def read_call_home_config(self) -> bytes:
        """`0x2C` → `0xD3`.  137 B — Series III's is 124, so do not reuse its map."""
        return self._read(SUB_CALL_HOME_CONFIG)
    # ── Setups ────────────────────────────────────────────────────────────────
    def read_active_setup_name(self) -> bytes:
        """`0x41` → `0xBE`.  266 B; carries the active `.MMB` name."""
        return self._read(SUB_SETUP_ACTIVE)
    def read_first_setup(self) -> bytes:
        """`0x3F` → `0xC0`.  Head of the setup-file list."""
        return self._read(SUB_SETUP_FIRST)
    def read_next_setup(self) -> bytes:
        """`0x40` → `0xBF`.  Repeat until the record carries an empty name.
        Stateful: the device holds the cursor, so the same request walks the
        list.  22 of these appear back to back in one captured session.
        """
        return self._read(SUB_SETUP_NEXT)
    # ── Event chain ───────────────────────────────────────────────────────────
    def arm_event(self) -> MicromateFrame:
        """`0x93` → `0x6C`.  THOR sends this before **every** `1E`/`1F`.
        It replaces Series III's `1E(token=0xFE)` arming step.  No params, no
        offset payload — an 11-byte ack.
        ⚠ Whether a unit actually requires it is untested.  Do it because it is
        known-good, not because it is known-necessary.
        """
        return self._exchange(SUB_ARM_EVENT, offset=READ_ALL)
    def read_event_first(self) -> bytes:
        """`0x1E` → `0xE1`.  First event key + size; 19 B."""
        return self._read(SUB_EVENT_FIRST, params=token_params())
    def read_event_next(self) -> bytes:
        """`0x1F` → `0xE0`.  Next key + size, or the all-zero null sentinel."""
        return self._read(SUB_EVENT_NEXT, params=token_params())
    def read_event_record(self, key4: bytes) -> bytes:
        """`0x0C` → `0xF3`.  221 B — project, client, operator, timestamp, peaks.
        ⚠ The peak float in here runs 2–5% above `max(T,V,L)` and is **not** the
        vector sum; its offset was inferred, not established.  Prefer decoded
        samples.
        """
        return self._read(SUB_EVENT_RECORD, params=key_params(key4))
    def read_monitor_log_next(self, key4: bytes) -> Optional[bytes]:
        """`0x0A` → `0xF5`.  One monitor-log record, or None at end of list.
        ⚠ Not the keyed single read the spec describes.  This is a **walk**:
        the same request repeated, the device advancing its own cursor, each
        response a 297-byte record carrying serial, mode and thresholds.  The
        list ends with a bare 11-byte ack — nine captured frames, eight records
        then the terminator.
        Series III reaches the same data through a record-type discriminator on
        its event walk (`0x2C` partial vs `0x46` full).  Here it is a separate
        cursor and the event chain does not see it at all.
        """
        data = self._read(SUB_MONITOR_LOG, params=key_lo_params(key4))
        return None if len(data) <= ACK_DATA_LEN else data
    # ── Bulk download ─────────────────────────────────────────────────────────
    def read_event_file(self, key4: bytes, size: int) -> bytes:
        """`0x5A` → `0xA5`.  The `.IDFW`/`.IDFH` file, byte for byte.
        `size` is the 4 bytes after the key in the `1E`/`1F` response.  Returns
        exactly that many bytes, or raises `ShortRead`.
        A bounded chunk walk — `ceil(size / 1024)` requests, each asking for
        `min(1024, remaining)` bytes:
            offset       = the byte count wanted (NOT an address)
            params       = the key on chunk 0, then a uint16 BE byte offset
            response     = exactly `offset + 11` bytes; file bytes are data[11:]
        Verified against THOR on all six bench events (4,076 → 13,424 B):
        `sum(offsets) == size` exactly, every time.
        ⚠ Do not port the Series III `5A` walk.  Its address arithmetic caused a
        5x over-read and a `>64 KB` page-boundary bug that is *still open* on
        that side.  Neither applies here — the cursor is a byte offset into the
        file, bounded by a size the device supplied, so it cannot run past the
        event.
        The result feeds `micromate.idf_file.read_idf_file()` and
        `/db/import/idf_file` unchanged; no new codec work is needed.
        """
        if size <= 0:
            raise ValueError(f"size must be positive, got {size}")
        out = bytearray()
        n_chunks = math.ceil(size / CHUNK_SIZE)
        for i in range(n_chunks):
            want = min(CHUNK_SIZE, size - i * CHUNK_SIZE)
            data = self._read(
                SUB_BULK_DOWNLOAD,
                offset=want,
                params=chunk_params(key4, i * CHUNK_SIZE),
            )
            if len(data) < _CHUNK_PREFIX:
                raise ShortRead(
                    f"chunk {i + 1}/{n_chunks} of {key4.hex()}: "
                    f"{len(data)} B is too short to hold a chunk header"
                )
            body = data[_CHUNK_PREFIX:]
            if len(body) != want:
                # Worth being loud: a silently short event is the failure mode
                # this project has been bitten by repeatedly on the Series III
                # side, and here the expected length is known up front.
                raise ShortRead(
                    f"chunk {i + 1}/{n_chunks} of {key4.hex()}: asked for "
                    f"{want} B, got {len(body)}"
                )
            out += body
        if len(out) != size:
            raise ShortRead(
                f"{key4.hex()}: assembled {len(out)} B, device promised {size}"
            )
        log.debug("downloaded %s: %d B in %d chunks", key4.hex(), len(out), n_chunks)
        return bytes(out)
    # ── Plumbing ──────────────────────────────────────────────────────────────
    def _read(
        self,
        sub: int,
        *,
        params: bytes = bytes(10),
        offset: Optional[int] = None,
        timeout: Optional[float] = None,
    ) -> bytes:
        """Send one read command, return the response's data section."""
        return self._exchange(sub, params=params, offset=offset, timeout=timeout).data
    def _exchange(
        self,
        sub: int,
        *,
        params: bytes = bytes(10),
        offset: Optional[int] = None,
        timeout: Optional[float] = None,
    ) -> MicromateFrame:
        if offset is None:
            offset = _OFFSETS.get(sub, READ_ALL)
        # Start every exchange clean: drop any half-frame and any frame left
        # stashed by the last one.  This is a strict request/response protocol,
        # so anything already buffered when we send is by definition stale, and
        # `expected_sub` would reject it anyway — better to discard it here than
        # to raise a confusing UnexpectedResponse one command later.
        self._parser.reset()
        self._pending.clear()
        self._send(build_request(sub, offset, params))
        return self._recv_one(expected_sub=0xFF - sub, timeout=timeout,
                              reset_parser=False)
    def _send(self, frame: bytes) -> None:
        log.debug("TX %d bytes: %s", len(frame), frame.hex())
        self._transport.write(frame)
    def _recv_one(
        self,
        expected_sub: Optional[int] = None,
        timeout: Optional[float] = None,
        reset_parser: bool = True,
    ) -> MicromateFrame:
        """Read until one complete frame is parsed."""
        deadline = time.monotonic() + (timeout or self._recv_timeout)
        if reset_parser:
            self._parser.reset()
            self._pending.clear()
        if self._pending:
            return self._validate(self._pending.pop(0), expected_sub)
        while time.monotonic() < deadline:
            chunk = self._transport.read(4096)
            if not chunk:
                time.sleep(0.005)
                continue
            log.debug("RX %d bytes", len(chunk))
            frames = self._parser.feed(chunk)
            if frames:
                self._pending.extend(frames[1:])
                return self._validate(frames[0], expected_sub)
        raise TimeoutError(
            f"no frame in {timeout or self._recv_timeout:.1f}s"
            + (f" (expected SUB 0x{expected_sub:02X})" if expected_sub is not None else "")
            + f"; {self._parser.bytes_fed} bytes were received"
            # That byte count is the whole point: it separates "nothing came
            # back at all" from "bytes arrived but never framed", and those have
            # completely different causes.  It earned its keep on Series III.
        )
    def _validate(
        self, frame: MicromateFrame, expected_sub: Optional[int]
    ) -> MicromateFrame:
        if not frame.checksum_valid:
            msg = (
                f"SUB 0x{frame.sub:02X}: checksum mismatch "
                f"(got 0x{frame.chk_byte:02X}, {len(frame.data)} B data)"
            )
            if self._strict:
                raise ChecksumError(msg)
            log.warning("%s — continuing (strict_checksums=False)", msg)
        if expected_sub is not None and frame.sub != expected_sub:
            raise UnexpectedResponse(
                f"expected SUB 0x{expected_sub:02X}, got 0x{frame.sub:02X}"
            )
        return frame
@@ -0,0 +1,33 @@
 """Pretend to be a Micromate on a serial port: log what arrives, reply to POLL.
 Proves the modem's return path (serial -> TCP) independently of the real unit.
 """
 import os, select, sys, termios, time
 path, baud = sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 115200
 B = {9600: termios.B9600, 38400: termios.B38400, 115200: termios.B115200}[baud]
 fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
 a = termios.tcgetattr(fd)
 a[0] = a[1] = a[3] = 0
 a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL
 a[4] = a[5] = B
 a[6] = list(a[6]); a[6][termios.VMIN] = 0; a[6][termios.VTIME] = 0
 termios.tcsetattr(fd, termios.TCSANOW, a)
 termios.tcflush(fd, termios.TCIOFLUSH)
 # A real POLL probe reply, captured from UM12947 on 2026-09-24.
 REPLY = bytes.fromhex("0200c5a4000000000000300000000000000099") + b"\x03"
 print(f"fake unit on {path} @ {baud}; will answer any inbound frame", flush=True)
 while True:
    r, _, _ = select.select([fd], [], [], 1.0)
    if not r:
        continue
    data = os.read(fd, 4096)
    if not data:
        continue
    ts = time.strftime("%H:%M:%S")
    print(f"{ts}  IN  {len(data):3} B  {data.hex(' ')}", flush=True)
    time.sleep(0.02)
    os.write(fd, REPLY)
    print(f"{ts}  OUT {len(REPLY):3} B  {REPLY.hex(' ')}  <- canned POLL reply", flush=True)
@@ -0,0 +1,223 @@
 #!/usr/bin/env python3
 """
 mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
 raw capture pair.
 Why this exists
 ---------------
 `minimateplus.framing.S3FrameParser` cannot see Micromate traffic.  It locates
 frames by scanning for `DLE STX`, and a Micromate response has **no leading
 DLE** — it starts at a bare `STX`.  It also expects `payload[1] == 0x10`, where
 the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
 The practical consequence, seen on the 9-24-26 setup-push capture: the
 Blastware-side requests parse fine (Thor emits Series III request frames), but
 **every device response is silently dropped or mis-framed** — so a capture that
 actually contains 12 acked writes looks like 12 unanswered requests.
 Destuffing
 ----------
 One rule covers both directions: after the leading doubled `BW_CMD`, every
 `10 XX` pair on the wire destuffs to `XX`.  That includes `10 03` — Thor
 escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
 exactly as Blastware does.
 A Micromate escapes exactly four byte values — `0x02 0x03 0x04 0x10` — and
 nothing else, which is what makes the uniform rule exact rather than merely
 convenient.  Established by re-stuffing all 502 captured frames and comparing
 to the wire: 251/251 each direction, where `{0x10}` alone gets 130 and 177.
 ⚠ Checksum, corrected 2026-09-27
 --------------------------------
 With uniform destuffing the checksum is **plain SUM8 of the destuffed
 payload**.  This script used to try SUM8 *and* a "DLE-aware" variant that
 excludes `0x10` bytes, and report whichever matched — which is why it never
 flagged a bad frame and why the protocol reference carried the wrong rule for
 two days.  The DLE-aware form belongs with *Series III* destuffing, where an
 escaped byte survives as two bytes; applying it after uniform destuffing
 subtracts the correction twice and disagrees with the wire on 55 of 251
 responses.
 The lesson generalises: a tool that accepts any of N candidate rules cannot
 falsify any of them.  It now validates against SUM8 alone, and reports
 `DLE-aware` only to name what a mismatch *would* have been — never as a pass.
 See `docs/micromate_protocol_reference.md` → *Checksum*, and
 `tests/test_micromate_framing.py`, which pins it.
 `micromate/framing.py` is the production implementation; this stays as the
 one-shot capture-inspection tool.
 Usage
 -----
    python scratch/mm_frame_parse.py <capture-dir>
    python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
    python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
 """
 from __future__ import annotations
 import argparse
 import sys
 from pathlib import Path
 DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
 # Request SUB -> short name.  Series III names where they carry over; the
 # Series IV additions are marked.
 SUBNAME = {
    0x01: "DEVICE_INFO",
    0x06: "STORAGE_RANGE",
    0x08: "EVENT_INDEX",
    0x0A: "WAVEFORM_HDR",
    0x0C: "WAVEFORM_REC",
    0x15: "SERIAL",
    0x1A: "COMPLIANCE_CFG",
    0x1C: "MONITOR_STATUS",
    0x1E: "EVENT_HDR",
    0x2C: "CALLHOME_CFG",
    0x2E: "TRIGGER_CFG_READ",     # Series IV
    0x3E: "OPERATOR",
    0x41: "SETUP_NAME_READ",      # Series IV
    0x5A: "BULK_DOWNLOAD",
    0x5B: "POLL",
    0x68: "EVENT_INDEX_WRITE",
    0x69: "WAVEFORM_WRITE",
    0x71: "COMPLIANCE_WRITE",
    0x72: "CONFIRM_A",
    0x73: "CONFIRM_B",
    0x74: "CONFIRM_C",
    0x82: "TRIGGER_WRITE",
    0x83: "TRIGGER_CONFIRM",
    0xDA: "SETUP_FILE_DECL",      # Series IV — names the target .MMB
    0xFE: "FULL_CFG",
 }
 def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
    """Destuff one frame starting at `start`.
    Returns (payload, checksum, index_of_terminating_ETX).  `payload` excludes
    the trailing checksum byte.  A request frame opens `ACK STX 10 10`; a
    response opens with a bare `STX`.
    """
    i = start + (2 if is_request else 1)
    out = bytearray()
    if is_request:
        # The doubled BW_CMD is the one guaranteed stuffed byte.
        if blob[i : i + 2] != bytes([DLE, DLE]):
            raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
        out.append(DLE)
        i += 2
    while i < len(blob):
        b = blob[i]
        if b == DLE and i + 1 < len(blob):
            out.append(blob[i + 1])
            i += 2
            continue
        if b == ETX:
            break
        out.append(b)
        i += 1
    if len(out) < 2:
        raise ValueError(f"@0x{start:04x}: frame too short")
    return bytes(out[:-1]), out[-1], i
 def frames(blob: bytes, *, is_request: bool):
    """Yield (offset, payload, chk, checksum_kind)."""
    i, n = 0, len(blob)
    while i < n:
        if is_request:
            if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
                i += 1
                continue
        elif blob[i] != STX:
            i += 1
            continue
        try:
            payload, chk, end = destuff(blob, i, is_request=is_request)
        except ValueError:
            i += 1
            continue
        # SUM8 of the destuffed payload is THE rule -- 502/502 captured frames.
        # The DLE-aware variant is reported only to name a near-miss; it is
        # never a pass.  See the module docstring.
        if (sum(payload) & 0xFF) == chk:
            kind = "ok"
        elif (sum(b for b in payload if b != DLE) & 0xFF) == chk:
            kind = "BAD(dle-aware)"
        else:
            kind = "BAD"
        yield i, payload, chk, kind
        i = end + 1
 def describe(payload: bytes, is_request: bool) -> str:
    if len(payload) < 3:
        return "??"
    sub = payload[2]
    if is_request:
        return SUBNAME.get(sub, f"SUB_{sub:02X}")
    req = 0xFF - sub
    return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
 def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
    blob = path.read_bytes()
    side = "Thor" if is_request else "unit"
    print(f"== {side:4}  {path.name}  ({len(blob)} bytes)")
    n_bad = 0
    for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
        if kind == "BAD":
            n_bad += 1
        sub = p[2] if len(p) > 2 else -1
        flags = p[1] if len(p) > 1 else -1
        # Requests carry offset at payload[4:6]; responses page at [3:5].
        word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
        data = len(p) - 16 if is_request else max(len(p) - 5, 0)
        print(
            f"  [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
            f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
            f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
        )
        if dump_sub is not None and sub == dump_sub:
            body = p[16:] if is_request else p[5:]
            print(f"       ---- data ({len(body)} bytes) ----")
            for o in range(0, len(body), 16):
                chunk = body[o : o + 16]
                txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
                print(f"       {o:06x}  {chunk.hex(' '):<47}  |{txt}|")
    print(f"  -- {idx + 1} frames, {n_bad} bad checksum\n")
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("paths", nargs="+",
                    help="a capture directory, or raw_bw.bin and raw_s3.bin")
    ap.add_argument("--dump", default=None,
                    help="hex-dump the data section of this SUB (e.g. 0x71)")
    args = ap.parse_args()
    dump_sub = int(args.dump, 0) if args.dump else None
    if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
        d = Path(args.paths[0])
        bw = sorted(d.glob("raw_bw_*.bin"))
        s3 = sorted(d.glob("raw_s3_*.bin"))
        if not bw or not s3:
            print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
            return 2
        pairs = [(bw[0], True), (s3[0], False)]
    elif len(args.paths) == 2:
        pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
    else:
        ap.error("pass a capture directory, or exactly two .bin files")
    for path, is_request in pairs:
        report(path, is_request=is_request, dump_sub=dump_sub)
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -0,0 +1,119 @@
 #!/usr/bin/env python3
 """
 socat_log_split.py — recover a capture pair from a `socat -x` relay log.
 Why this exists
 ---------------
 The bench relay that puts Thor in front of a USB-attached Micromate is:
    socat -d -d -x TCP-LISTEN:12345,reuseaddr,fork /dev/ttyACM0,raw,echo=0,b115200 \
        > ~/mm-captures/socat_<ts>.log 2>&1
 `-x` makes socat hex-dump every byte it forwards, in both directions, with
 timestamps.  That log is therefore a **complete second copy of every capture**
 taken through the relay — independent of whether seismo_lab was recording.
 On 2026-09-25 that mattered: a capture's `.bin` files never made it off the
 Windows machine, and the session was rebuilt from this log instead.  When the
 real bins turned up later, the reconstruction was **byte-for-byte identical in
 both directions** (3,595 and 4,004 bytes).  So this is a validated fallback, not
 a lossy approximation.
 Log format
 ----------
 ```
 > 2026/09/25 00:30:35.000276659  length=21 from=0 to=20
 41 02 10 10 00 5b 00 00 30 00 ...
 2026/09/25 00:30:35 socat[32190] N write(5, 0x..., 21) completed
 < 2026/09/25 00:30:35.000384100  length=64 from=0 to=63
 02 00 c5 a4 00 00 30 00 ...
 ```
 `>` is data heading toward the serial device (Thor → unit).  `<` is data coming
 back (unit → Thor).  Hex lines are space-separated and indented; socat's own
 status lines start with a date and carry no payload.
 Usage
 -----
    # whole log
    python scratch/socat_log_split.py socat_20260924_181248.log --out-dir ./recovered
    # one session — line numbers from the "accepting connection" markers
    grep -n "accepting connection" socat_*.log
    python scratch/socat_log_split.py socat_*.log --from-line 919 --out-dir ./recovered
 Then parse the result as usual:
    python scratch/mm_frame_parse.py recovered/raw_bw.bin recovered/raw_s3.bin
 ⚠ A log spanning several sessions concatenates them.  Split by line number using
 the `accepting connection` markers, or the frame walk will run sessions together.
 """
 from __future__ import annotations
 import argparse
 import re
 from pathlib import Path
 _HEX = re.compile(r"\A[0-9a-f]{2}\Z")
 _SOCAT_STATUS = re.compile(r"\A\d{4}/\d{2}/\d{2}")
 def split(lines) -> tuple[bytes, bytes]:
    """Return (to_device, from_device) byte streams."""
    to_dev, from_dev = bytearray(), bytearray()
    cur = None
    for line in lines:
        if line.startswith(">"):
            cur = to_dev
            continue
        if line.startswith("<"):
            cur = from_dev
            continue
        if _SOCAT_STATUS.match(line):
            # socat's own status line ends the current dump block.
            cur = None
            continue
        if cur is None or not line.startswith(" "):
            continue
        toks = line.split()
        if toks and all(_HEX.match(t) for t in toks):
            cur.extend(int(t, 16) for t in toks)
    return bytes(to_dev), bytes(from_dev)
 def main() -> int:
    ap = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    ap.add_argument("log", help="a socat -x log file")
    ap.add_argument("--out-dir", default=".", help="where to write the .bin pair")
    ap.add_argument("--from-line", type=int, default=1,
                    help="first log line to read (1-based) — use the "
                         "'accepting connection' marker of the session you want")
    ap.add_argument("--to-line", type=int, default=None,
                    help="last log line to read (1-based, inclusive)")
    ap.add_argument("--prefix", default="raw", help="output basename prefix")
    args = ap.parse_args()
    lines = Path(args.log).read_text(errors="replace").splitlines()
    lo = max(args.from_line - 1, 0)
    hi = args.to_line if args.to_line is not None else len(lines)
    to_dev, from_dev = split(lines[lo:hi])
    out = Path(args.out_dir)
    out.mkdir(parents=True, exist_ok=True)
    bw = out / f"{args.prefix}_bw.bin"
    s3 = out / f"{args.prefix}_s3.bin"
    bw.write_bytes(to_dev)
    s3.write_bytes(from_dev)
    print(f"Thor -> unit  {len(to_dev):>7} bytes  {bw}")
    print(f"unit -> Thor  {len(from_dev):>7} bytes  {s3}")
    if not to_dev or not from_dev:
        print("⚠ one direction is empty — check --from-line / --to-line")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -0,0 +1,493 @@
 """Client-layer tests for the Micromate (series-4) live client.
 Every response constant below is a **real data section**, captured from UM12947
 (firmware 11.0CB) in ``bridges/captures/9-24-26 - micromate2/``.  They are
 embedded as hex because the captures are gitignored.
 Where a decoded value can be checked against something outside the bytes, it is:
 the device clock against the capture's own filename timestamp, the battery
 against Thor's event reports (3.8 V), the setup list against what the unit
 displays.
 """
 from __future__ import annotations
 import datetime
 import os
 import sys
 import pytest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from micromate.client import CONTENT, MicromateClient, _content, _cstring
 from micromate.framing import ETX, STX, checksum, stuff
 from micromate.protocol import ProtocolError
 FLAGS_CB, FLAGS_THOR = 0xC5, 0x03
 # ── Captured response data sections ───────────────────────────────────────────
 # Generated from the captures by hand-free extraction -- the hex below is
 # verbatim response data, not reconstructed.  The trailing comment on each
 # names the capture it came from, which is what lets the clock assertions be
 # checked against a wall-clock timestamp.
 POLL = bytes.fromhex(  # 59 B, from_20260924_185113_
    "300000000000000000000000000050496e7374616e74656c"
    "000600c3f04a00e4194f0074024d4d2f495345452f532f49"
    "4f00001f603e7657603e76"
 )
 SERIAL = bytes.fromhex(  # 21 B, from_20260924_191214_
    "0a00000000000000000000554d3132393437003100"
 )
 STATE_IDLE = bytes.fromhex(  # 16 B, from_20260924_191214_
    "050000000000000000000000e8000b00"
 )
 STATE_MONITORING = bytes.fromhex(  # 16 B, from_20260924_191214_
    "050000000000000000000002e8000b00"
 )
 MS_MONITORING = bytes.fromhex(  # 55 B, from_20260924_191214_
    "2c00000000000000000000000e180907ea20130c19000000"
    "000001000000000000000000000000000000000000017c00"
    "e4e1c000e3f1c0"
 )
 MS_IDLE = bytes.fromhex(  # 55 B, from_20260924_191214_
    "2c000000000000000000000000180907ea64130d22000000"
    "000001000000000000000000000000000000000000017c00"
    "e4e1c000e3e1c0"
 )
 MS_LATE = bytes.fromhex(  # 55 B, from_20260925_011403_
    "2c000000000000000000000000190907ea74010e05000000"
    "000001000000000000000000000000000000000000017c00"
    "e4e1c000e3e1c0"
 )
 # ── Real 11.0BD bytes (UM20147, captured over USB 2026-09-30) ─────────────────
 #
 # The Thor firmware line, which was pure inference until this capture.  These
 # are the two responses where BD differs from CB.  Captured with
 # `mm_client_check.py --capture` and read out of the resulting pair with
 # scratch/mm_frame_parse.py, so the data sections are verbatim; only the frame
 # wrapper is rebuilt, and the framing is independently verified 251/251.
 POLL_BD = bytes.fromhex(  # 59 B -- flags 0x03, and the SHORTER model string
    "30000000000000000000000000005649"
    "6e7374616e74656c000600c3f04a00e4"
    "194f0052024d4d2f495345452f530058"
    "f406001f60c0755760c075"
 )
 MS_BD_MONITORING = bytes.fromhex(  # 59 B -- FOUR BYTES LONGER than CB's 55
    "3000000000000000000000000e1e0907"
    "ea600e00310000000000000000000000"
    "00000000000000000000000000015e00"
    "e4e1c000e291c00fa00004"
 )
 _SETUP_PAD = 266 - CONTENT
 def setup_response(name: str) -> bytes:
    """A 0x41/0x3F/0x40 response: 11-byte prefix then a null-padded name."""
    body = name.encode("ascii").ljust(_SETUP_PAD, b"\x00")
    return bytes([0xFF]) + bytes(10) + body
 # The real 22 names, in the order the unit walked them.
 SETUP_NAMES = [
    "factory.MMB", "TEST.MMB", "BUS TEST.MMB", "Walsh JV 241.mmb",
    "Walsh JV 008.mmb", "Hawbaker 322.mmb", "Hawbaker 322 blasting.mmb",
    "min.mmb", "Playhouse Loc 1.mmb", "Valley Rock Solution.MMB",
    "RecordingSetup.mmb", "UPMC.mmb", "UPMC Loc 3.mmb", "Residence Inn.mmb",
    "Micromate ext trigger.mmb", "Micromate remort alarm.mmb",
    "Tree of Life - Loc 1 - 5861 Solway.mmb", "Mele-PWSA-Carroll -Loc 4.mmb",
    "Micromate min trigger mmb.mmb", "Fay - Layton Bridge Project.mmb",
    "Default Micromate ISEE.mmb", "TEST1.mmb",
 ]
 # ── Test doubles ──────────────────────────────────────────────────────────────
 class ScriptedTransport:
    def __init__(self, responses: list[bytes]) -> None:
        self.queue = list(responses)
        self.written: list[bytes] = []
        self._connected = False
    def connect(self) -> None:
        self._connected = True
    def disconnect(self) -> None:
        self._connected = False
    def is_connected(self) -> bool:
        return self._connected
    def write(self, data: bytes) -> None:
        self.written.append(data)
    def read(self, n: int) -> bytes:
        return self.queue.pop(0) if self.queue else b""
 def frame(rsp_sub: int, data: bytes, *, flags: int = FLAGS_CB) -> bytes:
    payload = bytes([0x00, flags, rsp_sub, 0x00, 0x00]) + data
    return bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
 def client(responses: list[bytes], **kw) -> tuple[MicromateClient, ScriptedTransport]:
    t = ScriptedTransport(responses)
    return MicromateClient(t, recv_timeout=0.5, **kw), t
 # ── The captured constants are what we think they are ─────────────────────────
 def test_captured_constants_have_the_expected_lengths():
    assert len(POLL) == 59
    assert len(SERIAL) == 21
    assert len(STATE_IDLE) == len(STATE_MONITORING) == 16
    assert len(MS_MONITORING) == len(MS_IDLE) == len(MS_LATE) == 55
 def test_the_prefix_length_byte_is_only_the_low_byte():
    """⚠ data[0] is `content_length & 0xFF`, with no high byte anywhere.
    True for every response under 256 bytes, which is why it reads as a working
    length field -- and then loses 2,048 bytes on a setup block.  The client
    takes content as data[11:] for exactly this reason.
    """
    for data in (POLL, SERIAL, STATE_IDLE, MS_MONITORING):
        assert data[0] == (len(data) - CONTENT) & 0xFF
        assert data[1] == 0x00, "no high byte is stored"
    # The two that prove it is not a real length: a 2,092-byte setup block
    # reports 44, and a 1,024-byte download chunk reports 0.
    assert (2092 & 0xFF) == 44
    assert (1024 & 0xFF) == 0
 # ── Helpers ───────────────────────────────────────────────────────────────────
 def test_a_printable_byte_precedes_the_manufacturer_string():
    """content[2] is 0x50 -- "P".  This is why the POLL parse cannot be a scan."""
    c = _content(POLL)
    assert c[3] == 0x50 and chr(c[3]) == "P"
    assert c[4:13] == b"Instantel"
 def test_content_strips_exactly_eleven_bytes():
    assert _content(SERIAL) == bytes.fromhex("554d3132393437003100")
    assert _content(b"short") == b""
 def test_cstring_stops_at_the_null():
    assert _cstring(bytes.fromhex("554d3132393437003100")) == "UM12947"
    assert _cstring(b"\x00rest") == ""
 # ── connect() ─────────────────────────────────────────────────────────────────
 def test_connect_decodes_identity():
    mm, t = client([
        frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
        frame(0xBE, setup_response("TEST1.mmb")),
    ])
    info = mm.connect()
    assert info.serial == "UM12947"
    assert info.manufacturer == "Instantel"
    assert info.model == "MM/ISEE/S/IO"
    assert info.firmware_line == "blastware"
    assert info.monitoring is False
    assert info.active_setup == "TEST1.mmb"
    assert "UM12947" in str(info) and "idle" in str(info)
 def test_connect_sends_three_reads_not_thors_four():
    """⚠ Deliberately narrower than Thor's POLL -> SERIAL -> 0x49 -> POLL.
    The trailing POLL repeats the first; measuring all 8 captured sessions
    showed the four-command form is Thor's connection check (3 of 8 sessions),
    not a handshake.  Only "opens with POLL" is invariant.
    """
    mm, t = client([
        frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
        frame(0xBE, setup_response("TEST1.mmb")),
    ])
    mm.connect()
    subs = [w[5] for w in t.written]          # payload[2] lands at wire[5]
    assert subs == [0x5B, 0x15, 0x49, 0x41]
    assert 0x01 not in subs, "device info has no Thor precedent; do not read it"
 def test_connect_can_skip_the_active_setup():
    mm, t = client([frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE)])
    info = mm.connect(with_active_setup=False)
    assert info.active_setup is None
    assert len(t.written) == 3
 def test_connect_survives_an_unreadable_active_setup():
    """A unit with no setup loaded is a real state, not a failed connect."""
    mm, _ = client([
        frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
        frame(0x00, bytes(20)),          # wrong SUB -> UnexpectedResponse
    ])
    info = mm.connect()
    assert info.serial == "UM12947"
    assert info.active_setup is None
 def test_connect_reports_a_monitoring_unit():
    mm, _ = client([
        frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_MONITORING),
        frame(0xBE, setup_response("TEST1.mmb")),
    ])
    assert mm.connect().monitoring is True
 def test_the_state_flag_is_tested_for_non_zero():
    """⚠ Never compared against 0x02 -- this flag family is not a stable enum.
    Its sibling in SUB 0x1C has read both 0x0E and 0x0C while monitoring.
    """
    for value in (0x01, 0x02, 0x0C, 0x0E, 0xFF):
        data = bytearray(STATE_IDLE)
        data[CONTENT] = value
        mm, _ = client([
            frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, bytes(data)),
            frame(0xBE, setup_response("x.mmb")),
        ])
        assert mm.connect().monitoring is True, f"0x{value:02x} should read as monitoring"
 def test_the_model_string_is_anchored_on_MM_not_on_an_offset():
    """The Thor firmware line reports a SHORTER model string, "MM/ISEE/S".
    Anchoring on b"MM/" survives that; a fixed end offset would not.  A generic
    printable-run scan fails for a different reason -- see _parse_poll.
    ✅ CONFIRMED on real hardware 2026-09-30: UM20147 (11.0BD) read back
    model="MM/ISEE/S" over USB.  The frame below is still synthesised because
    no BD capture is in the repo, but the string it asserts is the real one.
    """
    bd = bytearray(POLL)
    assert bd[CONTENT + 26:CONTENT + 38] == b"MM/ISEE/S/IO"
    bd[CONTENT + 26:CONTENT + 38] = b"MM/ISEE/S\x00\x00\x00"
    mm, _ = client([
        frame(0xA4, bytes(bd), flags=FLAGS_THOR), frame(0xEA, SERIAL),
        frame(0xB6, STATE_IDLE), frame(0xBE, setup_response("x.mmb")),
    ])
    info = mm.connect()
    assert info.model == "MM/ISEE/S"
    assert info.manufacturer == "Instantel"
    assert info.firmware_line == "thor"
 # ── get_state() ───────────────────────────────────────────────────────────────
@pytest.mark.parametrize(
    "data, monitoring, when, free",
    [
        (MS_MONITORING, True,  datetime.datetime(2026, 9, 24, 19, 12, 25), 0x00E3F1C0),
        (MS_IDLE,       False, datetime.datetime(2026, 9, 24, 19, 13, 34), 0x00E3E1C0),
        (MS_LATE,       False, datetime.datetime(2026, 9, 25, 1, 14, 5),   0x00E3E1C0),
    ],
    ids=["monitoring", "idle", "after-midnight"],
 )
 def test_get_state_decodes_the_real_reads(data, monitoring, when, free):
    """⚠ There is an unidentified byte at content[6]; the hour is at content[7].
    The protocol reference's 0x1C section has this right.  Its one-line summary
    in the divergences list reads as six contiguous fields and does not.
    Each expected time is checked against the capture filename that produced the
    bytes: 19:12:14, 19:12:14 and 01:14:03.  All three decode to seconds-to-a-
    minute after their session opened, which is what a device clock should do.
    Reading content[6] as the hour gives 32, 100 and 116.
    """
    mm, _ = client([frame(0xE3, data)])
    st = mm.get_state()
    assert st.monitoring is monitoring
    assert st.device_time == when
    assert st.battery_volts == 3.80          # Thor's reports print 3.8 V
    assert st.memory_total_bytes == 15_000_000
    assert st.memory_free_bytes == free
    assert st.raw == data
 def test_content_6_is_not_the_hour():
    """The byte the reference implies is the hour reads 32, 100 and 116."""
    for data in (MS_MONITORING, MS_IDLE, MS_LATE):
        assert _content(data)[6] not in range(24)
 def test_memory_derivations():
    mm, _ = client([frame(0xE3, MS_MONITORING)])
    st = mm.get_state()
    assert st.memory_used_bytes == 15_000_000 - 0x00E3F1C0
    assert 0 < st.memory_used_fraction < 0.02
    assert "3.80 V" in str(st)
 def test_battery_and_memory_are_read_forward_from_content_start():
    """⚠ NOT backward from the end.
    This block is 4 bytes longer on the Thor firmware line, and Series III's
    from-the-end offsets give a 11.0BD unit a battery reading of 577.92 V.  The
    extra bytes are trailing, so appending four does not move anything.
    """
    bd = MS_MONITORING + bytes.fromhex("0fa00000")
    bd = bytes([0x30]) + bd[1:]              # low-byte length becomes 48
    mm, _ = client([frame(0xE3, bd)])
    st = mm.get_state()
    assert st.battery_volts == 3.80, "forward offsets must survive the 4 extra bytes"
    assert st.memory_total_bytes == 15_000_000
    # What the Series III from-the-end offsets would have produced:
    assert int.from_bytes(bd[-10:-8], "big") / 100 == pytest.approx(577.92, abs=0.01)
 def test_a_dead_clock_battery_does_not_fail_the_whole_read():
    """An impossible date is information; the rest of the block is still good."""
    broken = bytearray(MS_MONITORING)
    broken[CONTENT + 3] = 0xFF               # month 255
    mm, _ = client([frame(0xE3, bytes(broken))])
    st = mm.get_state()
    assert st.device_time is None
    assert st.battery_volts == 3.80
 def test_a_truncated_state_block_raises():
    mm, _ = client([frame(0xE3, bytes(20))])
    with pytest.raises(ProtocolError, match="need at least 44"):
        mm.get_state()
 # ── Setups ────────────────────────────────────────────────────────────────────
 def test_list_setups_walks_to_the_empty_terminator():
    """22 real names then an empty one, exactly as the unit walked them."""
    responses = [frame(0xC0, setup_response(SETUP_NAMES[0]))]
    responses += [frame(0xBF, setup_response(n)) for n in SETUP_NAMES[1:]]
    responses += [frame(0xBF, setup_response(""))]
    mm, t = client(responses)
    assert mm.list_setups() == SETUP_NAMES
    assert len(t.written) == 23, "22 names plus the terminator"
    assert t.written[0][5] == 0x3F
    assert {w[5] for w in t.written[1:]} == {0x40}
 def test_list_setups_handles_an_empty_unit():
    mm, _ = client([frame(0xC0, setup_response(""))])
    assert mm.list_setups() == []
 def test_list_setups_refuses_to_loop_forever():
    """A cursor that never advances is a bug, and must not hang the caller."""
    from micromate import client as C
    mm, _ = client([frame(0xC0, setup_response("a.mmb"))]
                   + [frame(0xBF, setup_response("a.mmb"))] * (C._MAX_SETUPS + 5))
    with pytest.raises(ProtocolError, match="not advancing"):
        mm.list_setups()
 def test_get_active_setup_handles_a_long_name():
    long_name = "Tree of Life - Loc 1 - 5861 Solway.mmb"
    mm, _ = client([frame(0xBE, setup_response(long_name))])
    assert mm.get_active_setup() == long_name
 # ── Lifecycle ─────────────────────────────────────────────────────────────────
 def test_the_client_owns_the_transport():
    mm, t = client([])
    assert not mm.is_open()
    mm.open()
    assert mm.is_open() and t.is_connected()
    mm.close()
    assert not mm.is_open()
 def test_context_manager_opens_and_closes():
    t = ScriptedTransport([frame(0xA4, POLL), frame(0xEA, SERIAL),
                           frame(0xB6, STATE_IDLE), frame(0xBE, setup_response("x.mmb"))])
    with MicromateClient(t, recv_timeout=0.5) as mm:
        assert t.is_connected()
        assert mm.connect().serial == "UM12947"
    assert not t.is_connected()
 # ── The Thor firmware line, on real bytes ─────────────────────────────────────
 def test_bd_constants_have_the_lengths_the_parser_reported():
    assert len(POLL_BD) == 59
    assert len(MS_BD_MONITORING) == 59, "CB is 55; BD is four bytes longer"
    assert MS_BD_MONITORING[0] == 0x30, "low-byte length 48 = 59 - 11"
    assert MS_IDLE[0] == 0x2C, "the CB equivalent declares 44"
 def test_connect_on_a_thor_line_unit():
    """UM20147, real POLL bytes.  flags 0x03 is ETX, so it arrives as `10 03`."""
    mm, _ = client([
        frame(0xA4, POLL_BD, flags=FLAGS_THOR), frame(0xEA, SERIAL),
        frame(0xB6, STATE_MONITORING), frame(0xBE, setup_response("test2.MMB")),
    ])
    info = mm.connect()
    assert info.firmware_line == "thor"
    assert info.model == "MM/ISEE/S", "the BD model string is shorter than CB's"
    assert info.manufacturer == "Instantel"
    assert info.monitoring is True
 def test_poll_content_3_is_not_a_constant():
    """⚠ 0x50 on UM12947, 0x56 on UM20147 -- it VARIES between units.
    This is why the manufacturer is read at the fixed offset content[4] and not
    by scanning: content[3] is printable in both cases ("P" and "V"), so a
    printable-run scan would return "PInstantel" on one unit and "VInstantel" on
    the other.  Whatever the byte is, it is not a stable marker to anchor on.
    """
    assert _content(POLL)[3] == 0x50
    assert _content(POLL_BD)[3] == 0x56
    assert chr(_content(POLL_BD)[3]) == "V"
 def test_get_state_on_a_thor_line_unit():
    """⚠ THE test for the forward-offset decision.  Real UM20147 bytes.
    The 0x1C block is four bytes longer here, and the extras are TRAILING, so
    offsets measured from the start of content are unmoved.  Series III reads
    battery and memory from the END of this block; test_series_iii_offsets_...
    below shows what that produces.
    """
    mm, _ = client([frame(0xE3, MS_BD_MONITORING, flags=FLAGS_THOR)])
    st = mm.get_state()
    assert st.monitoring is True
    assert st.device_time == datetime.datetime(2026, 9, 30, 14, 0, 49)
    assert st.battery_volts == 3.50
    assert st.memory_total_bytes == 15_000_000
    assert st.memory_free_bytes == 14_848_448
 def test_series_iii_from_the_end_offsets_give_577_volts_on_a_bd_unit():
    """The exact number the protocol reference warned about, now demonstrated.
    577.92 V is not a plausible battery reading for anything, which is what
    makes it a free self-check -- bridges/mm_client_check.py watches for it.
    """
    assert int.from_bytes(MS_BD_MONITORING[-10:-8], "big") / 100 == 577.92
 def test_the_four_extra_bd_bytes_are_not_all_zero():
    """⚠ The reference records them as `0f a0 00 00`; UM20147 sent `0f a0 00 04`.
    Only the first two bytes look fixed.  Nothing reads them, but a future
    decoder must not treat the last one as padding.
    """
    assert _content(MS_BD_MONITORING)[44:] == bytes.fromhex("0fa00004")
    assert _content(MS_IDLE)[44:] == b"", "the CB block has no such tail"
@@ -0,0 +1,343 @@
 """Event-chain tests for the Micromate (series-4) client.
 The load-bearing test here replays THOR's captured six-event download session
 through `MicromateClient.iter_events()` + `get_event()` and asserts **every byte
 we put on the wire matches what THOR put on the wire** — 99 frames — while also
 decoding all six events and cross-checking each waveform's peak vector sum
 against the one the device computed itself.
 That capture is gitignored, so those tests skip on a fresh clone; the offline
 tests below use embedded real response bytes and cover the same logic.
 """
 from __future__ import annotations
 import datetime
 import os
 import sys
 from pathlib import Path
 import pytest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from micromate.client import CONTENT, DecodeMismatch, MicromateClient
 from micromate.framing import ACK, DLE, ETX, STX, checksum, stuff
 from micromate.models import MicromateEventRef
 from micromate.protocol import ProtocolError
 from test_micromate_client import ScriptedTransport, SERIAL, frame
 # The real 0x0C record from UM20147 (11.0BD), captured over USB 2026-09-30.
 # A histogram: content[11] = 0x08.
 RECORD_BD = bytes.fromhex(
    "d200000000055d4a8100001e0907eab30d1b21000000084c6f636174696f6e00"
    "0000000000000000000000000074657374320000000000000000000000000000"
    "0000000000000000000000000000000000000000000000554d32303134370000"
    "003fcb3bde00000000053f000f5472616e00003e698cdb000300005665727400"
    "003fc76e65000300004c6f6e6700003e567c21000300004d69630000003956b9"
    "7c00050000000000000000000000000000000000000000000000000000000000"
    "0000000000000000000000000000000000000000000000000000000000"
 )
 def chain_entry(key: bytes, size: int) -> bytes:
    """A 1E/1F response data section: 11-byte prefix then key + size."""
    return (bytes([0x08]) + bytes(7) + bytes([0xFE]) + bytes(2)
            + key + size.to_bytes(4, "big"))
 def client(responses):
    t = ScriptedTransport(responses)
    return MicromateClient(t, recv_timeout=0.5), t
 # ── The chain walk ────────────────────────────────────────────────────────────
 def test_chain_walk_arms_before_every_entry():
    """⚠ THOR sends 0x93 before EVERY 1E/1F, and this mirrors that."""
    responses = [
        frame(0xEA, SERIAL),                                    # serial()
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 4076)),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes.fromhex("055d4a82"), 11032)),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),   # sentinel
    ]
    mm, t = client(responses)
    refs = mm.list_events(with_records=False)
    subs = [w[5] for w in t.written]
    assert subs == [0x15, 0x93, 0x1E, 0x93, 0x1F, 0x93, 0x1F]
    assert [r.key_hex for r in refs] == ["055d4a81", "055d4a82"]
    assert [r.size for r in refs] == [4076, 11032]
 def test_an_all_zero_key_ends_the_chain_and_is_not_an_error():
    mm, _ = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes(4), 0)),
    ])
    assert mm.list_events(with_records=False) == []
 def test_refs_carry_the_serial_because_a_key_alone_is_ambiguous():
    """⚠ UM12947 and UM20147 BOTH have an event 055d4a81.
    A store keyed on the event key alone treats one unit's event as a duplicate
    of the other's, and nothing raises.  `uid` is the safe identifier.
    """
    mm, _ = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 4076)),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),
    ])
    (ref,) = mm.list_events(with_records=False)
    assert ref.serial == "UM12947"
    assert ref.uid == "UM12947:055d4a81"
 def test_a_cursor_that_never_advances_raises_rather_than_hanging():
    from micromate import client as C
    responses = [frame(0xEA, SERIAL)]
    entry = chain_entry(bytes.fromhex("055d4a81"), 4076)
    responses += [frame(0x6C, bytes(11)), frame(0xE1, entry)]      # the 1E read
    for _ in range(C._MAX_EVENTS + 2):                             # then 1F forever
        responses += [frame(0x6C, bytes(11)), frame(0xE0, entry)]
    mm, _ = client(responses)
    with pytest.raises(ProtocolError, match="not advancing"):
        mm.list_events(with_records=False)
 def test_a_truncated_chain_entry_raises():
    mm, _ = client([
        frame(0xEA, SERIAL), frame(0x6C, bytes(11)), frame(0xE1, bytes(14)),
    ])
    with pytest.raises(ProtocolError, match="need 8"):
        mm.list_events(with_records=False)
 def test_iter_events_does_not_read_ahead():
    """It must yield at the cursor position, one arm/advance per event."""
    mm, t = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 4076)),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),
    ])
    it = mm.iter_events(with_records=False)
    first = next(it)
    assert [w[5] for w in t.written] == [0x15, 0x93, 0x1E], "no read-ahead"
    assert first.key_hex == "055d4a81"
    assert list(it) == []
 # ── The 0x0C record ───────────────────────────────────────────────────────────
 def test_record_decode_on_real_bd_bytes():
    mm, _ = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 4796)),
        frame(0xF3, RECORD_BD),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),
    ])
    (ref,) = mm.list_events()
    assert ref.record_type == "histogram"      # content[11] = 0x08
    assert ref.is_histogram is True
    assert ref.suffix == ".IDFH"
    assert ref.timestamp == datetime.datetime(2026, 9, 30, 13, 27, 33)
    assert ref.sensor_location == "Location"
    assert ref.setup == "test2"
    assert ref.serial == "UM20147", "the record's own serial wins over the cache"
    assert ref.peak_vector_sum_ips == pytest.approx(1.587765, abs=1e-5)
    assert ref.peaks_ips["Tran"] == pytest.approx(0.228076, abs=1e-5)
    assert ref.peaks_ips["Vert"] == pytest.approx(1.558056, abs=1e-5)
    assert ref.peaks_ips["Long"] == pytest.approx(0.209458, abs=1e-5)
 def test_the_pvs_is_not_reconstructible_from_the_reported_peaks():
    """⚠ Why the PVS field matters: it cannot be recomputed from the peaks.
    It is the PER-SAMPLE peak vector sum.  `sqrt(Σpeak²)` is only an upper
    bound, because the channel maxima do not occur at the same instant, and
    `max(T,V,L)` is a lower bound.  On THIS event the two happen to be within
    0.05%, which is exactly the coincidence that made the field look like
    `sqrt(Σpeak²)` on first inspection.  Six other events separated them.
    """
    import math
    mm, _ = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 4796)),
        frame(0xF3, RECORD_BD),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),
    ])
    (ref,) = mm.list_events()
    p = ref.peaks_ips
    upper = math.sqrt(p["Tran"] ** 2 + p["Vert"] ** 2 + p["Long"] ** 2)
    lower = max(p["Tran"], p["Vert"], p["Long"])
    assert lower < ref.peak_vector_sum_ips < upper
 def test_filename_matches_thors_convention():
    ref = MicromateEventRef(index=0, key=bytes.fromhex("055d4a82"), size=11032,
                            serial="UM12947", record_type="waveform",
                            timestamp=datetime.datetime(2026, 9, 23, 16, 33, 19))
    assert ref.filename == "UM12947_20260923163319.IDFW"
    ref.record_type = "histogram"
    assert ref.filename == "UM12947_20260923163319.IDFH"
 def test_filename_is_none_without_a_type_rather_than_guessing():
    """⚠ Guessing would file a histogram as a waveform, and read_idf_file()
    dispatches on exactly that suffix."""
    ref = MicromateEventRef(index=0, key=bytes(4), size=1, serial="UM12947",
                            timestamp=datetime.datetime(2026, 1, 1))
    assert ref.record_type is None
    assert ref.suffix is None
    assert ref.filename is None
 def test_an_unknown_record_type_warns_and_leaves_it_none(caplog):
    bad = bytearray(RECORD_BD)
    bad[CONTENT + 11] = 0x99
    mm, _ = client([
        frame(0xEA, SERIAL),
        frame(0x6C, bytes(11)), frame(0xE1, chain_entry(bytes.fromhex("055d4a81"), 10)),
        frame(0xF3, bytes(bad)),
        frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0)),
    ])
    with caplog.at_level("WARNING"):
        (ref,) = mm.list_events()
    assert ref.record_type is None
    assert "unknown record type 0x99" in caplog.text
 def test_get_event_refuses_without_a_record_type():
    mm, _ = client([])
    ref = MicromateEventRef(index=0, key=bytes.fromhex("055d4a81"), size=4076)
    with pytest.raises(ValueError, match="record_type is unknown"):
        mm.get_event(ref)
 # ── Against the real captured session ─────────────────────────────────────────
 _CAPTURES = (
    Path(__file__).resolve().parents[1]
    / "bridges" / "captures" / "9-24-26 - micromate2"
 )
 _DOWNLOAD = "raw_bw_20260925_011403_Download_events_then_delete_1_event.bin"
 def _destuffed(blob: bytes, is_req: bool):
    i, n = 0, len(blob)
    while i < n:
        if is_req:
            if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
                i += 1
                continue
            j = i + 2
        else:
            if blob[i] != STX:
                i += 1
                continue
            j = i + 1
        out = bytearray()
        while j < n:
            if blob[j] == DLE and j + 1 < n:
                out.append(blob[j + 1])
                j += 2
                continue
            if blob[j] == ETX:
                break
            out.append(blob[j])
            j += 1
        if len(out) >= 6:
            yield blob[i:j + 1], bytes(out[:-1])
        i = j + 1
@pytest.mark.skipif(
    not (_CAPTURES / _DOWNLOAD).is_file(),
    reason="capture is gitignored; present only on a dev box",
 )
 def test_replaying_thors_session_reproduces_every_byte_and_decodes_every_event():
    """The whole point of step 4.
    Feed THOR's own responses to `iter_events()` + `get_event()`, and assert:
      * every request byte we emit matches THOR's, in order
      * all six events decode
      * each waveform's decoded PVS matches the device's stored float
    """
    reqs = list(_destuffed((_CAPTURES / _DOWNLOAD).read_bytes(), True))
    rsps = list(_destuffed(
        (_CAPTURES / _DOWNLOAD.replace("raw_bw", "raw_s3")).read_bytes(), False))
    # THOR's session opens with commands our client does not send (POLL, 0x1C,
    # 0x06 …) and ends with a delete.  Take the contiguous run from the first
    # 0x93 to the last 0x5A -- that is the event walk, and it is what we mirror.
    subs = [p[2] for _, p in reqs]
    lo = subs.index(0x93)
    hi = len(subs) - 1 - subs[::-1].index(0x5A)
    want_wire = [w for w, _ in reqs[lo:hi + 1]]
    replay = [bytes([STX]) + stuff(p + bytes([checksum(p)])) + bytes([ETX])
              for _, p in rsps[lo:hi + 1]]
    # ⚠ THOR never reads the chain sentinel in this capture -- it downloaded
    # exactly six events and stopped, so it knew the count in advance.  It read
    # `SUB 0x06` (storage range) at frame 7, BEFORE the walk, and that response
    # begins `00 00 00 06` -- the event count.  Our generator walks until the
    # sentinel instead, so append one and compare only the overlapping frames.
    replay += [frame(0x6C, bytes(11)), frame(0xE0, chain_entry(bytes(4), 0))]
    # serial() would emit a 0x15 that is not in this slice, so prime the cache.
    t = ScriptedTransport(replay)
    mm = MicromateClient(t, recv_timeout=1.0)
    mm._serial = "UM12947"
    decoded, checked = 0, 0
    for ref in mm.iter_events():
        result = mm.get_event(ref)          # verify=True by default
        decoded += 1
        assert ref.serial == "UM12947"
        assert ref.filename and ref.filename.endswith(ref.suffix)
        if not ref.is_histogram:
            err = mm.decode_error(ref, result)
            assert err is not None
            assert abs(err) < 1e-4, f"{ref.uid}: PVS off by {100 * err:+.4f}%"
            checked += 1
    assert decoded == 6, "four waveforms and two histograms"
    assert checked == 4
    # Our trailing sentinel read is two frames THOR did not send; everything
    # up to it must match byte for byte, in order.
    assert t.written[:len(want_wire)] == want_wire, (
        f"we emitted {len(t.written)} frames, THOR emitted {len(want_wire)}"
    )
    assert len(t.written) == len(want_wire) + 2, "only the sentinel read is extra"
@pytest.mark.skipif(
    not (_CAPTURES / _DOWNLOAD).is_file(),
    reason="capture is gitignored; present only on a dev box",
 )
 def test_a_corrupted_stored_peak_is_caught_by_the_self_check():
    """Prove the verify path actually fires -- otherwise it is decoration."""
    reqs = list(_destuffed((_CAPTURES / _DOWNLOAD).read_bytes(), True))
    rsps = list(_destuffed(
        (_CAPTURES / _DOWNLOAD.replace("raw_bw", "raw_s3")).read_bytes(), False))
    subs = [p[2] for _, p in reqs]
    lo = subs.index(0x93)
    hi = len(subs) - 1 - subs[::-1].index(0x5A)
    replay = [bytes([STX]) + stuff(p + bytes([checksum(p)])) + bytes([ETX])
              for _, p in rsps[lo:hi + 1]]
    t = ScriptedTransport(replay)
    mm = MicromateClient(t, recv_timeout=1.0)
    mm._serial = "UM12947"
    for ref in mm.iter_events():
        if ref.is_histogram:
            mm.download_event(ref)          # keep the replay in step
            continue
        ref.peak_vector_sum_ips *= 1.5      # as a bad decode would look
        with pytest.raises(DecodeMismatch, match="decode is suspect"):
            mm.get_event(ref)
        return
    pytest.fail("no waveform event found in the capture")
@@ -0,0 +1,413 @@
 """Framing tests for the Micromate (series-4) live protocol.
 Every constant below is a **real frame**, lifted from
 ``bridges/captures/9-24-26 - micromate2/`` (UM12947, firmware 11.0CB) or from
 ``scratch/fake_unit.py``, which preserves a POLL probe reply.  Frames are
 embedded as hex rather than read from disk because both ``bridges/captures/``
 and ``tests/fixtures/`` are gitignored -- these tests must pass on a fresh
 clone.
 The few synthesised frames are marked ``SYNTH_`` and each says what it stands
 in for and why a captured frame was not available.
 Two of these tests exist because the first draft of
 ``docs/micromate_client_spec.md`` got the rule wrong, and both wrong rules
 fail quietly -- a frame the unit ignores, or a checksum that reads as bad:
  * ``test_builder_matches_thor_byte_for_byte`` -- the escape set.  Escaping
    only 0x10 (the series-3 rule) reproduces 161 of Thor's 218 read frames.
  * ``test_checksum_is_plain_sum8_over_destuffed_payload`` -- the checksum.
    The DLE-aware variant disagrees with the wire on 55 of 251 responses.
 """
 from __future__ import annotations
 import os
 import sys
 from pathlib import Path
 import pytest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from micromate.framing import (
    ACK,
    DLE,
    ETX,
    FLAGS_BLASTWARE,
    FLAGS_THOR,
    STX,
    MicromateFrame,
    MicromateFrameParser,
    build_request,
    checksum,
    stuff,
    unstuff,
 )
 # ── Captured response frames ──────────────────────────────────────────────────
 # POLL probe reply, 19 B on the wire -- the shortest valid frame there is.
 # Preserved in scratch/fake_unit.py, captured from UM12947 on 2026-09-24.
 # Its payload[8:10] is 0x0030, the data length POLL then asks for.
 RSP_POLL_PROBE = bytes.fromhex("0200c5a400000000000030000000000000009903")
 # SUB 0x49 -> 0xB6, the cheap state read.  Carries a literal 0x02, escaped.
 RSP_STATE = bytes.fromhex("0200c5b6000005000000000000000000001002e8000b007503")
 # SUB 0x48 -> 0xB7, a path-addressed file read.  offset_hi 0x04 arrives escaped,
 # so page_hi is only correct if the parser destuffs before indexing.
 RSP_FILE_READ = bytes.fromhex("0200c5b70000100400000000000000010000000000018203")
 # An 0x5A download chunk, 138 B on the wire.  THE checksum case: its payload
 # holds literal 0x10 bytes, so plain SUM8 (0xC1, correct) and the DLE-aware
 # variant (0x91) disagree.  Also holds a literal 0x41, which is NOT escaped.
 RSP_CHUNK_WITH_DLE = bytes.fromhex(
    "0200c5a500007000003400000000000000e3fd1f10020f0f0e2e1e1fd4f2d4c3d2f00f000"
    "e003fe03fe200d6e2d12d2e1c4e101011d101f3d0f3e0f2efe23d2b2e3c0d3e0e2c4101e0"
    "d4d3e21f1d311e202fe2e2f1e3f100210d201002ee2e22d2f2f0f30fd3f0101000f0111f1"
    "ff01e011010011f0f1002b4d200e0f0302c4010020001fffedb2dc103"
 )
 # ── Captured request frames (Thor -> unit) ────────────────────────────────────
 REQ_POLL       = bytes.fromhex("41021010005b000030000000000000000000009b03")
 REQ_SERIAL     = bytes.fromhex("41021010001500000a000000000000000000002f03")
 REQ_STATE      = bytes.fromhex("41021010004900ffff000000000000000000005703")
 REQ_COMPLIANCE = bytes.fromhex("41021010001a00ffff000000000000000000002803")
 REQ_ARM_EVENT  = bytes.fromhex("41021010009300ffff00000000000000000000a103")
 # ⚠ The two frames that break a 0x10-only escaper.
 # Scheduler enable: params[7] = 0x03, on the wire as `10 03`.
 REQ_SCHED_ON = bytes.fromhex("41021010004700ffff00000000000000100300005803")
 # Bulk download, first chunk of event 055d4a81: offset 0x0400 puts a literal
 # 0x04 in offset_hi, on the wire as `10 04`.  EVERY download frame needs this.
 REQ_DOWNLOAD_CHUNK0 = bytes.fromhex(
    "41021010005a00100400055d4a810000000000009b03"
 )
 # A later chunk of the same event: params[2:4] = 0x1000, doubled to `10 10`.
 REQ_DOWNLOAD_CHUNK4 = bytes.fromhex(
    "41021010005a0010040000001010000000000000007e03"
 )
 # ── Stuffing ──────────────────────────────────────────────────────────────────
 def test_escape_set_is_exactly_four_bytes():
    """0x02, 0x03, 0x04 and 0x10 -- and nothing else.
    ACK (0x41) in particular is NOT escaped; assuming it was reproduced only
    196 of 251 captured responses.
    """
    assert stuff(bytes([0x02, 0x03, 0x04, 0x10])) == bytes(
        [DLE, 0x02, DLE, 0x03, DLE, 0x04, DLE, 0x10]
    )
    for b in (0x00, 0x01, 0x05, 0x41, 0xC5, 0xFF):
        assert stuff(bytes([b])) == bytes([b]), f"0x{b:02x} must not be escaped"
 def test_unstuff_is_uniform_with_no_inner_frame_carve_out():
    """`10 XX` -> `XX` for any XX -- the series-3 DLE+ETX exception is absent."""
    assert unstuff(bytes.fromhex("1003")) == b"\x03"
    assert unstuff(bytes.fromhex("1010")) == b"\x10"
    assert unstuff(bytes.fromhex("001002ff")) == bytes.fromhex("0002ff")
 def test_stuff_unstuff_round_trips_over_every_byte_value():
    data = bytes(range(256))
    assert unstuff(stuff(data)) == data
 def test_a_trailing_dle_is_held_not_dropped():
    """A DLE as the last byte of a chunk must not consume nothing and vanish."""
    assert unstuff(b"\xff\x10") == b"\xff\x10"
 # ── Checksum ──────────────────────────────────────────────────────────────────
 def test_checksum_is_plain_sum8_over_destuffed_payload():
    """⚠ Plain SUM8 -- do NOT exclude 0x10 bytes.
    RSP_CHUNK_WITH_DLE is a real frame whose payload holds literal 0x10 bytes.
    The wire says 0xC1; plain SUM8 gives 0xC1 and the DLE-aware variant used by
    series-3 `5A`/write frames gives 0x91.  55 of 251 captured responses
    disagree the same way.
    """
    payload = unstuff(RSP_CHUNK_WITH_DLE[1:-1])[:-1]
    chk_on_wire = unstuff(RSP_CHUNK_WITH_DLE[1:-1])[-1]
    assert DLE in payload, "this frame is only interesting if it holds a 0x10"
    assert checksum(payload) == chk_on_wire == 0xC1
    assert (sum(b for b in payload if b != DLE) & 0xFF) == 0x91  # the wrong rule
 def test_every_captured_frame_validates():
    parser = MicromateFrameParser()
    frames = parser.feed(
        RSP_POLL_PROBE + RSP_STATE + RSP_FILE_READ + RSP_CHUNK_WITH_DLE
    )
    assert len(frames) == 4
    assert all(f.checksum_valid for f in frames)
 # ── Request builder ───────────────────────────────────────────────────────────
@pytest.mark.parametrize(
    "wire, sub, offset, params",
    [
        (REQ_POLL,            0x5B, 0x0030, bytes(10)),
        (REQ_SERIAL,          0x15, 0x000A, bytes(10)),
        (REQ_STATE,           0x49, 0xFFFF, bytes(10)),
        (REQ_COMPLIANCE,      0x1A, 0xFFFF, bytes(10)),
        (REQ_ARM_EVENT,       0x93, 0xFFFF, bytes(10)),
        (REQ_SCHED_ON,        0x47, 0xFFFF, bytes.fromhex("00000000000000030000")),
        (REQ_DOWNLOAD_CHUNK0, 0x5A, 0x0400, bytes.fromhex("055d4a81000000000000")),
        (REQ_DOWNLOAD_CHUNK4, 0x5A, 0x0400, bytes.fromhex("00001000000000000000")),
    ],
    ids="poll serial state compliance arm sched_on dl_chunk0 dl_chunk4".split(),
 )
 def test_builder_matches_thor_byte_for_byte(wire, sub, offset, params):
    assert build_request(sub, offset, params) == wire
 def test_a_0x10_in_params_needs_no_special_handling():
    """The spec's one open question.  Thor sends it; the wire doubles it."""
    frame = build_request(0x5A, 0x0400, bytes.fromhex("00001000000000000000"))
    assert bytes([DLE, DLE]) in frame
    assert frame == REQ_DOWNLOAD_CHUNK4
 def test_builder_rejects_malformed_arguments():
    with pytest.raises(ValueError):
        build_request(0x5B, 0, bytes(9))
    with pytest.raises(ValueError):
        build_request(0x5B, 0x10000)
    with pytest.raises(ValueError):
        build_request(0x100)
 # ── Parsing ───────────────────────────────────────────────────────────────────
 def test_poll_probe_reply_fields():
    (f,) = MicromateFrameParser().feed(RSP_POLL_PROBE)
    assert f.sub == 0xA4
    assert f.request_sub == 0x5B
    assert f.flags == FLAGS_BLASTWARE
    assert f.firmware_line == "blastware"
    assert f.checksum_valid
    assert f.probe_length == 0x0030
 def test_probe_length_is_a_uint16_not_a_byte():
    """⚠ Read as data[3] alone, SUB 0x1A's 0x082C (2092) reads as 44 -- 47x low.
    SYNTHESISED: no probe response survives in the captures on disk (the
    9-24-26 session uses single-step reads at offset 0xFFFF throughout, so it
    contains no probes at all).  The field position is taken from the captured
    POLL probe reply above, which does exercise it for real.
    """
    payload = bytes([0x00, FLAGS_BLASTWARE, 0xE5, 0x00, 0x00]) + bytes(
        [0x00, 0x00, 0x00, 0x08, 0x2C]
    )
    synth = bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
    (f,) = MicromateFrameParser().feed(synth)
    assert f.probe_length == 0x082C == 2092
    assert f.data[3] == 0x08, "the high byte is where a byte-wide read loses 2048"
 def test_escaped_bytes_land_in_the_right_field():
    """RSP_FILE_READ's first data byte is 0x04, which arrives as `10 04`.
    Without destuffing it reads as 0x10 and every field after it is one byte
    late -- the failure mode that put `SUB 0x02` in the log as `SUB_10` for an
    afternoon.
    """
    (f,) = MicromateFrameParser().feed(RSP_FILE_READ)
    assert f.sub == 0xB7
    assert f.request_sub == 0x48
    assert (f.page_hi, f.page_lo) == (0x00, 0x00)
    assert f.data[0] == 0x04
    assert len(f.data) == 15, "one byte shorter than the wire suggests"
    assert f.checksum_valid
 # The real 11.0BD POLL data section, UM20147 over USB 2026-09-30.  This
 # replaces a synthesised frame -- the Thor firmware line was inference-only
 # until this capture.
 _POLL_BD_DATA = bytes.fromhex(
    "30000000000000000000000000005649"
    "6e7374616e74656c000600c3f04a00e4"
    "194f0052024d4d2f495345452f530058"
    "f406001f60c0755760c075"
 )
 def test_thor_firmware_line_survives_destuffing():
    """⚠ flags = 0x03 is ETX, so it arrives as `10 03`.
    A parser that does not destuff ends the frame at byte 2 on half the fleet.
    The data section is REAL (UM20147, 11.0BD); the frame wrapper is rebuilt,
    which is sound because the framing is verified 251/251 elsewhere in this
    file.  scratch/mm_frame_parse.py reported payload=64 for this frame, and
    the assertion below pins that, so the reconstruction is checked rather than
    assumed.
    """
    payload = bytes([0x00, FLAGS_THOR, 0xA4, 0x00, 0x00]) + _POLL_BD_DATA
    assert len(payload) == 64, "the parser reported payload=64 for this frame"
    wire = bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
    assert bytes([DLE, ETX]) in wire, "flags 0x03 must be escaped on the wire"
    (f,) = MicromateFrameParser().feed(wire)
    assert f.flags == FLAGS_THOR
    assert f.firmware_line == "thor"
    assert f.sub == 0xA4
    assert f.request_sub == 0x5B
    assert f.checksum_valid
    assert b"MM/ISEE/S\x00" in f.data, "the shorter BD model string"
 def test_an_escaped_checksum_byte_is_read_correctly():
    """SYNTHESISED, but the behaviour is real: three captured responses have a
    checksum of 0x02/0x03/0x04 and all three escape it on the wire.  The
    shortest is 1,070 B (an 0x5A chunk in
    raw_s3_20260925_011403_Download_events_then_delete_1_event.bin, chk = 0x03),
    too long to embed for one byte's worth of assertion.
    """
    payload = bytes([0x00, FLAGS_BLASTWARE, 0xA4, 0x00, 0x00, 0x03])
    assert checksum(payload) == 0x03 + FLAGS_BLASTWARE + 0xA4 & 0xFF
    body = payload + bytes([checksum(payload)])
    synth = bytes([STX]) + stuff(body) + bytes([ETX])
    (f,) = MicromateFrameParser().feed(synth)
    assert f.chk_byte == checksum(payload)
    assert f.checksum_valid
 def test_a_corrupted_checksum_still_yields_a_frame():
    """Flag it, do not swallow it -- a dropped frame looks like a dead unit."""
    broken = bytearray(RSP_POLL_PROBE)
    broken[-2] ^= 0xFF
    (f,) = MicromateFrameParser().feed(bytes(broken))
    assert f.sub == 0xA4
    assert not f.checksum_valid
 def test_a_truncated_frame_yields_nothing():
    parser = MicromateFrameParser()
    assert parser.feed(RSP_POLL_PROBE[:-1]) == []
    assert parser.frames == []
    assert parser.bytes_fed == len(RSP_POLL_PROBE) - 1
 def test_a_frame_too_short_to_hold_a_header_is_rejected():
    assert MicromateFrameParser().feed(bytes([STX, 0x00, 0xC5, 0xA4, ETX])) == []
 def test_request_frames_are_not_mistaken_for_responses():
    """Feeding a bidirectional capture must yield only the unit's side."""
    parser = MicromateFrameParser()
    frames = parser.feed(REQ_POLL + RSP_POLL_PROBE + REQ_SERIAL)
    assert len(frames) == 1
    assert frames[0].request_sub == 0x5B
 def test_leading_noise_is_discarded():
    """Cold-boot banners and the RV55's RING/CONNECT chatter precede frames."""
    noise = b"\r\nRING\r\n\r\nCONNECT\r\n" + bytes([ACK])
    (f,) = MicromateFrameParser().feed(noise + RSP_POLL_PROBE)
    assert f.sub == 0xA4
    assert f.checksum_valid
 def test_frames_split_across_feeds_reassemble():
    """The transport hands over whatever the socket returned, DLE pairs and all."""
    whole = RSP_STATE + RSP_CHUNK_WITH_DLE
    for cut in (1, 2, 5, 17, 24, 25, 40, len(RSP_STATE), len(whole) - 1):
        parser = MicromateFrameParser()
        got = parser.feed(whole[:cut]) + parser.feed(whole[cut:])
        assert len(got) == 2, f"split at {cut} lost a frame"
        assert all(f.checksum_valid for f in got), f"split at {cut} broke a checksum"
 def test_reset_clears_partial_state():
    parser = MicromateFrameParser()
    parser.feed(RSP_POLL_PROBE[:6])
    parser.reset()
    assert parser.bytes_fed == 0
    (f,) = parser.feed(RSP_POLL_PROBE)
    assert f.checksum_valid
 def test_firmware_line_of_an_unknown_flags_byte():
    f = MicromateFrame(sub=0xA4, flags=0x99, page_hi=0, page_lo=0,
                       data=b"", checksum_valid=True)
    assert f.firmware_line == "unknown"
    assert f.probe_length is None
 # ── Whole-session round trip ──────────────────────────────────────────────────
 _CAPTURES = (
    Path(__file__).resolve().parents[1]
    / "bridges" / "captures" / "9-24-26 - micromate2"
 )
@pytest.mark.skipif(
    not _CAPTURES.is_dir(),
    reason="capture directory is gitignored; present only on a dev box",
 )
 def test_whole_captured_sessions_parse_with_no_bad_checksums():
    """Belt-and-braces against the real bytes when they happen to be here.
    scratch/mm_frame_parse.py reported zero bad checksums on these sessions
    only because it accepts a frame matching *either* checksum rule.  This
    asserts the single rule holds across all of them.
    """
    total = 0
    for path in sorted(_CAPTURES.rglob("raw_s3_*.bin")):
        parser = MicromateFrameParser()
        frames = parser.feed(path.read_bytes())
        assert frames, f"{path.name}: no frames parsed"
        bad = [f for f in frames if not f.checksum_valid]
        assert not bad, f"{path.name}: {len(bad)} bad checksums"
        total += len(frames)
    assert total == 251, f"expected 251 response frames across the corpus, got {total}"
@pytest.mark.skipif(
    not _CAPTURES.is_dir(),
    reason="capture directory is gitignored; present only on a dev box",
 )
 def test_builder_reproduces_every_captured_read_frame():
    """218/218.  This is the test that would have caught the escape-set error."""
    checked = 0
    for path in sorted(_CAPTURES.rglob("raw_bw_*.bin")):
        blob = path.read_bytes()
        i = 0
        while i < len(blob):
            if not (blob[i] == ACK and i + 1 < len(blob) and blob[i + 1] == STX):
                i += 1
                continue
            j = i + 2
            body = bytearray()
            while j < len(blob):
                if blob[j] == DLE and j + 1 < len(blob):
                    body.append(blob[j + 1])
                    j += 2
                    continue
                if blob[j] == ETX:
                    break
                body.append(blob[j])
                j += 1
            payload = bytes(body[:-1])
            if len(payload) == 16:      # a read frame; writes carry a data section
                sub = payload[2]
                offset = (payload[4] << 8) | payload[5]
                assert build_request(sub, offset, payload[6:16]) == blob[i:j + 1], (
                    f"{path.name} @0x{i:04x} SUB=0x{sub:02x} offset=0x{offset:04x}"
                )
                checked += 1
            i = j + 1
    assert checked == 218, f"expected 218 read frames, checked {checked}"
@@ -0,0 +1,460 @@
 """Protocol-layer tests for the Micromate (series-4) live client.
 The load-bearing assertion in here is not "our parser understands the device" —
 it is **"the bytes we put on the wire are the bytes THOR puts on the wire."**
 Every request constant below is lifted from
 ``bridges/captures/9-24-26 - micromate2/`` (UM12947, firmware 11.0CB), so a
 passing test means a real unit has already answered exactly that frame.
 Responses are replayed through a scripted transport.  No hardware, no network.
 """
 from __future__ import annotations
 import os
 import sys
 from pathlib import Path
 import pytest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 from micromate import protocol as P
 from micromate.framing import ETX, STX, checksum, stuff
 from micromate.protocol import (
    ACK_DATA_LEN,
    ChecksumError,
    MicromateProtocol,
    ShortRead,
    UnexpectedResponse,
    chunk_params,
    key_lo_params,
    key_params,
    token_params,
 )
 FLAGS_CB = 0xC5
 # ── Test doubles ──────────────────────────────────────────────────────────────
 class ScriptedTransport:
    """Hands back queued responses; records every byte written."""
    def __init__(self, responses: list[bytes] | None = None) -> None:
        self.queue = list(responses or [])
        self.written: list[bytes] = []
        self._connected = True
    # BaseTransport surface actually used by MicromateProtocol
    def connect(self) -> None:
        self._connected = True
    def disconnect(self) -> None:
        self._connected = False
    def is_connected(self) -> bool:
        return self._connected
    def write(self, data: bytes) -> None:
        self.written.append(data)
    def read(self, n: int) -> bytes:
        return self.queue.pop(0) if self.queue else b""
 def frame(rsp_sub: int, data: bytes, *, flags: int = FLAGS_CB, page: int = 0) -> bytes:
    """Build a response frame the way a unit would."""
    payload = bytes([0x00, flags, rsp_sub, (page >> 8) & 0xFF, page & 0xFF]) + data
    return bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
 def ack(rsp_sub: int) -> bytes:
    return frame(rsp_sub, bytes(ACK_DATA_LEN))
 def proto(responses: list[bytes], **kw) -> tuple[MicromateProtocol, ScriptedTransport]:
    t = ScriptedTransport(responses)
    return MicromateProtocol(t, recv_timeout=0.5, **kw), t
 # ── Captured THOR request frames ──────────────────────────────────────────────
 REQ = {
    "poll":        bytes.fromhex("41021010005b000030000000000000000000009b03"),
    "serial":      bytes.fromhex("41021010001500000a000000000000000000002f03"),
    "state":       bytes.fromhex("41021010004900ffff000000000000000000005703"),
    "compliance":  bytes.fromhex("41021010001a00ffff000000000000000000002803"),
    "arm":         bytes.fromhex("41021010009300ffff00000000000000000000a103"),
    "setup_first": bytes.fromhex("41021010003f00ffff000000000000000000004d03"),
    "setup_next":  bytes.fromhex("41021010004000ffff000000000000000000004e03"),
 }
 # The complete 0x5A sequence for event 055d4a81 (4,076 bytes → 4 chunks), as
 # THOR sent it.  Chunk 1's params hold a literal 0x04 and chunk 3's offset is
 # the exact remainder.
 REQ_CHUNKS_4A81 = [
    bytes.fromhex("41021010005a00100400055d4a810000000000009b03"),
    bytes.fromhex("41021010005a0010040000001004000000000000007203"),
    bytes.fromhex("41021010005a00100400000008000000000000007603"),
    bytes.fromhex("41021010005a001003ec00000c000000000000006503"),
 ]
 SIZE_4A81 = 4076
 # ── Params builders ───────────────────────────────────────────────────────────
 def test_event_token_sits_at_params_7():
    """⚠ THOR sends 0xFE here; the protocol reference documents all-zero params.
    That reference entry describes our own browse probing, not THOR's.
    """
    assert token_params() == bytes.fromhex("00000000000000fe0000")
 def test_event_record_takes_the_full_key_at_params_4():
    assert key_params(bytes.fromhex("055d4a81")) == bytes.fromhex("00000000055d4a810000")
 def test_monitor_log_takes_only_the_low_half_of_the_key():
    """⚠ Inferred from one key value -- see key_lo_params' docstring."""
    assert key_lo_params(bytes.fromhex("055d4a81")) == bytes.fromhex("0000000000004a810000")
 def test_chunk_params_switch_from_key_to_byte_offset():
    key = bytes.fromhex("055d4a81")
    assert chunk_params(key, 0) == bytes.fromhex("055d4a81000000000000")
    assert chunk_params(key, 1024) == bytes.fromhex("00000400000000000000")
    assert chunk_params(key, 13312) == bytes.fromhex("00003400000000000000")
@pytest.mark.parametrize("bad", [b"", b"\x01\x02\x03", b"\x01\x02\x03\x04\x05"])
 def test_params_builders_reject_a_wrong_length_key(bad):
    for fn in (key_params, key_lo_params):
        with pytest.raises(ValueError):
            fn(bad)
 # ── Each read emits the frame THOR emits ──────────────────────────────────────
@pytest.mark.parametrize(
    "name, method, rsp_sub, data_len",
    [
        ("poll",        "poll",              0xA4, 59),
        ("serial",      "read_serial",       0xEA, 21),
        ("state",       "read_state",        0xB6, 16),
        ("compliance",  "read_compliance_config", 0xE5, 2103),
        ("setup_first", "read_first_setup",  0xC0, 266),
        ("setup_next",  "read_next_setup",   0xBF, 266),
    ],
 )
 def test_reads_match_thors_wire_bytes(name, method, rsp_sub, data_len):
    p, t = proto([frame(rsp_sub, bytes(data_len))])
    getattr(p, method)()
    assert t.written == [REQ[name]]
 def test_arm_event_matches_thors_wire_bytes():
    p, t = proto([ack(0x6C)])
    p.arm_event()
    assert t.written == [REQ["arm"]]
 def test_poll_is_the_only_read_with_a_non_ffff_offset_besides_serial():
    """Reads are single-step at 0xFFFF; POLL and SERIAL are the exceptions."""
    assert set(P._OFFSETS) == {P.SUB_POLL, P.SUB_SERIAL}
    assert P._OFFSETS[P.SUB_POLL] == 0x0030
    assert P._OFFSETS[P.SUB_SERIAL] == 0x000A
 # ── The chunk walk ────────────────────────────────────────────────────────────
 def test_download_reproduces_thors_chunk_sequence_byte_for_byte():
    """The whole point of step 2.  Four chunks, 4,076 bytes, THOR's exact frames."""
    payload = bytes(range(256)) * 16          # 4096 B, we use the first 4076
    payload = payload[:SIZE_4A81]
    responses = []
    for i in range(4):
        want = min(P.CHUNK_SIZE, SIZE_4A81 - i * P.CHUNK_SIZE)
        body = payload[i * P.CHUNK_SIZE: i * P.CHUNK_SIZE + want]
        responses.append(frame(0xA5, bytes(11) + body, page=want // 256))
    p, t = proto(responses)
    got = p.read_event_file(bytes.fromhex("055d4a81"), SIZE_4A81)
    assert t.written == REQ_CHUNKS_4A81
    assert got == payload
    assert len(got) == SIZE_4A81
@pytest.mark.parametrize(
    "size, n_chunks, last_offset",
    [
        (4076, 4, 0x03EC), (11032, 11, 0x0318), (11502, 12, 0x00EE),
        (13424, 14, 0x0070), (8746, 9, 0x022A), (6092, 6, 0x03CC),
        (1024, 1, 0x0400), (1, 1, 0x0001), (1025, 2, 0x0001),
    ],
 )
 def test_chunk_count_and_final_offset(size, n_chunks, last_offset):
    """The first six rows are the six bench events, with THOR's real offsets."""
    responses = []
    for i in range(n_chunks):
        want = min(P.CHUNK_SIZE, size - i * P.CHUNK_SIZE)
        responses.append(frame(0xA5, bytes(11) + bytes(want)))
    p, t = proto(responses)
    p.read_event_file(bytes.fromhex("055d4a81"), size)
    assert len(t.written) == n_chunks
    # offset is payload[4:5] of the request; recover it from the built frame
    final = t.written[-1]
    assert final[7:9] in (
        bytes([last_offset >> 8, last_offset & 0xFF]),
        # a 0x02/0x03/0x04/0x10 high byte arrives escaped, shifting the pair
        bytes([0x10, last_offset >> 8]),
    )
 def test_a_short_chunk_raises_rather_than_truncating():
    """A silently short event is the failure mode this codebase keeps hitting."""
    p, _ = proto([frame(0xA5, bytes(11) + bytes(900))])   # asked for 1024
    with pytest.raises(ShortRead, match="asked for 1024 B, got 900"):
        p.read_event_file(bytes.fromhex("055d4a81"), 1024)
 def test_a_chunk_too_short_to_hold_its_header_raises():
    p, _ = proto([frame(0xA5, bytes(4))])
    with pytest.raises(ShortRead, match="too short to hold a chunk header"):
        p.read_event_file(bytes.fromhex("055d4a81"), 1024)
 def test_download_rejects_a_nonsense_size():
    p, _ = proto([])
    with pytest.raises(ValueError):
        p.read_event_file(bytes.fromhex("055d4a81"), 0)
 # ── The monitor-log walk ──────────────────────────────────────────────────────
 def test_monitor_log_walk_ends_on_a_short_response():
    """⚠ Not a keyed read -- the same request repeated, device-side cursor.
    Eight records then an 11-byte ack, which is what the capture shows.
    """
    key = bytes.fromhex("055d4a81")
    responses = [frame(0xF5, bytes(297)) for _ in range(8)] + [ack(0xF5)]
    p, t = proto(responses)
    records = []
    while (rec := p.read_monitor_log_next(key)) is not None:
        records.append(rec)
    assert len(records) == 8
    assert len(t.written) == 9
    assert len(set(t.written)) == 1, "every request in the walk is identical"
 # ── Error handling ────────────────────────────────────────────────────────────
 def test_a_bad_checksum_raises_by_default():
    """⚠ Deliberately stricter than the Series III sibling.
    That one logs and continues because its parser cannot always tell an
    inner-frame delimiter from a checksum byte.  The Micromate rule is exact on
    251/251 captured frames, so a mismatch here means something real.
    """
    bad = bytearray(frame(0xA4, bytes(59)))
    bad[-2] ^= 0xFF
    p, _ = proto([bytes(bad)])
    with pytest.raises(ChecksumError, match="checksum mismatch"):
        p.poll()
 def test_a_bad_checksum_can_be_downgraded_for_field_diagnosis():
    bad = bytearray(frame(0xA4, bytes(59)))
    bad[-2] ^= 0xFF
    p, _ = proto([bytes(bad)], strict_checksums=False)
    assert p.poll().sub == 0xA4
 def test_the_wrong_response_sub_raises():
    p, _ = proto([frame(0xE0, bytes(19))])      # 0xE0 answers 0x1F, not 0x1E
    with pytest.raises(UnexpectedResponse, match="expected SUB 0xE1"):
        p.read_event_first()
 def test_a_timeout_reports_how_many_bytes_arrived():
    """Separates "nothing came back" from "bytes arrived but never framed".
    Those have completely different causes -- and on a Micromate the second one
    is the signature of a modem forwarding a session it should not be.
    """
    p, _ = proto([])
    with pytest.raises(P.TimeoutError, match="0 bytes were received"):
        p.poll()
    unframed = b"\x02\x00\xc5\xa4garbage-no-terminator"
    p2, _ = proto([unframed])
    with pytest.raises(P.TimeoutError, match=f"{len(unframed)} bytes were received"):
        p2.poll()
 def test_a_leftover_frame_is_discarded_rather_than_answered_with():
    """If a read returns two frames, the extra must not answer the NEXT request.
    Every exchange resets the parser before sending, so anything already
    buffered is treated as stale.  Delivering it would be the worse failure:
    `expected_sub` happens to catch a mismatched SUB, but a same-SUB leftover
    would sail through and return data for the wrong key.
    """
    # Both frames arrive while answering arm_event(); the 0xE1 is left over.
    p, _ = proto([ack(0x6C) + frame(0xE1, b"\xaa" * 19)])
    p.arm_event()
    # The next request gets no bytes of its own, so it must time out rather
    # than hand back the stale 0xE1.
    with pytest.raises(P.TimeoutError):
        p.read_event_first()
 # ── Against the real capture, when it happens to be present ───────────────────
 _CAPTURES = (
    Path(__file__).resolve().parents[1]
    / "bridges" / "captures" / "9-24-26 - micromate2"
 )
 _DOWNLOAD = "raw_bw_20260925_011403_Download_events_then_delete_1_event.bin"
@pytest.mark.skipif(
    not (_CAPTURES / _DOWNLOAD).is_file(),
    reason="capture is gitignored; present only on a dev box",
 )
 def test_every_captured_download_frame_is_one_we_would_have_sent():
    """Replay the real session: for each event, assert our chunk walk emits
    exactly the frames THOR emitted -- all 50-odd of them, six events."""
    from micromate.framing import ACK, DLE
    def destuffed_frames(blob: bytes, is_req: bool):
        i, n = 0, len(blob)
        while i < n:
            if is_req:
                if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
                    i += 1
                    continue
                j = i + 2
            else:
                if blob[i] != STX:
                    i += 1
                    continue
                j = i + 1
            out = bytearray()
            while j < n:
                if blob[j] == DLE and j + 1 < n:
                    out.append(blob[j + 1])
                    j += 2
                    continue
                if blob[j] == ETX:
                    break
                out.append(blob[j])
                j += 1
            if len(out) >= 6:
                yield blob[i:j + 1], bytes(out[:-1])
            i = j + 1
    bw = list(destuffed_frames((_CAPTURES / _DOWNLOAD).read_bytes(), True))
    s3 = list(
        destuffed_frames(
            (_CAPTURES / _DOWNLOAD.replace("raw_bw", "raw_s3")).read_bytes(), False
        )
    )
    # Group THOR's 0x5A frames per event, taking each event's key+size from the
    # 1E/1F that preceded them.
    events, cur = [], None
    for (wire, req), (_, rsp) in zip(bw, s3):
        sub, data = req[2], rsp[5:]
        if sub in (0x1E, 0x1F) and len(data) >= 19:
            cur = {"key": data[11:15], "size": int.from_bytes(data[15:19], "big"),
                   "reqs": [], "rsps": []}
            if cur["size"]:
                events.append(cur)
        elif sub == 0x5A and cur is not None:
            cur["reqs"].append(wire)
            cur["rsps"].append(rsp)
    # The capture walks the chain twice (it deletes an event on the second
    # pass), so some 1E/1F hits carry a size but no download behind them.
    events = [e for e in events if e["reqs"]]
    assert len(events) == 6, f"expected 6 downloaded events, found {len(events)}"
    total = 0
    for e in events:
        p, t = proto([bytes([STX]) + stuff(r + bytes([checksum(r)])) + bytes([ETX])
                      for r in e["rsps"]])
        got = p.read_event_file(e["key"], e["size"])
        assert t.written == e["reqs"], (
            f"event {e['key'].hex()}: our {len(t.written)} frames differ from "
            f"THOR's {len(e['reqs'])}"
        )
        assert len(got) == e["size"]
        total += len(e["reqs"])
    assert total == 56, f"expected 56 download frames across the 6 events, saw {total}"
 # ── Above 64 KB: the limit UM20147 already exceeds ────────────────────────────
 def test_chunk_offsets_carry_past_64_kb():
    """⚠ The offset is a uint32 at params[0:4], not a uint16 at params[2:4].
    Every offset THOR was observed to send fits in two bytes (largest 0x3400),
    so params[0:2] was always zero and the field looks narrower than it is.
    Reading it as a uint16 caps a download at 65,536 B — and UM20147 holds a
    72,560 B event, so the cap is not hypothetical.
    This is the Series III 64 KB page-boundary bug in a new guise; that one is
    still open.  An address field whose high bytes are zero in every capture is
    not a narrow field, it is an untested one.
    """
    key = bytes.fromhex("055d4a83")
    # Below the old cap: byte-identical to the uint16 form, so nothing verified
    # against THOR's frames changes.
    for off in (1024, 13312, 65535):
        assert chunk_params(key, off) == bytes(2) + off.to_bytes(2, "big") + bytes(6)
    # Above it: the carry lands in params[1].
    assert chunk_params(key, 65536) == bytes.fromhex("00010000") + bytes(6)
    assert chunk_params(key, 71680) == bytes.fromhex("00011800") + bytes(6)
 def test_a_chunk_offset_with_0x10_in_it_is_escaped_on_the_wire():
    """At offset 1 MiB params[1] is 0x10, which must go out as `10 10`.
    Nothing below 64 KB can produce this, so it only became reachable with the
    uint32 offset — and an unescaped 0x10 is the bug class that cost the
    Series III `5A` walk a release.
    """
    params = chunk_params(bytes(4), 1 << 20)
    assert params[:4] == bytes.fromhex("00100000")
    from micromate.framing import build_request
    frame = build_request(P.SUB_BULK_DOWNLOAD, 0x0400, params)
    assert bytes.fromhex("1010") in frame
 def test_a_72kb_event_downloads_in_71_chunks():
    """UM20147's event 055d4a83, the one that broke the uint16 assumption."""
    size = 72560
    n = 71
    responses = []
    for i in range(n):
        want = min(P.CHUNK_SIZE, size - i * P.CHUNK_SIZE)
        responses.append(frame(0xA5, bytes(11) + bytes(want)))
    p, t = proto(responses)
    got = p.read_event_file(bytes.fromhex("055d4a83"), size)
    assert len(got) == size
    assert len(t.written) == n
    # Chunk 64 is the first past the old cap; its params must carry the 0x01.
    wire = t.written[64]
    assert bytes.fromhex("000100") in wire, "the carry into params[1] is on the wire"