docs(changelog): Series-4 live protocol, THOR behaviour, and bench tooling

Written on dev as part of finishing the merge, per the repo convention, and
folded into the existing Unreleased sections rather than adding duplicates.

Added: the Micromate wire protocol (framing divergences, read path, event chain,
0x5A verbatim file streaming), setup management as read-modify-write with 0xDA
creating files, the decoded scheduler file, the path-addressed file transfer that
retracts an earlier "no such command" conclusion, monitoring control and
per-event delete, and four new tools -- mm_probe, mm_link, mm_frame_parse,
socat_log_split and fake_unit.

Changed: what THOR actually does on the wire (eleven-command status check, ~2.2 KB
a time, 18.8 MB/day/unit at 10 s), the connection interval setting a count rather
than a period, two reproduced THOR defects (silent poll death with a false
"Connected", and a subscription leak growing 1 -> 12 handlers over ten hours), and
the Micromate USB host supporting FTDI and CDC-ACM but not Prolific.

Migration extended to state the operational consequence for the whole section:
none.  No codec, store, DB or TOOL_VERSION change; the Series-4 work touches only
docs/, bridges/ and scratch/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL

CHANGELOG.md

@@ -46,6 +46,50 @@ All notable changes to seismo-relay are documented here.
   500s.  Reproduced on BE12599.  Production is unaffected: it ingests complete
   files via the watcher path and never runs this walk.
 
+- **The Micromate (Series IV) live wire protocol, reverse-engineered end to
+  end** — `docs/micromate_protocol_reference.md`.  Worked out against a bench
+  UM12947 over USB and a recording relay, with THOR driving every write so that
+  no command has ever been originated against a unit by this project.  **A
+  Micromate answers Series III command frames**, with three framing differences:
+  responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
+  or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
+  **uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
+  `SUB 0x1A` by 47x.  Read path, event chain, and `SUB 0x5A` streaming the
+  `.IDFW` file verbatim are all confirmed.
+- **Series IV setup management, fully mapped.**  `0x41` reads the active setup
+  name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
+  write it back.  **Setups are read-modify-write** — the written block is the
+  read block, 91% byte-identical at a fixed 11-byte shift.  `0xDA` **creates**
+  files rather than only overwriting, confirmed on the unit's own screen, and an
+  overwrite is protocol-identical to a create: no handshake, no warning, and no
+  protection even over the *active* setup of a monitoring unit.
+- **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
+  records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
+  half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
+  name.  Verified entry-for-entry against the operator's own THOR screen.
+- **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
+  `0x8D`/`0x8E` write.  This **retracts** an earlier conclusion in the same
+  document that no such command existed; that was inferred from absent firmware
+  strings and was wrong.
+- **Monitoring control and per-event delete.**  `0x96`/`0x97` start and stop as
+  on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
+  for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
+  constant.  Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
+  which is safer than Series III's erase-everything.  `SUB 0x1C` also carries the
+  device clock.
+- **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
+  identically as "disconnected": refused, connect timeout (the silent-drop
+  signature of a trusted-IP whitelist), **connected but no reply** (the modem
+  answered and the unit did not), and replied.  Each verdict names what to try
+  next.  `--slots N` tests single-session modem behaviour.
+- **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
+  decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
+  `onewaydev`) driven by a control file.  No pyserial; stdlib `termios` only.
+- **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
+  — a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
+  all, since it scans for `DLE+STX`), byte-exact capture recovery from a
+  `socat -x` relay log, and a serial-port stand-in that answers as a unit.
+
 ### Changed
 
 - **Connecting to a unit no longer walks its event chain.**  `/device/events`
@@ -58,11 +102,32 @@ All notable changes to seismo-relay are documented here.
   toolbar.  Knowing whether a unit's ACH is on no longer requires reading every
   event it has stored.
 
+- **Recorded what THOR actually does on the wire**, measured rather than assumed.
+  A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
+  MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
+  `MONITOR_STATUS` check at 60 s.  The **status interval is honoured; the
+  connection interval is not** — it sets `(status / connection) - 1` checks per
+  cycle, so equal values yield *zero* cheap checks and every connection becomes
+  the expensive one.
+- **Two THOR defects reproduced with timestamps.**  After a connection drops
+  mid-download it retries **once**, stops polling entirely and **never resumes**,
+  while displaying `Connected` for as long as it is left alone — and `Idle` for a
+  unit that is actively recording.  Separately, THOR's own log shows a
+  **subscription leak**: one logical event dispatched to a growing number of
+  handlers, **1 to 12 over ten hours** of uptime, consistent with the field
+  report that only a restart recovers it.
+- **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
+  either firmware line.  A PL2303 cable (Benfei) leaves a unit with no working
+  modem port; an FTDI cable (Sabrent) works.  Both are in circulation and
+  indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
+
 ### Migration
 
-**None.**  Frontend and documentation only — no codec, waveform-store or DB
-change, no schema change, and no `TOOL_VERSION` bump.  The webapp is served
-from the image, so the change appears after the next `sfm` rebuild.
+**None.**  Frontend, documentation and bench tooling only — no codec,
+waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump.  The
+webapp is served from the image, so its changes appear after the next `sfm`
+rebuild.  The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only;
+nothing under `sfm/`, `minimateplus/` or `micromate/` was touched.
 
 ---
 

bridges/mm_link.py

@@ -0,0 +1,338 @@
+#!/usr/bin/env python3
+"""
+mm_link.py — a "perfect modem" between THOR and a Micromate, with a readable
+log and deliberate fault injection.
+
+Why
+---
+THOR gives almost no visibility into a connection: a refresh button, two poll
+intervals, and no way to see whether a check succeeded, timed out, or was never
+sent.  When a unit "won't stay connected" there is nothing to look at.
+
+This sits where the cellular modem would sit and answers the question directly:
+
+  * **What is THOR actually doing?**  Every frame is decoded and timestamped —
+    `POLL`, `MONITOR_STATUS`, `SETUP_NAME_READ` — not a hex dump.
+  * **Is it even trying?**  Silence is visible: the log shows gaps.
+  * **How does it behave when the link misbehaves?**  Faults can be injected on
+    demand, which a real cell link will not do on cue.
+
+Point THOR at this host and port exactly as if it were a modem (Communication:
+TCP, IP: <this host>, Port: <--listen>).
+
+Fault injection
+---------------
+Write a mode into the control file (default `mm_link.ctl`) and it takes effect
+on the next byte:
+
+    echo pass      > mm_link.ctl    # normal relay
+    echo blackhole > mm_link.ctl    # TCP stays up, bytes are swallowed
+    echo drop      > mm_link.ctl    # close the connection abruptly (RST-ish)
+    echo delay:2.0 > mm_link.ctl    # forward, but 2 s late in both directions
+    echo onewaydev > mm_link.ctl    # THOR->unit passes, unit->THOR is swallowed
+
+**`blackhole` is the one that matters.**  It reproduces the classic cellular
+failure: the socket is still open as far as both ends are concerned, but nothing
+crosses.  A client that relies on TCP to tell it the peer is gone will sit there
+until the OS keepalive fires — which by default is about two hours.
+
+Usage
+-----
+    python3 bridges/mm_link.py --serial /dev/ttyACM0 --baud 115200 \\
+        --listen 12345 --logdir ~/mm-captures
+
+Writes, per session:
+    <logdir>/mmlink_<ts>/session.log   decoded, timestamped, human-readable
+    <logdir>/mmlink_<ts>/raw_bw.bin    THOR -> unit, raw
+    <logdir>/mmlink_<ts>/raw_s3.bin    unit -> THOR, raw
+
+The raw pair loads straight into `scratch/mm_frame_parse.py`.
+"""
+
+from __future__ import annotations
+
+import argparse
+import datetime
+import os
+import socket
+import sys
+import threading
+import time
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scratch"))
+try:
+    from mm_frame_parse import SUBNAME, destuff  # noqa: F401
+except Exception:                                 # pragma: no cover
+    SUBNAME = {}
+
+import errno
+import select
+import termios
+
+DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
+
+_BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
+         57600: termios.B57600, 115200: termios.B115200}
+
+
+class SerialPort:
+    """Minimal raw serial port on stdlib termios — no pyserial dependency.
+
+    The bench hosts are whatever is to hand; requiring a pip install on someone
+    else's machine is a poor trade for the ~30 lines this saves.
+    """
+
+    def __init__(self, path: str, baud: int):
+        if baud not in _BAUD:
+            raise ValueError(f"unsupported baud {baud}; pick one of {sorted(_BAUD)}")
+        self.fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
+        a = termios.tcgetattr(self.fd)
+        a[0] = 0                                             # iflag: no translation
+        a[1] = 0                                             # oflag: raw
+        a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL  # cflag: 8N1, ignore modem lines
+        a[3] = 0                                             # lflag: non-canonical, no echo
+        a[4] = a[5] = _BAUD[baud]
+        a[6] = list(a[6])
+        a[6][termios.VMIN] = 0
+        a[6][termios.VTIME] = 0
+        termios.tcsetattr(self.fd, termios.TCSANOW, a)
+        termios.tcflush(self.fd, termios.TCIOFLUSH)
+
+    def read(self, n: int) -> bytes:
+        r, _, _ = select.select([self.fd], [], [], 0.2)
+        if not r:
+            return b""
+        try:
+            return os.read(self.fd, n)
+        except OSError as e:
+            if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
+                return b""
+            raise
+
+    def write(self, data: bytes) -> None:
+        while data:
+            try:
+                data = data[os.write(self.fd, data):]
+            except OSError as e:
+                if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
+                    select.select([], [self.fd], [], 0.2)
+                    continue
+                raise
+
+    def close(self) -> None:
+        try:
+            os.close(self.fd)
+        except OSError:
+            pass
+
+
+def name_of(sub: int, is_request: bool) -> str:
+    if is_request:
+        return SUBNAME.get(sub, f"SUB_{sub:02X}")
+    return "rsp " + SUBNAME.get(0xFF - sub, f"SUB_{0xFF - sub:02X}")
+
+
+class FrameSniffer:
+    """Accumulate bytes and report complete frames, without altering the stream."""
+
+    def __init__(self, is_request: bool):
+        self.is_request = is_request
+        self.buf = bytearray()
+
+    def feed(self, data: bytes):
+        """Yield (sub, payload_len) for each complete frame seen."""
+        self.buf.extend(data)
+        while True:
+            start = -1
+            for i, b in enumerate(self.buf):
+                if self.is_request and b == ACK and i + 1 < len(self.buf) and self.buf[i + 1] == STX:
+                    start = i
+                    break
+                if not self.is_request and b == STX:
+                    start = i
+                    break
+            if start < 0:
+                if len(self.buf) > 8192:
+                    del self.buf[:-16]
+                return
+            j = start + (2 if self.is_request else 1)
+            end = -1
+            while j < len(self.buf):
+                if self.buf[j] == DLE and j + 1 < len(self.buf):
+                    j += 2
+                    continue
+                if self.buf[j] == ETX:
+                    end = j
+                    break
+                j += 1
+            if end < 0:
+                return                      # wait for more bytes
+            body = self.buf[start:end + 1]
+            del self.buf[:end + 1]
+            # SUB sits at a fixed spot past the leading framing -- but it is
+            # DLE-escaped when its own value is 0x02/0x03/0x04/0x10, so a raw
+            # read reports 0x10 for those.  SUB 0x02 was being logged as
+            # "SUB_10" until this was handled.
+            off = 5 if self.is_request else 3
+            if len(body) > off:
+                sub = body[off]
+                if sub == DLE and len(body) > off + 1:
+                    sub = body[off + 1]
+                yield sub, len(body)
+
+
+class Link:
+    def __init__(self, args):
+        self.args = args
+        self.mode = "pass"
+        self.delay = 0.0
+        self.ctl = Path(args.control)
+        self.session: Path | None = None
+        self.log_fh = None
+        self.raw = {}
+        self.t0 = time.time()
+        self.counts = {}
+
+    # ── logging ────────────────────────────────────────────────────────────
+    def open_session(self):
+        ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+        self.session = Path(self.args.logdir) / f"mmlink_{ts}"
+        self.session.mkdir(parents=True, exist_ok=True)
+        self.log_fh = open(self.session / "session.log", "a", buffering=1)
+        self.raw = {
+            "bw": open(self.session / "raw_bw.bin", "ab"),
+            "s3": open(self.session / "raw_s3.bin", "ab"),
+        }
+        self.say(f"=== session {ts} — serial {self.args.serial} @ {self.args.baud} ===")
+
+    def say(self, text: str):
+        line = f"{datetime.datetime.now().strftime('%H:%M:%S.%f')[:-3]}  {text}"
+        print(line, flush=True)
+        if self.log_fh:
+            self.log_fh.write(line + "\n")
+
+    # ── control file ───────────────────────────────────────────────────────
+    def poll_control(self):
+        while True:
+            try:
+                if self.ctl.exists():
+                    want = self.ctl.read_text().strip().lower()
+                    if want.startswith("delay:"):
+                        d = float(want.split(":", 1)[1])
+                        if ("delay", d) != (self.mode, self.delay):
+                            self.mode, self.delay = "delay", d
+                            self.say(f"*** MODE -> delay {d}s ***")
+                    elif want and want != self.mode:
+                        self.mode, self.delay = want, 0.0
+                        self.say(f"*** MODE -> {want} ***")
+            except Exception:
+                pass
+            time.sleep(0.25)
+
+    # ── the relay ──────────────────────────────────────────────────────────
+    def pump(self, src, dst, tag: str, is_request: bool, stop: threading.Event):
+        sniff = FrameSniffer(is_request)
+        arrow = "THOR->unit" if is_request else "unit->THOR"
+        last = time.time()
+        while not stop.is_set():
+            timed_out = False
+            try:
+                data = src.recv(4096) if isinstance(src, socket.socket) else src.read(4096)
+            except TimeoutError:
+                timed_out = True
+                # socket.timeout subclasses OSError, so it MUST be caught first.
+                # Treating it as a dead socket closes the connection after 200 ms
+                # of quiet -- which is exactly what `blackhole` produces, so the
+                # relay killed the link it was supposed to be faking a fault on.
+                data = b""
+            except OSError:
+                break
+            if isinstance(src, socket.socket) and data == b"" and not timed_out:
+                self.say(f"{arrow}: peer closed the connection")
+                break
+            if not data:
+                if time.time() - last > self.args.quiet_after and self.counts:
+                    self.say(f"--- {self.args.quiet_after:.0f}s with no traffic ---")
+                    last = time.time()
+                continue
+            last = time.time()
+
+            self.raw[tag].write(data)
+            self.raw[tag].flush()
+            for sub, ln in sniff.feed(data):
+                label = name_of(sub, is_request)
+                self.counts[label] = self.counts.get(label, 0) + 1
+                self.say(f"{arrow}  {label:<20} ({ln} B)"
+                         + ("" if self.mode == "pass" else f"   [mode={self.mode}]"))
+
+            mode = self.mode
+            if mode == "drop":
+                self.say(f"{arrow}: DROPPING the connection (fault injection)")
+                stop.set()
+                break
+            if mode == "blackhole":
+                continue                      # swallow, keep the socket open
+            if mode == "onewaydev" and not is_request:
+                continue                      # unit's replies never reach THOR
+            if mode == "delay" and self.delay:
+                time.sleep(self.delay)
+            try:
+                if isinstance(dst, socket.socket):
+                    dst.sendall(data)
+                else:
+                    dst.write(data)
+            except OSError:
+                break
+        stop.set()
+
+    def serve(self):
+        srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        srv.bind(("0.0.0.0", self.args.listen))
+        srv.listen(5)
+        self.open_session()
+        self.say(f"listening on 0.0.0.0:{self.args.listen}   control file: {self.ctl}")
+        self.say("point THOR at this host/port as Communication=TCP")
+        threading.Thread(target=self.poll_control, daemon=True).start()
+
+        while True:
+            conn, addr = srv.accept()
+            conn.settimeout(0.2)
+            self.say(f"+++ THOR connected from {addr[0]}:{addr[1]} +++")
+            try:
+                ser = SerialPort(self.args.serial, self.args.baud)
+            except OSError as e:
+                self.say(f"!!! cannot open {self.args.serial}: {e}")
+                conn.close()
+                continue
+            stop = threading.Event()
+            ts = [
+                threading.Thread(target=self.pump, args=(conn, ser, "bw", True, stop), daemon=True),
+                threading.Thread(target=self.pump, args=(ser, conn, "s3", False, stop), daemon=True),
+            ]
+            for t in ts:
+                t.start()
+            for t in ts:
+                t.join()
+            conn.close()
+            ser.close()
+            summary = ", ".join(f"{k}x{v}" for k, v in sorted(self.counts.items()))
+            self.say(f"--- connection closed.  frames this session: {summary or 'none'} ---")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--serial", default="/dev/ttyACM0")
+    ap.add_argument("--baud", type=int, default=115200)
+    ap.add_argument("--listen", type=int, default=12345)
+    ap.add_argument("--logdir", default=os.path.expanduser("~/mm-captures"))
+    ap.add_argument("--control", default="mm_link.ctl")
+    ap.add_argument("--quiet-after", type=float, default=30.0,
+                    help="log a marker after this many seconds of silence")
+    Link(ap.parse_args()).serve()
+
+
+if __name__ == "__main__":
+    main()

bridges/mm_probe.py

@@ -0,0 +1,226 @@
+#!/usr/bin/env python3
+"""
+mm_probe.py — answer "why can't we reach this unit?" in one command.
+
+THOR reports a failed connection as "disconnected" and nothing else.  That single
+word covers at least four completely different faults with four different fixes,
+and telling them apart is the difference between a modem reboot and a site visit:
+
+  * **connection refused**   something answered and said no — wrong port, or the
+                             modem is refusing a further session
+  * **connect timed out**    nothing answered at all — trusted-IP whitelist,
+                             firewall, or the modem is off the network
+  * **connected, no reply**  the MODEM answered but the unit did not.  The TCP
+                             path is fine; the modem is not forwarding to serial.
+                             This is the signature of a wedged transparent-TCP
+                             session, and it is the one THOR cannot distinguish
+                             from any of the others
+  * **replied**              the unit is alive; the problem is upstream software
+
+Read-only.  It sends `POLL`, then optionally `SERIAL` and the state read — the
+same three commands THOR's own connection check uses — and never writes.
+
+Usage
+-----
+    python3 bridges/mm_probe.py 63.45.161.30:9034
+    python3 bridges/mm_probe.py 10.0.0.8:12345 --timeout 5
+    python3 bridges/mm_probe.py <host:port> --slots 3
+
+`--slots N` opens N connections at once and reports how many the far end accepts.
+A transparent-TCP modem typically serves **one** session; if the first succeeds
+and the rest are refused or hang, that confirms the single-slot behaviour and
+explains why a leaked session takes a unit offline until the slot frees.
+
+Works for both series: a Series III reply opens `DLE STX`, a Micromate reply
+opens with a bare `STX`, so the probe also tells you which one answered.
+"""
+
+from __future__ import annotations
+
+import argparse
+import socket
+import sys
+import time
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+from minimateplus.framing import build_bw_frame  # noqa: E402
+
+DLE, STX, ETX = 0x10, 0x02, 0x03
+
+
+def destuff(raw: bytes) -> bytes:
+    """Strip framing and DLE escapes; return the payload without its checksum."""
+    i = 1 if raw and raw[0] == STX else (2 if len(raw) > 1 and raw[1] == STX else 0)
+    out = bytearray()
+    while i < len(raw):
+        b = raw[i]
+        if b == DLE and i + 1 < len(raw):
+            out.append(raw[i + 1])
+            i += 2
+            continue
+        if b == ETX:
+            break
+        out.append(b)
+        i += 1
+    return bytes(out[:-1]) if len(out) > 1 else b""
+
+
+# Reads are two-step on Series III: a probe at offset 0, then a data read at the
+# block's length.  THOR sends these offsets, and they also work on a Micromate.
+OFFSETS = {0x5B: 0x0030, 0x15: 0x000A, 0x49: 0xFFFF}
+
+
+def exchange(sock: socket.socket, sub: int, timeout: float) -> tuple[bytes, float]:
+    sock.sendall(build_bw_frame(sub, OFFSETS.get(sub, 0)))
+    t0 = time.time()
+    buf, deadline = b"", t0 + timeout
+    sock.settimeout(0.3)
+    while time.time() < deadline:
+        try:
+            chunk = sock.recv(4096)
+            if not chunk:
+                break
+            buf += chunk
+            if buf.endswith(bytes([ETX])) and len(buf) > 8:
+                break
+        except TimeoutError:
+            continue
+        except OSError:
+            break
+    return buf, time.time() - t0
+
+
+def step(n: int, label: str, result: str) -> None:
+    print(f"  [{n}] {label:.<28} {result}")
+
+
+def probe(host: str, port: int, timeout: float) -> int:
+    print(f"\ntarget {host}:{port}   (read-only: POLL, SERIAL, state)\n")
+
+    # ── 1. TCP ────────────────────────────────────────────────────────────
+    t0 = time.time()
+    try:
+        sock = socket.create_connection((host, port), timeout=timeout)
+    except ConnectionRefusedError:
+        step(1, "TCP connect", f"REFUSED after {1000*(time.time()-t0):.0f} ms")
+        print("\nverdict: something answered and actively refused.")
+        print("  Not a silent firewall drop — the host is reachable.")
+        print("  Wrong port, the service is down, or the modem is refusing")
+        print("  an additional session because its one slot is in use.")
+        return 2
+    except (TimeoutError, socket.timeout):
+        step(1, "TCP connect", f"TIMED OUT after {time.time()-t0:.1f} s")
+        print("\nverdict: nothing answered at all.")
+        print("  A silent drop, which is what a trusted-IP whitelist looks")
+        print("  like — it discards rather than refuses.  Check the modem's")
+        print("  Trusted IPs (and note a VPN changes the IP you arrive from),")
+        print("  the firewall, and whether the modem is on the network.")
+        return 3
+    except OSError as e:
+        step(1, "TCP connect", f"FAILED: {e}")
+        return 4
+    step(1, "TCP connect", f"ok ({1000*(time.time()-t0):.0f} ms)")
+
+    # ── 2. POLL ───────────────────────────────────────────────────────────
+    try:
+        raw, dt = exchange(sock, 0x5B, timeout)
+    except OSError as e:
+        step(2, "POLL", f"send failed: {e}")
+        sock.close()
+        return 4
+
+    if not raw:
+        step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
+        print("\nverdict: the MODEM answered but the unit did not.")
+        print("  TCP is fine end to end — something accepted the connection.")
+        print("  What is missing is the serial side.  Most likely the modem is")
+        print("  not forwarding to its serial port, which is what a wedged")
+        print("  transparent-TCP session looks like: the slot is held by a")
+        print("  connection that never closed.")
+        print("\n  Try, in order:")
+        print("    1. ACEmanager -> TCP Idle Timeout.  If 0/disabled, a stale")
+        print("       session holds the slot forever.  2 minutes is the value")
+        print("       this project standardised on.")
+        print("    2. Reboot the modem.  If that fixes it, the modem was")
+        print("       holding state and the timeout is the permanent fix.")
+        print("    3. Check the unit's own screen — serial cable, power.")
+        sock.close()
+        return 5
+
+    series = "Series III (DLE STX)" if raw[0] == DLE else "Micromate (bare STX)"
+    step(2, "POLL", f"reply {len(raw)} B in {1000*dt:.0f} ms")
+    p = destuff(raw)
+    ok = len(p) > 3 and p[2] == 0xFF - 0x5B
+    step(3, "frame", f"{'valid' if ok else 'MALFORMED'}, {series}")
+    if not ok:
+        print("\nverdict: something replied, but not a seismograph.")
+        print("  Another service is on this port, or the modem is in a mode")
+        print("  that injects its own text (check Quiet Mode / AT echo).")
+        print(f"  first bytes: {raw[:16].hex(' ')}")
+        sock.close()
+        return 6
+
+    # ── 3. identity + state ───────────────────────────────────────────────
+    for n, (sub, label) in enumerate(((0x15, "serial"), (0x49, "state")), start=4):
+        try:
+            r, dt = exchange(sock, sub, timeout)
+            d = destuff(r)[5:]
+            if sub == 0x15:
+                # serial is a null-terminated run; a further field follows it
+                serial = bytes(d[11:]).split(b"\x00")[0]
+                step(n, label, serial.decode("ascii", "replace") or "(empty)")
+            else:
+                step(n, label, "MONITORING" if len(d) > 11 and d[11] else "idle")
+        except OSError:
+            step(n, label, "no reply")
+
+    sock.close()
+    print("\nverdict: the unit is alive and answering.")
+    print("  If THOR still shows it disconnected, the fault is in THOR, not")
+    print("  the network or the device.")
+    return 0
+
+
+def slots(host: str, port: int, n: int, timeout: float) -> None:
+    print(f"\nopening {n} simultaneous connections to {host}:{port}\n")
+    held = []
+    for i in range(n):
+        try:
+            s = socket.create_connection((host, port), timeout=timeout)
+            held.append(s)
+            step(i + 1, f"connection {i+1}", "accepted")
+        except ConnectionRefusedError:
+            step(i + 1, f"connection {i+1}", "REFUSED")
+        except (TimeoutError, socket.timeout):
+            step(i + 1, f"connection {i+1}", "timed out")
+        except OSError as e:
+            step(i + 1, f"connection {i+1}", f"failed: {e}")
+    print(f"\n{len(held)} of {n} accepted.")
+    if len(held) == 1:
+        print("  Single-slot behaviour confirmed — this far end serves ONE")
+        print("  session at a time.  A connection that is never closed takes")
+        print("  the unit offline until the idle timeout frees the slot.")
+    for s in held:
+        s.close()
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("target", help="host:port, e.g. 63.45.161.30:9034")
+    ap.add_argument("--timeout", type=float, default=10.0)
+    ap.add_argument("--slots", type=int, metavar="N",
+                    help="open N simultaneous connections to test single-slot behaviour")
+    a = ap.parse_args()
+    host, _, port = a.target.rpartition(":")
+    if not host:
+        ap.error("target must be host:port")
+    if a.slots:
+        slots(host, int(port), a.slots, a.timeout)
+        return 0
+    return probe(host, int(port), a.timeout)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scratch/fake_unit.py

@@ -0,0 +1,33 @@
+"""Pretend to be a Micromate on a serial port: log what arrives, reply to POLL.
+
+Proves the modem's return path (serial -> TCP) independently of the real unit.
+"""
+import os, select, sys, termios, time
+
+path, baud = sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 115200
+B = {9600: termios.B9600, 38400: termios.B38400, 115200: termios.B115200}[baud]
+fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
+a = termios.tcgetattr(fd)
+a[0] = a[1] = a[3] = 0
+a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL
+a[4] = a[5] = B
+a[6] = list(a[6]); a[6][termios.VMIN] = 0; a[6][termios.VTIME] = 0
+termios.tcsetattr(fd, termios.TCSANOW, a)
+termios.tcflush(fd, termios.TCIOFLUSH)
+
+# A real POLL probe reply, captured from UM12947 on 2026-09-24.
+REPLY = bytes.fromhex("0200c5a4000000000000300000000000000099") + b"\x03"
+
+print(f"fake unit on {path} @ {baud}; will answer any inbound frame", flush=True)
+while True:
+    r, _, _ = select.select([fd], [], [], 1.0)
+    if not r:
+        continue
+    data = os.read(fd, 4096)
+    if not data:
+        continue
+    ts = time.strftime("%H:%M:%S")
+    print(f"{ts}  IN  {len(data):3} B  {data.hex(' ')}", flush=True)
+    time.sleep(0.02)
+    os.write(fd, REPLY)
+    print(f"{ts}  OUT {len(REPLY):3} B  {REPLY.hex(' ')}  <- canned POLL reply", flush=True)

scratch/mm_frame_parse.py

@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+"""
+mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
+raw capture pair.
+
+Why this exists
+---------------
+`minimateplus.framing.S3FrameParser` cannot see Micromate traffic.  It locates
+frames by scanning for `DLE STX`, and a Micromate response has **no leading
+DLE** — it starts at a bare `STX`.  It also expects `payload[1] == 0x10`, where
+the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
+
+The practical consequence, seen on the 9-24-26 setup-push capture: the
+Blastware-side requests parse fine (Thor emits Series III request frames), but
+**every device response is silently dropped or mis-framed** — so a capture that
+actually contains 12 acked writes looks like 12 unanswered requests.
+
+Destuffing
+----------
+One rule covers both directions: after the leading doubled `BW_CMD`, every
+`10 XX` pair on the wire destuffs to `XX`.  That includes `10 03` — Thor
+escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
+exactly as Blastware does.
+
+That rule was chosen by evidence, not assumption: of the four candidates tried
+against the 9-24-26 capture's four data-carrying write frames, it is the only
+one under which all four checksums validate.  See
+`docs/micromate_protocol_reference.md` → *The write path*.
+
+Usage
+-----
+    python scratch/mm_frame_parse.py <capture-dir>
+    python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
+    python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
+
+# Request SUB -> short name.  Series III names where they carry over; the
+# Series IV additions are marked.
+SUBNAME = {
+    0x01: "DEVICE_INFO",
+    0x06: "STORAGE_RANGE",
+    0x08: "EVENT_INDEX",
+    0x0A: "WAVEFORM_HDR",
+    0x0C: "WAVEFORM_REC",
+    0x15: "SERIAL",
+    0x1A: "COMPLIANCE_CFG",
+    0x1C: "MONITOR_STATUS",
+    0x1E: "EVENT_HDR",
+    0x2C: "CALLHOME_CFG",
+    0x2E: "TRIGGER_CFG_READ",     # Series IV
+    0x3E: "OPERATOR",
+    0x41: "SETUP_NAME_READ",      # Series IV
+    0x5A: "BULK_DOWNLOAD",
+    0x5B: "POLL",
+    0x68: "EVENT_INDEX_WRITE",
+    0x69: "WAVEFORM_WRITE",
+    0x71: "COMPLIANCE_WRITE",
+    0x72: "CONFIRM_A",
+    0x73: "CONFIRM_B",
+    0x74: "CONFIRM_C",
+    0x82: "TRIGGER_WRITE",
+    0x83: "TRIGGER_CONFIRM",
+    0xDA: "SETUP_FILE_DECL",      # Series IV — names the target .MMB
+    0xFE: "FULL_CFG",
+}
+
+
+def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
+    """Destuff one frame starting at `start`.
+
+    Returns (payload, checksum, index_of_terminating_ETX).  `payload` excludes
+    the trailing checksum byte.  A request frame opens `ACK STX 10 10`; a
+    response opens with a bare `STX`.
+    """
+    i = start + (2 if is_request else 1)
+    out = bytearray()
+    if is_request:
+        # The doubled BW_CMD is the one guaranteed stuffed byte.
+        if blob[i : i + 2] != bytes([DLE, DLE]):
+            raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
+        out.append(DLE)
+        i += 2
+    while i < len(blob):
+        b = blob[i]
+        if b == DLE and i + 1 < len(blob):
+            out.append(blob[i + 1])
+            i += 2
+            continue
+        if b == ETX:
+            break
+        out.append(b)
+        i += 1
+    if len(out) < 2:
+        raise ValueError(f"@0x{start:04x}: frame too short")
+    return bytes(out[:-1]), out[-1], i
+
+
+def frames(blob: bytes, *, is_request: bool):
+    """Yield (offset, payload, chk, checksum_kind)."""
+    i, n = 0, len(blob)
+    while i < n:
+        if is_request:
+            if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
+                i += 1
+                continue
+        elif blob[i] != STX:
+            i += 1
+            continue
+        try:
+            payload, chk, end = destuff(blob, i, is_request=is_request)
+        except ValueError:
+            i += 1
+            continue
+        sum8 = sum(payload) & 0xFF
+        dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
+        if sum8 == chk:
+            kind = "SUM8"
+        elif dle_aware == chk:
+            kind = "DLE-aware"
+        else:
+            kind = "BAD"
+        yield i, payload, chk, kind
+        i = end + 1
+
+
+def describe(payload: bytes, is_request: bool) -> str:
+    if len(payload) < 3:
+        return "??"
+    sub = payload[2]
+    if is_request:
+        return SUBNAME.get(sub, f"SUB_{sub:02X}")
+    req = 0xFF - sub
+    return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
+
+
+def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
+    blob = path.read_bytes()
+    side = "Thor" if is_request else "unit"
+    print(f"== {side:4}  {path.name}  ({len(blob)} bytes)")
+    n_bad = 0
+    for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
+        if kind == "BAD":
+            n_bad += 1
+        sub = p[2] if len(p) > 2 else -1
+        flags = p[1] if len(p) > 1 else -1
+        # Requests carry offset at payload[4:6]; responses page at [3:5].
+        word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
+        data = len(p) - 16 if is_request else max(len(p) - 5, 0)
+        print(
+            f"  [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
+            f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
+            f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
+        )
+        if dump_sub is not None and sub == dump_sub:
+            body = p[16:] if is_request else p[5:]
+            print(f"       ---- data ({len(body)} bytes) ----")
+            for o in range(0, len(body), 16):
+                chunk = body[o : o + 16]
+                txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
+                print(f"       {o:06x}  {chunk.hex(' '):<47}  |{txt}|")
+    print(f"  -- {idx + 1} frames, {n_bad} bad checksum\n")
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("paths", nargs="+",
+                    help="a capture directory, or raw_bw.bin and raw_s3.bin")
+    ap.add_argument("--dump", default=None,
+                    help="hex-dump the data section of this SUB (e.g. 0x71)")
+    args = ap.parse_args()
+
+    dump_sub = int(args.dump, 0) if args.dump else None
+
+    if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
+        d = Path(args.paths[0])
+        bw = sorted(d.glob("raw_bw_*.bin"))
+        s3 = sorted(d.glob("raw_s3_*.bin"))
+        if not bw or not s3:
+            print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
+            return 2
+        pairs = [(bw[0], True), (s3[0], False)]
+    elif len(args.paths) == 2:
+        pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
+    else:
+        ap.error("pass a capture directory, or exactly two .bin files")
+
+    for path, is_request in pairs:
+        report(path, is_request=is_request, dump_sub=dump_sub)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scratch/socat_log_split.py

@@ -0,0 +1,119 @@
+#!/usr/bin/env python3
+"""
+socat_log_split.py — recover a capture pair from a `socat -x` relay log.
+
+Why this exists
+---------------
+The bench relay that puts Thor in front of a USB-attached Micromate is:
+
+    socat -d -d -x TCP-LISTEN:12345,reuseaddr,fork /dev/ttyACM0,raw,echo=0,b115200 \
+        > ~/mm-captures/socat_<ts>.log 2>&1
+
+`-x` makes socat hex-dump every byte it forwards, in both directions, with
+timestamps.  That log is therefore a **complete second copy of every capture**
+taken through the relay — independent of whether seismo_lab was recording.
+
+On 2026-09-25 that mattered: a capture's `.bin` files never made it off the
+Windows machine, and the session was rebuilt from this log instead.  When the
+real bins turned up later, the reconstruction was **byte-for-byte identical in
+both directions** (3,595 and 4,004 bytes).  So this is a validated fallback, not
+a lossy approximation.
+
+Log format
+----------
+```
+> 2026/09/25 00:30:35.000276659  length=21 from=0 to=20
+ 41 02 10 10 00 5b 00 00 30 00 ...
+2026/09/25 00:30:35 socat[32190] N write(5, 0x..., 21) completed
+< 2026/09/25 00:30:35.000384100  length=64 from=0 to=63
+ 02 00 c5 a4 00 00 30 00 ...
+```
+
+`>` is data heading toward the serial device (Thor → unit).  `<` is data coming
+back (unit → Thor).  Hex lines are space-separated and indented; socat's own
+status lines start with a date and carry no payload.
+
+Usage
+-----
+    # whole log
+    python scratch/socat_log_split.py socat_20260924_181248.log --out-dir ./recovered
+
+    # one session — line numbers from the "accepting connection" markers
+    grep -n "accepting connection" socat_*.log
+    python scratch/socat_log_split.py socat_*.log --from-line 919 --out-dir ./recovered
+
+Then parse the result as usual:
+
+    python scratch/mm_frame_parse.py recovered/raw_bw.bin recovered/raw_s3.bin
+
+⚠ A log spanning several sessions concatenates them.  Split by line number using
+the `accepting connection` markers, or the frame walk will run sessions together.
+"""
+
+from __future__ import annotations
+
+import argparse
+import re
+from pathlib import Path
+
+_HEX = re.compile(r"\A[0-9a-f]{2}\Z")
+_SOCAT_STATUS = re.compile(r"\A\d{4}/\d{2}/\d{2}")
+
+
+def split(lines) -> tuple[bytes, bytes]:
+    """Return (to_device, from_device) byte streams."""
+    to_dev, from_dev = bytearray(), bytearray()
+    cur = None
+    for line in lines:
+        if line.startswith(">"):
+            cur = to_dev
+            continue
+        if line.startswith("<"):
+            cur = from_dev
+            continue
+        if _SOCAT_STATUS.match(line):
+            # socat's own status line ends the current dump block.
+            cur = None
+            continue
+        if cur is None or not line.startswith(" "):
+            continue
+        toks = line.split()
+        if toks and all(_HEX.match(t) for t in toks):
+            cur.extend(int(t, 16) for t in toks)
+    return bytes(to_dev), bytes(from_dev)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    ap.add_argument("log", help="a socat -x log file")
+    ap.add_argument("--out-dir", default=".", help="where to write the .bin pair")
+    ap.add_argument("--from-line", type=int, default=1,
+                    help="first log line to read (1-based) — use the "
+                         "'accepting connection' marker of the session you want")
+    ap.add_argument("--to-line", type=int, default=None,
+                    help="last log line to read (1-based, inclusive)")
+    ap.add_argument("--prefix", default="raw", help="output basename prefix")
+    args = ap.parse_args()
+
+    lines = Path(args.log).read_text(errors="replace").splitlines()
+    lo = max(args.from_line - 1, 0)
+    hi = args.to_line if args.to_line is not None else len(lines)
+    to_dev, from_dev = split(lines[lo:hi])
+
+    out = Path(args.out_dir)
+    out.mkdir(parents=True, exist_ok=True)
+    bw = out / f"{args.prefix}_bw.bin"
+    s3 = out / f"{args.prefix}_s3.bin"
+    bw.write_bytes(to_dev)
+    s3.write_bytes(from_dev)
+    print(f"Thor -> unit  {len(to_dev):>7} bytes  {bw}")
+    print(f"unit -> Thor  {len(from_dev):>7} bytes  {s3}")
+    if not to_dev or not from_dev:
+        print("⚠ one direction is empty — check --from-line / --to-line")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())