docs(offset): CONFIRMED — BE12599 is a geophone-assembly fault, not the recorder

The first mechanism this investigation has established physically rather than inferred. Brian pulled BE12599 and bench-tested it; a geophone swap settles it. Same recorder, twenty minutes apart: Vert overswing 2026.0 with its own geophone, 3.4 with BE9888's known-good one. Frequency 2.2 -> 7.4 Hz. All three channels Passed, no offsets, no self-triggering, and a clean damped impulse with broadband content. The recorder is functional and merely out of calibration. A bench control -- trigger forced by slapping the microphone, geophone untouched -- shows Vert producing 0.19 in/s of bipolar wander with no mechanical input at all. The fault is continuous, not episodic. All three channels are affected: Tran and Long sit on a small DC while their coils still pass the swing test, Vert has lost damping entirely. That is leakage on several contacts plus one gone open, not three elements failing in sequence. The consequence that reaches past this unit: an autozero adjusts the RECORDER's zero reference and cannot fix a geophone. If most offsets are geophone-assembly faults, a re-zero succeeds only on the minority where the recorder really is at fault -- which is exactly the ~10% rate recorded in section 5 and unexplained since August. Procedure change: swap the geophone first. Corrects 8e's title, which called it a connector rather than a geophone: the swap narrows the fault to the assembly, and element / cable / connector remain open within it. And reweights the cause -- the geophone is buried, not in the enclosure with the nest, and the unit sat ~6 ft from I-80 WB, so road salt now outranks the mice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
2026-09-23 02:57:34 +00:00
9 changed files with 121 additions and 4020 deletions
@@ -10,38 +10,10 @@ pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
 ---
-## Where things stand (updated 2026-09-26)
+## Where things stand (updated 2026-08-28)
 Read this first when picking the project back up.
 - **The Series-4 LIVE wire protocol is reverse-engineered end to end
  (2026-09-25).** `docs/micromate_protocol_reference.md` is the Series-4
  Rosetta Stone, sibling to `instantel_protocol_reference.md`. **A Micromate
  answers Series III command frames** — three framing differences: responses
  have **no leading `DLE`** (a bare `STX`), `payload[1]` is `0xC5` (Blastware
  firmware) or `0x03` (Thor firmware) rather than `0x10`, and the data length
  is a **uint16 BE at `payload[8:10]`** (as a byte it under-reads `SUB 0x1A`
  by 47x). Read path, event chain, setups, scheduler, monitoring control and
  per-event delete are all mapped; **the inbound call-home session is the only
  protocol unknown left.**
  ⚠ **No command has ever been originated against a unit by this project.**
  Every write was performed by THOR while we recorded. That line is worth
  keeping.
  ⚠ `micromate/` still has **no live client** — it is codec-only. The
  `minimateplus/` stack (transport/framing/protocol/client) has no Series-4
  counterpart yet. `minimateplus.transport` is protocol-agnostic and reusable.
 - **Bench tooling for device diagnosis (2026-09-25).** `bridges/mm_probe.py`
  distinguishes the four faults THOR reports identically as "disconnected"
  (refused / connect timeout / **connected but no reply** / replied) and names
  what to try next. `bridges/mm_link.py` is a stand-in for a cellular modem
  with a decoded log and fault injection. `scratch/mm_frame_parse.py` exists
  because **`S3FrameParser` cannot see Micromate responses at all** — it scans
  for `DLE+STX`, which never appears in Series-4 traffic.
 - **A Micromate's USB-A host port drives FTDI and CDC-ACM only** — no Prolific,
  in either firmware line. TMI buys both Sabrent (FTDI) and Benfei (PL2303)
  cables and they are indistinguishable by eye. A PL2303 cable leaves a unit
  with **no working modem port at all**; identify by `lsusb` VID, `0403` vs
  `067b`. This accounted for a unit that could not be deployed.
 - **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
  archive decodes **14,338 / 14,338** paired files exactly against their
  preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
@@ -120,9 +92,6 @@ Read this first when picking the project back up.
  **v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
  fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
  files are archive-only and were never ingested).
  ✅ **The v0.30.0 Series-4 backfill HAS been run on prod (2026-09-25).** Every
  stored Series-4 geophone value was ~3.3% low until then; that is corrected and
  the job does not need repeating.
 - **The "offset" hardware fault has its own journal** --
  `docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
  **persistent** — it stays until the geophone is serviced. Detect it with
@@ -134,18 +103,7 @@ Read this first when picking the project back up.
  `SUB 0x0E` (unimplemented), which may carry those very numbers.
-When new information about a protocol is discovered, record it in the matching
+When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
 reference **in addition to** this document:
 | series | document |
 |---|---|
 | Series III (MiniMate Plus / BlastMate) | `docs/instantel_protocol_reference.md` |
 | **Series IV (Micromate / THOR)** | **`docs/micromate_protocol_reference.md`** |
 | Thor IDF file format | `docs/idf_protocol_reference.md` |
 Both protocol references carry retractions in place rather than deleting what
 turned out to be wrong — that convention has already saved re-deriving the same
 mistakes twice, so keep it.
 ---
@@ -318,15 +276,8 @@ minimateplus/         ← Python client library (primary focus)
 sfm/server.py         ← FastAPI REST server exposing device data over HTTP
 seismo_lab.py         ← Tkinter GUI (Bridge + Analyzer + Console tabs)
 bridges/
  mm_probe.py         ←   name the fault behind a dead unit (4 verdicts, read-only)
  mm_link.py          ←   bench stand-in for a cellular modem, with fault injection
  ach_mitm.py         ←   TCP relay for recording a Series-3 ACH session
 docs/
-  instantel_protocol_reference.md  ← Series III protocol spec ("the Rosetta Stone")
+  instantel_protocol_reference.md  ← reverse-engineered protocol spec ("the Rosetta Stone")
  micromate_protocol_reference.md  ← Series IV protocol spec + THOR's measured behaviour
  idf_protocol_reference.md        ← Thor IDF file format
 CHANGELOG.md          ← version history
 ```
@@ -1,338 +0,0 @@
 #!/usr/bin/env python3
 """
 mm_link.py — a "perfect modem" between THOR and a Micromate, with a readable
 log and deliberate fault injection.
 Why
 ---
 THOR gives almost no visibility into a connection: a refresh button, two poll
 intervals, and no way to see whether a check succeeded, timed out, or was never
 sent.  When a unit "won't stay connected" there is nothing to look at.
 This sits where the cellular modem would sit and answers the question directly:
  * **What is THOR actually doing?**  Every frame is decoded and timestamped —
    `POLL`, `MONITOR_STATUS`, `SETUP_NAME_READ` — not a hex dump.
  * **Is it even trying?**  Silence is visible: the log shows gaps.
  * **How does it behave when the link misbehaves?**  Faults can be injected on
    demand, which a real cell link will not do on cue.
 Point THOR at this host and port exactly as if it were a modem (Communication:
 TCP, IP: <this host>, Port: <--listen>).
 Fault injection
 ---------------
 Write a mode into the control file (default `mm_link.ctl`) and it takes effect
 on the next byte:
    echo pass      > mm_link.ctl    # normal relay
    echo blackhole > mm_link.ctl    # TCP stays up, bytes are swallowed
    echo drop      > mm_link.ctl    # close the connection abruptly (RST-ish)
    echo delay:2.0 > mm_link.ctl    # forward, but 2 s late in both directions
    echo onewaydev > mm_link.ctl    # THOR->unit passes, unit->THOR is swallowed
 **`blackhole` is the one that matters.**  It reproduces the classic cellular
 failure: the socket is still open as far as both ends are concerned, but nothing
 crosses.  A client that relies on TCP to tell it the peer is gone will sit there
 until the OS keepalive fires — which by default is about two hours.
 Usage
 -----
    python3 bridges/mm_link.py --serial /dev/ttyACM0 --baud 115200 \\
        --listen 12345 --logdir ~/mm-captures
 Writes, per session:
    <logdir>/mmlink_<ts>/session.log   decoded, timestamped, human-readable
    <logdir>/mmlink_<ts>/raw_bw.bin    THOR -> unit, raw
    <logdir>/mmlink_<ts>/raw_s3.bin    unit -> THOR, raw
 The raw pair loads straight into `scratch/mm_frame_parse.py`.
 """
 from __future__ import annotations
 import argparse
 import datetime
 import os
 import socket
 import sys
 import threading
 import time
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scratch"))
 try:
    from mm_frame_parse import SUBNAME, destuff  # noqa: F401
 except Exception:                                 # pragma: no cover
    SUBNAME = {}
 import errno
 import select
 import termios
 DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
 _BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
         57600: termios.B57600, 115200: termios.B115200}
 class SerialPort:
    """Minimal raw serial port on stdlib termios — no pyserial dependency.
    The bench hosts are whatever is to hand; requiring a pip install on someone
    else's machine is a poor trade for the ~30 lines this saves.
    """
    def __init__(self, path: str, baud: int):
        if baud not in _BAUD:
            raise ValueError(f"unsupported baud {baud}; pick one of {sorted(_BAUD)}")
        self.fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
        a = termios.tcgetattr(self.fd)
        a[0] = 0                                             # iflag: no translation
        a[1] = 0                                             # oflag: raw
        a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL  # cflag: 8N1, ignore modem lines
        a[3] = 0                                             # lflag: non-canonical, no echo
        a[4] = a[5] = _BAUD[baud]
        a[6] = list(a[6])
        a[6][termios.VMIN] = 0
        a[6][termios.VTIME] = 0
        termios.tcsetattr(self.fd, termios.TCSANOW, a)
        termios.tcflush(self.fd, termios.TCIOFLUSH)
    def read(self, n: int) -> bytes:
        r, _, _ = select.select([self.fd], [], [], 0.2)
        if not r:
            return b""
        try:
            return os.read(self.fd, n)
        except OSError as e:
            if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
                return b""
            raise
    def write(self, data: bytes) -> None:
        while data:
            try:
                data = data[os.write(self.fd, data):]
            except OSError as e:
                if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
                    select.select([], [self.fd], [], 0.2)
                    continue
                raise
    def close(self) -> None:
        try:
            os.close(self.fd)
        except OSError:
            pass
 def name_of(sub: int, is_request: bool) -> str:
    if is_request:
        return SUBNAME.get(sub, f"SUB_{sub:02X}")
    return "rsp " + SUBNAME.get(0xFF - sub, f"SUB_{0xFF - sub:02X}")
 class FrameSniffer:
    """Accumulate bytes and report complete frames, without altering the stream."""
    def __init__(self, is_request: bool):
        self.is_request = is_request
        self.buf = bytearray()
    def feed(self, data: bytes):
        """Yield (sub, payload_len) for each complete frame seen."""
        self.buf.extend(data)
        while True:
            start = -1
            for i, b in enumerate(self.buf):
                if self.is_request and b == ACK and i + 1 < len(self.buf) and self.buf[i + 1] == STX:
                    start = i
                    break
                if not self.is_request and b == STX:
                    start = i
                    break
            if start < 0:
                if len(self.buf) > 8192:
                    del self.buf[:-16]
                return
            j = start + (2 if self.is_request else 1)
            end = -1
            while j < len(self.buf):
                if self.buf[j] == DLE and j + 1 < len(self.buf):
                    j += 2
                    continue
                if self.buf[j] == ETX:
                    end = j
                    break
                j += 1
            if end < 0:
                return                      # wait for more bytes
            body = self.buf[start:end + 1]
            del self.buf[:end + 1]
            # SUB sits at a fixed spot past the leading framing -- but it is
            # DLE-escaped when its own value is 0x02/0x03/0x04/0x10, so a raw
            # read reports 0x10 for those.  SUB 0x02 was being logged as
            # "SUB_10" until this was handled.
            off = 5 if self.is_request else 3
            if len(body) > off:
                sub = body[off]
                if sub == DLE and len(body) > off + 1:
                    sub = body[off + 1]
                yield sub, len(body)
 class Link:
    def __init__(self, args):
        self.args = args
        self.mode = "pass"
        self.delay = 0.0
        self.ctl = Path(args.control)
        self.session: Path | None = None
        self.log_fh = None
        self.raw = {}
        self.t0 = time.time()
        self.counts = {}
    # ── logging ────────────────────────────────────────────────────────────
    def open_session(self):
        ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
        self.session = Path(self.args.logdir) / f"mmlink_{ts}"
        self.session.mkdir(parents=True, exist_ok=True)
        self.log_fh = open(self.session / "session.log", "a", buffering=1)
        self.raw = {
            "bw": open(self.session / "raw_bw.bin", "ab"),
            "s3": open(self.session / "raw_s3.bin", "ab"),
        }
        self.say(f"=== session {ts} — serial {self.args.serial} @ {self.args.baud} ===")
    def say(self, text: str):
        line = f"{datetime.datetime.now().strftime('%H:%M:%S.%f')[:-3]}  {text}"
        print(line, flush=True)
        if self.log_fh:
            self.log_fh.write(line + "\n")
    # ── control file ───────────────────────────────────────────────────────
    def poll_control(self):
        while True:
            try:
                if self.ctl.exists():
                    want = self.ctl.read_text().strip().lower()
                    if want.startswith("delay:"):
                        d = float(want.split(":", 1)[1])
                        if ("delay", d) != (self.mode, self.delay):
                            self.mode, self.delay = "delay", d
                            self.say(f"*** MODE -> delay {d}s ***")
                    elif want and want != self.mode:
                        self.mode, self.delay = want, 0.0
                        self.say(f"*** MODE -> {want} ***")
            except Exception:
                pass
            time.sleep(0.25)
    # ── the relay ──────────────────────────────────────────────────────────
    def pump(self, src, dst, tag: str, is_request: bool, stop: threading.Event):
        sniff = FrameSniffer(is_request)
        arrow = "THOR->unit" if is_request else "unit->THOR"
        last = time.time()
        while not stop.is_set():
            timed_out = False
            try:
                data = src.recv(4096) if isinstance(src, socket.socket) else src.read(4096)
            except TimeoutError:
                timed_out = True
                # socket.timeout subclasses OSError, so it MUST be caught first.
                # Treating it as a dead socket closes the connection after 200 ms
                # of quiet -- which is exactly what `blackhole` produces, so the
                # relay killed the link it was supposed to be faking a fault on.
                data = b""
            except OSError:
                break
            if isinstance(src, socket.socket) and data == b"" and not timed_out:
                self.say(f"{arrow}: peer closed the connection")
                break
            if not data:
                if time.time() - last > self.args.quiet_after and self.counts:
                    self.say(f"--- {self.args.quiet_after:.0f}s with no traffic ---")
                    last = time.time()
                continue
            last = time.time()
            self.raw[tag].write(data)
            self.raw[tag].flush()
            for sub, ln in sniff.feed(data):
                label = name_of(sub, is_request)
                self.counts[label] = self.counts.get(label, 0) + 1
                self.say(f"{arrow}  {label:<20} ({ln} B)"
                         + ("" if self.mode == "pass" else f"   [mode={self.mode}]"))
            mode = self.mode
            if mode == "drop":
                self.say(f"{arrow}: DROPPING the connection (fault injection)")
                stop.set()
                break
            if mode == "blackhole":
                continue                      # swallow, keep the socket open
            if mode == "onewaydev" and not is_request:
                continue                      # unit's replies never reach THOR
            if mode == "delay" and self.delay:
                time.sleep(self.delay)
            try:
                if isinstance(dst, socket.socket):
                    dst.sendall(data)
                else:
                    dst.write(data)
            except OSError:
                break
        stop.set()
    def serve(self):
        srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        srv.bind(("0.0.0.0", self.args.listen))
        srv.listen(5)
        self.open_session()
        self.say(f"listening on 0.0.0.0:{self.args.listen}   control file: {self.ctl}")
        self.say("point THOR at this host/port as Communication=TCP")
        threading.Thread(target=self.poll_control, daemon=True).start()
        while True:
            conn, addr = srv.accept()
            conn.settimeout(0.2)
            self.say(f"+++ THOR connected from {addr[0]}:{addr[1]} +++")
            try:
                ser = SerialPort(self.args.serial, self.args.baud)
            except OSError as e:
                self.say(f"!!! cannot open {self.args.serial}: {e}")
                conn.close()
                continue
            stop = threading.Event()
            ts = [
                threading.Thread(target=self.pump, args=(conn, ser, "bw", True, stop), daemon=True),
                threading.Thread(target=self.pump, args=(ser, conn, "s3", False, stop), daemon=True),
            ]
            for t in ts:
                t.start()
            for t in ts:
                t.join()
            conn.close()
            ser.close()
            summary = ", ".join(f"{k}x{v}" for k, v in sorted(self.counts.items()))
            self.say(f"--- connection closed.  frames this session: {summary or 'none'} ---")
 def main():
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--serial", default="/dev/ttyACM0")
    ap.add_argument("--baud", type=int, default=115200)
    ap.add_argument("--listen", type=int, default=12345)
    ap.add_argument("--logdir", default=os.path.expanduser("~/mm-captures"))
    ap.add_argument("--control", default="mm_link.ctl")
    ap.add_argument("--quiet-after", type=float, default=30.0,
                    help="log a marker after this many seconds of silence")
    Link(ap.parse_args()).serve()
 if __name__ == "__main__":
    main()
@@ -1,239 +0,0 @@
 #!/usr/bin/env python3
 """
 mm_probe.py — answer "why can't we reach this unit?" in one command.
 THOR reports a failed connection as "disconnected" and nothing else.  That single
 word covers at least four completely different faults with four different fixes,
 and telling them apart is the difference between a modem reboot and a site visit:
  * **connection refused**   something answered and said no — wrong port, or the
                             modem is refusing a further session
  * **connect timed out**    nothing answered at all — trusted-IP whitelist,
                             firewall, or the modem is off the network
  * **connected, no reply**  the MODEM answered but the unit did not.  The TCP
                             path is fine; the modem is not forwarding to serial.
                             This is the signature of a wedged transparent-TCP
                             session, and it is the one THOR cannot distinguish
                             from any of the others
  * **replied**              the unit is alive; the problem is upstream software
 Read-only.  It sends `POLL`, then optionally `SERIAL` and the state read — the
 same three commands THOR's own connection check uses — and never writes.
 Usage
 -----
    python3 bridges/mm_probe.py 63.45.161.30:9034
    python3 bridges/mm_probe.py 10.0.0.8:12345 --timeout 5
    python3 bridges/mm_probe.py <host:port> --slots 3
 `--slots N` opens N connections at once and reports how many the far end accepts.
 A transparent-TCP modem typically serves **one** session; if the first succeeds
 and the rest are refused or hang, that confirms the single-slot behaviour and
 explains why a leaked session takes a unit offline until the slot frees.
 Works for both series: a Series III reply opens `DLE STX`, a Micromate reply
 opens with a bare `STX`, so the probe also tells you which one answered.
 """
 from __future__ import annotations
 import argparse
 import socket
 import sys
 import time
 from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
 from minimateplus.framing import build_bw_frame  # noqa: E402
 DLE, STX, ETX = 0x10, 0x02, 0x03
 def destuff(raw: bytes) -> bytes:
    """Strip framing and DLE escapes; return the payload without its checksum."""
    i = 1 if raw and raw[0] == STX else (2 if len(raw) > 1 and raw[1] == STX else 0)
    out = bytearray()
    while i < len(raw):
        b = raw[i]
        if b == DLE and i + 1 < len(raw):
            out.append(raw[i + 1])
            i += 2
            continue
        if b == ETX:
            break
        out.append(b)
        i += 1
    return bytes(out[:-1]) if len(out) > 1 else b""
 # Reads are two-step on Series III: a probe at offset 0, then a data read at the
 # block's length.  THOR sends these offsets, and they also work on a Micromate.
 OFFSETS = {0x5B: 0x0030, 0x15: 0x000A, 0x49: 0xFFFF}
 def exchange(sock: socket.socket, sub: int, timeout: float) -> tuple[bytes, float]:
    sock.sendall(build_bw_frame(sub, OFFSETS.get(sub, 0)))
    t0 = time.time()
    buf, deadline = b"", t0 + timeout
    sock.settimeout(0.3)
    while time.time() < deadline:
        try:
            chunk = sock.recv(4096)
            if not chunk:
                break
            buf += chunk
            if buf.endswith(bytes([ETX])) and len(buf) > 8:
                break
        except TimeoutError:
            continue
        except OSError:
            break
    return buf, time.time() - t0
 def step(n: int, label: str, result: str) -> None:
    print(f"  [{n}] {label:.<28} {result}")
 def probe(host: str, port: int, timeout: float) -> int:
    print(f"\ntarget {host}:{port}   (read-only: POLL, SERIAL, state)\n")
    # ── 1. TCP ────────────────────────────────────────────────────────────
    t0 = time.time()
    try:
        sock = socket.create_connection((host, port), timeout=timeout)
    except ConnectionRefusedError:
        step(1, "TCP connect", f"REFUSED after {1000*(time.time()-t0):.0f} ms")
        print("\nverdict: something answered and actively refused.")
        print("  Not a silent firewall drop — the host is reachable.")
        print("  Wrong port, the service is down, or the modem is refusing")
        print("  an additional session because its one slot is in use.")
        return 2
    except (TimeoutError, socket.timeout):
        step(1, "TCP connect", f"TIMED OUT after {time.time()-t0:.1f} s")
        print("\nverdict: nothing answered at all.")
        print("  A silent drop, which is what a trusted-IP whitelist looks")
        print("  like — it discards rather than refuses.  Check the modem's")
        print("  Trusted IPs (and note a VPN changes the IP you arrive from),")
        print("  the firewall, and whether the modem is on the network.")
        return 3
    except OSError as e:
        step(1, "TCP connect", f"FAILED: {e}")
        return 4
    step(1, "TCP connect", f"ok ({1000*(time.time()-t0):.0f} ms)")
    # ── 2. POLL ───────────────────────────────────────────────────────────
    try:
        raw, dt = exchange(sock, 0x5B, timeout)
    except OSError as e:
        step(2, "POLL", f"send failed: {e}")
        sock.close()
        return 4
    if not raw:
        step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
        print("\nverdict: the MODEM answered but the unit did not.")
        print("  TCP is fine end to end — something accepted the connection.")
        print("  What is missing is the serial side.  Two quite different")
        print("  causes produce this, and they are NOT distinguishable from")
        print("  here:")
        print("\n  1. SOMEONE ELSE HOLDS THE SESSION.  These modems bridge ONE")
        print("     TCP session to serial at a time.  A second connection is")
        print("     accepted and then simply not forwarded.  Confirmed 2026-09-26:")
        print("     with THOR connected this probe saw exactly this; the moment")
        print("     THOR disconnected the same probe returned the serial number.")
        print("     ** Check whether THOR (or anything else) has the unit first. **")
        print("\n  2. The serial path is genuinely broken — a stale session the")
        print("     modem never released, a cable the unit cannot enumerate, or")
        print("     a unit that is off.")
        print("\n  Try, in order:")
        print("    1. Disconnect any other client and re-probe.  If it answers,")
        print("       it was contention, not a fault.")
        print("    2. The cable's chipset.  A Micromate drives FTDI and CDC-ACM")
        print("       only — a Prolific PL2303 gives it no serial port at all.")
        print("       lsusb: FTDI is 0403, Prolific 067b.")
        print("    3. Power-cycle the UNIT with the cable attached (hold power")
        print("       5 s, through the two-stage prompt).  Its USB host rescans")
        print("       on cold boot; it may not on hot-swap.")
        print("    4. AirLink OS -> TCP Idle Timeout.  If 0/disabled, a stale")
        print("       session holds the slot indefinitely.  2 minutes is the")
        print("       value this project standardised on.")
        sock.close()
        return 5
    series = "Series III (DLE STX)" if raw[0] == DLE else "Micromate (bare STX)"
    step(2, "POLL", f"reply {len(raw)} B in {1000*dt:.0f} ms")
    p = destuff(raw)
    ok = len(p) > 3 and p[2] == 0xFF - 0x5B
    step(3, "frame", f"{'valid' if ok else 'MALFORMED'}, {series}")
    if not ok:
        print("\nverdict: something replied, but not a seismograph.")
        print("  Another service is on this port, or the modem is in a mode")
        print("  that injects its own text (check Quiet Mode / AT echo).")
        print(f"  first bytes: {raw[:16].hex(' ')}")
        sock.close()
        return 6
    # ── 3. identity + state ───────────────────────────────────────────────
    for n, (sub, label) in enumerate(((0x15, "serial"), (0x49, "state")), start=4):
        try:
            r, dt = exchange(sock, sub, timeout)
            d = destuff(r)[5:]
            if sub == 0x15:
                # serial is a null-terminated run; a further field follows it
                serial = bytes(d[11:]).split(b"\x00")[0]
                step(n, label, serial.decode("ascii", "replace") or "(empty)")
            else:
                step(n, label, "MONITORING" if len(d) > 11 and d[11] else "idle")
        except OSError:
            step(n, label, "no reply")
    sock.close()
    print("\nverdict: the unit is alive and answering.")
    print("  If THOR still shows it disconnected, the fault is in THOR, not")
    print("  the network or the device.")
    return 0
 def slots(host: str, port: int, n: int, timeout: float) -> None:
    print(f"\nopening {n} simultaneous connections to {host}:{port}\n")
    held = []
    for i in range(n):
        try:
            s = socket.create_connection((host, port), timeout=timeout)
            held.append(s)
            step(i + 1, f"connection {i+1}", "accepted")
        except ConnectionRefusedError:
            step(i + 1, f"connection {i+1}", "REFUSED")
        except (TimeoutError, socket.timeout):
            step(i + 1, f"connection {i+1}", "timed out")
        except OSError as e:
            step(i + 1, f"connection {i+1}", f"failed: {e}")
    print(f"\n{len(held)} of {n} accepted.")
    if len(held) == 1:
        print("  Single-slot behaviour confirmed — this far end serves ONE")
        print("  session at a time.  A connection that is never closed takes")
        print("  the unit offline until the idle timeout frees the slot.")
    for s in held:
        s.close()
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("target", help="host:port, e.g. 63.45.161.30:9034")
    ap.add_argument("--timeout", type=float, default=10.0)
    ap.add_argument("--slots", type=int, metavar="N",
                    help="open N simultaneous connections to test single-slot behaviour")
    a = ap.parse_args()
    host, _, port = a.target.rpartition(":")
    if not host:
        ap.error("target must be host:port")
    if a.slots:
        slots(host, int(port), a.slots, a.timeout)
        return 0
    return probe(host, int(port), a.timeout)
 if __name__ == "__main__":
    raise SystemExit(main())
@@ -1,291 +0,0 @@
 # Spec — a live client for Series IV (Micromate)
 Drafted 2026-09-26, ahead of implementation.  The protocol work is finished; this
 is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
 run.
 **Read that document first.**  Everything here assumes it, and every constant
 below is sourced from it rather than restated with justification.
 ---
 ## Goal and scope
 `micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
 writers.  There is no way to talk to a unit.  This adds the live half, mirroring
 `minimateplus/`.
 **In scope, first pass:**
 - connect over TCP (a field modem) or serial/USB (a bench unit)
 - identify a unit, read its state, clock, memory and setups
 - walk the event chain and download events
 - return `Event` objects the existing codec already understands
 **Explicitly out of scope, first pass:**
 - ⚠ **Any write.**  Setups, schedules, call-home config, monitoring start/stop,
  and per-event delete are all mapped, and none of them will be implemented
  here.  **No command has ever been originated against a unit by this project**
  — every write observed was performed by THOR while we recorded.  Keeping that
  true through the read client is deliberate: it means the first thing we ever
  send to a customer's instrument is a decision someone made on purpose, not a
  side effect of a client that happened to grow a method.
 - the inbound call-home session — still the one protocol unknown
 ---
 ## Layout
 ```
 micromate/
  framing.py     NEW   frame building, response parsing, checksum
  protocol.py    NEW   one method per wire command, returns raw payloads
  client.py      NEW   high-level API, returns models
  idf_file.py    (existing — decodes what 0x5A returns, unchanged)
  models.py      (existing — extend, do not fork)
 ```
 **Transport is reused, not rewritten.**  `minimateplus/transport.py` is
 byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
 `TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
 habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller.  Import it.
 ⚠ Do **not** import `minimateplus.framing`.  The two framings differ in ways
 that look small and are not, and a shared module would accumulate `if series ==`
 branches until neither case is readable.
 ---
 ## `micromate/framing.py`
 ### Requests
 Series IV accepts Series III request frames unmodified.  The simplest correct
 implementation re-exports the builder rather than duplicating it:
 ```python
 from minimateplus.framing import build_bw_frame   # requests are identical
 ```
 ⚠ One open question, flagged in the protocol reference and **not** settled:
 whether `0x10` bytes inside request *params* need stuffing.  No probe we sent
 carried one.  Until it is settled, assert on it rather than guessing:
 ```python
 def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
    if 0x10 in params:
        raise NotImplementedError(
            "params containing 0x10 — stuffing rule unconfirmed; see "
            "micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'"
        )
    return build_bw_frame(sub, offset, params)
 ```
 That turns an unknown into a loud failure instead of a corrupt frame.
 ### Responses — where Series III's parser cannot follow
 | | Series III | Micromate |
 |---|---|---|
 | frame start | `DLE STX` | **bare `STX`** |
 | `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
 | destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |
 The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
 it scans for `DLE+STX`, which never appears.
 The third is a genuine **simplification** — no inner-frame carve-out.  Validated
 by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
 four candidate destuffing rules were tried, and only this one makes all frames
 validate.
 ### Checksum
 ```python
 def checksum(payload: bytes) -> int:
    return sum(b for b in payload if b != 0x10) & 0xFF
 ```
 The DLE-aware variant, same as Series III's `5A` and write frames — not the plain
 SUM8 of ordinary Series III reads.
 ### ⚠ The SUB byte can be escaped
 When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
 Reading it positionally without destuffing reports `0x10`.  This bit once
 already — `SUB 0x02` was logged as `SUB_10` for an afternoon.  Destuff first,
 then index.
 ### Response shape
 ```python
@dataclass
 class MicromateFrame:
    sub: int              # response SUB;  request = 0xFF - sub
    flags: int            # 0xC5 Blastware line, 0x03 Thor line
    page_hi: int
    page_lo: int
    data: bytes           # payload[5:], checksum stripped
    checksum_valid: bool
    @property
    def firmware_line(self) -> str:      # "blastware" | "thor" | "unknown"
    @property
    def declared_length(self) -> int:    # uint16 BE at data[3:5]  (= payload[8:10])
 ```
 ⚠ **`declared_length` is a uint16 BE.**  Read as a single byte it under-reads
 `SUB 0x1A` by 47x — 44 against a true 2092.  This is the single most expensive
 mistake available in this protocol and it has already been made once.
 `MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
 accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
 what distinguishes "no bytes at all" from "bytes but no complete frame" on a
 timeout, and that distinction earned its keep during the Series III work).
 ---
 ## `micromate/protocol.py`
 One method per command, returning raw payload bytes.  No interpretation — that
 belongs in `client.py`.
 **Reads use `offset = 0xFFFF`** and return the whole block in one response;
 Series III's two-step probe/data dance is unnecessary.  `POLL` is the exception,
 taking its data length.  Per-command offsets, all observed:
 | command | SUB | rsp | offset | returns |
 |---|---|---|---|---|
 | poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
 | serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
 | device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
 | state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
 | monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
 | storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
 | active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
 | first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
 | next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
 | compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
 | call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
 | arm event | `0x93` | `0x6C` | — | before every event |
 | first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
 | next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
 | event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks |
 | event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record |
 | bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** |
 ⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
 Parse **forward** from `declared_length`, never backward from the end — Series
 III reads battery and memory from the end of that block, and doing so on a BD
 unit yields a battery voltage of **577.92 V**.
 ⚠ **Test the monitoring flag for non-zero**, never against a constant.  It has
 read both `0x0E` and `0x0C` while monitoring.
 ### `0x5A` — simpler than Series III, deliberately
 No arming ritual, no chunk loop, no `STRT` end-offset parsing, no `TERM` frame.
 One request returns the whole event:
 ```python
 offset_word = 0x1000 + 2 * ceil(size / 512)     # size from the chain walk
 ```
 The payload **is** the `.IDFW` file, byte for byte — so it feeds
 `micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.
 ⚠ Do not port the Series III `5A` walk.  Its address arithmetic caused a 5x
 over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
 III side.  None of that applies here.
 ---
 ## `micromate/client.py`
 ```python
 class MicromateClient:
    def __init__(self, transport: BaseTransport): ...
    def open(self) / close(self) / is_open(self)
    # identity and state
    def connect(self) -> DeviceInfo          # poll → serial → device info → state
    def get_state(self) -> UnitState         # monitoring?, clock, battery, memory
    def get_active_setup(self) -> str
    def list_setups(self) -> list[str]       # 0x3F → 0x40… until empty
    # events
    def list_events(self) -> list[EventRef]  # 0x93 → 0x1E → 0x1F… (key + size)
    def download_event(self, ref) -> bytes   # raw .IDFW/.IDFH
    def get_event(self, ref) -> Event        # download + decode via idf_file
 ```
 `connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
 ⚠ but note the reference records that **whether the unit requires it is
 untested**.  Do it because it is known-good, not because it is known-necessary,
 and say so in the docstring.
 `list_events()` returns the key *and* the size, because `download_event()` needs
 the size to compute its offset word.
 ---
 ## Tests
 **Offline, from captured bytes — no hardware.**  This is the part worth doing
 first, because it can be fully verified tonight's-captures-style before any unit
 is involved.
 ```
 tests/test_micromate_framing.py
 ```
 ⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
 not depend on files being present.  **Embed the frames as hex constants** — they
 are 16–68 bytes each, and a handful covers every case:
 | case | why |
 |---|---|
 | POLL probe reply, 19 B | shortest valid frame |
 | POLL data reply, 68 B | contains a literal `0x10` — only the DLE-aware checksum matches |
 | `0x1A` response, 2108 B | exercises `declared_length` as a true uint16 (`0x082C`) |
 | a Thor-line reply, `flags = 0x03` | `0x03` is ETX; proves destuffing before framing |
 | a frame whose SUB is `0x02` | arrives as `10 02`; proves destuff-then-index |
 | a truncated frame | parser must return nothing, not a bad frame |
 | a corrupted checksum | `checksum_valid == False`, frame still returned |
 Then a round-trip assertion: feed a whole captured session through the parser and
 assert the frame count and every SUB, against `scratch/mm_frame_parse.py`'s
 output — which is already known good, having parsed 24, 38 and 40-frame sessions
 with zero bad checksums.
 **Live, second:** against the bench unit on mint-mac via `mm_link.py`.
 `connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
 then `download_event()` and assert the bytes decode and match a
 `/db/import/idf_file` ingest of the same event.
 ---
 ## Order of work
 1. `framing.py` + its tests — offline, verifiable immediately
 2. `protocol.py` — reads only, one method per row of the table above
 3. `client.py` — `connect()`, `get_state()`, `list_setups()`
 4. the event chain and `download_event()`
 5. decode end-to-end and compare against a store event
 Steps 1–2 need no hardware at all.
 ---
 ## Open questions to settle while implementing
 - **Request param stuffing** — raise `NotImplementedError` rather than guess.
 - **Is THOR's preamble required?**  Try one command cold and find out; it is a
  two-minute test with the bench unit and it removes a ritual if unnecessary.
 - **`Event` model fit** — Series IV carries fields Series III lacks (setup file
  name, `LMic`/`SMic` channels).  Extend `micromate/models.py`; do not fork the
  shared `Event`.
 - **Which `0x0C` fields to trust.**  The peak float there runs 2–5% above
  `max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
  established.  The reference marks it do-not-rely-on — prefer decoded samples.
@@ -934,6 +934,120 @@ which looks like a different failure mode wearing the same signature.
 ---
 ### 8f. BE12599 — CONFIRMED on the bench: the geophone assembly, not the recorder (2026-09-23)
 **The first mechanism in this investigation established physically rather than
 inferred.** Brian pulled BE12599 from I-80 Abut 1 West on 2026-09-21 and bench-
 tested it. A geophone swap settles it.
 #### The swap test
 Same recorder, twenty minutes apart, sensor self-check:
 | geophone fitted | Tran | **Vert** | Long |
 |---|---|---|---|
 | BE12599's own (2026-09-22 21:10) | 4.2 | **2026.0** | 4.8 |
 | BE9888's known-good (2026-09-22 22:48) | 3.6 | **3.4** | 3.6 |
 Vert frequency likewise **2.2 Hz -> 7.4 Hz**. Fleet reference: overswing 3.3–5.1
 (median 3.8, max 5.1 across 42,979 passing records), frequency 7.4–7.8 Hz.
 With the good geophone the unit shows **no offsets, no self-triggering, all three
 channels Passed**, and records a clean damped impulse with broadband 5–100 Hz
 content. **BE12599 the recorder is functional** — it is only out of calibration
 (due 2025-08-12).
 **The fault is in the geophone assembly: element, cable, or its connector.**
 The swap does not separate those three, because the cable is permanently glanded
 into the geophone.
 #### The bench evidence, with ground motion removed
 17 events recorded on a workbench with nothing touching the sensor.
 **The control** — `N599LRY0.4B0`, trigger forced by slapping the *microphone*,
 geophone untouched:
 | channel | peak in/s | frac < 0 | mean/peak |
 |---|---|---|---|
 | Tran | 0.070 | **0.00** | 0.87 |
 | **Vert** | **0.190** | **0.35** | 0.13 |
 | Long | 0.070 | **0.00** | 0.84 |
 Vert produces 0.19 in/s of bipolar wander with **no mechanical input at all**.
 The fault is *continuous*, not episodic — every field event was simply the
 moment it crossed the trigger.
 **All three channels are affected**, consistently across all 17 events:
 Tran and Long parked at a small positive DC (`frac<0 = 0.00`, mp 0.64–0.95);
 Vert wandering at 0.89–1.78 Hz, peaks 0.14–0.54. Yet the swing test calls Tran
 and Long **Passed** (4.2 / 4.8). So their *coils* are healthy and merely sitting
 on an offset, while Vert's circuit has lost damping entirely.
 That reads as **leakage on several contacts plus one gone open** — which is what
 progressive corrosion at a shared connection does. Three independent geophone
 elements do not fail in sequence.
 **It is still progressing.** Tran's DC offset was ~0.011 in/s on 2026-09-14 and
 ~0.064 on 2026-09-22 — 6x in eight days. Vert's overswing went 7.5 -> 2026.0 over
 the same window.
 #### ⚠ This explains the autozero's 10% success rate
 The journal has framed this fault around the **recorder** throughout: Instantel's
 autozero, the X1/X8 gains, the 2027–2069 window, `SUB 0x0E` as the best lead.
 All of that adjusts the recorder's zero reference.
 **An autozero cannot fix a geophone.** If most offsets are geophone-assembly
 faults, a re-zero would succeed only on the minority where the recorder really is
 at fault — which is exactly the ~10% Brian's father reported and which has sat
 unexplained in §5 since 2026-08-28.
 **Procedure change: swap the geophone first.** It is a five-minute field test
 costing nothing. On this unit it would have answered the question in August.
 ⚠ Scoped to one unit. The other four confirmed units are the obvious check.
 #### Correction to §8e
 §8e is titled "a connector, not a geophone" and concluded the element was fine
 because the fault hopped channels. The channel-hopping observation stands and the
 swap test supports the general shape, but **that title overstated what was known**:
 the swap narrows the fault to the *assembly*, and element / cable / connector are
 still open within it. §8e's reasoning was sound; its confidence was not.
 #### Cause — road salt now outranks the mice
 The enclosure held a mouse nest on the battery ([[§8e]]), and that framed the
 2026-09-22 analysis. But **the geophone is buried in the ground, not in the
 enclosure** — they are two independent contamination paths, and the fault is in
 the buried half.
 The unit sat **~6 feet from I-80 westbound**. Winter road salt spray and highway
 runoff put a buried geophone in chloride-rich water for months a year. Chloride
 drives exactly this failure: corrosion at terminations, leakage across
 insulation, eventually an open circuit.
 The mice remain a real finding for the *enclosure* and the recorder end. They are
 probably not the cause of this fault.
 **Testable consequence:** road salt is a property of the **site**, not the unit.
 Units deployed near salted highways should carry higher geophone-assembly failure
 rates than units on quiet sites. Nothing in the archive records proximity to a
 road, so this needs the deployment records rather than the event data.
 #### Open — to be settled by opening the geophone
 1. **Meter at the geophone plug** before disassembly: three coil pairs, and each
   to shell. Prediction: **Vert open or very high** (that is the 2026 overswing);
   Tran and Long matched and normal; **insulation to shell finite on all three**,
   Vert worst (that is the leakage putting DC on healthy coils).
 2. **Then open it.** Look for a water line, white/green corrosion at the coil
   terminations, the state of the damping shunt, and the cable gland from the
   inside. Measure the coils again **at the internal terminals** — if Vert reads
   fine there, the fault is in the internal wiring or the gland, not the element.
 ---
 ### ⚠ Serial prefixes — four of these units are BlastMates, not MiniMates
 Corrected 2026-09-06, after Brian queried "BA10895?" against a report that
@@ -985,3 +1099,7 @@ decoder lost the geo threshold along with the serial. See commit `9ceff65`.
 | 2026-09-06 | The spread gate is **blind to onsets by construction** — it rejects a moving floor, which is what an onset is. It rejected the very record in which the ramp is visible. |
 | 2026-09-07 | **The non-motion test** (Brian: "it doesn't cross zero"). `\|mean\|/peak` is bimodal with a 1.09% dead zone and returns exactly the 5 confirmed units from physics, not a threshold. Independent corroboration of the unit list. **BE11007 settled as NOT an offset** — a one-sided 9 in/s blast. |
 | 2026-09-07 | **BE12599 is a connector fault, not a geophone fault.** Unipolar spikes with a 26→118 ms RC tail progressing to a flat pedestal, and the fault MOVES between Long and Tran while the sensor self-check passes on every event. An element cannot hop channels; a connector can. Inspect before reseating. |
 | 2026-09-23 | **CONFIRMED: BE12599's fault is the geophone assembly, not the recorder.** Geophone swap on the same unit: Vert overswing **2026.0 -> 3.4**, frequency 2.2 -> 7.4 Hz, all three channels Passed, no self-triggering. First physically established mechanism in this investigation. |
 | 2026-09-23 | **This explains the autozero's ~10% success rate** (§5): an autozero adjusts the *recorder's* zero reference and cannot fix a geophone. **Procedure change: swap the geophone first.** |
 | 2026-09-23 | Bench control event (mic-slap trigger, geophone untouched) shows Vert wandering **0.19 in/s with no mechanical input** — the fault is continuous, not episodic. All three channels affected; Tran/Long parked on DC while their coils pass the swing test. |
 | 2026-09-23 | **Cause: road salt now outranks the mice.** The geophone is buried, not in the enclosure — two independent contamination paths, and the fault is in the buried half. The unit sat ~6 ft from I-80 WB. Salt is a *site* property, so this predicts higher failure rates near salted highways. |
@@ -1,33 +0,0 @@
 """Pretend to be a Micromate on a serial port: log what arrives, reply to POLL.
 Proves the modem's return path (serial -> TCP) independently of the real unit.
 """
 import os, select, sys, termios, time
 path, baud = sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 115200
 B = {9600: termios.B9600, 38400: termios.B38400, 115200: termios.B115200}[baud]
 fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
 a = termios.tcgetattr(fd)
 a[0] = a[1] = a[3] = 0
 a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL
 a[4] = a[5] = B
 a[6] = list(a[6]); a[6][termios.VMIN] = 0; a[6][termios.VTIME] = 0
 termios.tcsetattr(fd, termios.TCSANOW, a)
 termios.tcflush(fd, termios.TCIOFLUSH)
 # A real POLL probe reply, captured from UM12947 on 2026-09-24.
 REPLY = bytes.fromhex("0200c5a4000000000000300000000000000099") + b"\x03"
 print(f"fake unit on {path} @ {baud}; will answer any inbound frame", flush=True)
 while True:
    r, _, _ = select.select([fd], [], [], 1.0)
    if not r:
        continue
    data = os.read(fd, 4096)
    if not data:
        continue
    ts = time.strftime("%H:%M:%S")
    print(f"{ts}  IN  {len(data):3} B  {data.hex(' ')}", flush=True)
    time.sleep(0.02)
    os.write(fd, REPLY)
    print(f"{ts}  OUT {len(REPLY):3} B  {REPLY.hex(' ')}  <- canned POLL reply", flush=True)
@@ -1,202 +0,0 @@
 #!/usr/bin/env python3
 """
 mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
 raw capture pair.
 Why this exists
 ---------------
 `minimateplus.framing.S3FrameParser` cannot see Micromate traffic.  It locates
 frames by scanning for `DLE STX`, and a Micromate response has **no leading
 DLE** — it starts at a bare `STX`.  It also expects `payload[1] == 0x10`, where
 the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
 The practical consequence, seen on the 9-24-26 setup-push capture: the
 Blastware-side requests parse fine (Thor emits Series III request frames), but
 **every device response is silently dropped or mis-framed** — so a capture that
 actually contains 12 acked writes looks like 12 unanswered requests.
 Destuffing
 ----------
 One rule covers both directions: after the leading doubled `BW_CMD`, every
 `10 XX` pair on the wire destuffs to `XX`.  That includes `10 03` — Thor
 escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
 exactly as Blastware does.
 That rule was chosen by evidence, not assumption: of the four candidates tried
 against the 9-24-26 capture's four data-carrying write frames, it is the only
 one under which all four checksums validate.  See
 `docs/micromate_protocol_reference.md` → *The write path*.
 Usage
 -----
    python scratch/mm_frame_parse.py <capture-dir>
    python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
    python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
 """
 from __future__ import annotations
 import argparse
 import sys
 from pathlib import Path
 DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
 # Request SUB -> short name.  Series III names where they carry over; the
 # Series IV additions are marked.
 SUBNAME = {
    0x01: "DEVICE_INFO",
    0x06: "STORAGE_RANGE",
    0x08: "EVENT_INDEX",
    0x0A: "WAVEFORM_HDR",
    0x0C: "WAVEFORM_REC",
    0x15: "SERIAL",
    0x1A: "COMPLIANCE_CFG",
    0x1C: "MONITOR_STATUS",
    0x1E: "EVENT_HDR",
    0x2C: "CALLHOME_CFG",
    0x2E: "TRIGGER_CFG_READ",     # Series IV
    0x3E: "OPERATOR",
    0x41: "SETUP_NAME_READ",      # Series IV
    0x5A: "BULK_DOWNLOAD",
    0x5B: "POLL",
    0x68: "EVENT_INDEX_WRITE",
    0x69: "WAVEFORM_WRITE",
    0x71: "COMPLIANCE_WRITE",
    0x72: "CONFIRM_A",
    0x73: "CONFIRM_B",
    0x74: "CONFIRM_C",
    0x82: "TRIGGER_WRITE",
    0x83: "TRIGGER_CONFIRM",
    0xDA: "SETUP_FILE_DECL",      # Series IV — names the target .MMB
    0xFE: "FULL_CFG",
 }
 def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
    """Destuff one frame starting at `start`.
    Returns (payload, checksum, index_of_terminating_ETX).  `payload` excludes
    the trailing checksum byte.  A request frame opens `ACK STX 10 10`; a
    response opens with a bare `STX`.
    """
    i = start + (2 if is_request else 1)
    out = bytearray()
    if is_request:
        # The doubled BW_CMD is the one guaranteed stuffed byte.
        if blob[i : i + 2] != bytes([DLE, DLE]):
            raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
        out.append(DLE)
        i += 2
    while i < len(blob):
        b = blob[i]
        if b == DLE and i + 1 < len(blob):
            out.append(blob[i + 1])
            i += 2
            continue
        if b == ETX:
            break
        out.append(b)
        i += 1
    if len(out) < 2:
        raise ValueError(f"@0x{start:04x}: frame too short")
    return bytes(out[:-1]), out[-1], i
 def frames(blob: bytes, *, is_request: bool):
    """Yield (offset, payload, chk, checksum_kind)."""
    i, n = 0, len(blob)
    while i < n:
        if is_request:
            if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
                i += 1
                continue
        elif blob[i] != STX:
            i += 1
            continue
        try:
            payload, chk, end = destuff(blob, i, is_request=is_request)
        except ValueError:
            i += 1
            continue
        sum8 = sum(payload) & 0xFF
        dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
        if sum8 == chk:
            kind = "SUM8"
        elif dle_aware == chk:
            kind = "DLE-aware"
        else:
            kind = "BAD"
        yield i, payload, chk, kind
        i = end + 1
 def describe(payload: bytes, is_request: bool) -> str:
    if len(payload) < 3:
        return "??"
    sub = payload[2]
    if is_request:
        return SUBNAME.get(sub, f"SUB_{sub:02X}")
    req = 0xFF - sub
    return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
 def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
    blob = path.read_bytes()
    side = "Thor" if is_request else "unit"
    print(f"== {side:4}  {path.name}  ({len(blob)} bytes)")
    n_bad = 0
    for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
        if kind == "BAD":
            n_bad += 1
        sub = p[2] if len(p) > 2 else -1
        flags = p[1] if len(p) > 1 else -1
        # Requests carry offset at payload[4:6]; responses page at [3:5].
        word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
        data = len(p) - 16 if is_request else max(len(p) - 5, 0)
        print(
            f"  [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
            f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
            f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
        )
        if dump_sub is not None and sub == dump_sub:
            body = p[16:] if is_request else p[5:]
            print(f"       ---- data ({len(body)} bytes) ----")
            for o in range(0, len(body), 16):
                chunk = body[o : o + 16]
                txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
                print(f"       {o:06x}  {chunk.hex(' '):<47}  |{txt}|")
    print(f"  -- {idx + 1} frames, {n_bad} bad checksum\n")
 def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("paths", nargs="+",
                    help="a capture directory, or raw_bw.bin and raw_s3.bin")
    ap.add_argument("--dump", default=None,
                    help="hex-dump the data section of this SUB (e.g. 0x71)")
    args = ap.parse_args()
    dump_sub = int(args.dump, 0) if args.dump else None
    if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
        d = Path(args.paths[0])
        bw = sorted(d.glob("raw_bw_*.bin"))
        s3 = sorted(d.glob("raw_s3_*.bin"))
        if not bw or not s3:
            print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
            return 2
        pairs = [(bw[0], True), (s3[0], False)]
    elif len(args.paths) == 2:
        pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
    else:
        ap.error("pass a capture directory, or exactly two .bin files")
    for path, is_request in pairs:
        report(path, is_request=is_request, dump_sub=dump_sub)
    return 0
 if __name__ == "__main__":
    sys.exit(main())
@@ -1,119 +0,0 @@
 #!/usr/bin/env python3
 """
 socat_log_split.py — recover a capture pair from a `socat -x` relay log.
 Why this exists
 ---------------
 The bench relay that puts Thor in front of a USB-attached Micromate is:
    socat -d -d -x TCP-LISTEN:12345,reuseaddr,fork /dev/ttyACM0,raw,echo=0,b115200 \
        > ~/mm-captures/socat_<ts>.log 2>&1
 `-x` makes socat hex-dump every byte it forwards, in both directions, with
 timestamps.  That log is therefore a **complete second copy of every capture**
 taken through the relay — independent of whether seismo_lab was recording.
 On 2026-09-25 that mattered: a capture's `.bin` files never made it off the
 Windows machine, and the session was rebuilt from this log instead.  When the
 real bins turned up later, the reconstruction was **byte-for-byte identical in
 both directions** (3,595 and 4,004 bytes).  So this is a validated fallback, not
 a lossy approximation.
 Log format
 ----------
 ```
 > 2026/09/25 00:30:35.000276659  length=21 from=0 to=20
 41 02 10 10 00 5b 00 00 30 00 ...
 2026/09/25 00:30:35 socat[32190] N write(5, 0x..., 21) completed
 < 2026/09/25 00:30:35.000384100  length=64 from=0 to=63
 02 00 c5 a4 00 00 30 00 ...
 ```
 `>` is data heading toward the serial device (Thor → unit).  `<` is data coming
 back (unit → Thor).  Hex lines are space-separated and indented; socat's own
 status lines start with a date and carry no payload.
 Usage
 -----
    # whole log
    python scratch/socat_log_split.py socat_20260924_181248.log --out-dir ./recovered
    # one session — line numbers from the "accepting connection" markers
    grep -n "accepting connection" socat_*.log
    python scratch/socat_log_split.py socat_*.log --from-line 919 --out-dir ./recovered
 Then parse the result as usual:
    python scratch/mm_frame_parse.py recovered/raw_bw.bin recovered/raw_s3.bin
 ⚠ A log spanning several sessions concatenates them.  Split by line number using
 the `accepting connection` markers, or the frame walk will run sessions together.
 """
 from __future__ import annotations
 import argparse
 import re
 from pathlib import Path
 _HEX = re.compile(r"\A[0-9a-f]{2}\Z")
 _SOCAT_STATUS = re.compile(r"\A\d{4}/\d{2}/\d{2}")
 def split(lines) -> tuple[bytes, bytes]:
    """Return (to_device, from_device) byte streams."""
    to_dev, from_dev = bytearray(), bytearray()
    cur = None
    for line in lines:
        if line.startswith(">"):
            cur = to_dev
            continue
        if line.startswith("<"):
            cur = from_dev
            continue
        if _SOCAT_STATUS.match(line):
            # socat's own status line ends the current dump block.
            cur = None
            continue
        if cur is None or not line.startswith(" "):
            continue
        toks = line.split()
        if toks and all(_HEX.match(t) for t in toks):
            cur.extend(int(t, 16) for t in toks)
    return bytes(to_dev), bytes(from_dev)
 def main() -> int:
    ap = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
    )
    ap.add_argument("log", help="a socat -x log file")
    ap.add_argument("--out-dir", default=".", help="where to write the .bin pair")
    ap.add_argument("--from-line", type=int, default=1,
                    help="first log line to read (1-based) — use the "
                         "'accepting connection' marker of the session you want")
    ap.add_argument("--to-line", type=int, default=None,
                    help="last log line to read (1-based, inclusive)")
    ap.add_argument("--prefix", default="raw", help="output basename prefix")
    args = ap.parse_args()
    lines = Path(args.log).read_text(errors="replace").splitlines()
    lo = max(args.from_line - 1, 0)
    hi = args.to_line if args.to_line is not None else len(lines)
    to_dev, from_dev = split(lines[lo:hi])
    out = Path(args.out_dir)
    out.mkdir(parents=True, exist_ok=True)
    bw = out / f"{args.prefix}_bw.bin"
    s3 = out / f"{args.prefix}_s3.bin"
    bw.write_bytes(to_dev)
    s3.write_bytes(from_dev)
    print(f"Thor -> unit  {len(to_dev):>7} bytes  {bw}")
    print(f"unit -> Thor  {len(from_dev):>7} bytes  {s3}")
    if not to_dev or not from_dev:
        print("⚠ one direction is empty — check --from-line / --to-line")
    return 0
 if __name__ == "__main__":
    raise SystemExit(main())