merge: Series-4 live client -- framing, protocol and read client, verified on hardware
The Series-4 live half of SFM: micromate/{framing,protocol,client}.py with 98
offline tests, driven against two real units over both physical transports.
Read-only throughout. No command has ever been originated against a unit by
this project -- every write in the protocol reference was performed by THOR
while we recorded, and keeping that true through the read client is deliberate.
Six errors in docs/micromate_client_spec.md were found by measuring against the
captures before writing code rather than after. All six fail silently:
* requests need FOUR escaped bytes, not one (161/218 frames without)
* the response checksum is plain SUM8, not the DLE-aware variant (55/251)
* SUB 0x5A is a 1024-byte chunk loop, not one request per event
* SUB 0x0A is the monitor-log walk, not a keyed event-header read
* 1E/1F carry token 0xFE at params[7]
* SUB 0x01 has no THOR precedent at all
Hardware verification: identical wire bytes over USB CDC-ACM and an RX55 in PAD
mode; every 11.0BD inference confirmed on UM20147; a monitoring unit answers
reads with no SESSION_RESET; and all six bench events decode with the existing
IDF codec.
Two findings that constrain SFM's design rather than just documenting a unit:
cellular costs ~0.65 s PER ROUND TRIP regardless of payload, and event keys
COLLIDE ACROSS UNITS, so any store must key on (serial, event_key).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
Claude Opus 5
@@ -10,10 +10,38 @@ pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
|
||||
|
||||
---
|
||||
|
||||
## Where things stand (updated 2026-08-28)
|
||||
## Where things stand (updated 2026-09-26)
|
||||
|
||||
Read this first when picking the project back up.
|
||||
|
||||
- **The Series-4 LIVE wire protocol is reverse-engineered end to end
|
||||
(2026-09-25).** `docs/micromate_protocol_reference.md` is the Series-4
|
||||
Rosetta Stone, sibling to `instantel_protocol_reference.md`. **A Micromate
|
||||
answers Series III command frames** — three framing differences: responses
|
||||
have **no leading `DLE`** (a bare `STX`), `payload[1]` is `0xC5` (Blastware
|
||||
firmware) or `0x03` (Thor firmware) rather than `0x10`, and the data length
|
||||
is a **uint16 BE at `payload[8:10]`** (as a byte it under-reads `SUB 0x1A`
|
||||
by 47x). Read path, event chain, setups, scheduler, monitoring control and
|
||||
per-event delete are all mapped; **the inbound call-home session is the only
|
||||
protocol unknown left.**
|
||||
⚠ **No command has ever been originated against a unit by this project.**
|
||||
Every write was performed by THOR while we recorded. That line is worth
|
||||
keeping.
|
||||
⚠ `micromate/` still has **no live client** — it is codec-only. The
|
||||
`minimateplus/` stack (transport/framing/protocol/client) has no Series-4
|
||||
counterpart yet. `minimateplus.transport` is protocol-agnostic and reusable.
|
||||
- **Bench tooling for device diagnosis (2026-09-25).** `bridges/mm_probe.py`
|
||||
distinguishes the four faults THOR reports identically as "disconnected"
|
||||
(refused / connect timeout / **connected but no reply** / replied) and names
|
||||
what to try next. `bridges/mm_link.py` is a stand-in for a cellular modem
|
||||
with a decoded log and fault injection. `scratch/mm_frame_parse.py` exists
|
||||
because **`S3FrameParser` cannot see Micromate responses at all** — it scans
|
||||
for `DLE+STX`, which never appears in Series-4 traffic.
|
||||
- **A Micromate's USB-A host port drives FTDI and CDC-ACM only** — no Prolific,
|
||||
in either firmware line. TMI buys both Sabrent (FTDI) and Benfei (PL2303)
|
||||
cables and they are indistinguishable by eye. A PL2303 cable leaves a unit
|
||||
with **no working modem port at all**; identify by `lsusb` VID, `0403` vs
|
||||
`067b`. This accounted for a unit that could not be deployed.
|
||||
- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
|
||||
archive decodes **14,338 / 14,338** paired files exactly against their
|
||||
preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
|
||||
@@ -92,6 +120,9 @@ Read this first when picking the project back up.
|
||||
**v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
|
||||
fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
|
||||
files are archive-only and were never ingested).
|
||||
✅ **The v0.30.0 Series-4 backfill HAS been run on prod (2026-09-25).** Every
|
||||
stored Series-4 geophone value was ~3.3% low until then; that is corrected and
|
||||
the job does not need repeating.
|
||||
- **The "offset" hardware fault has its own journal** --
|
||||
`docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
|
||||
**persistent** — it stays until the geophone is serviced. Detect it with
|
||||
@@ -103,7 +134,18 @@ Read this first when picking the project back up.
|
||||
`SUB 0x0E` (unimplemented), which may carry those very numbers.
|
||||
|
||||
|
||||
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
|
||||
When new information about a protocol is discovered, record it in the matching
|
||||
reference **in addition to** this document:
|
||||
|
||||
| series | document |
|
||||
|---|---|
|
||||
| Series III (MiniMate Plus / BlastMate) | `docs/instantel_protocol_reference.md` |
|
||||
| **Series IV (Micromate / THOR)** | **`docs/micromate_protocol_reference.md`** |
|
||||
| Thor IDF file format | `docs/idf_protocol_reference.md` |
|
||||
|
||||
Both protocol references carry retractions in place rather than deleting what
|
||||
turned out to be wrong — that convention has already saved re-deriving the same
|
||||
mistakes twice, so keep it.
|
||||
|
||||
---
|
||||
|
||||
@@ -276,8 +318,15 @@ minimateplus/ ← Python client library (primary focus)
|
||||
|
||||
sfm/server.py ← FastAPI REST server exposing device data over HTTP
|
||||
seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs)
|
||||
bridges/
|
||||
mm_probe.py ← name the fault behind a dead unit (4 verdicts, read-only)
|
||||
mm_link.py ← bench stand-in for a cellular modem, with fault injection
|
||||
ach_mitm.py ← TCP relay for recording a Series-3 ACH session
|
||||
|
||||
docs/
|
||||
instantel_protocol_reference.md ← reverse-engineered protocol spec ("the Rosetta Stone")
|
||||
instantel_protocol_reference.md ← Series III protocol spec ("the Rosetta Stone")
|
||||
micromate_protocol_reference.md ← Series IV protocol spec + THOR's measured behaviour
|
||||
idf_protocol_reference.md ← Thor IDF file format
|
||||
CHANGELOG.md ← version history
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,363 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_client_check.py — exercise the Micromate read client against a real unit.
|
||||
|
||||
**Read-only.** It sends POLL, SERIAL, state, monitor status, the setup walk and
|
||||
(optionally) one event download. It never writes, never erases, never starts or
|
||||
stops monitoring.
|
||||
|
||||
Why it exists
|
||||
-------------
|
||||
`micromate/{framing,protocol,client}.py` are verified against captures taken
|
||||
**over USB**, on **one firmware line** (`11.0CB`). Two things that cannot be
|
||||
verified that way:
|
||||
|
||||
* **the modem path.** An RX55/RV55 bridges serial to TCP transparently, but
|
||||
it buffers up to ~1 s before forwarding, so a single logical response can
|
||||
arrive as many small reads. The client reads to frame completion rather
|
||||
than using idle-gap detection, which should be strictly more robust — but
|
||||
"should be" is the point of this script.
|
||||
* **the other firmware line.** `11.0BD` reports `flags = 0x03`, a shorter
|
||||
model string, and a `SUB 0x1C` block 4 bytes longer. Everything about that
|
||||
is currently inference from one 2026-09-23 sweep whose captures never
|
||||
landed in the repo.
|
||||
|
||||
Run it over both paths and diff the two reports. Anything that differs beyond
|
||||
timings is a finding.
|
||||
|
||||
Usage
|
||||
-----
|
||||
# over the modem
|
||||
python3 bridges/mm_client_check.py 63.45.161.30:9034
|
||||
|
||||
# over USB / direct serial
|
||||
python3 bridges/mm_client_check.py /dev/ttyACM0 --baud 115200
|
||||
|
||||
# include one event download (still read-only)
|
||||
python3 bridges/mm_client_check.py <target> --download
|
||||
|
||||
⚠ These modems bridge ONE TCP session to serial at a time. If THOR holds the
|
||||
unit, this will connect and then see nothing — that is contention, not a fault.
|
||||
`bridges/mm_probe.py` explains that case; disconnect THOR first.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import errno
|
||||
import os
|
||||
import select
|
||||
import sys
|
||||
import termios
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
|
||||
from micromate.client import MicromateClient, _content # noqa: E402
|
||||
from micromate.protocol import ProtocolError # noqa: E402
|
||||
from minimateplus.transport import TcpTransport # noqa: E402
|
||||
|
||||
|
||||
class StdlibSerial:
|
||||
"""Raw serial on stdlib `termios` — no pyserial.
|
||||
|
||||
`minimateplus.SerialTransport` needs pyserial, and a bench host is whatever
|
||||
is to hand. On a PEP 668 distro (Mint 22, Ubuntu 24.04, Debian 12) a plain
|
||||
`pip install pyserial` is refused outright, so a diagnostic that depends on
|
||||
it is one you cannot run at the moment you need it. `bridges/mm_link.py`
|
||||
and `scratch/fake_unit.py` already take this approach; this is the same
|
||||
~30 lines, and it means the tool runs on a stock Python 3 anywhere.
|
||||
|
||||
Not a general replacement for SerialTransport — no flow control, no
|
||||
parity options, Linux/macOS only. Enough for a Micromate, which is 8N1
|
||||
with no handshaking.
|
||||
"""
|
||||
|
||||
_BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
|
||||
57600: termios.B57600, 115200: termios.B115200}
|
||||
|
||||
def __init__(self, path: str, baud: int = 115200) -> None:
|
||||
if baud not in self._BAUD:
|
||||
raise ValueError(f"unsupported baud {baud}; pick from {sorted(self._BAUD)}")
|
||||
self.path, self.baud, self.fd = path, baud, None
|
||||
|
||||
def connect(self) -> None:
|
||||
if self.fd is not None:
|
||||
return
|
||||
self.fd = os.open(self.path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
a = termios.tcgetattr(self.fd)
|
||||
a[0] = a[1] = a[3] = 0 # raw in/out, non-canonical
|
||||
a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL # 8N1, ignore modem lines
|
||||
a[4] = a[5] = self._BAUD[self.baud]
|
||||
a[6] = list(a[6])
|
||||
a[6][termios.VMIN] = 0
|
||||
a[6][termios.VTIME] = 0
|
||||
termios.tcsetattr(self.fd, termios.TCSANOW, a)
|
||||
termios.tcflush(self.fd, termios.TCIOFLUSH)
|
||||
|
||||
def disconnect(self) -> None:
|
||||
if self.fd is not None:
|
||||
os.close(self.fd)
|
||||
self.fd = None
|
||||
|
||||
def is_connected(self) -> bool:
|
||||
return self.fd is not None
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
if self.fd is None:
|
||||
return b""
|
||||
r, _, _ = select.select([self.fd], [], [], 0.05)
|
||||
if not r:
|
||||
return b""
|
||||
try:
|
||||
return os.read(self.fd, n)
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
return b""
|
||||
raise
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
if self.fd is None:
|
||||
raise OSError("port is not open")
|
||||
while data:
|
||||
data = data[os.write(self.fd, data):]
|
||||
|
||||
|
||||
class _Timed:
|
||||
"""Count bytes and time each read, so the two transports can be compared.
|
||||
|
||||
With `capture`, also writes the raw byte streams to a `raw_bw_*` /
|
||||
`raw_s3_*` pair in the layout `scratch/mm_frame_parse.py` already reads --
|
||||
so a run on an unfamiliar unit can be turned into test fixtures without
|
||||
setting up a relay.
|
||||
"""
|
||||
|
||||
def __init__(self, inner, capture: str | None = None) -> None:
|
||||
self._inner = inner
|
||||
self.reads = 0
|
||||
self.bytes_in = 0
|
||||
self.bytes_out = 0
|
||||
self._bw = self._s3 = None
|
||||
if capture:
|
||||
stamp = time.strftime("%Y%m%d_%H%M%S")
|
||||
d = Path(capture)
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
self.bw_path = d / f"raw_bw_{stamp}_mm_client_check.bin"
|
||||
self.s3_path = d / f"raw_s3_{stamp}_mm_client_check.bin"
|
||||
self._bw = open(self.bw_path, "wb")
|
||||
self._s3 = open(self.s3_path, "wb")
|
||||
|
||||
def close_capture(self) -> None:
|
||||
for f in (self._bw, self._s3):
|
||||
if f:
|
||||
f.close()
|
||||
|
||||
def connect(self):
|
||||
return self._inner.connect()
|
||||
|
||||
def disconnect(self):
|
||||
return self._inner.disconnect()
|
||||
|
||||
def is_connected(self):
|
||||
return self._inner.is_connected()
|
||||
|
||||
def write(self, data: bytes):
|
||||
self.bytes_out += len(data)
|
||||
if self._bw:
|
||||
self._bw.write(data); self._bw.flush()
|
||||
return self._inner.write(data)
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
chunk = self._inner.read(n)
|
||||
if chunk:
|
||||
self.reads += 1
|
||||
self.bytes_in += len(chunk)
|
||||
if self._s3:
|
||||
self._s3.write(chunk); self._s3.flush()
|
||||
return chunk
|
||||
|
||||
|
||||
# ⚠ HYPOTHESIS, 6 events. content[11] of the 0x0C record separated 4 waveforms
|
||||
# from 2 histograms cleanly and was constant within each group. A 4/2 split is
|
||||
# thin evidence for a byte that could be anything, so _decode() below does NOT
|
||||
# trust it -- it tries the other suffix on failure and says when the guess was
|
||||
# wrong. The protocol reference states no type field is known; this may be it.
|
||||
_TYPE_BYTE = 11
|
||||
_TYPES = {0x07: ".IDFW", 0x08: ".IDFH"}
|
||||
|
||||
|
||||
def _event_type(record: bytes) -> str:
|
||||
if len(record) <= _TYPE_BYTE:
|
||||
return "?"
|
||||
b = record[_TYPE_BYTE]
|
||||
return {0x07: "waveform", 0x08: "histogram"}.get(b, f"unknown(0x{b:02x})")
|
||||
|
||||
|
||||
def _decode(blob: bytes, key: bytes, record: bytes) -> None:
|
||||
"""Decode the downloaded bytes, proving they are a real event file.
|
||||
|
||||
read_idf_file() picks waveform vs histogram from the FILENAME SUFFIX, and a
|
||||
wire download has no filename -- so the suffix has to come from somewhere.
|
||||
This tries the 0x0C type byte first and the other suffix second; getting a
|
||||
decode either way proves the chunk assembly, and which one worked is itself
|
||||
the finding.
|
||||
"""
|
||||
import tempfile
|
||||
from micromate.idf_file import read_idf_file
|
||||
|
||||
guess = _TYPES.get(record[_TYPE_BYTE] if len(record) > _TYPE_BYTE else -1, ".IDFW")
|
||||
order = [guess] + [e for e in (".IDFW", ".IDFH") if e != guess]
|
||||
|
||||
for n, ext in enumerate(order):
|
||||
with tempfile.NamedTemporaryFile(suffix=ext, delete=False) as f:
|
||||
f.write(blob)
|
||||
tmp = f.name
|
||||
try:
|
||||
res = read_idf_file(tmp)
|
||||
samples = sum(len(v) for v in getattr(res, "samples", {}).values())
|
||||
note = "" if n == 0 else f" *** the 0x0C type byte guessed {guess} — WRONG ***"
|
||||
print(f" decoded OK as {ext}: {samples} samples{note}")
|
||||
os.unlink(tmp)
|
||||
return
|
||||
except Exception as e:
|
||||
last = f"{ext}: {type(e).__name__}: {e}"
|
||||
finally:
|
||||
if os.path.exists(tmp):
|
||||
os.unlink(tmp)
|
||||
|
||||
print(f" decode failed BOTH ways — last: {last}")
|
||||
out = Path(f"./{key.hex()}.bin")
|
||||
out.write_bytes(blob)
|
||||
print(f" saved to {out} for offline analysis")
|
||||
|
||||
|
||||
def step(label: str, fn):
|
||||
"""Run one read, report how long it took and what it returned."""
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
value = fn()
|
||||
except Exception as e:
|
||||
print(f" {label:.<26} FAILED {type(e).__name__}: {e}")
|
||||
return None
|
||||
ms = 1000 * (time.monotonic() - t0)
|
||||
shown = value if isinstance(value, str) else repr(value)
|
||||
if isinstance(value, list):
|
||||
shown = f"{len(value)} entries"
|
||||
print(f" {label:.<26} {ms:7.0f} ms {shown}")
|
||||
return value
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(
|
||||
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
|
||||
)
|
||||
ap.add_argument("target", help="host:port for TCP, or a serial device path")
|
||||
ap.add_argument("--baud", type=int, default=115200,
|
||||
help="serial only; the USB-A/FTDI path runs at 115200. "
|
||||
"Ignored by the USB-B 'PC' port, which is CDC-ACM "
|
||||
"and negotiates its own rate.")
|
||||
ap.add_argument("--timeout", type=float, default=10.0)
|
||||
ap.add_argument("--download", action="store_true",
|
||||
help="also download the first stored event (read-only)")
|
||||
ap.add_argument("--capture", metavar="DIR",
|
||||
help="also write a raw_bw_*/raw_s3_*.bin pair to DIR, so "
|
||||
"this run can become a test fixture. Worth doing on "
|
||||
"any unit whose firmware line is new to us.")
|
||||
ap.add_argument("--lenient", action="store_true",
|
||||
help="do not raise on a bad checksum — for diagnosis only")
|
||||
a = ap.parse_args()
|
||||
|
||||
if ":" in a.target and not Path(a.target).exists():
|
||||
host, _, port = a.target.rpartition(":")
|
||||
inner = TcpTransport(host, int(port), connect_timeout=a.timeout)
|
||||
path = f"TCP {host}:{port}"
|
||||
else:
|
||||
inner = StdlibSerial(a.target, baud=a.baud)
|
||||
path = f"serial {a.target} @ {a.baud}"
|
||||
|
||||
transport = _Timed(inner, capture=a.capture)
|
||||
mm = MicromateClient(transport, recv_timeout=a.timeout,
|
||||
strict_checksums=not a.lenient)
|
||||
|
||||
print(f"\n{path} (read-only: POLL, SERIAL, state, status, setups)\n")
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
mm.open()
|
||||
except OSError as e:
|
||||
print(f" connect.................... FAILED {e}")
|
||||
return 2
|
||||
print(f" {'connect':.<26} {1000*(time.monotonic()-t0):7.0f} ms")
|
||||
|
||||
try:
|
||||
info = step("connect() identity", mm.connect)
|
||||
if info:
|
||||
print(f" serial={info.serial} model={info.model} "
|
||||
f"fw={info.firmware_line} monitoring={info.monitoring}")
|
||||
print(f" active setup={info.active_setup!r}")
|
||||
if info.firmware_line == "thor":
|
||||
print(" *** 11.0BD unit — the FIRST one this code has met. ***")
|
||||
print(" *** Check the battery and clock below carefully: ***")
|
||||
print(" *** its 0x1C block is 4 bytes longer. ***")
|
||||
|
||||
state = step("get_state()", mm.get_state)
|
||||
if state:
|
||||
print(f" {state}")
|
||||
if state.battery_volts and not 2.5 < state.battery_volts < 9.0:
|
||||
print(f" *** battery {state.battery_volts} V is impossible — "
|
||||
f"this is the from-the-end offset bug. ***")
|
||||
if state.device_time is None:
|
||||
print(" *** device clock did not decode — dump raw below. ***")
|
||||
print(f" raw 0x1C content: {_content(state.raw).hex(' ')}")
|
||||
|
||||
setups = step("list_setups()", mm.list_setups)
|
||||
if setups:
|
||||
print(f" first={setups[0]!r} last={setups[-1]!r}")
|
||||
|
||||
if a.download:
|
||||
print("\n event chain (read-only):")
|
||||
proto = mm.protocol
|
||||
proto.arm_event()
|
||||
hdr = _content(proto.read_event_first())
|
||||
key, size = hdr[0:4], int.from_bytes(hdr[4:8], "big")
|
||||
if not size:
|
||||
print(" no events stored")
|
||||
else:
|
||||
rec = _content(proto.read_event_record(key))
|
||||
print(f" first event key={key.hex()} size={size} B "
|
||||
f"type={_event_type(rec)}")
|
||||
t1 = time.monotonic()
|
||||
blob = proto.read_event_file(key, size)
|
||||
dt = time.monotonic() - t1
|
||||
print(f" downloaded {len(blob)} B in {dt:.1f} s "
|
||||
f"({len(blob)/dt/1024:.1f} KiB/s)")
|
||||
assert len(blob) == size
|
||||
_decode(blob, key, rec)
|
||||
|
||||
except ProtocolError as e:
|
||||
print(f"\n ABORTED {type(e).__name__}: {e}")
|
||||
return 3
|
||||
finally:
|
||||
mm.close()
|
||||
transport.close_capture()
|
||||
if a.capture:
|
||||
print(f"\n capture written:\n {transport.bw_path}\n {transport.s3_path}")
|
||||
print(" parse it with: python3 scratch/mm_frame_parse.py "
|
||||
f"{transport.bw_path} {transport.s3_path}")
|
||||
|
||||
elapsed = time.monotonic() - t0
|
||||
print(f"\n transport: {transport.reads} reads, "
|
||||
f"{transport.bytes_in} B in, {transport.bytes_out} B out, "
|
||||
f"{elapsed:.1f} s total")
|
||||
print(" Measured 2026-09-29, UM12947, same unit both ways:")
|
||||
print(" USB-B (CDC-ACM) 83 reads list_setups 0.46 s download 394 KiB/s")
|
||||
print(" RX55 (TCP) 36 reads list_setups 16.05 s download 1.6 KiB/s")
|
||||
print(" The modem needs FEWER reads, not more -- it buffers ~1 s and then")
|
||||
print(" forwards one large segment, where CDC-ACM delivers many small ones.")
|
||||
print(" Cost is ~0.65 s PER ROUND TRIP regardless of payload size, so what")
|
||||
print(" matters over cellular is the number of commands, not the bytes.\n")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
+23
-10
@@ -134,17 +134,30 @@ def probe(host: str, port: int, timeout: float) -> int:
|
||||
step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
|
||||
print("\nverdict: the MODEM answered but the unit did not.")
|
||||
print(" TCP is fine end to end — something accepted the connection.")
|
||||
print(" What is missing is the serial side. Most likely the modem is")
|
||||
print(" not forwarding to its serial port, which is what a wedged")
|
||||
print(" transparent-TCP session looks like: the slot is held by a")
|
||||
print(" connection that never closed.")
|
||||
print(" What is missing is the serial side. Two quite different")
|
||||
print(" causes produce this, and they are NOT distinguishable from")
|
||||
print(" here:")
|
||||
print("\n 1. SOMEONE ELSE HOLDS THE SESSION. These modems bridge ONE")
|
||||
print(" TCP session to serial at a time. A second connection is")
|
||||
print(" accepted and then simply not forwarded. Confirmed 2026-09-26:")
|
||||
print(" with THOR connected this probe saw exactly this; the moment")
|
||||
print(" THOR disconnected the same probe returned the serial number.")
|
||||
print(" ** Check whether THOR (or anything else) has the unit first. **")
|
||||
print("\n 2. The serial path is genuinely broken — a stale session the")
|
||||
print(" modem never released, a cable the unit cannot enumerate, or")
|
||||
print(" a unit that is off.")
|
||||
print("\n Try, in order:")
|
||||
print(" 1. ACEmanager -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot forever. 2 minutes is the value")
|
||||
print(" this project standardised on.")
|
||||
print(" 2. Reboot the modem. If that fixes it, the modem was")
|
||||
print(" holding state and the timeout is the permanent fix.")
|
||||
print(" 3. Check the unit's own screen — serial cable, power.")
|
||||
print(" 1. Disconnect any other client and re-probe. If it answers,")
|
||||
print(" it was contention, not a fault.")
|
||||
print(" 2. The cable's chipset. A Micromate drives FTDI and CDC-ACM")
|
||||
print(" only — a Prolific PL2303 gives it no serial port at all.")
|
||||
print(" lsusb: FTDI is 0403, Prolific 067b.")
|
||||
print(" 3. Power-cycle the UNIT with the cable attached (hold power")
|
||||
print(" 5 s, through the two-stage prompt). Its USB host rescans")
|
||||
print(" on cold boot; it may not on hot-swap.")
|
||||
print(" 4. AirLink OS -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot indefinitely. 2 minutes is the")
|
||||
print(" value this project standardised on.")
|
||||
sock.close()
|
||||
return 5
|
||||
|
||||
|
||||
@@ -0,0 +1,408 @@
|
||||
# Spec — a live client for Series IV (Micromate)
|
||||
|
||||
Drafted 2026-09-26, ahead of implementation. The protocol work is finished; this
|
||||
is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
|
||||
run.
|
||||
|
||||
**Read that document first.** Everything here assumes it, and every constant
|
||||
below is sourced from it rather than restated with justification.
|
||||
|
||||
---
|
||||
|
||||
## Goal and scope
|
||||
|
||||
`micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
|
||||
writers. There is no way to talk to a unit. This adds the live half, mirroring
|
||||
`minimateplus/`.
|
||||
|
||||
**In scope, first pass:**
|
||||
|
||||
- connect over TCP (a field modem) or serial/USB (a bench unit)
|
||||
- identify a unit, read its state, clock, memory and setups
|
||||
- walk the event chain and download events
|
||||
- return `Event` objects the existing codec already understands
|
||||
|
||||
**Explicitly out of scope, first pass:**
|
||||
|
||||
- ⚠ **Any write.** Setups, schedules, call-home config, monitoring start/stop,
|
||||
and per-event delete are all mapped, and none of them will be implemented
|
||||
here. **No command has ever been originated against a unit by this project**
|
||||
— every write observed was performed by THOR while we recorded. Keeping that
|
||||
true through the read client is deliberate: it means the first thing we ever
|
||||
send to a customer's instrument is a decision someone made on purpose, not a
|
||||
side effect of a client that happened to grow a method.
|
||||
- the inbound call-home session — still the one protocol unknown
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
micromate/
|
||||
framing.py NEW frame building, response parsing, checksum
|
||||
protocol.py NEW one method per wire command, returns raw payloads
|
||||
client.py NEW high-level API, returns models
|
||||
idf_file.py (existing — decodes what 0x5A returns, unchanged)
|
||||
models.py (existing — extend, do not fork)
|
||||
```
|
||||
|
||||
**Transport is reused, not rewritten.** `minimateplus/transport.py` is
|
||||
byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
|
||||
`TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
|
||||
habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller. Import it.
|
||||
|
||||
⚠ Do **not** import `minimateplus.framing`. The two framings differ in ways
|
||||
that look small and are not, and a shared module would accumulate `if series ==`
|
||||
branches until neither case is readable.
|
||||
|
||||
---
|
||||
|
||||
## `micromate/framing.py` — ✅ BUILT 2026-09-27
|
||||
|
||||
Implemented, with `tests/test_micromate_framing.py` (31 tests, all passing).
|
||||
**Two things in this section as originally drafted were wrong**, and both were
|
||||
caught by measuring against the captures before writing code rather than after.
|
||||
They are left in place below, struck through, because both are the kind of
|
||||
mistake that would be made again.
|
||||
|
||||
### Requests
|
||||
|
||||
~~Series IV accepts Series III request frames unmodified. The simplest correct
|
||||
implementation re-exports the builder rather than duplicating it:~~
|
||||
|
||||
```python
|
||||
from minimateplus.framing import build_bw_frame # ✗ WRONG — 161/218
|
||||
```
|
||||
|
||||
⚠ **`build_bw_frame` reproduces only 161 of Thor's 218 captured read frames.**
|
||||
The payload layout is identical; the stuffing is not. A Micromate escapes
|
||||
**four** byte values — `0x02`, `0x03`, `0x04`, `0x10` — where Series III
|
||||
escapes one. The frames this breaks are **every `SUB 0x5A` download**
|
||||
(`offset = 0x0400` → a literal `0x04` in `offset_hi`) and the scheduler enable.
|
||||
An unescaped `0x03`/`0x04` terminates the frame early, so the unit just does not
|
||||
answer.
|
||||
|
||||
`build_request()` with the correct escape set reproduces **218/218**.
|
||||
|
||||
✅ **Settled: `0x10` inside params needs no special handling.** The planned
|
||||
`NotImplementedError` guard is unnecessary — Thor sends
|
||||
`params = 00 00 10 00 …` on `SUB 0x5A` in five captured frames and the wire
|
||||
carries an ordinary doubled `10 10`. One rule covers the whole payload.
|
||||
|
||||
### Responses — where Series III's parser cannot follow
|
||||
|
||||
| | Series III | Micromate |
|
||||
|---|---|---|
|
||||
| frame start | `DLE STX` | **bare `STX`** |
|
||||
| `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
|
||||
| destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |
|
||||
|
||||
The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
|
||||
it scans for `DLE+STX`, which never appears.
|
||||
|
||||
The third is a genuine **simplification** — no inner-frame carve-out. Validated
|
||||
by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
|
||||
four candidate destuffing rules were tried, and only this one makes all frames
|
||||
validate.
|
||||
|
||||
### Checksum
|
||||
|
||||
```python
|
||||
def checksum(payload: bytes) -> int:
|
||||
return sum(payload) & 0xFF # payload already de-stuffed
|
||||
```
|
||||
|
||||
~~The DLE-aware variant, same as Series III's `5A` and write frames.~~
|
||||
|
||||
⚠ **Plain SUM8, not the DLE-aware variant** — 251/251 both directions. The
|
||||
DLE-aware form is the right answer paired with *Series III* de-stuffing, which
|
||||
leaves an escaped byte in the payload as two bytes. De-stuffing `10 XX → XX`
|
||||
already removes the `0x10`, so excluding it again subtracts the correction
|
||||
twice, and the result disagrees with the wire on **55 of 251** captured
|
||||
responses — every frame holding a literal `0x10`.
|
||||
|
||||
`scratch/mm_frame_parse.py` shipped with exactly that pairing. It looked clean
|
||||
only because it accepts a frame matching *either* rule, so it labelled those 55
|
||||
`SUM8` and never flagged one bad. "Zero bad checksums" was true and carried no
|
||||
information. Fixed there too.
|
||||
|
||||
### ⚠ The SUB byte can be escaped
|
||||
|
||||
When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
|
||||
Reading it positionally without destuffing reports `0x10`. This bit once
|
||||
already — `SUB 0x02` was logged as `SUB_10` for an afternoon. Destuff first,
|
||||
then index.
|
||||
|
||||
### Response shape
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class MicromateFrame:
|
||||
sub: int # response SUB; request = 0xFF - sub
|
||||
flags: int # 0xC5 Blastware line, 0x03 Thor line
|
||||
page_hi: int
|
||||
page_lo: int
|
||||
data: bytes # payload[5:], checksum stripped
|
||||
checksum_valid: bool
|
||||
|
||||
@property
|
||||
def request_sub(self) -> int: # 0xFF - sub
|
||||
@property
|
||||
def page_key(self) -> int: # uint16 BE at payload[3:5]
|
||||
@property
|
||||
def firmware_line(self) -> str: # "blastware" | "thor" | "unknown"
|
||||
@property
|
||||
def probe_length(self) -> int | None: # uint16 BE at data[3:5] (= payload[8:10])
|
||||
```
|
||||
|
||||
⚠ **`probe_length` is a uint16 BE.** Read as a single byte it under-reads
|
||||
`SUB 0x1A` by 47x — 44 against a true 2092. This is the single most expensive
|
||||
mistake available in this protocol and it has already been made once.
|
||||
|
||||
Renamed from `declared_length`, because it is **only meaningful in the reply to
|
||||
an `offset = 0` probe** — and Thor never probes. Across all 251 captured
|
||||
responses the field reads 0 or a page count, never a length, precisely because
|
||||
that session is single-step reads throughout. `page_key` is the field that
|
||||
carries meaning there. The only genuine probe reply we hold is the POLL one
|
||||
preserved in `scratch/fake_unit.py`.
|
||||
|
||||
`MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
|
||||
accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
|
||||
what distinguishes "no bytes at all" from "bytes but no complete frame" on a
|
||||
timeout, and that distinction earned its keep during the Series III work).
|
||||
|
||||
---
|
||||
|
||||
## `micromate/protocol.py` — ✅ BUILT 2026-09-27
|
||||
|
||||
Implemented, with `tests/test_micromate_protocol.py` (35 tests). Reads only;
|
||||
nothing here writes, erases or changes monitoring state.
|
||||
|
||||
The tests replay Thor's captured responses through a scripted transport and
|
||||
assert **the bytes we emit are the bytes Thor emits** — including a full replay
|
||||
of the six-event download session, all 56 `0x5A` frames byte-for-byte. That is
|
||||
a stronger guarantee than "our parser understands the device": a passing test
|
||||
means a real unit has already answered exactly that frame.
|
||||
|
||||
One method per command, returning raw payload bytes. No interpretation — that
|
||||
belongs in `client.py`.
|
||||
|
||||
**Reads use `offset = 0xFFFF`** and return the whole block in one response;
|
||||
Series III's two-step probe/data dance is unnecessary. `POLL` is the exception,
|
||||
taking its data length. Per-command offsets, all observed:
|
||||
|
||||
| command | SUB | rsp | offset | returns |
|
||||
|---|---|---|---|---|
|
||||
| poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
|
||||
| serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
|
||||
| device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
|
||||
| state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
|
||||
| monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
|
||||
| storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
|
||||
| active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
|
||||
| first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
|
||||
| next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
|
||||
| compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
|
||||
| call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
|
||||
| arm event | `0x93` | `0x6C` | — | before every event |
|
||||
| first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
|
||||
| next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
|
||||
| event record | `0x0C` | `0xF3` | `0xFFFF` | 221 B — project, location, peaks |
|
||||
| ~~event header~~ **monitor log** | `0x0A` | `0xF5` | `0xFFFF` | ⚠ 297 B, a **walk** — see below |
|
||||
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim**, 1024 B at a time |
|
||||
|
||||
⚠ **Corrected 2026-09-27, from Thor's frames.** Three rows of the table above
|
||||
were wrong or incomplete, and the last one is a different command than labelled:
|
||||
|
||||
- **`0x0A` is the monitor-log walk**, not a keyed "30 B list record" read. The
|
||||
*same request repeated* returns successive 297-byte records — serial, mode,
|
||||
thresholds — until an 11-byte ack ends the list. The device holds the cursor;
|
||||
nothing in the request selects a record. Series III reaches this data through
|
||||
a record-type discriminator on its event chain; here it has its own cursor and
|
||||
the event chain never sees it.
|
||||
- **`0x1E`/`0x1F` carry token `0xFE` at `params[7]`.** The reference documents
|
||||
all-zero params (our own probing, which also worked). Thor's form is the one
|
||||
with mileage.
|
||||
- **`0x01` has no Thor frame behind it** — it is never read in any captured
|
||||
session. Its `0xFFFF` comes from our probes.
|
||||
|
||||
And one useful negative: **no `SESSION_RESET` (`41 03`)**. Series III needs that
|
||||
2-byte signal or a monitoring unit will not answer `POLL` over TCP. Thor never
|
||||
sends it — zero occurrences across 8 sessions, including 40 frames exchanged
|
||||
with a unit that *was* monitoring.
|
||||
|
||||
⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
|
||||
Parse **forward** from `declared_length`, never backward from the end — Series
|
||||
III reads battery and memory from the end of that block, and doing so on a BD
|
||||
unit yields a battery voltage of **577.92 V**.
|
||||
|
||||
⚠ **Test the monitoring flag for non-zero**, never against a constant. It has
|
||||
read both `0x0E` and `0x0C` while monitoring.
|
||||
|
||||
### `0x5A` — a bounded chunk loop, and much simpler than Series III
|
||||
|
||||
⚠ **Corrected 2026-09-27.** This section said "no chunk loop — one request
|
||||
returns the whole event", with `offset_word = 0x1000 + 2 * ceil(size / 512)`.
|
||||
That describes our own 2026-09-23 probes, which set `offset_hi = 0x10`. **Thor
|
||||
chunks**, and Thor's form is the one verified from bytes on disk:
|
||||
|
||||
```python
|
||||
n = ceil(size / 1024) # size from the chain walk
|
||||
for i in range(n):
|
||||
offset = min(1024, size - 1024 * i) # a BYTE COUNT
|
||||
params = key4 + bytes(6) if i == 0 else bytes(2) + pack(">H", 1024*i) + bytes(6)
|
||||
file_bytes += response.data[11:] # response data is exactly offset + 11
|
||||
```
|
||||
|
||||
Verified on all six bench events (4,076 → 13,424 B): `sum(offsets) == size`
|
||||
exactly, with the predicted chunk count and final offset every time.
|
||||
|
||||
Still no arming ritual for `0x5A` itself, no `STRT` end-offset parsing and no
|
||||
`TERM` frame — the simplification the original claim celebrated is real, it just
|
||||
is not single-shot. (`SUB 0x93` arms the *chain walk*, before `1E`/`1F`, not
|
||||
the download.)
|
||||
|
||||
The concatenated payload **is** the `.IDFW` file, byte for byte — so it feeds
|
||||
`micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.
|
||||
|
||||
⚠ Do not port the Series III `5A` walk. Its address arithmetic caused a 5x
|
||||
over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
|
||||
III side. None of that applies here: the chunk index is a byte offset into the
|
||||
file, bounded by a size the device told us, and it cannot run past the event.
|
||||
|
||||
⚠ **`assert sum(len(chunk) - 11 for chunk in chunks) == size`.** A silently
|
||||
short event is the failure mode this project has been bitten by repeatedly on
|
||||
the Series III side, and here the check is free because the size is known up
|
||||
front.
|
||||
|
||||
---
|
||||
|
||||
## `micromate/client.py` — ✅ BUILT (read half) 2026-09-27
|
||||
|
||||
`connect()`, `get_state()`, `get_active_setup()`, `list_setups()` plus
|
||||
`MicromateDeviceInfo` / `MicromateState` in `models.py`. 26 tests, every
|
||||
response constant a real captured data section.
|
||||
|
||||
⚠ **`connect()` is deliberately narrower than this spec asked for.** The spec
|
||||
said to mirror Thor's `POLL → SERIAL → 0x49 → POLL` "because it is known-good".
|
||||
Measurement showed the four-command form is Thor's *connection check*, present
|
||||
in 3 of 8 sessions, and its fourth frame repeats its first — so `connect()`
|
||||
sends the three reads that gather something. `0x01` is not read at all: Thor
|
||||
never reads it, its layout is unmapped, and `firmware_line` comes free from any
|
||||
response's flags byte.
|
||||
|
||||
Event-chain methods (`list_events`, `download_event`, `get_event`) are step 4
|
||||
and not yet written; `MicromateProtocol.read_event_file()` already does the
|
||||
download.
|
||||
|
||||
```python
|
||||
class MicromateClient:
|
||||
def __init__(self, transport: BaseTransport): ...
|
||||
def open(self) / close(self) / is_open(self)
|
||||
|
||||
# identity and state
|
||||
def connect(self) -> DeviceInfo # poll → serial → device info → state
|
||||
def get_state(self) -> UnitState # monitoring?, clock, battery, memory
|
||||
def get_active_setup(self) -> str
|
||||
def list_setups(self) -> list[str] # 0x3F → 0x40… until empty
|
||||
|
||||
# events
|
||||
def list_events(self) -> list[EventRef] # 0x93 → 0x1E → 0x1F… (key + size)
|
||||
def download_event(self, ref) -> bytes # raw .IDFW/.IDFH
|
||||
def get_event(self, ref) -> Event # download + decode via idf_file
|
||||
```
|
||||
|
||||
`connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
|
||||
⚠ but note the reference records that **whether the unit requires it is
|
||||
untested**. Do it because it is known-good, not because it is known-necessary,
|
||||
and say so in the docstring.
|
||||
|
||||
`list_events()` returns the key *and* the size, because `download_event()` needs
|
||||
the size to compute its offset word.
|
||||
|
||||
---
|
||||
|
||||
## Tests
|
||||
|
||||
**Offline, from captured bytes — no hardware.** This is the part worth doing
|
||||
first, because it can be fully verified tonight's-captures-style before any unit
|
||||
is involved.
|
||||
|
||||
```
|
||||
tests/test_micromate_framing.py
|
||||
```
|
||||
|
||||
⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
|
||||
not depend on files being present. **Embed the frames as hex constants** — they
|
||||
are 19–138 bytes each. ✅ Done; what actually landed:
|
||||
|
||||
| case | source | why |
|
||||
|---|---|---|
|
||||
| POLL probe reply, 19 B | captured (via `fake_unit.py`) | shortest valid frame; the only real probe reply we hold |
|
||||
| `0x5A` chunk, 138 B | captured | holds literal `0x10` **and** literal `0x41` — the checksum case, and proves ACK is not escaped |
|
||||
| `0x49` state reply, 25 B | captured | a literal `0x02`, escaped |
|
||||
| `0x48` file reply, 24 B | captured | escaped `0x04` in `data[0]` — one byte late without destuffing |
|
||||
| 8 Thor request frames | captured | byte-for-byte against `build_request()`, incl. both `0x5A` forms and the scheduler enable |
|
||||
| `probe_length = 0x082C` | **synthesised** | no probe reply for `0x1A` exists on disk — the 9-24-26 session never probes |
|
||||
| Thor-line reply, `flags = 0x03` | **synthesised** | no 11.0BD capture is in the repo; built by flipping one byte of the real POLL reply |
|
||||
| escaped checksum byte | **synthesised** | the shortest real one is 1,070 B, too long to embed for one assertion |
|
||||
| truncated / corrupt / split-across-feeds | derived | parser must return nothing, flag rather than swallow, and survive any split point |
|
||||
|
||||
⚠ Synthesised frames are marked `SYNTH_`-style in the test and each says what it
|
||||
stands in for and why no capture was available. Do not let that set grow
|
||||
quietly: the `flags = 0x03` case in particular is the **only** coverage of half
|
||||
the fleet, and it deserves a real 11.0BD capture the next time UM20147 is on a
|
||||
bench.
|
||||
|
||||
Two corpus-backed tests run when the captures happen to be on the box and skip
|
||||
cleanly otherwise: **251 response frames parse with zero bad checksums**, and
|
||||
**`build_request()` reproduces 218/218 read frames**. The second is the test
|
||||
that would have caught the escape-set error, so it is worth the skip marker.
|
||||
|
||||
⚠ Do **not** assert against `scratch/mm_frame_parse.py`'s output as the original
|
||||
plan proposed. That script accepts either checksum rule and is wrong about
|
||||
which one is right — using it as an oracle would have pinned the bug.
|
||||
|
||||
**Live, second:** against the bench unit on mint-mac via `mm_link.py`.
|
||||
`connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
|
||||
then `download_event()` and assert the bytes decode and match a
|
||||
`/db/import/idf_file` ingest of the same event.
|
||||
|
||||
---
|
||||
|
||||
## Order of work
|
||||
|
||||
1. ✅ `framing.py` + its tests — **done 2026-09-27**, 31 tests, offline
|
||||
2. ✅ `protocol.py` + its tests — **done 2026-09-27**, 35 tests, offline
|
||||
3. ✅ `client.py` + its tests — **done 2026-09-27**, 26 tests, offline
|
||||
4. the event chain and `download_event()`
|
||||
5. decode end-to-end and compare against a store event
|
||||
|
||||
Steps 1–2 need no hardware at all.
|
||||
|
||||
**Worth carrying forward.** Both steps began by measuring against the captures
|
||||
rather than trusting this document, and both found errors in it — three in the
|
||||
framing rules (the escape set, 26% of frames; the checksum, 22%; the `0x5A`
|
||||
chunk model) and three more in the command table (`0x0A`'s meaning, the
|
||||
`1E`/`1F` token, `0x01`'s provenance). All six fail quietly. The captures are on
|
||||
disk and a measure-then-write loop costs about two minutes per rule, so keep
|
||||
doing it for `client.py`'s field offsets — and treat this spec as a plan, not a
|
||||
source.
|
||||
|
||||
---
|
||||
|
||||
## Open questions to settle while implementing
|
||||
|
||||
- ~~**Request param stuffing**~~ — ✅ settled 2026-09-27; no special handling.
|
||||
- **Is the single-request `0x5A` form real?** Our 2026-09-23 probes set
|
||||
`offset_hi = 0x10` and appeared to get a whole 11 KB event back, where Thor
|
||||
chunks at 1024 B. Plausibly a distinct streaming mode that returns several
|
||||
frames. One bench test settles it; implement Thor's form regardless.
|
||||
- **Is THOR's preamble required?** Try one command cold and find out; it is a
|
||||
two-minute test with the bench unit and it removes a ritual if unnecessary.
|
||||
- **`Event` model fit** — Series IV carries fields Series III lacks (setup file
|
||||
name, `LMic`/`SMic` channels). Extend `micromate/models.py`; do not fork the
|
||||
shared `Event`.
|
||||
- **Which `0x0C` fields to trust.** The peak float there runs 2–5% above
|
||||
`max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
|
||||
established. The reference marks it do-not-rely-on — prefer decoded samples.
|
||||
@@ -150,12 +150,38 @@ Every frame in this session was produced by `build_bw_frame(sub, offset)` with
|
||||
no Series IV changes, and the device accepted all of them:
|
||||
|
||||
```
|
||||
[ACK 0x41] [STX 0x02] [10 10] [flags 00] [SUB] [00] [00] [offset] [params×10] [chk] [ETX 0x03]
|
||||
[ACK 0x41] [STX 0x02] [10 10] [flags 00] [SUB] [00] [offset_hi] [offset_lo] [params×10] [chk] [ETX 0x03]
|
||||
```
|
||||
|
||||
⚠ Only the doubled `BW_CMD` (`10 10`) form has been exercised. Whether other
|
||||
literal `0x10` bytes inside params require stuffing is **untested** — none of
|
||||
the probes sent carried one.
|
||||
The *payload layout* is identical to Series III. The *stuffing* is not.
|
||||
|
||||
> #### ⚠ Correction, 2026-09-27 — `build_bw_frame()` is NOT sufficient
|
||||
>
|
||||
> This section used to say requests were Series III frames "unmodified", and
|
||||
> the client spec accordingly planned to re-export
|
||||
> `minimateplus.framing.build_bw_frame`. Measured against Thor's own frames,
|
||||
> that builder reproduces **161 of 218** captured read frames. It escapes only
|
||||
> `0x10`; a Micromate escapes four bytes (see *The escape set* below).
|
||||
>
|
||||
> The 57 it gets wrong are not obscure:
|
||||
>
|
||||
> | frame | why it breaks |
|
||||
> |---|---|
|
||||
> | **every `SUB 0x5A` download** | `offset = 0x0400` puts a literal `0x04` in `offset_hi`, which must go out as `10 04` |
|
||||
> | `SUB 0x47` scheduler enable | `params[7] = 0x03` → `10 03` |
|
||||
>
|
||||
> An unescaped `0x03` or `0x04` *is* a frame terminator, so the unit sees a
|
||||
> short frame and does not answer — indistinguishable from a dead unit, and
|
||||
> event download would have hit it on the first request ever sent.
|
||||
>
|
||||
> `micromate/framing.py:build_request()` reproduces **218/218**. Pinned per
|
||||
> frame in `tests/test_micromate_framing.py`.
|
||||
|
||||
✅ **Settled 2026-09-27: a `0x10` inside params needs no special handling.**
|
||||
Previously flagged untested. Thor sends `SUB 0x5A` with
|
||||
`params = 00 00 10 00 …` in five captured frames and the wire carries the
|
||||
ordinary doubled `10 10`. There is no Series-III-style partial-stuffing
|
||||
carve-out here — one rule covers the whole payload including the checksum.
|
||||
|
||||
### Responses — Series III *minus the DLE prefix*
|
||||
|
||||
@@ -181,18 +207,75 @@ mismatch.
|
||||
[5+] data
|
||||
```
|
||||
|
||||
### Checksum — the DLE-aware variant
|
||||
### Checksum — plain SUM8 of the de-stuffed payload
|
||||
|
||||
```python
|
||||
chk = sum(b for b in payload if b != 0x10) & 0xFF
|
||||
chk = sum(payload) & 0xFF # payload already de-stuffed
|
||||
```
|
||||
|
||||
Confirmed on every frame captured. The `POLL` probe response contains no
|
||||
`0x10` and so cannot distinguish plain SUM8 from the DLE-aware form; the
|
||||
`POLL` **data** response contains a `0x10` at payload offset 42, and only the
|
||||
DLE-aware rule matches there. This is the same checksum Series III uses for
|
||||
its `5A` bulk-stream and write frames — not the plain SUM8 of ordinary
|
||||
Series III reads.
|
||||
**251/251 responses and 251/251 requests**, across every capture in
|
||||
`bridges/captures/9-24-26 - micromate2/`.
|
||||
|
||||
> #### ⚠ Correction, 2026-09-27 — "the DLE-aware variant"
|
||||
>
|
||||
> This section previously specified
|
||||
> `chk = sum(b for b in payload if b != 0x10) & 0xFF`, and that is wrong **as
|
||||
> paired with the uniform de-stuffing rule below.**
|
||||
>
|
||||
> The earlier claim was not a misreading; it was a correct rule attached to the
|
||||
> wrong convention. The DLE-aware form belongs with *Series III* de-stuffing,
|
||||
> which leaves an escaped byte in the payload as **two** bytes — there,
|
||||
> skipping the `0x10` is the necessary correction. De-stuffing `10 XX → XX`
|
||||
> already removes it, so excluding `0x10` as well **subtracts the correction
|
||||
> twice**.
|
||||
>
|
||||
> Cost of the pairing, measured: it disagrees with the wire on **55 of 251**
|
||||
> response frames — every frame whose payload holds a literal `0x10`. A clean
|
||||
> example, from the `0x5A` chunk at offset `0x0070` of
|
||||
> `raw_s3_…_Download_events_then_delete_1_event.bin`: the wire says `0xC1`,
|
||||
> plain SUM8 says `0xC1`, the DLE-aware form says `0x91`.
|
||||
>
|
||||
> **Why nobody noticed:** `scratch/mm_frame_parse.py` accepts a frame matching
|
||||
> *either* rule and labels which one hit. It reported those 55 frames as
|
||||
> `SUM8` and never flagged one bad, so "zero bad checksums across 24, 38 and
|
||||
> 40-frame sessions" was true and told us nothing about which rule was right.
|
||||
> A tool that tries every candidate cannot falsify any of them — if it is going
|
||||
> to stay permissive, it has to *report the split*, not just the pass.
|
||||
>
|
||||
> Pinned by `tests/test_micromate_framing.py`.
|
||||
|
||||
### The escape set — exactly four bytes
|
||||
|
||||
A Micromate escapes `0x02`, `0x03`, `0x04` and `0x10`, each prefixed with a
|
||||
`DLE`, **and nothing else** — in both directions.
|
||||
|
||||
Established by re-stuffing every captured frame and comparing to the wire:
|
||||
|
||||
| candidate escape set | responses reproduced | requests reproduced |
|
||||
|---|---|---|
|
||||
| `{0x10}` — the Series III rule | 130/251 | 177/251 |
|
||||
| `{0x10, 0x03}` | 135/251 | 184/251 |
|
||||
| `{0x10, 0x02, 0x03}` | 178/251 | 192/251 |
|
||||
| **`{0x10, 0x02, 0x03, 0x04}`** | **251/251** | **251/251** |
|
||||
| `{0x10, 0x02, 0x03, 0x04, 0x41}` | 196/251 | — |
|
||||
|
||||
Corroborated independently by the byte that follows a wire `DLE`: across all
|
||||
251 responses it is only ever `0x02` (933×), `0x03` (510×), `0x04` (655×) or
|
||||
`0x10` (1179×). Nothing else ever appears there.
|
||||
|
||||
The last row matters because ACK looks like it ought to be escaped and is not —
|
||||
a literal `0x41` in the data goes out bare.
|
||||
|
||||
**The checksum byte is escaped too.** Three captured responses have a checksum
|
||||
of `0x02`/`0x03`/`0x04` and all three arrive as `10 XX` immediately before the
|
||||
terminating `ETX`. No captured *request* happened to land on one, so Thor's
|
||||
behaviour there is unobserved — but a device and its host share one framing
|
||||
routine, and the alternative is a frame the far end truncates, so escape it.
|
||||
|
||||
This supersedes nothing: the de-stuffing rule `10 XX → XX` stays exactly as
|
||||
documented. Knowing only four values are ever escaped is what makes that
|
||||
uniform rule *exact* rather than merely convenient — and it is what an
|
||||
**encoder** needs, which the write path will.
|
||||
|
||||
### The probe response carries the data length
|
||||
|
||||
@@ -573,8 +656,63 @@ Series III ignores a `5A` probe unless preceded by
|
||||
answers a **bare `5A` request** with nothing before it. That whole ritual is
|
||||
gone.
|
||||
|
||||
### ⚠ Correction, 2026-09-27 — THOR uses a 1024-byte chunk loop
|
||||
|
||||
The section below ("One request returns the entire event; there is no chunk
|
||||
loop") describes **our own probes**, and it is not how THOR downloads an event.
|
||||
Read from the wire, THOR's sequence per event is:
|
||||
|
||||
```
|
||||
0x93 (arm) → 0x1E / 0x1F → key + size
|
||||
0x0C → the 210-byte record
|
||||
0x5A × n → n = ceil(size / 1024)
|
||||
```
|
||||
|
||||
and the `0x5A` frames are a plain bounded chunk walk:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| chunk `i` offset | `min(1024, size − 1024·i)` — a **byte count** |
|
||||
| chunk 0 params | `[key4][6 × 0x00]` — the key means "from the start" |
|
||||
| chunk `i>0` params | `[00 00][uint16 BE of 1024·i][6 × 0x00]` |
|
||||
| response data | exactly `offset + 11` bytes; the file bytes are `data[11:]` |
|
||||
| response `page_key` | `offset // 256` — a page count, not an address |
|
||||
|
||||
**Verified on all six bench events**, sizes 4,076 → 13,424 bytes:
|
||||
`sum(offsets) == size` **exactly** in every case, with the predicted chunk
|
||||
count and predicted final offset. No `STRT` parsing, no `TERM` frame, no
|
||||
over-read — that part of the original claim holds, and it is still far simpler
|
||||
than the Series III walk.
|
||||
|
||||
```
|
||||
key size(1E) chunks sum(offsets) last offset
|
||||
055d4a81 4076 4 4076 0x03ec
|
||||
055d4a82 11032 11 11032 0x0318
|
||||
055d4a83 11502 12 11502 0x00ee
|
||||
055d4a84 13424 14 13424 0x0070
|
||||
055d4a85 8746 9 8746 0x022a
|
||||
055d4a86 6092 6 6092 0x03cc
|
||||
```
|
||||
|
||||
**These are probably two different modes, not a contradiction.** THOR's
|
||||
`offset_hi` is the chunk length (`0x04`, `0x03`, `0x00` …). Our single-request
|
||||
probes set `offset_hi = 0x10` — which in Series III is precisely the
|
||||
bulk-stream marker `build_5a_frame()` writes raw. So `0x10XX` plausibly means
|
||||
"stream until done" and returns **several** frames, which the parser of the day
|
||||
concatenated into the 11,049 bytes recorded below. That reconciles both
|
||||
observations, but it is a hypothesis: those 2026-09-23 captures never landed in
|
||||
the repo, so it cannot be re-derived from bytes on disk.
|
||||
|
||||
**Implement THOR's chunked form.** It is verified byte-exact across six events
|
||||
and five distinct sizes, and it is what the firmware runs every day. The
|
||||
single-request form is worth one bench test as an optimisation — `offset_hi =
|
||||
0x10` and count the frames — but not worth depending on first.
|
||||
|
||||
### The offset word is a LENGTH, not a position
|
||||
|
||||
⚠ Superseded as the implementation path by the correction above; the *reading*
|
||||
of the field is right and is what makes THOR's chunk walk make sense.
|
||||
|
||||
This is the key divergence. Series III walks chunks by absolute flash
|
||||
address, stepping `0x0200` per request. On the Micromate the offset word
|
||||
requests *how much to send*:
|
||||
@@ -591,10 +729,11 @@ offset_word = 0x1000 + 2 × pages pages = ceil(event_size / 512)
|
||||
| `0x102C` | 22 | **11,033 — the whole event** |
|
||||
|
||||
`event_size` comes from the chain walk (the 4 bytes after the key in
|
||||
`1E`/`1F`). **One request returns the entire event**; there is no chunk loop,
|
||||
no `STRT` end-offset parsing, and no `TERM` frame. Over-requesting is safe —
|
||||
`0x1030` (24 pages) returned exactly the same bytes as `0x102C`, so the device
|
||||
caps at the real size.
|
||||
`1E`/`1F`). **One request returns the entire event** ⚠ *— true of our
|
||||
`offset_hi = 0x10` probes; see the 2026-09-27 correction above. THOR chunks* —
|
||||
there is no `STRT` end-offset parsing and no `TERM` frame in either form.
|
||||
Over-requesting is safe — `0x1030` (24 pages) returned exactly the same bytes
|
||||
as `0x102C`, so the device caps at the real size.
|
||||
|
||||
Params are the Series III *probe* form: `[0x00][key4][6 × 0x00]`.
|
||||
|
||||
@@ -636,12 +775,231 @@ per-sample-exact applies.
|
||||
### What a full read now looks like
|
||||
|
||||
```
|
||||
0x93 → arm (THOR sends this before every 1E/1F)
|
||||
1E → first key + size
|
||||
0C(key) → project/client/operator, timestamp, peaks
|
||||
5A(key, 0x1000+2×ceil(size/512)) → the whole .IDFW
|
||||
1F → next key + size (until null sentinel)
|
||||
5A × ceil(size/1024) → the .IDFW, 1024 bytes at a time
|
||||
0x93 → 1F → next key + size (until null sentinel)
|
||||
```
|
||||
|
||||
## ✅ Verified against real hardware, both transports (2026-09-29)
|
||||
|
||||
`micromate/{framing,protocol,client}.py` driven against **UM12947** by
|
||||
`bridges/mm_client_check.py`, read-only, over both physical paths:
|
||||
|
||||
| | USB-B "PC" (CDC-ACM) | RX55 → TCP :9034 |
|
||||
|---|---|---|
|
||||
| identity, state, setups | identical | identical |
|
||||
| bytes in / out | **11,580 / 761** | **11,580 / 761** |
|
||||
| `list_setups()` (24 entries) | 0.46 s | **16.05 s** |
|
||||
| 4,076 B download | 394 KiB/s | 1.6 KiB/s |
|
||||
| transport reads | **83** | **36** |
|
||||
|
||||
**The wire bytes are identical on both paths** — same counts to the byte. The
|
||||
protocol does not care which physical layer it runs over, which is what the
|
||||
modem path needed to prove.
|
||||
|
||||
### ⚠ The modem needs FEWER reads, not more — the prediction was backwards
|
||||
|
||||
`mm_client_check.py` originally printed "a modem path should show MORE reads
|
||||
for the same bytes". It shows **36 against USB's 83**. The modem *coalesces*:
|
||||
it buffers for up to ~1 s and then forwards one large TCP segment, where
|
||||
CDC-ACM delivers many small chunks as they arrive.
|
||||
|
||||
That is reassuring rather than alarming — the risk was a frame arriving split
|
||||
across reads, and the modem splits **less** than USB does. The client reads to
|
||||
frame completion rather than using `read_until_idle`'s idle-gap detection, and
|
||||
handled both without a retry.
|
||||
|
||||
### 🔑 ~0.65 s per round trip over cellular, independent of payload
|
||||
|
||||
This is the number that matters for SFM's design.
|
||||
|
||||
```
|
||||
list_setups() 24 commands 16.05 s ≈ 0.67 s each
|
||||
get_state() 1 command 0.64 s
|
||||
connect() 4 commands 2.53 s ≈ 0.63 s each
|
||||
download 4 chunks 2.40 s ≈ 0.60 s each (1024 B per chunk)
|
||||
```
|
||||
|
||||
The cost is per *command*, not per byte: a 1,024-byte chunk and a 16-byte
|
||||
state read cost the same. **Over cellular, minimise round trips, not bytes.**
|
||||
|
||||
Concrete consequences:
|
||||
|
||||
- Enumerating setups costs **16 seconds** on a unit with 24 of them. Cache it;
|
||||
do not refresh it on a timer.
|
||||
- A 13 KB event is 14 chunks ≈ 8.4 s of latency against ~0.03 s of data. If
|
||||
the single-request `offset_hi = 0x10` streaming mode is real (see
|
||||
*`SUB 0x5A`*), it would cut a download to **one** round trip — that is worth
|
||||
the two-minute bench test on its own.
|
||||
- THOR's measured polling cost should be re-read in this light: its per-unit
|
||||
status sweep is round trips, and round trips are what cellular charges for.
|
||||
|
||||
### End to end: downloaded events decode with the existing codec
|
||||
|
||||
All six bench events, assembled by `MicromateProtocol.read_event_file()` from
|
||||
captured `0x5A` responses and fed to `micromate.idf_file.read_idf_file()`:
|
||||
|
||||
| key | size | type | decode |
|
||||
|---|---|---|---|
|
||||
| `055d4a81` | 4,076 | histogram | ✅ |
|
||||
| `055d4a82` | 11,032 | waveform | ✅ 12,288 samples |
|
||||
| `055d4a83` | 11,502 | waveform | ✅ 12,288 samples |
|
||||
| `055d4a84` | 13,424 | waveform | ✅ 12,288 samples |
|
||||
| `055d4a85` | 8,746 | waveform | ✅ 8,192 samples |
|
||||
| `055d4a86` | 6,092 | histogram | ✅ |
|
||||
|
||||
**No new codec work is needed.** The bytes off the wire are the bytes
|
||||
`thor-watcher` forwards today, so `/db/import/idf_file` ingests a directly
|
||||
downloaded event unchanged.
|
||||
|
||||
### ✅ `11.0BD` confirmed on real hardware (2026-09-30)
|
||||
|
||||
UM20147 read over USB. **Every inference about the Thor firmware line held**,
|
||||
and two of the three were covered only by *synthesised* test frames until now:
|
||||
|
||||
| inference | predicted | UM20147 reported |
|
||||
|---|---|---|
|
||||
| `flags = 0x03` identifies the line | `thor` | ✅ `thor` |
|
||||
| `0x03` is ETX, so it arrives as `10 03` | destuffs before indexing | ✅ implied — the line could not be identified otherwise |
|
||||
| model string is shorter | `MM/ISEE/S` | ✅ `MM/ISEE/S` |
|
||||
| `0x1C` is 4 bytes longer, extras **trailing** | forward offsets survive | ✅ battery **3.55 V**, clock correct to the second |
|
||||
|
||||
That last row is the one that mattered. Series III's from-the-end offsets would
|
||||
have produced **577.92 V** on this unit — a number so obviously wrong it is a
|
||||
cheap self-check, which is why `mm_client_check.py` watches for it. Reading a
|
||||
plausible 3.55 V and a correct clock confirms the four extra bytes really are
|
||||
trailing.
|
||||
|
||||
Also read cleanly: serial `UM20147`, 5 setups (`factory.MMB` → `test2.mmb`),
|
||||
active setup `test2.mmb`, 15,000,000 bytes total and free. 21 reads, 2,165 B in.
|
||||
|
||||
**One protocol stack drives the whole fleet regardless of firmware line** — the
|
||||
headline finding of 2026-09-23 — is now demonstrated by a working client, not
|
||||
just by matching response SUBs.
|
||||
|
||||
### ✅ A MONITORING unit answers reads normally (2026-09-30)
|
||||
|
||||
UM20147 was read while actively monitoring — `monitoring=True` on **both**
|
||||
indicators (`0x49` content[0] and `0x1C` content[1], which agree) — and
|
||||
answered POLL, serial, state, monitor status and a 5-entry setup walk with no
|
||||
special handling.
|
||||
|
||||
**This confirms the `SESSION_RESET` finding with our own client.** Series III
|
||||
*requires* a bare `41 03` before POLL or a monitoring unit will not answer over
|
||||
TCP. That was previously inferred from its absence in THOR's captures; it is now
|
||||
demonstrated directly, on the other firmware line, by a client that never sends
|
||||
one.
|
||||
|
||||
Note THOR *refuses to send a setup* to a monitoring unit — that restriction is
|
||||
about writes. Reads are unaffected.
|
||||
|
||||
Monitoring also moves two values, so neither is a stable thing to compare
|
||||
against: battery drifted 3.55 → 3.50 V, and memory free dropped 15,000,000 →
|
||||
14,848,448 (monitoring allocates as it runs).
|
||||
|
||||
### ⚠ Setup-name case is NOT consistent between `0x41` and `0x40`
|
||||
|
||||
Same unit, same session, same file:
|
||||
|
||||
```
|
||||
0x41 active setup name -> test2.MMB
|
||||
0x40 setup-list entry -> test2.mmb
|
||||
```
|
||||
|
||||
An earlier run 45 minutes before reported `test2.mmb` from *both*, so the case
|
||||
is not fixed per command either. What changed in between is that the unit
|
||||
started monitoring — plausibly it rewrites the active-setup name in a canonical
|
||||
form when loading a setup, but that is a guess from two samples.
|
||||
|
||||
**Practical consequence, which does not depend on the cause: compare setup file
|
||||
names case-insensitively.** An exact-string test of "is the active setup one I
|
||||
know about?" answers *no* on this unit. Worth fixing before it is a field bug —
|
||||
Thor's own UI couples the scheduler to a setup name, so name matching is on the
|
||||
path for anything that replaces it.
|
||||
|
||||
### ✅ Real BD bytes, and the exact numbers (2026-09-30)
|
||||
|
||||
Captured with `mm_client_check.py --capture` and read out with
|
||||
`scratch/mm_frame_parse.py`: 11 frames each direction, 0 bad checksums. Both
|
||||
BD-specific responses are now real fixtures in `tests/`, replacing synthesised
|
||||
ones.
|
||||
|
||||
**`SUB 0x1C`, BD, 59 B data — the block the from-the-end offsets break on:**
|
||||
|
||||
```
|
||||
content[ 1] 0x0e monitoring
|
||||
content[ 2:6] 1e 09 07ea 30 Sep 2026
|
||||
content[ 6] 0x60 = 96 ⚠ still unidentified (32/100/116 on CB)
|
||||
content[ 7:10] 0e 00 31 14:00:49 ✓ matches the wall clock
|
||||
content[34:36] 01 5e 3.50 V
|
||||
content[36:40] 00 e4 e1 c0 15,000,000
|
||||
content[40:44] 00 e2 91 c0 14,848,448
|
||||
content[44:48] 0f a0 00 04 ⚠ the four extra bytes
|
||||
```
|
||||
|
||||
`data[0] = 0x30` = 48 = 59 − 11, against CB's `0x2C` = 44. The low-byte length
|
||||
rule and the +4 both hold.
|
||||
|
||||
⚠ **The extra four bytes are `0f a0 00 04`, not `0f a0 00 00`** as recorded
|
||||
earlier. Only the first two look fixed. Nothing reads them, but do not treat
|
||||
the last as padding.
|
||||
|
||||
**And the warning is now demonstrated on the unit it predicted:**
|
||||
`data[-10:-8]` on this block is `e1 c0` → **577.92 V**. Exactly the figure in
|
||||
the A/B section. Forward offsets give 3.50 V.
|
||||
|
||||
**`SUB 0x5B` POLL, BD:**
|
||||
|
||||
```
|
||||
content[ 3] 0x56 ⚠ 0x50 on UM12947 — THIS BYTE VARIES BETWEEN UNITS
|
||||
content[ 4] "Instantel\0"
|
||||
content[26] "MM/ISEE/S\0" (CB: "MM/ISEE/S/IO\0", same offset)
|
||||
```
|
||||
|
||||
⚠ **content[3] is not a constant.** It is printable on both units — `P` and `V`
|
||||
— which is the second reason a printable-run scan is the wrong way to read this
|
||||
block: it would return `PInstantel` on one unit and `VInstantel` on the other.
|
||||
Read the vendor at the fixed offset content[4]; the model is at content[26] on
|
||||
both lines, but anchor on `MM/` since only its tail varies.
|
||||
|
||||
### ✅ Download from a BD unit (2026-09-30)
|
||||
|
||||
UM20147, event `055d4a81`, 4,796 B, **type read as histogram from the `0x0C`
|
||||
byte and decoded as `.IDFH` on the first try.** The chunk walk, the offset
|
||||
arithmetic, the 11-byte chunk prefix and the record-type byte all hold on the
|
||||
Thor firmware line unchanged.
|
||||
|
||||
The type byte is now **7 events across both firmware lines** — 4 waveform,
|
||||
3 histogram — and has predicted correctly every time. Still not a large
|
||||
sample, and the fallback stays.
|
||||
|
||||
### 🔑 EVENT KEYS COLLIDE ACROSS UNITS — dedup must be (serial, key)
|
||||
|
||||
UM20147's first event is `055d4a81`. So is UM12947's. **Different units,
|
||||
different sizes (4,796 B vs 4,076 B), different contents, identical key.**
|
||||
|
||||
This confirms *keys are a sequential counter* (already recorded) and adds the
|
||||
consequence: **the counter starts from the same value on every unit, so a key is
|
||||
meaningless without its serial.**
|
||||
|
||||
⚠ **Anything that stores, deduplicates or addresses Series IV events must key on
|
||||
`(serial, event_key)`.** A store keyed on the event key alone will silently
|
||||
treat one unit's event as a duplicate of another's — and the failure is invisible,
|
||||
because the second event is simply never ingested.
|
||||
|
||||
Series III has a related hazard the ACH server already handles: after an erase
|
||||
its counter resets, so keys are reused *within* a unit, which is why
|
||||
`ach_state.json` tracks `max_downloaded_key` per serial. Series IV inherits that
|
||||
and adds cross-unit collision on top.
|
||||
|
||||
### Still not covered
|
||||
- **A unit that is monitoring**, and a unit with a nearly-full event buffer.
|
||||
- **The inbound call-home session** — still the one protocol unknown.
|
||||
|
||||
---
|
||||
|
||||
## Setups are FILES, not a config block
|
||||
|
||||
Series III has one compliance config you overwrite. Series IV keeps **named
|
||||
@@ -1076,6 +1434,143 @@ the protocol's requirements*. Thor sends it before trivial reads too, so it may
|
||||
be habit rather than handshake. Do not assume it is mandatory. Note `POLL` here carries `offset = 0x0030` (its data
|
||||
length), not `0xFFFF` — `POLL` is the one read Thor still addresses by length.
|
||||
|
||||
> #### ⚠ Narrowed 2026-09-27 — there is no *universal* preamble
|
||||
>
|
||||
> Across all 8 captured sessions, the only invariant is that **the session opens
|
||||
> with `POLL`**. What follows depends on the operation:
|
||||
>
|
||||
> | opening sequence | sessions | operation |
|
||||
> |---|---|---|
|
||||
> | `5b 15 49 5b …` | 3 | status refresh / monitoring / ACH change |
|
||||
> | `5b 41 08 2e 1a da …` | 3 | setup push |
|
||||
> | `5b 94 48 48 48 …` | 2 | scheduler read |
|
||||
>
|
||||
> So `POLL → SERIAL → 0x49 → POLL` is Thor's **connection check**, not a
|
||||
> handshake the protocol demands — it appears where Thor wants to refresh what
|
||||
> it displays. Treat `POLL` as the one thing to send first.
|
||||
|
||||
### 🔑 No `SESSION_RESET` — the Series III requirement does not carry over
|
||||
|
||||
Series III needs a bare `41 03` (ACK + ETX, no STX) to wake a unit that is
|
||||
actively monitoring; without it the unit will not answer `POLL` over TCP, and
|
||||
`protocol.startup()` sends it before and between the POLL frames.
|
||||
|
||||
**Thor never sends it to a Micromate.** Zero occurrences across all 8 sessions
|
||||
— including `raw_bw_20260924_191214_turn_on_monitormode_…`, which exchanges 40
|
||||
frames with a unit that *was* monitoring at the time.
|
||||
|
||||
### Measured offsets and response lengths (all read off Thor's frames)
|
||||
|
||||
`offset = 0xFFFF` for everything except two commands. Data lengths are from
|
||||
UM12947 (`11.0CB`) and are **orientation, not assertions** — `0x1C` is 4 bytes
|
||||
longer on the Thor line.
|
||||
|
||||
| SUB | rsp | offset | data | notes |
|
||||
|---|---|---|---|---|
|
||||
| `0x5B` POLL | `0xA4` | **`0x0030`** | 59 | the one length-addressed read |
|
||||
| `0x15` serial | `0xEA` | **`0x000A`** | 21 | |
|
||||
| `0x49` state | `0xB6` | `0xFFFF` | 16 | |
|
||||
| `0x1C` monitor status | `0xE3` | `0xFFFF` | 55 | +4 on `11.0BD` |
|
||||
| `0x06` storage range | `0xF9` | `0xFFFF` | 47 | |
|
||||
| `0x08` event index | `0xF7` | `0xFFFF` | 101 | contents unmapped |
|
||||
| `0x2E` trigger config | `0xD1` | `0xFFFF` | 39 | |
|
||||
| `0x1A` compliance | `0xE5` | `0xFFFF` | 2103 | one frame, not Series III's four |
|
||||
| `0x2C` call-home | `0xD3` | `0xFFFF` | 137 | |
|
||||
| `0x3F`/`0x40`/`0x41` setups | `0xC0`/`0xBF`/`0xBE` | `0xFFFF` | 266 | |
|
||||
| `0x93` arm | `0x6C` | `0xFFFF` | 11 | ack only |
|
||||
| `0x1E`/`0x1F` chain | `0xE1`/`0xE0` | `0xFFFF` | 19 | ⚠ **token `0xFE` at `params[7]`** |
|
||||
| `0x0C` event record | `0xF3` | `0xFFFF` | 221 | full key at `params[4:8]` |
|
||||
| `0x0A` monitor log | `0xF5` | `0xFFFF` | 297 | ⚠ a **walk** — see below |
|
||||
| `0x5A` download | `0xA5` | computed | offset+11 | 1024-byte chunk loop |
|
||||
|
||||
An acknowledgement is an **11-byte data section**, and that doubles as the
|
||||
end-of-list signal on the walks.
|
||||
|
||||
#### ⚠ `0x1E`/`0x1F` carry token `0xFE`
|
||||
|
||||
`params = 00 00 00 00 00 00 00 fe 00 00` on all 7 captured chain reads — the
|
||||
same `token_params(0xFE)` form Series III uses to arm its bulk stream. The
|
||||
event-chain section above documents **all-zero params**; that was our own browse
|
||||
probing, which also worked. Both evidently do, but Thor's form is the one with
|
||||
mileage on it, and it is sent on browse and download alike.
|
||||
|
||||
#### 🔑 `SUB 0x0A` is the monitor-log walk, not a keyed read
|
||||
|
||||
The command table long described `0x0A` as a keyed "waveform header / partial
|
||||
record" read, by analogy with Series III. What the bytes show is a **cursor
|
||||
walk**: the *same request repeated*, the device advancing its own position.
|
||||
|
||||
```
|
||||
0x93 → 1E → 0x0A ×8 (297 B each: "UM12947", "Histo…", "Ver…", " 0.49", " 28.4")
|
||||
0x0A (11 B ack = end of list)
|
||||
```
|
||||
|
||||
All nine frames carry identical params (`…00 00 4a 81 00 00`), so nothing in the
|
||||
request selects the record. Terminate on a response of `ACK_DATA_LEN` (11).
|
||||
|
||||
⚠ The `4a 81` is the **low two bytes** of the event key then in play
|
||||
(`055d4a81`). One key cannot distinguish "the key's low half" from "a cursor
|
||||
handle that happened to equal it" — both produce those bytes. It does not
|
||||
matter operationally, since the walk works with the params held constant.
|
||||
|
||||
This is a genuine structural divergence: Series III reaches the same data
|
||||
through a record-type discriminator (`0x2C` partial vs `0x46` full) *on its
|
||||
event walk*, so partial records and events share one chain. Here the monitor
|
||||
log has its own cursor and the event chain never sees it.
|
||||
|
||||
#### 🔑 Every response has an 11-byte prefix — and its length byte lies
|
||||
|
||||
**Content starts at `data[11]`.** One rule, every command.
|
||||
|
||||
⚠ **`data[0]` is the content length `& 0xFF`, and there is no high byte
|
||||
anywhere in the prefix.** `data[1]` is zero on every frame examined. So it
|
||||
reads as a perfectly good length field for any response under 256 bytes — which
|
||||
is most of them — and then:
|
||||
|
||||
| command | true content | `data[0]` says |
|
||||
|---|---|---|
|
||||
| `0x1A` compliance | 2,092 | **44** |
|
||||
| `0x0A` monitor log | 286 | **30** |
|
||||
| `0x5A` download chunk | 1,024 | **0** |
|
||||
| `0x41` setup name | 255 | 255 ✓ (only just) |
|
||||
|
||||
182 of 251 captured responses agree with a naive uint16 LE read; the 69 that do
|
||||
not are exactly the ones ≥ 256 bytes.
|
||||
|
||||
**This is the third time a length in this protocol has been read too narrow** —
|
||||
after `payload[9]` vs `payload[8:10]` in the probe response, and after
|
||||
`data[0]` here. The pattern is worth naming rather than fixing case by case:
|
||||
*take the content as `data[11:]` and let the frame's own length bound it.*
|
||||
Nothing needs the declared length; the frame already knows how long it is.
|
||||
|
||||
#### `SUB 0x5B` POLL content — where the strings actually sit
|
||||
|
||||
```
|
||||
content[3] 0x50 ⚠ printable as "P", immediately before the vendor string
|
||||
content[4] "Instantel\0"
|
||||
content[17:26] binary — 06 00 c3 f0 4a 00 e4 19 4f 00 74 02
|
||||
content[26] "MM/ISEE/S/IO\0" ← "MM/ISEE/S" on the Thor line
|
||||
```
|
||||
|
||||
⚠ **Do not parse this by scanning for printable runs.** That `0x50` at
|
||||
content[3] is printable, so a run scan returns `PInstantel`. Nothing
|
||||
distinguishes a length or tag byte from text by inspection. Read the vendor
|
||||
from the fixed offset and find the model by searching for `MM/` — structural
|
||||
rather than positional, which is what survives the model string's
|
||||
firmware-line difference.
|
||||
|
||||
#### `SUB 0x3F`/`0x40` setup walk — the terminator is an empty name
|
||||
|
||||
23 responses on the bench unit: 22 names, then a record whose name field is all
|
||||
zeros. The empty name **is** the end of list — not an error, not a setup.
|
||||
`factory.MMB` first, `TEST1.mmb` last.
|
||||
|
||||
#### `SUB 0x01` has no Thor frame behind it
|
||||
|
||||
Thor never reads device info in any captured session. `0xFFFF` for `0x01` comes
|
||||
from our own 2026-09-23 probes — it answered correctly on both firmware lines,
|
||||
but it is the only read in the table with no Thor precedent.
|
||||
|
||||
### `SUB 0x96` / `0x97` — start and stop monitoring ✅
|
||||
|
||||
Identical to Series III, including the acks:
|
||||
@@ -1249,6 +1744,33 @@ length, and the timestamps are sequential across the recording session.
|
||||
|
||||
### Record type + filename: generate it, don't detect it
|
||||
|
||||
> #### 🔑 Update 2026-09-29 — there IS a type field, in the `0x0C` record
|
||||
>
|
||||
> **`0x0C` content[11]: `0x07` = waveform, `0x08` = histogram.**
|
||||
>
|
||||
> Found by diffing the six bench events' `0x0C` records against their known
|
||||
> types: exactly one byte separates the two groups and is constant within each.
|
||||
> It then predicted the right suffix for **6 of 6** on a blind re-run.
|
||||
>
|
||||
> ⚠ **Six events, split 4/2.** That is thin evidence for a byte that could be
|
||||
> a counter, a channel count or a mode. Treat it as a strong candidate, not a
|
||||
> settled field, and **keep a fallback**: `bridges/mm_client_check.py` tries the
|
||||
> other suffix on failure and reports when the guess was wrong, which is how
|
||||
> this would be caught rather than silently mis-filing an event.
|
||||
>
|
||||
> This does not overturn the section below — the *filename* still has to be
|
||||
> generated, and the type still comes out of the protocol rather than the
|
||||
> payload. It just means the type is available from a command we already send
|
||||
> for every event, instead of needing to be carried out of the chain walk
|
||||
> separately.
|
||||
>
|
||||
> ⚠ The claim below that the type comes from "`SUB 0x0A` length `0x1E` =
|
||||
> histogram, `0x00` = waveform" **is not visible in the 2026-09-24 download
|
||||
> capture**, where `0x0A` appears once as a standalone monitor-log walk after
|
||||
> the last chain entry, not per event. Either it was observed in a session not
|
||||
> in the repo, or the two are being conflated. The `0x0C` byte is reproducible
|
||||
> from bytes on disk; prefer it.
|
||||
|
||||
`read_idf_file()` decides waveform vs histogram from the **filename suffix** —
|
||||
and there is no filename when downloading over the wire.
|
||||
|
||||
@@ -1289,7 +1811,55 @@ given it, and `/db/import/idf_file` needs no change at all.
|
||||
must carry it out of the chain walk. Losing it means losing the ability to
|
||||
name the file correctly.
|
||||
|
||||
### ⚠ Unresolved: the `0x0C` peak float
|
||||
### ✅ RESOLVED 2026-09-30: the `0x0C` float is the PER-SAMPLE peak vector sum
|
||||
|
||||
**`0x0C` content[`Tran` label − 12], float32 BE, = the peak vector sum computed
|
||||
sample by sample.** Exact to 0.000% on all four bench waveforms, against a PVS
|
||||
recomputed from the decoded samples:
|
||||
|
||||
| key | `0x0C` float | per-sample PVS | err | `sqrt(Σpeak²)` | `max(T,V,L)` |
|
||||
|---|---|---|---|---|---|
|
||||
| `…82` | 1.37198 | 1.37198 | **+0.000%** | 1.41746 | 1.37063 |
|
||||
| `…83` | 2.35414 | 2.35414 | **+0.000%** | 2.37824 | 2.22274 |
|
||||
| `…84` | 3.51515 | 3.51515 | **−0.000%** | 3.78731 | 3.44194 |
|
||||
| `…85` | 0.42267 | 0.42267 | **−0.000%** | 0.46678 | 0.41985 |
|
||||
|
||||
> #### ⚠ Retraction — why this looked unresolved
|
||||
>
|
||||
> The previous entry (below, kept) observed the float running 2–5% above
|
||||
> `max(T,V,L)` and concluded it was **not** the peak vector sum, because
|
||||
> "computed PVS runs *higher* than both".
|
||||
>
|
||||
> That computed PVS was `sqrt(T² + V² + L²)` **from the three reported channel
|
||||
> peaks** — and that is an *upper bound* on the real thing, not the real thing.
|
||||
> The channel maxima do not occur at the same instant, so combining them
|
||||
> overstates the true peak. Every measurement in the old table is consistent
|
||||
> with the correct answer: `max(T,V,L) ≤ stored ≤ sqrt(Σpeak²)`, in all five rows.
|
||||
>
|
||||
> A near-miss worth recording: on UM20147's event the stored float matched
|
||||
> `sqrt(Σpeak²)` to **0.048%**, which briefly looked like proof it *was* that
|
||||
> quantity. It is not — the peaks on that event simply happen to be nearly
|
||||
> coincident in time. **One sample agreeing with a formula is not evidence when
|
||||
> a second formula fits it equally well;** the six CB events separated them
|
||||
> immediately (errors to −9.5%).
|
||||
|
||||
**The offset is now established, not inferred** — four independent confirmations
|
||||
at 0.000%, plus a fifth on the other firmware line.
|
||||
|
||||
**Why this is worth having: it is a free self-check on the decoder.** The device
|
||||
computed this number from the same samples we decode, independently of our
|
||||
codec. If a decoded event's per-sample PVS does not match the stored float, the
|
||||
decode is wrong. Given this codebase's history of *silent* channel truncation —
|
||||
walker bugs that shorten a channel and raise nothing — a per-event invariant
|
||||
that costs one float comparison is cheap insurance. Worth wiring into the
|
||||
download path.
|
||||
|
||||
⚠ It remains true that `max(T,V,L)` and `sqrt(Σpeak²)` are **not**
|
||||
interchangeable with it, so a consumer wanting the vector sum must use this
|
||||
field or decode the samples — it cannot be reconstructed from the three
|
||||
per-channel peaks.
|
||||
|
||||
#### Superseded: the original "unresolved" entry
|
||||
|
||||
The float32 extracted from `0x0C` runs 2–5% above `max(Tran, Vert, Long)` from
|
||||
the decoded samples:
|
||||
@@ -1302,15 +1872,38 @@ the decoded samples:
|
||||
| `…84` | 3.5152 | 3.4419 |
|
||||
| `…85` | 0.4227 | 0.4198 |
|
||||
|
||||
It is not peak vector sum either (computed PVS runs *higher* than both). The
|
||||
~~It is not peak vector sum either (computed PVS runs *higher* than both). The
|
||||
field may not be the peak at all — its offset was inferred from a byte marker,
|
||||
not established. **Do not rely on it** until it is pinned properly.
|
||||
not established. **Do not rely on it** until it is pinned properly.~~
|
||||
|
||||
Worth noting the histogram (`…81`) and the loudest waveform (`…84`) report
|
||||
*identical* peaks to four decimals, in both measures. That is self-consistent:
|
||||
the histogram's single 1-minute interval spans the whole thumping session, so
|
||||
its maximum should equal the loudest event in it.
|
||||
|
||||
### `0x0C` field map, from UM20147 (11.0BD, 2026-09-30)
|
||||
|
||||
Real bytes, 221 B response → 210 B content declared at `data[0] = 0xd2`:
|
||||
|
||||
```
|
||||
content[ 0] day 0x1e = 30
|
||||
content[ 1] month 0x09
|
||||
content[ 2:4] year BE 0x07ea = 2026
|
||||
content[ 4] ⚠ 0xb3 = 179 — unidentified, same slot as 0x1C's content[6]
|
||||
content[ 5:8] h:m:s 13:27:33
|
||||
content[11] RECORD TYPE 0x08 histogram / 0x07 waveform
|
||||
content[12] "Location\0" sensor-location label
|
||||
content[34] "test2\0" the setup file name, without its extension
|
||||
content[76] "UM20147\0" serial
|
||||
content[86] float32 BE the per-sample PEAK VECTOR SUM (= Tran − 12)
|
||||
content[98] "Tran\0\0" + float32 BE at +6
|
||||
content[112] "Vert" … content[126] "Long" … content[140] "Mic" …
|
||||
```
|
||||
|
||||
⚠ The setup name here is `test2` — no extension — where `0x41` reported
|
||||
`test2.MMB` and `0x40` reported `test2.mmb` in the same session. Three commands,
|
||||
three spellings of one file. See *Setup-name case*.
|
||||
|
||||
## Event download, per-event delete, and ACH config (2026-09-25)
|
||||
|
||||
Three captures with operator-supplied ground truth, including Thor screenshots of
|
||||
@@ -2119,20 +2712,94 @@ FTDI is VID `0403`, Prolific is `067b`. Note also that counterfeit FTDI chips
|
||||
are common in cheap cables — they carry FTDI's VID but may not behave like one,
|
||||
and an embedded host with a single driver is far less forgiving than Linux.
|
||||
|
||||
### Does this explain the 2026-09-22 field outage?
|
||||
### ✅ This accounts for the 2026-09-22 failure
|
||||
|
||||
⚠ **Not on the timeline as reported.** A Prolific cable does not fail
|
||||
*gradually* — it never enumerates, so the modem port never works at all. The
|
||||
unit was described as working at first and degrading.
|
||||
An earlier draft of this section said the timeline did not fit, because the unit
|
||||
was described as *working at first and then degrading* — which a Prolific cable
|
||||
cannot do, since it never enumerates at all.
|
||||
|
||||
There is one story where it fits: if the initial success was over a **different
|
||||
path** — the USB **PC** port, or a bench test before deployment — that would work
|
||||
regardless of which serial cable was attached. The modem path would then have
|
||||
been broken from the moment it was deployed, and "it worked and then stopped"
|
||||
would be a recollection conflating two connection types days after the fact.
|
||||
**The timeline was the thing that needed correcting, not the finding.** From the
|
||||
operator, 2026-09-25:
|
||||
|
||||
Plausible, unverified, and recorded as such. Identifying that unit's cable would
|
||||
settle it.
|
||||
1. Initial setup — **one** modem, **one** cable. Worked: configs sent, status
|
||||
read, no trouble.
|
||||
2. Two Micromates then had to be deployed, so a **second cable** was fetched to
|
||||
run both at once.
|
||||
3. **Only one of the two ever worked.** The other would not talk over its modem
|
||||
at any point.
|
||||
4. That unit was never deployed — a **MiniMate Plus was put out in its place**,
|
||||
and the Micromate came back to the bench.
|
||||
5. The cable on that bench unit is a **Benfei (Prolific PL2303)**.
|
||||
|
||||
So nothing degraded. The "worked at first" was the original single-cable setup;
|
||||
the failure began when the pair was split across two cables and one of them was
|
||||
the wrong chipset. **One working and one not, set up side by side, is the
|
||||
signature of a mixed cable supply** — not of a unit fault.
|
||||
|
||||
✅ **The cable travelled home with the unit** (confirmed 2026-09-25). So the
|
||||
Benfei on the bench is physically the cable from the failed office setup — not a
|
||||
substitute picked up later. There is no inference left in the chain:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| the cable from the failed unit | **is** a Benfei / Prolific PL2303 |
|
||||
| the Micromate's USB host | has **no** Prolific driver, in either firmware line |
|
||||
| a laptop on that same cable + modem | round-trips perfectly |
|
||||
| the Micromate on it | never answered |
|
||||
|
||||
The FTDI cable arriving 2026-09-26 is now a confirmation rather than a test.
|
||||
|
||||
**Corollary worth knowing.** The standing workaround — *"just deploy a Series III
|
||||
instead"* — works partly because a **MiniMate Plus has a DB-9 port directly on the
|
||||
unit.** No USB-to-serial adapter anywhere in the path, so there is no chipset to
|
||||
get wrong. That workaround has been quietly routing around this exact failure
|
||||
mode.
|
||||
|
||||
### ✅ RESOLVED (2026-09-26) — and it took a cold boot as well as the cable
|
||||
|
||||
Swapping to a genuine FTDI cable (`lsusb`: `0403:6001`, FT232) **did not work on
|
||||
its own.** A probe immediately after the swap returned the same *connected, no
|
||||
reply*.
|
||||
|
||||
**A cold boot of the unit was also required.** Holding the power button for five
|
||||
seconds — through the two-stage prompt that disconnects the internal battery —
|
||||
and powering back up **with the cable already attached**, brought it straight up.
|
||||
THOR at the office connected the moment it finished booting, and an independent
|
||||
probe returned `UM12947, idle` over cellular.
|
||||
|
||||
⚠ **The USB host does not rescan on hot-swap.** The Micromate enumerates its
|
||||
USB-A host port at boot; once it has failed to identify a device there it does
|
||||
not appear to try again. Changing that cable is a power-cycle operation, and
|
||||
anyone swapping one in the field who probes immediately will conclude the new
|
||||
cable is faulty too.
|
||||
|
||||
So the full chain for a Micromate on a modem is:
|
||||
|
||||
1. an **FTDI** or CDC-ACM cable — a Prolific PL2303 gives the unit no serial port
|
||||
2. a **cold boot** with that cable attached
|
||||
|
||||
### ✅ These modems bridge ONE session at a time — confirmed
|
||||
|
||||
Long suspected, never demonstrated. Demonstrated now, by accident:
|
||||
|
||||
| | probe result |
|
||||
|---|---|
|
||||
| THOR connected to the unit | `TCP connect ok` … **`NO REPLY`** |
|
||||
| THOR disconnected, nothing else changed | `reply 68 B`, **`UM12947`**, idle |
|
||||
|
||||
The PAD **accepts** a second TCP connection and then does not forward it. It
|
||||
does not refuse, and it does not close — it simply never bridges.
|
||||
|
||||
⚠ **This means "connected but no reply" has two causes that are indistinguishable
|
||||
from the client side:** a genuinely broken serial path, and *somebody else
|
||||
already has the unit*. `bridges/mm_probe.py` originally reported only the first,
|
||||
which would send a diagnosis in exactly the wrong direction; its verdict now
|
||||
names both and tells you to eliminate contention first.
|
||||
|
||||
**For SFM this is a design constraint, not a footnote.** Our receiver and THOR
|
||||
cannot both hold a unit, and during any migration both will exist. "Another
|
||||
client holds this unit" needs to be a distinct, visible state — not folded into
|
||||
a failure, and certainly not into a green tick.
|
||||
|
||||
### How this was isolated — the method is reusable
|
||||
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
"""
|
||||
client.py — high-level API for a live Micromate (Series IV).
|
||||
|
||||
Owns the transport, turns raw payloads into models. Read-only, like the layer
|
||||
below it: nothing here writes, erases, or changes monitoring state.
|
||||
|
||||
with MicromateClient(TcpTransport("63.45.161.30", 9034)) as mm:
|
||||
info = mm.connect()
|
||||
print(info) # UM12947 MM/ISEE/S/IO blastware fw idle
|
||||
print(mm.get_state()) # idle 2026-09-25 01:14:05 3.80 V memory 0.4% used
|
||||
for name in mm.list_setups():
|
||||
print(name)
|
||||
|
||||
The response layout, measured rather than assumed
|
||||
-------------------------------------------------
|
||||
**Every response carries an 11-byte prefix, and the content starts at
|
||||
``data[11]``.** That one rule covers every command.
|
||||
|
||||
⚠ **``data[0]`` looks like the content length and is only its low byte.** A
|
||||
2,092-byte setup block (`SUB 0x1A`) reports 44, and a 1,024-byte download chunk
|
||||
reports 0. It happens to be right for every response shorter than 256 bytes,
|
||||
which is most of them — so it reads as a working length field right up until it
|
||||
silently loses 2,048 bytes. There is no high byte anywhere in the prefix; it is
|
||||
``length & 0xFF`` and nothing more.
|
||||
|
||||
This is the same trap as ``MicromateFrame.probe_length``, in a different place,
|
||||
and it is now the third time a length in this protocol has been read too narrow.
|
||||
**Take the content as ``data[11:]`` and let the frame's own length bound it.**
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
from minimateplus.transport import BaseTransport
|
||||
|
||||
from .models import MicromateDeviceInfo, MicromateState
|
||||
from .protocol import MicromateProtocol, ProtocolError
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# The content of every response begins here; the first 11 bytes are a prefix
|
||||
# whose only decoded field is an unreliable low-byte length (see module docstring).
|
||||
CONTENT = 11
|
||||
|
||||
# Field offsets, relative to the start of content. Sources are named because
|
||||
# two of them disagree with docs/micromate_protocol_reference.md.
|
||||
_STATE_FLAG = 0 # 0x49: 0x00 idle, 0x02 monitoring
|
||||
|
||||
_MS_FLAG = 1 # 0x1C: monitoring flag — test NON-ZERO
|
||||
_MS_DAY, _MS_MONTH, _MS_YEAR = 2, 3, slice(4, 6)
|
||||
_MS_UNKNOWN_6 = 6 # ⚠ NOT the hour — see read note below
|
||||
_MS_HOUR, _MS_MIN, _MS_SEC = 7, 8, 9
|
||||
_MS_BATTERY = slice(34, 36) # uint16 BE, volts × 100
|
||||
_MS_MEM_TOTAL = slice(36, 40) # uint32 BE
|
||||
_MS_MEM_FREE = slice(40, 44) # uint32 BE
|
||||
|
||||
# A setup-list walk that does not terminate is a bug, not a big fleet. The
|
||||
# bench unit holds 22 setups; this is a generous ceiling, not a limit.
|
||||
_MAX_SETUPS = 512
|
||||
|
||||
def _content(data: bytes) -> bytes:
|
||||
"""Strip the 11-byte response prefix."""
|
||||
return data[CONTENT:] if len(data) > CONTENT else b""
|
||||
|
||||
|
||||
def _cstring(buf: bytes, offset: int = 0) -> str:
|
||||
"""A null-terminated ASCII run, stripped."""
|
||||
return buf[offset:].split(b"\x00")[0].decode("ascii", "replace").strip()
|
||||
|
||||
|
||||
class MicromateClient:
|
||||
"""High-level read-only client for one Micromate.
|
||||
|
||||
Owns the transport, unlike ``MicromateProtocol``, which borrows it.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
transport: BaseTransport,
|
||||
recv_timeout: float = 10.0,
|
||||
strict_checksums: bool = True,
|
||||
) -> None:
|
||||
self._transport = transport
|
||||
self._proto = MicromateProtocol(
|
||||
transport, recv_timeout=recv_timeout, strict_checksums=strict_checksums
|
||||
)
|
||||
self._firmware_line: Optional[str] = None
|
||||
|
||||
# ── Lifecycle ─────────────────────────────────────────────────────────────
|
||||
|
||||
def open(self) -> None:
|
||||
self._transport.connect()
|
||||
|
||||
def close(self) -> None:
|
||||
self._transport.disconnect()
|
||||
|
||||
def is_open(self) -> bool:
|
||||
return self._transport.is_connected()
|
||||
|
||||
def __enter__(self) -> "MicromateClient":
|
||||
self.open()
|
||||
return self
|
||||
|
||||
def __exit__(self, *_) -> None:
|
||||
self.close()
|
||||
|
||||
@property
|
||||
def protocol(self) -> MicromateProtocol:
|
||||
"""The wire layer, for anything this class does not wrap yet."""
|
||||
return self._proto
|
||||
|
||||
# ── Identity ──────────────────────────────────────────────────────────────
|
||||
|
||||
def connect(self, *, with_active_setup: bool = True) -> MicromateDeviceInfo:
|
||||
"""`POLL → SERIAL → state`, plus the active setup name.
|
||||
|
||||
⚠ **This is deliberately not Thor's full preamble.** Thor sends
|
||||
`POLL → SERIAL → 0x49 → POLL` and the client spec said to copy it
|
||||
verbatim on the grounds that it is known-good. Measuring all 8 captured
|
||||
sessions showed the only invariant is that a session **opens with
|
||||
POLL** — the four-command form appears in 3 of 8 and is Thor's
|
||||
*connection check*, run where it wants to refresh what it displays. The
|
||||
trailing POLL is a repeat of the first.
|
||||
|
||||
So this sends the three reads that actually gather something. Dropping
|
||||
the fourth is a judgement call on measured evidence, not a proof that
|
||||
nothing depends on it; if a unit ever refuses the next command after a
|
||||
cold connect, put it back and say so in the protocol reference.
|
||||
|
||||
`SUB 0x01` (device info) is **not** read. Thor never reads it in any
|
||||
captured session, its field layout is unmapped beyond eight `1.0f`
|
||||
floats, and `firmware_line` — the one thing we would want from it — comes
|
||||
free from the flags byte of any response.
|
||||
"""
|
||||
poll = self._proto.poll()
|
||||
self._firmware_line = poll.firmware_line
|
||||
|
||||
manufacturer, model = self._parse_poll(poll.data)
|
||||
serial = _cstring(_content(self._proto.read_serial()))
|
||||
monitoring = self._parse_state(self._proto.read_state())
|
||||
|
||||
info = MicromateDeviceInfo(
|
||||
serial=serial,
|
||||
manufacturer=manufacturer,
|
||||
model=model,
|
||||
firmware_line=poll.firmware_line,
|
||||
monitoring=monitoring,
|
||||
)
|
||||
if with_active_setup:
|
||||
try:
|
||||
info.active_setup = self.get_active_setup()
|
||||
except ProtocolError as e:
|
||||
# Not worth failing a connect over: a unit with no setup loaded
|
||||
# is a real state, and the caller can still read everything else.
|
||||
log.warning("active setup unreadable: %s", e)
|
||||
log.info("connected: %s", info)
|
||||
return info
|
||||
|
||||
@staticmethod
|
||||
def _parse_poll(data: bytes) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Manufacturer and model out of the POLL block.
|
||||
|
||||
`Instantel` sits at content[4] and the model at content[26], with 13
|
||||
binary bytes between them.
|
||||
|
||||
⚠ A generic "find the printable runs" scan does **not** work here, which
|
||||
cost a test failure before it cost anything worse. content[3] is `0x50`
|
||||
— printable as `P` — sitting immediately before `Instantel`, so a run
|
||||
scan returns `PInstantel`. Nothing distinguishes a length or tag byte
|
||||
from text by inspection.
|
||||
|
||||
So: the manufacturer comes from a fixed offset, and the model is found by
|
||||
searching for `MM/`. That anchor is structural rather than positional,
|
||||
which matters because the model string **differs by firmware line** —
|
||||
`MM/ISEE/S/IO` on the Blastware build, `MM/ISEE/S` on the Thor build —
|
||||
and only its tail changes.
|
||||
"""
|
||||
c = _content(data)
|
||||
manufacturer = _cstring(c, 4) or None
|
||||
|
||||
idx = c.find(b"MM/")
|
||||
model = _cstring(c, idx) if idx >= 0 else None
|
||||
return manufacturer, model
|
||||
|
||||
@staticmethod
|
||||
def _parse_state(data: bytes) -> Optional[bool]:
|
||||
"""`SUB 0x49` content[0]: 0x00 idle, 0x02 monitoring.
|
||||
|
||||
⚠ Tested for non-zero, never against `0x02`. The sibling flag in
|
||||
`SUB 0x1C` has read both `0x0E` and `0x0C` while monitoring, so this
|
||||
family of flags is not a stable enum.
|
||||
"""
|
||||
c = _content(data)
|
||||
return bool(c[_STATE_FLAG]) if c else None
|
||||
|
||||
# ── State ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def get_state(self) -> MicromateState:
|
||||
"""`SUB 0x1C` — monitoring, device clock, battery, memory.
|
||||
|
||||
⚠ Every offset here is **forward from the start of content**, never
|
||||
backward from the end. Series III reads battery and memory from the end
|
||||
of this block, and this block is **4 bytes longer on the Thor firmware
|
||||
line** — applying from-the-end offsets to a `11.0BD` unit yields a
|
||||
battery voltage of 577.92 V. The four extra bytes are trailing, so
|
||||
from-the-start offsets hold for both lines.
|
||||
|
||||
✅ **Confirmed on `11.0BD` 2026-09-30.** UM20147 read back 3.55 V and
|
||||
a clock correct to the second over USB, so the from-the-start offsets do
|
||||
survive the four extra trailing bytes. Had they not, the battery would
|
||||
have read 577.92 V — which is what makes this cheap to check.
|
||||
"""
|
||||
data = self._proto.read_monitor_status()
|
||||
c = _content(data)
|
||||
if len(c) < 44:
|
||||
raise ProtocolError(
|
||||
f"monitor status content is {len(c)} B, need at least 44"
|
||||
)
|
||||
|
||||
battery = int.from_bytes(c[_MS_BATTERY], "big") / 100.0
|
||||
return MicromateState(
|
||||
monitoring=bool(c[_MS_FLAG]),
|
||||
device_time=self._parse_clock(c),
|
||||
battery_volts=battery,
|
||||
memory_total_bytes=int.from_bytes(c[_MS_MEM_TOTAL], "big"),
|
||||
memory_free_bytes=int.from_bytes(c[_MS_MEM_FREE], "big"),
|
||||
raw=data,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _parse_clock(c: bytes) -> Optional[datetime.datetime]:
|
||||
"""The unit's own clock, in its own local time.
|
||||
|
||||
⚠ **content[6] is not part of the time.** The layout is day, month,
|
||||
year, *one unidentified byte*, then h/m/s — so the hour is at content[7].
|
||||
The protocol reference's `SUB 0x1C` section has this right and names
|
||||
`data[17]` as unidentified; its one-line summary in the divergences list
|
||||
("day/month/year/h/m/s at `data[13:21]`") reads as six contiguous fields
|
||||
and is the version worth not trusting.
|
||||
|
||||
Re-measured here across three captures: content[6] read 32, 100 and 116,
|
||||
none a valid hour, while content[7:10] gave 19:12:25, 19:13:34 and
|
||||
01:14:05 against capture filenames stamped 19:12:14, 19:12:14 and
|
||||
01:14:03 — each seconds to a minute after its session opened, which is
|
||||
what a device clock should do.
|
||||
|
||||
content[6] is undecoded and deliberately not exposed.
|
||||
"""
|
||||
try:
|
||||
return datetime.datetime(
|
||||
year=int.from_bytes(c[_MS_YEAR], "big"),
|
||||
month=c[_MS_MONTH],
|
||||
day=c[_MS_DAY],
|
||||
hour=c[_MS_HOUR],
|
||||
minute=c[_MS_MIN],
|
||||
second=c[_MS_SEC],
|
||||
)
|
||||
except ValueError as e:
|
||||
# A unit with a dead clock battery reports an impossible date. That
|
||||
# is information, not a reason to fail the whole state read.
|
||||
log.warning("device clock unreadable (%s): %s", e, c[2:10].hex(" "))
|
||||
return None
|
||||
|
||||
# ── Setups ────────────────────────────────────────────────────────────────
|
||||
|
||||
def get_active_setup(self) -> str:
|
||||
"""`SUB 0x41` — the loaded `.MMB` file name, e.g. `TEST1.mmb`."""
|
||||
return _cstring(_content(self._proto.read_active_setup_name()))
|
||||
|
||||
def list_setups(self) -> list[str]:
|
||||
"""`0x3F` then `0x40`… — every setup file stored on the unit.
|
||||
|
||||
A cursor walk: the device holds the position, so the same `0x40` request
|
||||
returns the next name. **An empty name terminates the list** — it is
|
||||
not an error and not a real setup.
|
||||
|
||||
Measured on the bench unit: 23 responses, 22 names then the empty one,
|
||||
`factory.MMB` first through `TEST1.mmb` last.
|
||||
"""
|
||||
names: list[str] = []
|
||||
raw = self._proto.read_first_setup()
|
||||
for _ in range(_MAX_SETUPS):
|
||||
name = _cstring(_content(raw))
|
||||
if not name:
|
||||
return names
|
||||
names.append(name)
|
||||
raw = self._proto.read_next_setup()
|
||||
|
||||
raise ProtocolError(
|
||||
f"setup list did not terminate after {_MAX_SETUPS} entries — the "
|
||||
f"device cursor is not advancing"
|
||||
)
|
||||
@@ -0,0 +1,304 @@
|
||||
"""
|
||||
framing.py — frame codec for the Instantel Micromate (Series IV) wire protocol.
|
||||
|
||||
A Micromate answers Series III *command* frames, so the request side looks
|
||||
familiar. The framing underneath is not the same, and the differences are all
|
||||
of the kind that produce a silently-ignored frame rather than an error:
|
||||
|
||||
Series III response: [DLE 0x10] [STX 0x02] … [chk] [ETX 0x03]
|
||||
Micromate response: [STX 0x02] … [chk] [ETX 0x03]
|
||||
^ no leading DLE
|
||||
|
||||
That missing byte is why `minimateplus.framing.S3FrameParser` returns *nothing*
|
||||
on Micromate traffic — it locates frames by scanning for `DLE STX`, which never
|
||||
occurs. A capture holding 12 acknowledged writes reads as 12 unanswered
|
||||
requests.
|
||||
|
||||
De-stuffed payload layout (both directions):
|
||||
|
||||
request response
|
||||
[0] CMD 0x10 [0] CMD 0x00
|
||||
[1] flags 0x00 [1] flags 0xC5 / 0x03 ← firmware line
|
||||
[2] SUB [2] SUB 0xFF − request_SUB
|
||||
[3] 0x00 [3] PAGE_HI
|
||||
[4] offset_hi [4] PAGE_LO
|
||||
[5] offset_lo [5+] data
|
||||
[6:16] params (10 bytes)
|
||||
|
||||
Everything below was established against the 251 request and 251 response
|
||||
frames in `bridges/captures/9-24-26 - micromate2/` (UM12947, firmware 11.0CB).
|
||||
Where a rule is asserted, the number of frames it was checked on is given — the
|
||||
two rules that look like small details cost 26% and 22% of frames respectively
|
||||
when guessed wrong, so the counts are the point.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
# ── Protocol byte constants ───────────────────────────────────────────────────
|
||||
|
||||
DLE = 0x10 # Data Link Escape
|
||||
STX = 0x02 # Start of text — begins a frame
|
||||
ETX = 0x03 # End of text — ends a frame
|
||||
ACK = 0x41 # Frame-start marker on the request side
|
||||
|
||||
MM_CMD = 0x10 # payload[0] in a request
|
||||
MM_RSP_CMD = 0x00 # payload[0] in a response
|
||||
|
||||
# payload[1] of a response identifies the firmware line it came from.
|
||||
# ⚠ Two units, one of each — a strong hypothesis, not a proven encoding.
|
||||
FLAGS_BLASTWARE = 0xC5 # the 11.0CB line (UM12947)
|
||||
FLAGS_THOR = 0x03 # the 11.0BD line (UM20147)
|
||||
|
||||
# ⚠ THE ESCAPE SET. A Micromate escapes exactly these four byte values,
|
||||
# prefixing each with a DLE — and nothing else. Established by re-stuffing
|
||||
# every captured frame and comparing to the wire: 251/251 responses and 251/251
|
||||
# requests reproduce byte-for-byte with this set, and no other candidate set
|
||||
# reproduces even 200 of either.
|
||||
#
|
||||
# The two near-misses are worth naming, because both look plausible:
|
||||
# * `{0x10}` alone — the Series III rule — reproduces 130/251 responses and
|
||||
# 177/251 requests.
|
||||
# * adding ACK (0x41) reproduces only 196/251 responses: a literal 0x41 in
|
||||
# the data is NOT escaped.
|
||||
_ESCAPED = frozenset({STX, ETX, 0x04, DLE})
|
||||
|
||||
# A response header is 5 bytes; a frame must also carry its checksum.
|
||||
_MIN_PAYLOAD = 5
|
||||
_REQUEST_PAYLOAD_SIZE = 16
|
||||
|
||||
|
||||
# ── Stuffing ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def stuff(data: bytes) -> bytes:
|
||||
"""Escape every byte the Micromate escapes: `XX` → `10 XX` for the four."""
|
||||
out = bytearray()
|
||||
for b in data:
|
||||
if b in _ESCAPED:
|
||||
out.append(DLE)
|
||||
out.append(b)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def unstuff(data: bytes) -> bytes:
|
||||
"""Reverse `stuff()`: `10 XX` → `XX`, for any XX.
|
||||
|
||||
Uniform, with no inner-frame carve-out — which is a real simplification
|
||||
over Series III, where `DLE+ETX` inside a frame is literal data that must
|
||||
survive de-stuffing. Since only four byte values are ever escaped, taking
|
||||
*any* `10 XX` as `XX` is exact rather than merely convenient.
|
||||
"""
|
||||
out = bytearray()
|
||||
i = 0
|
||||
while i < len(data):
|
||||
if data[i] == DLE and i + 1 < len(data):
|
||||
out.append(data[i + 1])
|
||||
i += 2
|
||||
else:
|
||||
out.append(data[i])
|
||||
i += 1
|
||||
return bytes(out)
|
||||
|
||||
|
||||
# ── Checksum ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def checksum(payload: bytes) -> int:
|
||||
"""SUM8 of the **de-stuffed** payload, mod 256. 251/251 both directions.
|
||||
|
||||
⚠ Do NOT exclude `0x10` bytes from this sum. The DLE-aware checksum that
|
||||
Series III uses for its `5A` and write frames is the right answer to a
|
||||
*different* question: it pairs with Series III de-stuffing, which leaves
|
||||
escaped bytes in the payload as two bytes. De-stuffing uniformly already
|
||||
removes the DLE, so excluding `0x10` as well subtracts the correction
|
||||
twice.
|
||||
|
||||
That combination — uniform de-stuffing *and* an exclusive sum — is what
|
||||
`scratch/mm_frame_parse.py` shipped with. It disagrees with the wire on
|
||||
**55 of 251** captured response frames, all of them frames whose payload
|
||||
holds a literal `0x10`. The script only ever looked correct because it
|
||||
accepts a frame that matches *either* rule, so it reported those 55 as
|
||||
plain SUM8 and never flagged one bad.
|
||||
"""
|
||||
return sum(payload) & 0xFF
|
||||
|
||||
|
||||
# ── Request builder ───────────────────────────────────────────────────────────
|
||||
|
||||
def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
|
||||
"""Build a host→unit command frame.
|
||||
|
||||
⚠ Do **not** substitute `minimateplus.framing.build_bw_frame()` here, even
|
||||
though the payload layout is identical. That builder escapes only `0x10`,
|
||||
so it reproduces just **161 of Thor's 218** captured read frames. The 57 it
|
||||
gets wrong are not edge cases:
|
||||
|
||||
* every `SUB 0x5A` bulk download — `offset = 0x0400` puts a literal
|
||||
`0x04` in `offset_hi`, which must go out as `10 04`
|
||||
* `SUB 0x47` (scheduler enable), whose params carry a `0x03`
|
||||
|
||||
An unescaped `0x03` or `0x04` reads as a frame terminator, so the unit sees
|
||||
a truncated frame and simply does not answer. That is indistinguishable
|
||||
from a dead unit, and event download would have hit it on the first try.
|
||||
|
||||
With the correct escape set this builder reproduces **218/218**.
|
||||
|
||||
Args:
|
||||
sub: command SUB byte.
|
||||
offset: uint16 at payload[4:5]. Micromate reads are single-step —
|
||||
Thor asks for `0xFFFF` and gets the whole block — so this is
|
||||
usually `0xFFFF`, not Series III's probe-then-data pair.
|
||||
params: exactly 10 bytes at payload[6:16].
|
||||
|
||||
A `0x10` inside `params` is fine and needs no special handling: Thor sends
|
||||
`SUB 0x5A` with `params = 00 00 10 00 …` and the wire carries `10 10`.
|
||||
(This was the spec's one open question; five captured frames settle it.)
|
||||
"""
|
||||
if len(params) != 10:
|
||||
raise ValueError(f"params must be exactly 10 bytes, got {len(params)}")
|
||||
if not 0 <= offset <= 0xFFFF:
|
||||
raise ValueError(f"offset must fit in uint16, got {offset:#x}")
|
||||
if not 0 <= sub <= 0xFF:
|
||||
raise ValueError(f"sub must be a single byte, got {sub:#x}")
|
||||
|
||||
payload = bytes([MM_CMD, 0x00, sub, 0x00, (offset >> 8) & 0xFF, offset & 0xFF]) + params
|
||||
body = payload + bytes([checksum(payload)])
|
||||
return bytes([ACK, STX]) + stuff(body) + bytes([ETX])
|
||||
|
||||
|
||||
# ── Response frame ────────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class MicromateFrame:
|
||||
"""A parsed, de-stuffed unit→host response frame."""
|
||||
|
||||
sub: int # response SUB; the request was 0xFF − this
|
||||
flags: int # payload[1] — 0xC5 Blastware line, 0x03 Thor line
|
||||
page_hi: int
|
||||
page_lo: int
|
||||
data: bytes # payload[5:], checksum stripped
|
||||
checksum_valid: bool
|
||||
chk_byte: int = 0 # the checksum byte as received
|
||||
|
||||
@property
|
||||
def request_sub(self) -> int:
|
||||
"""The SUB this is answering. No known exception to `0xFF − SUB`."""
|
||||
return 0xFF - self.sub
|
||||
|
||||
@property
|
||||
def page_key(self) -> int:
|
||||
"""payload[3:5] as a uint16 BE — a page/address on `0x5A` responses."""
|
||||
return (self.page_hi << 8) | self.page_lo
|
||||
|
||||
@property
|
||||
def firmware_line(self) -> str:
|
||||
return {FLAGS_BLASTWARE: "blastware", FLAGS_THOR: "thor"}.get(self.flags, "unknown")
|
||||
|
||||
@property
|
||||
def probe_length(self) -> Optional[int]:
|
||||
"""Data length declared by a **probe** response: uint16 BE at data[3:5].
|
||||
|
||||
⚠ Only meaningful in the reply to an `offset = 0` probe. Series III
|
||||
hardcodes a `DATA_LENGTHS` table; a Micromate will tell you instead,
|
||||
which already caught one divergence (call-home config is `0x7E`, where
|
||||
Series III has `0x7C`).
|
||||
|
||||
⚠ It is a **uint16 BE**, not a byte. Read as `data[3]` alone it is
|
||||
right only while the high byte is zero, and wrong by 47x for
|
||||
`SUB 0x1A`: a true `0x082C` (2092) reads as 44.
|
||||
|
||||
Returns None on a frame too short to hold the field. Note this reads
|
||||
as 0 on the single-step reads Thor actually uses — those are not probes,
|
||||
and `page_key` is the meaningful field there.
|
||||
"""
|
||||
if len(self.data) < 5:
|
||||
return None
|
||||
return (self.data[3] << 8) | self.data[4]
|
||||
|
||||
|
||||
# ── Streaming parser ──────────────────────────────────────────────────────────
|
||||
|
||||
class MicromateFrameParser:
|
||||
"""Incremental parser for unit→host frames. Mirrors `S3FrameParser`.
|
||||
|
||||
Feed bytes with `feed()`; completed frames are returned and also collected
|
||||
in `.frames`.
|
||||
|
||||
IDLE — scanning for a bare STX
|
||||
IN_FRAME — collecting; bare ETX terminates
|
||||
AFTER_DLE — the next byte is literal, whatever it is
|
||||
|
||||
Request frames are rejected rather than parsed: a frame whose `payload[0]`
|
||||
is not `0x00` is dropped, so feeding a bidirectional capture yields only
|
||||
the responses.
|
||||
"""
|
||||
|
||||
_IDLE, _IN_FRAME, _AFTER_DLE = 0, 1, 2
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._state = self._IDLE
|
||||
self._body = bytearray()
|
||||
self.frames: list[MicromateFrame] = []
|
||||
# Distinguishes "no bytes at all" from "bytes but no complete frame" on
|
||||
# a timeout. That distinction earned its keep during the Series III
|
||||
# work and costs one integer here.
|
||||
self.bytes_fed: int = 0
|
||||
|
||||
def reset(self) -> None:
|
||||
self._state = self._IDLE
|
||||
self._body.clear()
|
||||
self.bytes_fed = 0
|
||||
|
||||
def feed(self, data: bytes) -> list[MicromateFrame]:
|
||||
self.bytes_fed += len(data)
|
||||
completed: list[MicromateFrame] = []
|
||||
for b in data:
|
||||
frame = self._step(b)
|
||||
if frame is not None:
|
||||
completed.append(frame)
|
||||
self.frames.append(frame)
|
||||
return completed
|
||||
|
||||
def _step(self, b: int) -> Optional[MicromateFrame]:
|
||||
if self._state == self._IDLE:
|
||||
if b == STX:
|
||||
self._body.clear()
|
||||
self._state = self._IN_FRAME
|
||||
# Boot strings, modem RING/CONNECT chatter and stray ACKs land here
|
||||
# and are discarded.
|
||||
|
||||
elif self._state == self._IN_FRAME:
|
||||
if b == DLE:
|
||||
self._state = self._AFTER_DLE
|
||||
elif b == ETX:
|
||||
self._state = self._IDLE
|
||||
return self._finalise()
|
||||
else:
|
||||
self._body.append(b)
|
||||
|
||||
elif self._state == self._AFTER_DLE:
|
||||
# Uniform rule: the escaped byte is itself, including 0x03.
|
||||
self._body.append(b)
|
||||
self._state = self._IN_FRAME
|
||||
|
||||
return None
|
||||
|
||||
def _finalise(self) -> Optional[MicromateFrame]:
|
||||
body = bytes(self._body)
|
||||
if len(body) < _MIN_PAYLOAD + 1:
|
||||
return None
|
||||
|
||||
payload, chk_received = body[:-1], body[-1]
|
||||
if payload[0] != MM_RSP_CMD:
|
||||
return None # a request frame, or garbage that framed by accident
|
||||
|
||||
return MicromateFrame(
|
||||
sub = payload[2],
|
||||
flags = payload[1],
|
||||
page_hi = payload[3],
|
||||
page_lo = payload[4],
|
||||
data = payload[5:],
|
||||
checksum_valid = (chk_received == checksum(payload)),
|
||||
chk_byte = chk_received,
|
||||
)
|
||||
@@ -396,3 +396,85 @@ class IdfEvent:
|
||||
)
|
||||
ev._waveform_key = waveform_key
|
||||
return ev
|
||||
|
||||
|
||||
# ── Live-device models (2026-09-27) ───────────────────────────────────────────
|
||||
#
|
||||
# These describe what a unit reports over the wire, not what Thor wrote to a
|
||||
# file. Everything above this line came out of Thor's exports; everything below
|
||||
# came out of Thor's *traffic*. Field offsets are recorded in
|
||||
# ``micromate/client.py`` next to the code that reads them.
|
||||
|
||||
|
||||
@dataclass
|
||||
class MicromateDeviceInfo:
|
||||
"""Identity gathered by ``MicromateClient.connect()``.
|
||||
|
||||
Sourced from three reads:
|
||||
``0x5B`` POLL → manufacturer, model
|
||||
``0x15`` SERIAL → serial
|
||||
``0x49`` STATE → monitoring
|
||||
plus ``firmware_line``, which comes free from the flags byte of any
|
||||
response and needs no read of its own.
|
||||
"""
|
||||
|
||||
serial: str
|
||||
manufacturer: Optional[str] = None # "Instantel"
|
||||
model: Optional[str] = None # "MM/ISEE/S/IO" (CB) / "MM/ISEE/S" (BD)
|
||||
firmware_line: Optional[str] = None # "blastware" | "thor" | "unknown"
|
||||
monitoring: Optional[bool] = None
|
||||
active_setup: Optional[str] = None # e.g. "TEST1.mmb"
|
||||
|
||||
def __str__(self) -> str:
|
||||
bits = [self.serial]
|
||||
if self.model:
|
||||
bits.append(self.model)
|
||||
if self.firmware_line:
|
||||
bits.append(f"{self.firmware_line} fw")
|
||||
if self.monitoring is not None:
|
||||
bits.append("MONITORING" if self.monitoring else "idle")
|
||||
if self.active_setup:
|
||||
bits.append(f"setup={self.active_setup}")
|
||||
return " ".join(bits)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MicromateState:
|
||||
"""A unit's live state, from ``SUB 0x1C``.
|
||||
|
||||
``device_time`` is the unit's own clock, in its own local timezone — it is
|
||||
NOT converted. Nothing else this protocol exposes reports the unit's time,
|
||||
which makes it the only way to detect a drifted clock before it lands in
|
||||
event timestamps.
|
||||
"""
|
||||
|
||||
monitoring: bool
|
||||
device_time: Optional[datetime.datetime] = None
|
||||
battery_volts: Optional[float] = None
|
||||
memory_total_bytes: Optional[int] = None
|
||||
memory_free_bytes: Optional[int] = None
|
||||
raw: Optional[bytes] = field(default=None, repr=False)
|
||||
|
||||
@property
|
||||
def memory_used_bytes(self) -> Optional[int]:
|
||||
if self.memory_total_bytes is None or self.memory_free_bytes is None:
|
||||
return None
|
||||
return self.memory_total_bytes - self.memory_free_bytes
|
||||
|
||||
@property
|
||||
def memory_used_fraction(self) -> Optional[float]:
|
||||
used = self.memory_used_bytes
|
||||
if used is None or not self.memory_total_bytes:
|
||||
return None
|
||||
return used / self.memory_total_bytes
|
||||
|
||||
def __str__(self) -> str:
|
||||
bits = ["MONITORING" if self.monitoring else "idle"]
|
||||
if self.device_time:
|
||||
bits.append(self.device_time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
if self.battery_volts is not None:
|
||||
bits.append(f"{self.battery_volts:.2f} V")
|
||||
frac = self.memory_used_fraction
|
||||
if frac is not None:
|
||||
bits.append(f"memory {frac * 100:.1f}% used")
|
||||
return " ".join(bits)
|
||||
|
||||
@@ -0,0 +1,501 @@
|
||||
"""
|
||||
protocol.py — one method per Micromate (Series IV) wire command.
|
||||
|
||||
Returns raw payload bytes. Interpretation belongs in ``client.py``; this layer
|
||||
knows frames, offsets and sequencing, and nothing about what a field means.
|
||||
|
||||
Scope: **reads only.** Nothing here writes, erases, or changes monitoring
|
||||
state. That is deliberate and worth keeping — no command has ever been
|
||||
originated against a unit by this project; every write in
|
||||
``docs/micromate_protocol_reference.md`` was performed by THOR while we
|
||||
recorded. The first thing this codebase ever sends to a customer's instrument
|
||||
should be a decision someone made on purpose, not a side effect of a client
|
||||
that grew a method.
|
||||
|
||||
Every offset and params layout below was read off THOR's own frames in
|
||||
``bridges/captures/9-24-26 - micromate2/`` rather than taken from the spec
|
||||
table, because the same exercise during the framing work found three of that
|
||||
table's rules wrong. It found three more here:
|
||||
|
||||
* ``0x0A`` is the **monitor-log walk** — the same request repeated, the
|
||||
device advancing its own cursor, terminated by a short response — not the
|
||||
keyed "event header, 30 B list record" the spec describes.
|
||||
* ``0x1E``/``0x1F`` carry **token 0xFE** at ``params[7]``. The protocol
|
||||
reference documents all-zero params for the browse walk; that was our own
|
||||
probing, and THOR does not do it that way.
|
||||
* There is **no fixed preamble**. Sessions open with ``POLL`` and go
|
||||
straight to the operation. ``POLL → SERIAL → 0x49 → POLL`` appears in 3 of
|
||||
8 captured sessions and is THOR's *connection check*, not a handshake.
|
||||
|
||||
And one useful negative: **no ``SESSION_RESET`` (``41 03``).** Series III
|
||||
needs that 2-byte signal to wake a monitoring unit or it will not answer POLL
|
||||
over TCP. THOR never sends it — 0 occurrences across all 8 sessions, including
|
||||
40 frames exchanged with a unit that *was* monitoring.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import math
|
||||
import struct
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from minimateplus.transport import BaseTransport
|
||||
|
||||
from .framing import MicromateFrame, MicromateFrameParser, build_request
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_RECV_TIMEOUT = 10.0
|
||||
|
||||
# An acknowledgement carries an 11-byte data section and nothing else. It is
|
||||
# also how the monitor-log walk says "no more records".
|
||||
ACK_DATA_LEN = 11
|
||||
|
||||
|
||||
# ── Command SUBs ──────────────────────────────────────────────────────────────
|
||||
|
||||
SUB_DEVICE_INFO = 0x01
|
||||
SUB_STORAGE_RANGE = 0x06
|
||||
SUB_EVENT_INDEX = 0x08
|
||||
SUB_MONITOR_LOG = 0x0A
|
||||
SUB_EVENT_RECORD = 0x0C
|
||||
SUB_SERIAL = 0x15
|
||||
SUB_COMPLIANCE_CONFIG = 0x1A
|
||||
SUB_MONITOR_STATUS = 0x1C
|
||||
SUB_EVENT_FIRST = 0x1E
|
||||
SUB_EVENT_NEXT = 0x1F
|
||||
SUB_CALL_HOME_CONFIG = 0x2C
|
||||
SUB_TRIGGER_CONFIG = 0x2E
|
||||
SUB_SETUP_FIRST = 0x3F
|
||||
SUB_SETUP_NEXT = 0x40
|
||||
SUB_SETUP_ACTIVE = 0x41
|
||||
SUB_STATE = 0x49
|
||||
SUB_BULK_DOWNLOAD = 0x5A
|
||||
SUB_POLL = 0x5B
|
||||
SUB_ARM_EVENT = 0x93
|
||||
|
||||
# ⚠ Reads are SINGLE-STEP. Series III probes at offset 0 to learn the length,
|
||||
# then reads again at that length; a Micromate returns the whole block when
|
||||
# asked for 0xFFFF. THOR never probes, which is why `MicromateFrame.probe_length`
|
||||
# reads 0 on live traffic.
|
||||
READ_ALL = 0xFFFF
|
||||
|
||||
# The two commands that do NOT use READ_ALL, and the data length each returned
|
||||
# on UM12947 (firmware 11.0CB).
|
||||
_OFFSETS = {
|
||||
SUB_POLL: 0x0030, # 59 B — the one offset THOR treats as a constant
|
||||
SUB_SERIAL: 0x000A, # 21 B
|
||||
}
|
||||
|
||||
# Data-section lengths observed, for orientation only — deliberately NOT
|
||||
# asserted. `SUB 0x1C` is 4 bytes longer on the Thor firmware line (0x30 vs
|
||||
# 0x2C declared), so a length check here would fire spuriously on half the
|
||||
# fleet. See the protocol reference, "A/B: Blastware build vs Thor build".
|
||||
OBSERVED_DATA_LEN = {
|
||||
SUB_STORAGE_RANGE: 47, SUB_EVENT_INDEX: 101, SUB_MONITOR_LOG: 297,
|
||||
SUB_EVENT_RECORD: 221, SUB_SERIAL: 21, SUB_COMPLIANCE_CONFIG: 2103,
|
||||
SUB_MONITOR_STATUS: 55, SUB_EVENT_FIRST: 19, SUB_EVENT_NEXT: 19,
|
||||
SUB_CALL_HOME_CONFIG: 137, SUB_TRIGGER_CONFIG: 39,
|
||||
SUB_SETUP_FIRST: 266, SUB_SETUP_NEXT: 266, SUB_SETUP_ACTIVE: 266,
|
||||
SUB_STATE: 16, SUB_POLL: 59, SUB_ARM_EVENT: ACK_DATA_LEN,
|
||||
}
|
||||
|
||||
# `1E`/`1F` carry this at params[7]. Series III uses the same value to arm its
|
||||
# bulk stream; here THOR sends it on every chain read, browse or download.
|
||||
EVENT_TOKEN = 0xFE
|
||||
|
||||
# `SUB 0x5A` chunk size, in bytes of file payload per response.
|
||||
CHUNK_SIZE = 1024
|
||||
|
||||
# Every `0x5A` response prefixes the file bytes with 11 bytes of header.
|
||||
_CHUNK_PREFIX = 11
|
||||
|
||||
|
||||
# ── Exceptions ────────────────────────────────────────────────────────────────
|
||||
|
||||
class ProtocolError(Exception):
|
||||
"""The device violated the expected protocol."""
|
||||
|
||||
|
||||
class TimeoutError(ProtocolError):
|
||||
"""No response arrived within the allowed time."""
|
||||
|
||||
|
||||
class ChecksumError(ProtocolError):
|
||||
"""A received frame failed its checksum."""
|
||||
|
||||
|
||||
class UnexpectedResponse(ProtocolError):
|
||||
"""The response SUB did not match the request."""
|
||||
|
||||
|
||||
class ShortRead(ProtocolError):
|
||||
"""A bulk download returned fewer bytes than the device promised."""
|
||||
|
||||
|
||||
# ── Params builders ───────────────────────────────────────────────────────────
|
||||
|
||||
def token_params(token: int = EVENT_TOKEN) -> bytes:
|
||||
"""`1E`/`1F`: the token sits at params[7]."""
|
||||
return bytes(7) + bytes([token]) + bytes(2)
|
||||
|
||||
|
||||
def key_params(key4: bytes) -> bytes:
|
||||
"""`0x0C`: the full 4-byte event key at params[4:8]."""
|
||||
if len(key4) != 4:
|
||||
raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
|
||||
return bytes(4) + key4 + bytes(2)
|
||||
|
||||
|
||||
def key_lo_params(key4: bytes) -> bytes:
|
||||
"""`0x0A`: only the key's **low two bytes**, at params[6:8].
|
||||
|
||||
⚠ Inferred from a single key value. All nine captured `0x0A` frames carry
|
||||
`4a 81`, and the event key in play was `055d4a81` — so this is consistent
|
||||
with "the low half of the current key" and equally consistent with "a
|
||||
cursor handle that happened to equal it". Both readings produce the same
|
||||
bytes for that key, so one event cannot separate them.
|
||||
|
||||
It does not matter much in practice: the walk works with the same params
|
||||
repeated, so whichever it is, passing the current key is right.
|
||||
"""
|
||||
if len(key4) != 4:
|
||||
raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
|
||||
return bytes(6) + key4[2:4] + bytes(2)
|
||||
|
||||
|
||||
def chunk_params(key4: bytes, byte_offset: int) -> bytes:
|
||||
"""`0x5A`: the key opens the file, then a byte offset walks it.
|
||||
|
||||
Chunk 0 carries the event key at params[0:4] — that is what says "from the
|
||||
beginning". Later chunks carry a uint16 BE byte offset at params[2:4].
|
||||
"""
|
||||
if byte_offset == 0:
|
||||
if len(key4) != 4:
|
||||
raise ValueError(f"key4 must be 4 bytes, got {len(key4)}")
|
||||
return key4 + bytes(6)
|
||||
if not 0 <= byte_offset <= 0xFFFF:
|
||||
raise ValueError(f"byte_offset must fit in uint16, got {byte_offset}")
|
||||
return bytes(2) + struct.pack(">H", byte_offset) + bytes(6)
|
||||
|
||||
|
||||
# ── Protocol ──────────────────────────────────────────────────────────────────
|
||||
|
||||
class MicromateProtocol:
|
||||
"""Wire-level command set for one open connection to a Micromate.
|
||||
|
||||
Does not own the transport; lifetime belongs to the client.
|
||||
|
||||
proto = MicromateProtocol(transport)
|
||||
proto.poll()
|
||||
serial = proto.read_serial()
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
transport: BaseTransport,
|
||||
recv_timeout: float = DEFAULT_RECV_TIMEOUT,
|
||||
strict_checksums: bool = True,
|
||||
) -> None:
|
||||
"""
|
||||
Args:
|
||||
strict_checksums: raise on a bad checksum. **Defaults to True,
|
||||
unlike the Series III sibling**, which logs and continues
|
||||
because its parser cannot reliably tell an inner-frame
|
||||
delimiter from a checksum byte. That excuse does not apply
|
||||
here: the Micromate rule is plain SUM8 over the de-stuffed
|
||||
payload and it holds on 251 of 251 captured frames, so a
|
||||
mismatch means something real — line noise, a desync, or a rule
|
||||
we have wrong — and all three are worth hearing about.
|
||||
|
||||
The lenient Series III default is instructive: it hid the fact
|
||||
that the documented checksum rule was wrong for two days. Set
|
||||
False only to get a field diagnosis unstuck.
|
||||
"""
|
||||
self._transport = transport
|
||||
self._recv_timeout = recv_timeout
|
||||
self._strict = strict_checksums
|
||||
self._parser = MicromateFrameParser()
|
||||
self._pending: list[MicromateFrame] = []
|
||||
|
||||
# ── Identity and state ────────────────────────────────────────────────────
|
||||
|
||||
def poll(self) -> MicromateFrame:
|
||||
"""`0x5B` → `0xA4`. Handshake; carries the ID block and model string.
|
||||
|
||||
Every captured session opens with this and nothing before it.
|
||||
"""
|
||||
return self._exchange(SUB_POLL)
|
||||
|
||||
def read_serial(self) -> bytes:
|
||||
"""`0x15` → `0xEA`. ASCII, null-terminated — e.g. `UM12947`."""
|
||||
return self._read(SUB_SERIAL)
|
||||
|
||||
def read_device_info(self) -> bytes:
|
||||
"""`0x01` → `0xFE`. Firmware, calibration, per-channel float block.
|
||||
|
||||
⚠ THOR never sends this in any captured session, so the `0xFFFF` offset
|
||||
is from our own 2026-09-23 probes rather than from THOR's behaviour. It
|
||||
answered correctly on both firmware lines, but it is the one read here
|
||||
with no THOR frame behind it.
|
||||
"""
|
||||
return self._read(SUB_DEVICE_INFO)
|
||||
|
||||
def read_state(self) -> bytes:
|
||||
"""`0x49` → `0xB6`. A cheap monitoring check; 16 B.
|
||||
|
||||
⚠ Test `data[11]` for **non-zero**, never against a constant — it has
|
||||
read both `0x0E` and `0x0C` while monitoring.
|
||||
"""
|
||||
return self._read(SUB_STATE)
|
||||
|
||||
def read_monitor_status(self) -> bytes:
|
||||
"""`0x1C` → `0xE3`. Flag, **device clock**, battery, memory.
|
||||
|
||||
⚠ Parse **forward** from the declared length, never backward from the
|
||||
end. This block is 4 bytes longer on the Thor firmware line, and Series
|
||||
III's relative-to-end offsets yield a battery voltage of 577.92 V on a
|
||||
`11.0BD` unit.
|
||||
"""
|
||||
return self._read(SUB_MONITOR_STATUS)
|
||||
|
||||
def read_storage_range(self) -> bytes:
|
||||
"""`0x06` → `0xF9`. Event storage extent; 47 B."""
|
||||
return self._read(SUB_STORAGE_RANGE)
|
||||
|
||||
def read_event_index(self) -> bytes:
|
||||
"""`0x08` → `0xF7`. 101 B. Contents not yet mapped."""
|
||||
return self._read(SUB_EVENT_INDEX)
|
||||
|
||||
def read_trigger_config(self) -> bytes:
|
||||
"""`0x2E` → `0xD1`. 39 B. Series IV only; no Series III equivalent."""
|
||||
return self._read(SUB_TRIGGER_CONFIG)
|
||||
|
||||
def read_compliance_config(self) -> bytes:
|
||||
"""`0x1A` → `0xE5`. The whole active setup — 2103 B on UM12947.
|
||||
|
||||
One response. Series III needs a 4-frame sequence for the same thing.
|
||||
"""
|
||||
return self._read(SUB_COMPLIANCE_CONFIG)
|
||||
|
||||
def read_call_home_config(self) -> bytes:
|
||||
"""`0x2C` → `0xD3`. 137 B — Series III's is 124, so do not reuse its map."""
|
||||
return self._read(SUB_CALL_HOME_CONFIG)
|
||||
|
||||
# ── Setups ────────────────────────────────────────────────────────────────
|
||||
|
||||
def read_active_setup_name(self) -> bytes:
|
||||
"""`0x41` → `0xBE`. 266 B; carries the active `.MMB` name."""
|
||||
return self._read(SUB_SETUP_ACTIVE)
|
||||
|
||||
def read_first_setup(self) -> bytes:
|
||||
"""`0x3F` → `0xC0`. Head of the setup-file list."""
|
||||
return self._read(SUB_SETUP_FIRST)
|
||||
|
||||
def read_next_setup(self) -> bytes:
|
||||
"""`0x40` → `0xBF`. Repeat until the record carries an empty name.
|
||||
|
||||
Stateful: the device holds the cursor, so the same request walks the
|
||||
list. 22 of these appear back to back in one captured session.
|
||||
"""
|
||||
return self._read(SUB_SETUP_NEXT)
|
||||
|
||||
# ── Event chain ───────────────────────────────────────────────────────────
|
||||
|
||||
def arm_event(self) -> MicromateFrame:
|
||||
"""`0x93` → `0x6C`. THOR sends this before **every** `1E`/`1F`.
|
||||
|
||||
It replaces Series III's `1E(token=0xFE)` arming step. No params, no
|
||||
offset payload — an 11-byte ack.
|
||||
|
||||
⚠ Whether a unit actually requires it is untested. Do it because it is
|
||||
known-good, not because it is known-necessary.
|
||||
"""
|
||||
return self._exchange(SUB_ARM_EVENT, offset=READ_ALL)
|
||||
|
||||
def read_event_first(self) -> bytes:
|
||||
"""`0x1E` → `0xE1`. First event key + size; 19 B."""
|
||||
return self._read(SUB_EVENT_FIRST, params=token_params())
|
||||
|
||||
def read_event_next(self) -> bytes:
|
||||
"""`0x1F` → `0xE0`. Next key + size, or the all-zero null sentinel."""
|
||||
return self._read(SUB_EVENT_NEXT, params=token_params())
|
||||
|
||||
def read_event_record(self, key4: bytes) -> bytes:
|
||||
"""`0x0C` → `0xF3`. 221 B — project, client, operator, timestamp, peaks.
|
||||
|
||||
⚠ The peak float in here runs 2–5% above `max(T,V,L)` and is **not** the
|
||||
vector sum; its offset was inferred, not established. Prefer decoded
|
||||
samples.
|
||||
"""
|
||||
return self._read(SUB_EVENT_RECORD, params=key_params(key4))
|
||||
|
||||
def read_monitor_log_next(self, key4: bytes) -> Optional[bytes]:
|
||||
"""`0x0A` → `0xF5`. One monitor-log record, or None at end of list.
|
||||
|
||||
⚠ Not the keyed single read the spec describes. This is a **walk**:
|
||||
the same request repeated, the device advancing its own cursor, each
|
||||
response a 297-byte record carrying serial, mode and thresholds. The
|
||||
list ends with a bare 11-byte ack — nine captured frames, eight records
|
||||
then the terminator.
|
||||
|
||||
Series III reaches the same data through a record-type discriminator on
|
||||
its event walk (`0x2C` partial vs `0x46` full). Here it is a separate
|
||||
cursor and the event chain does not see it at all.
|
||||
"""
|
||||
data = self._read(SUB_MONITOR_LOG, params=key_lo_params(key4))
|
||||
return None if len(data) <= ACK_DATA_LEN else data
|
||||
|
||||
# ── Bulk download ─────────────────────────────────────────────────────────
|
||||
|
||||
def read_event_file(self, key4: bytes, size: int) -> bytes:
|
||||
"""`0x5A` → `0xA5`. The `.IDFW`/`.IDFH` file, byte for byte.
|
||||
|
||||
`size` is the 4 bytes after the key in the `1E`/`1F` response. Returns
|
||||
exactly that many bytes, or raises `ShortRead`.
|
||||
|
||||
A bounded chunk walk — `ceil(size / 1024)` requests, each asking for
|
||||
`min(1024, remaining)` bytes:
|
||||
|
||||
offset = the byte count wanted (NOT an address)
|
||||
params = the key on chunk 0, then a uint16 BE byte offset
|
||||
response = exactly `offset + 11` bytes; file bytes are data[11:]
|
||||
|
||||
Verified against THOR on all six bench events (4,076 → 13,424 B):
|
||||
`sum(offsets) == size` exactly, every time.
|
||||
|
||||
⚠ Do not port the Series III `5A` walk. Its address arithmetic caused a
|
||||
5x over-read and a `>64 KB` page-boundary bug that is *still open* on
|
||||
that side. Neither applies here — the cursor is a byte offset into the
|
||||
file, bounded by a size the device supplied, so it cannot run past the
|
||||
event.
|
||||
|
||||
The result feeds `micromate.idf_file.read_idf_file()` and
|
||||
`/db/import/idf_file` unchanged; no new codec work is needed.
|
||||
"""
|
||||
if size <= 0:
|
||||
raise ValueError(f"size must be positive, got {size}")
|
||||
|
||||
out = bytearray()
|
||||
n_chunks = math.ceil(size / CHUNK_SIZE)
|
||||
for i in range(n_chunks):
|
||||
want = min(CHUNK_SIZE, size - i * CHUNK_SIZE)
|
||||
data = self._read(
|
||||
SUB_BULK_DOWNLOAD,
|
||||
offset=want,
|
||||
params=chunk_params(key4, i * CHUNK_SIZE),
|
||||
)
|
||||
if len(data) < _CHUNK_PREFIX:
|
||||
raise ShortRead(
|
||||
f"chunk {i + 1}/{n_chunks} of {key4.hex()}: "
|
||||
f"{len(data)} B is too short to hold a chunk header"
|
||||
)
|
||||
body = data[_CHUNK_PREFIX:]
|
||||
if len(body) != want:
|
||||
# Worth being loud: a silently short event is the failure mode
|
||||
# this project has been bitten by repeatedly on the Series III
|
||||
# side, and here the expected length is known up front.
|
||||
raise ShortRead(
|
||||
f"chunk {i + 1}/{n_chunks} of {key4.hex()}: asked for "
|
||||
f"{want} B, got {len(body)}"
|
||||
)
|
||||
out += body
|
||||
|
||||
if len(out) != size:
|
||||
raise ShortRead(
|
||||
f"{key4.hex()}: assembled {len(out)} B, device promised {size}"
|
||||
)
|
||||
log.debug("downloaded %s: %d B in %d chunks", key4.hex(), len(out), n_chunks)
|
||||
return bytes(out)
|
||||
|
||||
# ── Plumbing ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _read(
|
||||
self,
|
||||
sub: int,
|
||||
*,
|
||||
params: bytes = bytes(10),
|
||||
offset: Optional[int] = None,
|
||||
timeout: Optional[float] = None,
|
||||
) -> bytes:
|
||||
"""Send one read command, return the response's data section."""
|
||||
return self._exchange(sub, params=params, offset=offset, timeout=timeout).data
|
||||
|
||||
def _exchange(
|
||||
self,
|
||||
sub: int,
|
||||
*,
|
||||
params: bytes = bytes(10),
|
||||
offset: Optional[int] = None,
|
||||
timeout: Optional[float] = None,
|
||||
) -> MicromateFrame:
|
||||
if offset is None:
|
||||
offset = _OFFSETS.get(sub, READ_ALL)
|
||||
# Start every exchange clean: drop any half-frame and any frame left
|
||||
# stashed by the last one. This is a strict request/response protocol,
|
||||
# so anything already buffered when we send is by definition stale, and
|
||||
# `expected_sub` would reject it anyway — better to discard it here than
|
||||
# to raise a confusing UnexpectedResponse one command later.
|
||||
self._parser.reset()
|
||||
self._pending.clear()
|
||||
self._send(build_request(sub, offset, params))
|
||||
return self._recv_one(expected_sub=0xFF - sub, timeout=timeout,
|
||||
reset_parser=False)
|
||||
|
||||
def _send(self, frame: bytes) -> None:
|
||||
log.debug("TX %d bytes: %s", len(frame), frame.hex())
|
||||
self._transport.write(frame)
|
||||
|
||||
def _recv_one(
|
||||
self,
|
||||
expected_sub: Optional[int] = None,
|
||||
timeout: Optional[float] = None,
|
||||
reset_parser: bool = True,
|
||||
) -> MicromateFrame:
|
||||
"""Read until one complete frame is parsed."""
|
||||
deadline = time.monotonic() + (timeout or self._recv_timeout)
|
||||
if reset_parser:
|
||||
self._parser.reset()
|
||||
self._pending.clear()
|
||||
|
||||
if self._pending:
|
||||
return self._validate(self._pending.pop(0), expected_sub)
|
||||
|
||||
while time.monotonic() < deadline:
|
||||
chunk = self._transport.read(4096)
|
||||
if not chunk:
|
||||
time.sleep(0.005)
|
||||
continue
|
||||
log.debug("RX %d bytes", len(chunk))
|
||||
frames = self._parser.feed(chunk)
|
||||
if frames:
|
||||
self._pending.extend(frames[1:])
|
||||
return self._validate(frames[0], expected_sub)
|
||||
|
||||
raise TimeoutError(
|
||||
f"no frame in {timeout or self._recv_timeout:.1f}s"
|
||||
+ (f" (expected SUB 0x{expected_sub:02X})" if expected_sub is not None else "")
|
||||
+ f"; {self._parser.bytes_fed} bytes were received"
|
||||
# That byte count is the whole point: it separates "nothing came
|
||||
# back at all" from "bytes arrived but never framed", and those have
|
||||
# completely different causes. It earned its keep on Series III.
|
||||
)
|
||||
|
||||
def _validate(
|
||||
self, frame: MicromateFrame, expected_sub: Optional[int]
|
||||
) -> MicromateFrame:
|
||||
if not frame.checksum_valid:
|
||||
msg = (
|
||||
f"SUB 0x{frame.sub:02X}: checksum mismatch "
|
||||
f"(got 0x{frame.chk_byte:02X}, {len(frame.data)} B data)"
|
||||
)
|
||||
if self._strict:
|
||||
raise ChecksumError(msg)
|
||||
log.warning("%s — continuing (strict_checksums=False)", msg)
|
||||
if expected_sub is not None and frame.sub != expected_sub:
|
||||
raise UnexpectedResponse(
|
||||
f"expected SUB 0x{expected_sub:02X}, got 0x{frame.sub:02X}"
|
||||
)
|
||||
return frame
|
||||
+31
-10
@@ -22,10 +22,30 @@ One rule covers both directions: after the leading doubled `BW_CMD`, every
|
||||
escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
|
||||
exactly as Blastware does.
|
||||
|
||||
That rule was chosen by evidence, not assumption: of the four candidates tried
|
||||
against the 9-24-26 capture's four data-carrying write frames, it is the only
|
||||
one under which all four checksums validate. See
|
||||
`docs/micromate_protocol_reference.md` → *The write path*.
|
||||
A Micromate escapes exactly four byte values — `0x02 0x03 0x04 0x10` — and
|
||||
nothing else, which is what makes the uniform rule exact rather than merely
|
||||
convenient. Established by re-stuffing all 502 captured frames and comparing
|
||||
to the wire: 251/251 each direction, where `{0x10}` alone gets 130 and 177.
|
||||
|
||||
⚠ Checksum, corrected 2026-09-27
|
||||
--------------------------------
|
||||
With uniform destuffing the checksum is **plain SUM8 of the destuffed
|
||||
payload**. This script used to try SUM8 *and* a "DLE-aware" variant that
|
||||
excludes `0x10` bytes, and report whichever matched — which is why it never
|
||||
flagged a bad frame and why the protocol reference carried the wrong rule for
|
||||
two days. The DLE-aware form belongs with *Series III* destuffing, where an
|
||||
escaped byte survives as two bytes; applying it after uniform destuffing
|
||||
subtracts the correction twice and disagrees with the wire on 55 of 251
|
||||
responses.
|
||||
|
||||
The lesson generalises: a tool that accepts any of N candidate rules cannot
|
||||
falsify any of them. It now validates against SUM8 alone, and reports
|
||||
`DLE-aware` only to name what a mismatch *would* have been — never as a pass.
|
||||
See `docs/micromate_protocol_reference.md` → *Checksum*, and
|
||||
`tests/test_micromate_framing.py`, which pins it.
|
||||
|
||||
`micromate/framing.py` is the production implementation; this stays as the
|
||||
one-shot capture-inspection tool.
|
||||
|
||||
Usage
|
||||
-----
|
||||
@@ -119,12 +139,13 @@ def frames(blob: bytes, *, is_request: bool):
|
||||
except ValueError:
|
||||
i += 1
|
||||
continue
|
||||
sum8 = sum(payload) & 0xFF
|
||||
dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
|
||||
if sum8 == chk:
|
||||
kind = "SUM8"
|
||||
elif dle_aware == chk:
|
||||
kind = "DLE-aware"
|
||||
# SUM8 of the destuffed payload is THE rule -- 502/502 captured frames.
|
||||
# The DLE-aware variant is reported only to name a near-miss; it is
|
||||
# never a pass. See the module docstring.
|
||||
if (sum(payload) & 0xFF) == chk:
|
||||
kind = "ok"
|
||||
elif (sum(b for b in payload if b != DLE) & 0xFF) == chk:
|
||||
kind = "BAD(dle-aware)"
|
||||
else:
|
||||
kind = "BAD"
|
||||
yield i, payload, chk, kind
|
||||
|
||||
@@ -0,0 +1,493 @@
|
||||
"""Client-layer tests for the Micromate (series-4) live client.
|
||||
|
||||
Every response constant below is a **real data section**, captured from UM12947
|
||||
(firmware 11.0CB) in ``bridges/captures/9-24-26 - micromate2/``. They are
|
||||
embedded as hex because the captures are gitignored.
|
||||
|
||||
Where a decoded value can be checked against something outside the bytes, it is:
|
||||
the device clock against the capture's own filename timestamp, the battery
|
||||
against Thor's event reports (3.8 V), the setup list against what the unit
|
||||
displays.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from micromate.client import CONTENT, MicromateClient, _content, _cstring
|
||||
from micromate.framing import ETX, STX, checksum, stuff
|
||||
from micromate.protocol import ProtocolError
|
||||
|
||||
FLAGS_CB, FLAGS_THOR = 0xC5, 0x03
|
||||
|
||||
|
||||
# ── Captured response data sections ───────────────────────────────────────────
|
||||
|
||||
# Generated from the captures by hand-free extraction -- the hex below is
|
||||
# verbatim response data, not reconstructed. The trailing comment on each
|
||||
# names the capture it came from, which is what lets the clock assertions be
|
||||
# checked against a wall-clock timestamp.
|
||||
|
||||
POLL = bytes.fromhex( # 59 B, from_20260924_185113_
|
||||
"300000000000000000000000000050496e7374616e74656c"
|
||||
"000600c3f04a00e4194f0074024d4d2f495345452f532f49"
|
||||
"4f00001f603e7657603e76"
|
||||
)
|
||||
SERIAL = bytes.fromhex( # 21 B, from_20260924_191214_
|
||||
"0a00000000000000000000554d3132393437003100"
|
||||
)
|
||||
STATE_IDLE = bytes.fromhex( # 16 B, from_20260924_191214_
|
||||
"050000000000000000000000e8000b00"
|
||||
)
|
||||
STATE_MONITORING = bytes.fromhex( # 16 B, from_20260924_191214_
|
||||
"050000000000000000000002e8000b00"
|
||||
)
|
||||
MS_MONITORING = bytes.fromhex( # 55 B, from_20260924_191214_
|
||||
"2c00000000000000000000000e180907ea20130c19000000"
|
||||
"000001000000000000000000000000000000000000017c00"
|
||||
"e4e1c000e3f1c0"
|
||||
)
|
||||
MS_IDLE = bytes.fromhex( # 55 B, from_20260924_191214_
|
||||
"2c000000000000000000000000180907ea64130d22000000"
|
||||
"000001000000000000000000000000000000000000017c00"
|
||||
"e4e1c000e3e1c0"
|
||||
)
|
||||
MS_LATE = bytes.fromhex( # 55 B, from_20260925_011403_
|
||||
"2c000000000000000000000000190907ea74010e05000000"
|
||||
"000001000000000000000000000000000000000000017c00"
|
||||
"e4e1c000e3e1c0"
|
||||
)
|
||||
|
||||
# ── Real 11.0BD bytes (UM20147, captured over USB 2026-09-30) ─────────────────
|
||||
#
|
||||
# The Thor firmware line, which was pure inference until this capture. These
|
||||
# are the two responses where BD differs from CB. Captured with
|
||||
# `mm_client_check.py --capture` and read out of the resulting pair with
|
||||
# scratch/mm_frame_parse.py, so the data sections are verbatim; only the frame
|
||||
# wrapper is rebuilt, and the framing is independently verified 251/251.
|
||||
|
||||
POLL_BD = bytes.fromhex( # 59 B -- flags 0x03, and the SHORTER model string
|
||||
"30000000000000000000000000005649"
|
||||
"6e7374616e74656c000600c3f04a00e4"
|
||||
"194f0052024d4d2f495345452f530058"
|
||||
"f406001f60c0755760c075"
|
||||
)
|
||||
MS_BD_MONITORING = bytes.fromhex( # 59 B -- FOUR BYTES LONGER than CB's 55
|
||||
"3000000000000000000000000e1e0907"
|
||||
"ea600e00310000000000000000000000"
|
||||
"00000000000000000000000000015e00"
|
||||
"e4e1c000e291c00fa00004"
|
||||
)
|
||||
|
||||
_SETUP_PAD = 266 - CONTENT
|
||||
|
||||
|
||||
def setup_response(name: str) -> bytes:
|
||||
"""A 0x41/0x3F/0x40 response: 11-byte prefix then a null-padded name."""
|
||||
body = name.encode("ascii").ljust(_SETUP_PAD, b"\x00")
|
||||
return bytes([0xFF]) + bytes(10) + body
|
||||
|
||||
|
||||
# The real 22 names, in the order the unit walked them.
|
||||
SETUP_NAMES = [
|
||||
"factory.MMB", "TEST.MMB", "BUS TEST.MMB", "Walsh JV 241.mmb",
|
||||
"Walsh JV 008.mmb", "Hawbaker 322.mmb", "Hawbaker 322 blasting.mmb",
|
||||
"min.mmb", "Playhouse Loc 1.mmb", "Valley Rock Solution.MMB",
|
||||
"RecordingSetup.mmb", "UPMC.mmb", "UPMC Loc 3.mmb", "Residence Inn.mmb",
|
||||
"Micromate ext trigger.mmb", "Micromate remort alarm.mmb",
|
||||
"Tree of Life - Loc 1 - 5861 Solway.mmb", "Mele-PWSA-Carroll -Loc 4.mmb",
|
||||
"Micromate min trigger mmb.mmb", "Fay - Layton Bridge Project.mmb",
|
||||
"Default Micromate ISEE.mmb", "TEST1.mmb",
|
||||
]
|
||||
|
||||
|
||||
# ── Test doubles ──────────────────────────────────────────────────────────────
|
||||
|
||||
class ScriptedTransport:
|
||||
def __init__(self, responses: list[bytes]) -> None:
|
||||
self.queue = list(responses)
|
||||
self.written: list[bytes] = []
|
||||
self._connected = False
|
||||
|
||||
def connect(self) -> None:
|
||||
self._connected = True
|
||||
|
||||
def disconnect(self) -> None:
|
||||
self._connected = False
|
||||
|
||||
def is_connected(self) -> bool:
|
||||
return self._connected
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
self.written.append(data)
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
return self.queue.pop(0) if self.queue else b""
|
||||
|
||||
|
||||
def frame(rsp_sub: int, data: bytes, *, flags: int = FLAGS_CB) -> bytes:
|
||||
payload = bytes([0x00, flags, rsp_sub, 0x00, 0x00]) + data
|
||||
return bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
|
||||
|
||||
|
||||
def client(responses: list[bytes], **kw) -> tuple[MicromateClient, ScriptedTransport]:
|
||||
t = ScriptedTransport(responses)
|
||||
return MicromateClient(t, recv_timeout=0.5, **kw), t
|
||||
|
||||
|
||||
# ── The captured constants are what we think they are ─────────────────────────
|
||||
|
||||
def test_captured_constants_have_the_expected_lengths():
|
||||
assert len(POLL) == 59
|
||||
assert len(SERIAL) == 21
|
||||
assert len(STATE_IDLE) == len(STATE_MONITORING) == 16
|
||||
assert len(MS_MONITORING) == len(MS_IDLE) == len(MS_LATE) == 55
|
||||
|
||||
|
||||
def test_the_prefix_length_byte_is_only_the_low_byte():
|
||||
"""⚠ data[0] is `content_length & 0xFF`, with no high byte anywhere.
|
||||
|
||||
True for every response under 256 bytes, which is why it reads as a working
|
||||
length field -- and then loses 2,048 bytes on a setup block. The client
|
||||
takes content as data[11:] for exactly this reason.
|
||||
"""
|
||||
for data in (POLL, SERIAL, STATE_IDLE, MS_MONITORING):
|
||||
assert data[0] == (len(data) - CONTENT) & 0xFF
|
||||
assert data[1] == 0x00, "no high byte is stored"
|
||||
|
||||
# The two that prove it is not a real length: a 2,092-byte setup block
|
||||
# reports 44, and a 1,024-byte download chunk reports 0.
|
||||
assert (2092 & 0xFF) == 44
|
||||
assert (1024 & 0xFF) == 0
|
||||
|
||||
|
||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_a_printable_byte_precedes_the_manufacturer_string():
|
||||
"""content[2] is 0x50 -- "P". This is why the POLL parse cannot be a scan."""
|
||||
c = _content(POLL)
|
||||
assert c[3] == 0x50 and chr(c[3]) == "P"
|
||||
assert c[4:13] == b"Instantel"
|
||||
|
||||
|
||||
def test_content_strips_exactly_eleven_bytes():
|
||||
assert _content(SERIAL) == bytes.fromhex("554d3132393437003100")
|
||||
assert _content(b"short") == b""
|
||||
|
||||
|
||||
def test_cstring_stops_at_the_null():
|
||||
assert _cstring(bytes.fromhex("554d3132393437003100")) == "UM12947"
|
||||
assert _cstring(b"\x00rest") == ""
|
||||
|
||||
|
||||
# ── connect() ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_connect_decodes_identity():
|
||||
mm, t = client([
|
||||
frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
|
||||
frame(0xBE, setup_response("TEST1.mmb")),
|
||||
])
|
||||
info = mm.connect()
|
||||
|
||||
assert info.serial == "UM12947"
|
||||
assert info.manufacturer == "Instantel"
|
||||
assert info.model == "MM/ISEE/S/IO"
|
||||
assert info.firmware_line == "blastware"
|
||||
assert info.monitoring is False
|
||||
assert info.active_setup == "TEST1.mmb"
|
||||
assert "UM12947" in str(info) and "idle" in str(info)
|
||||
|
||||
|
||||
def test_connect_sends_three_reads_not_thors_four():
|
||||
"""⚠ Deliberately narrower than Thor's POLL -> SERIAL -> 0x49 -> POLL.
|
||||
|
||||
The trailing POLL repeats the first; measuring all 8 captured sessions
|
||||
showed the four-command form is Thor's connection check (3 of 8 sessions),
|
||||
not a handshake. Only "opens with POLL" is invariant.
|
||||
"""
|
||||
mm, t = client([
|
||||
frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
|
||||
frame(0xBE, setup_response("TEST1.mmb")),
|
||||
])
|
||||
mm.connect()
|
||||
subs = [w[5] for w in t.written] # payload[2] lands at wire[5]
|
||||
assert subs == [0x5B, 0x15, 0x49, 0x41]
|
||||
assert 0x01 not in subs, "device info has no Thor precedent; do not read it"
|
||||
|
||||
|
||||
def test_connect_can_skip_the_active_setup():
|
||||
mm, t = client([frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE)])
|
||||
info = mm.connect(with_active_setup=False)
|
||||
assert info.active_setup is None
|
||||
assert len(t.written) == 3
|
||||
|
||||
|
||||
def test_connect_survives_an_unreadable_active_setup():
|
||||
"""A unit with no setup loaded is a real state, not a failed connect."""
|
||||
mm, _ = client([
|
||||
frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_IDLE),
|
||||
frame(0x00, bytes(20)), # wrong SUB -> UnexpectedResponse
|
||||
])
|
||||
info = mm.connect()
|
||||
assert info.serial == "UM12947"
|
||||
assert info.active_setup is None
|
||||
|
||||
|
||||
def test_connect_reports_a_monitoring_unit():
|
||||
mm, _ = client([
|
||||
frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, STATE_MONITORING),
|
||||
frame(0xBE, setup_response("TEST1.mmb")),
|
||||
])
|
||||
assert mm.connect().monitoring is True
|
||||
|
||||
|
||||
def test_the_state_flag_is_tested_for_non_zero():
|
||||
"""⚠ Never compared against 0x02 -- this flag family is not a stable enum.
|
||||
|
||||
Its sibling in SUB 0x1C has read both 0x0E and 0x0C while monitoring.
|
||||
"""
|
||||
for value in (0x01, 0x02, 0x0C, 0x0E, 0xFF):
|
||||
data = bytearray(STATE_IDLE)
|
||||
data[CONTENT] = value
|
||||
mm, _ = client([
|
||||
frame(0xA4, POLL), frame(0xEA, SERIAL), frame(0xB6, bytes(data)),
|
||||
frame(0xBE, setup_response("x.mmb")),
|
||||
])
|
||||
assert mm.connect().monitoring is True, f"0x{value:02x} should read as monitoring"
|
||||
|
||||
|
||||
def test_the_model_string_is_anchored_on_MM_not_on_an_offset():
|
||||
"""The Thor firmware line reports a SHORTER model string, "MM/ISEE/S".
|
||||
|
||||
Anchoring on b"MM/" survives that; a fixed end offset would not. A generic
|
||||
printable-run scan fails for a different reason -- see _parse_poll.
|
||||
|
||||
✅ CONFIRMED on real hardware 2026-09-30: UM20147 (11.0BD) read back
|
||||
model="MM/ISEE/S" over USB. The frame below is still synthesised because
|
||||
no BD capture is in the repo, but the string it asserts is the real one.
|
||||
"""
|
||||
bd = bytearray(POLL)
|
||||
assert bd[CONTENT + 26:CONTENT + 38] == b"MM/ISEE/S/IO"
|
||||
bd[CONTENT + 26:CONTENT + 38] = b"MM/ISEE/S\x00\x00\x00"
|
||||
mm, _ = client([
|
||||
frame(0xA4, bytes(bd), flags=FLAGS_THOR), frame(0xEA, SERIAL),
|
||||
frame(0xB6, STATE_IDLE), frame(0xBE, setup_response("x.mmb")),
|
||||
])
|
||||
info = mm.connect()
|
||||
assert info.model == "MM/ISEE/S"
|
||||
assert info.manufacturer == "Instantel"
|
||||
assert info.firmware_line == "thor"
|
||||
|
||||
|
||||
# ── get_state() ───────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"data, monitoring, when, free",
|
||||
[
|
||||
(MS_MONITORING, True, datetime.datetime(2026, 9, 24, 19, 12, 25), 0x00E3F1C0),
|
||||
(MS_IDLE, False, datetime.datetime(2026, 9, 24, 19, 13, 34), 0x00E3E1C0),
|
||||
(MS_LATE, False, datetime.datetime(2026, 9, 25, 1, 14, 5), 0x00E3E1C0),
|
||||
],
|
||||
ids=["monitoring", "idle", "after-midnight"],
|
||||
)
|
||||
def test_get_state_decodes_the_real_reads(data, monitoring, when, free):
|
||||
"""⚠ There is an unidentified byte at content[6]; the hour is at content[7].
|
||||
|
||||
The protocol reference's 0x1C section has this right. Its one-line summary
|
||||
in the divergences list reads as six contiguous fields and does not.
|
||||
|
||||
Each expected time is checked against the capture filename that produced the
|
||||
bytes: 19:12:14, 19:12:14 and 01:14:03. All three decode to seconds-to-a-
|
||||
minute after their session opened, which is what a device clock should do.
|
||||
Reading content[6] as the hour gives 32, 100 and 116.
|
||||
"""
|
||||
mm, _ = client([frame(0xE3, data)])
|
||||
st = mm.get_state()
|
||||
|
||||
assert st.monitoring is monitoring
|
||||
assert st.device_time == when
|
||||
assert st.battery_volts == 3.80 # Thor's reports print 3.8 V
|
||||
assert st.memory_total_bytes == 15_000_000
|
||||
assert st.memory_free_bytes == free
|
||||
assert st.raw == data
|
||||
|
||||
|
||||
def test_content_6_is_not_the_hour():
|
||||
"""The byte the reference implies is the hour reads 32, 100 and 116."""
|
||||
for data in (MS_MONITORING, MS_IDLE, MS_LATE):
|
||||
assert _content(data)[6] not in range(24)
|
||||
|
||||
|
||||
def test_memory_derivations():
|
||||
mm, _ = client([frame(0xE3, MS_MONITORING)])
|
||||
st = mm.get_state()
|
||||
assert st.memory_used_bytes == 15_000_000 - 0x00E3F1C0
|
||||
assert 0 < st.memory_used_fraction < 0.02
|
||||
assert "3.80 V" in str(st)
|
||||
|
||||
|
||||
def test_battery_and_memory_are_read_forward_from_content_start():
|
||||
"""⚠ NOT backward from the end.
|
||||
|
||||
This block is 4 bytes longer on the Thor firmware line, and Series III's
|
||||
from-the-end offsets give a 11.0BD unit a battery reading of 577.92 V. The
|
||||
extra bytes are trailing, so appending four does not move anything.
|
||||
"""
|
||||
bd = MS_MONITORING + bytes.fromhex("0fa00000")
|
||||
bd = bytes([0x30]) + bd[1:] # low-byte length becomes 48
|
||||
mm, _ = client([frame(0xE3, bd)])
|
||||
st = mm.get_state()
|
||||
|
||||
assert st.battery_volts == 3.80, "forward offsets must survive the 4 extra bytes"
|
||||
assert st.memory_total_bytes == 15_000_000
|
||||
# What the Series III from-the-end offsets would have produced:
|
||||
assert int.from_bytes(bd[-10:-8], "big") / 100 == pytest.approx(577.92, abs=0.01)
|
||||
|
||||
|
||||
def test_a_dead_clock_battery_does_not_fail_the_whole_read():
|
||||
"""An impossible date is information; the rest of the block is still good."""
|
||||
broken = bytearray(MS_MONITORING)
|
||||
broken[CONTENT + 3] = 0xFF # month 255
|
||||
mm, _ = client([frame(0xE3, bytes(broken))])
|
||||
st = mm.get_state()
|
||||
assert st.device_time is None
|
||||
assert st.battery_volts == 3.80
|
||||
|
||||
|
||||
def test_a_truncated_state_block_raises():
|
||||
mm, _ = client([frame(0xE3, bytes(20))])
|
||||
with pytest.raises(ProtocolError, match="need at least 44"):
|
||||
mm.get_state()
|
||||
|
||||
|
||||
# ── Setups ────────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_list_setups_walks_to_the_empty_terminator():
|
||||
"""22 real names then an empty one, exactly as the unit walked them."""
|
||||
responses = [frame(0xC0, setup_response(SETUP_NAMES[0]))]
|
||||
responses += [frame(0xBF, setup_response(n)) for n in SETUP_NAMES[1:]]
|
||||
responses += [frame(0xBF, setup_response(""))]
|
||||
|
||||
mm, t = client(responses)
|
||||
assert mm.list_setups() == SETUP_NAMES
|
||||
assert len(t.written) == 23, "22 names plus the terminator"
|
||||
assert t.written[0][5] == 0x3F
|
||||
assert {w[5] for w in t.written[1:]} == {0x40}
|
||||
|
||||
|
||||
def test_list_setups_handles_an_empty_unit():
|
||||
mm, _ = client([frame(0xC0, setup_response(""))])
|
||||
assert mm.list_setups() == []
|
||||
|
||||
|
||||
def test_list_setups_refuses_to_loop_forever():
|
||||
"""A cursor that never advances is a bug, and must not hang the caller."""
|
||||
from micromate import client as C
|
||||
|
||||
mm, _ = client([frame(0xC0, setup_response("a.mmb"))]
|
||||
+ [frame(0xBF, setup_response("a.mmb"))] * (C._MAX_SETUPS + 5))
|
||||
with pytest.raises(ProtocolError, match="not advancing"):
|
||||
mm.list_setups()
|
||||
|
||||
|
||||
def test_get_active_setup_handles_a_long_name():
|
||||
long_name = "Tree of Life - Loc 1 - 5861 Solway.mmb"
|
||||
mm, _ = client([frame(0xBE, setup_response(long_name))])
|
||||
assert mm.get_active_setup() == long_name
|
||||
|
||||
|
||||
# ── Lifecycle ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_the_client_owns_the_transport():
|
||||
mm, t = client([])
|
||||
assert not mm.is_open()
|
||||
mm.open()
|
||||
assert mm.is_open() and t.is_connected()
|
||||
mm.close()
|
||||
assert not mm.is_open()
|
||||
|
||||
|
||||
def test_context_manager_opens_and_closes():
|
||||
t = ScriptedTransport([frame(0xA4, POLL), frame(0xEA, SERIAL),
|
||||
frame(0xB6, STATE_IDLE), frame(0xBE, setup_response("x.mmb"))])
|
||||
with MicromateClient(t, recv_timeout=0.5) as mm:
|
||||
assert t.is_connected()
|
||||
assert mm.connect().serial == "UM12947"
|
||||
assert not t.is_connected()
|
||||
|
||||
|
||||
# ── The Thor firmware line, on real bytes ─────────────────────────────────────
|
||||
|
||||
def test_bd_constants_have_the_lengths_the_parser_reported():
|
||||
assert len(POLL_BD) == 59
|
||||
assert len(MS_BD_MONITORING) == 59, "CB is 55; BD is four bytes longer"
|
||||
assert MS_BD_MONITORING[0] == 0x30, "low-byte length 48 = 59 - 11"
|
||||
assert MS_IDLE[0] == 0x2C, "the CB equivalent declares 44"
|
||||
|
||||
|
||||
def test_connect_on_a_thor_line_unit():
|
||||
"""UM20147, real POLL bytes. flags 0x03 is ETX, so it arrives as `10 03`."""
|
||||
mm, _ = client([
|
||||
frame(0xA4, POLL_BD, flags=FLAGS_THOR), frame(0xEA, SERIAL),
|
||||
frame(0xB6, STATE_MONITORING), frame(0xBE, setup_response("test2.MMB")),
|
||||
])
|
||||
info = mm.connect()
|
||||
assert info.firmware_line == "thor"
|
||||
assert info.model == "MM/ISEE/S", "the BD model string is shorter than CB's"
|
||||
assert info.manufacturer == "Instantel"
|
||||
assert info.monitoring is True
|
||||
|
||||
|
||||
def test_poll_content_3_is_not_a_constant():
|
||||
"""⚠ 0x50 on UM12947, 0x56 on UM20147 -- it VARIES between units.
|
||||
|
||||
This is why the manufacturer is read at the fixed offset content[4] and not
|
||||
by scanning: content[3] is printable in both cases ("P" and "V"), so a
|
||||
printable-run scan would return "PInstantel" on one unit and "VInstantel" on
|
||||
the other. Whatever the byte is, it is not a stable marker to anchor on.
|
||||
"""
|
||||
assert _content(POLL)[3] == 0x50
|
||||
assert _content(POLL_BD)[3] == 0x56
|
||||
assert chr(_content(POLL_BD)[3]) == "V"
|
||||
|
||||
|
||||
def test_get_state_on_a_thor_line_unit():
|
||||
"""⚠ THE test for the forward-offset decision. Real UM20147 bytes.
|
||||
|
||||
The 0x1C block is four bytes longer here, and the extras are TRAILING, so
|
||||
offsets measured from the start of content are unmoved. Series III reads
|
||||
battery and memory from the END of this block; test_series_iii_offsets_...
|
||||
below shows what that produces.
|
||||
"""
|
||||
mm, _ = client([frame(0xE3, MS_BD_MONITORING, flags=FLAGS_THOR)])
|
||||
st = mm.get_state()
|
||||
|
||||
assert st.monitoring is True
|
||||
assert st.device_time == datetime.datetime(2026, 9, 30, 14, 0, 49)
|
||||
assert st.battery_volts == 3.50
|
||||
assert st.memory_total_bytes == 15_000_000
|
||||
assert st.memory_free_bytes == 14_848_448
|
||||
|
||||
|
||||
def test_series_iii_from_the_end_offsets_give_577_volts_on_a_bd_unit():
|
||||
"""The exact number the protocol reference warned about, now demonstrated.
|
||||
|
||||
577.92 V is not a plausible battery reading for anything, which is what
|
||||
makes it a free self-check -- bridges/mm_client_check.py watches for it.
|
||||
"""
|
||||
assert int.from_bytes(MS_BD_MONITORING[-10:-8], "big") / 100 == 577.92
|
||||
|
||||
|
||||
def test_the_four_extra_bd_bytes_are_not_all_zero():
|
||||
"""⚠ The reference records them as `0f a0 00 00`; UM20147 sent `0f a0 00 04`.
|
||||
|
||||
Only the first two bytes look fixed. Nothing reads them, but a future
|
||||
decoder must not treat the last one as padding.
|
||||
"""
|
||||
assert _content(MS_BD_MONITORING)[44:] == bytes.fromhex("0fa00004")
|
||||
assert _content(MS_IDLE)[44:] == b"", "the CB block has no such tail"
|
||||
@@ -0,0 +1,413 @@
|
||||
"""Framing tests for the Micromate (series-4) live protocol.
|
||||
|
||||
Every constant below is a **real frame**, lifted from
|
||||
``bridges/captures/9-24-26 - micromate2/`` (UM12947, firmware 11.0CB) or from
|
||||
``scratch/fake_unit.py``, which preserves a POLL probe reply. Frames are
|
||||
embedded as hex rather than read from disk because both ``bridges/captures/``
|
||||
and ``tests/fixtures/`` are gitignored -- these tests must pass on a fresh
|
||||
clone.
|
||||
|
||||
The few synthesised frames are marked ``SYNTH_`` and each says what it stands
|
||||
in for and why a captured frame was not available.
|
||||
|
||||
Two of these tests exist because the first draft of
|
||||
``docs/micromate_client_spec.md`` got the rule wrong, and both wrong rules
|
||||
fail quietly -- a frame the unit ignores, or a checksum that reads as bad:
|
||||
|
||||
* ``test_builder_matches_thor_byte_for_byte`` -- the escape set. Escaping
|
||||
only 0x10 (the series-3 rule) reproduces 161 of Thor's 218 read frames.
|
||||
* ``test_checksum_is_plain_sum8_over_destuffed_payload`` -- the checksum.
|
||||
The DLE-aware variant disagrees with the wire on 55 of 251 responses.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from micromate.framing import (
|
||||
ACK,
|
||||
DLE,
|
||||
ETX,
|
||||
FLAGS_BLASTWARE,
|
||||
FLAGS_THOR,
|
||||
STX,
|
||||
MicromateFrame,
|
||||
MicromateFrameParser,
|
||||
build_request,
|
||||
checksum,
|
||||
stuff,
|
||||
unstuff,
|
||||
)
|
||||
|
||||
# ── Captured response frames ──────────────────────────────────────────────────
|
||||
|
||||
# POLL probe reply, 19 B on the wire -- the shortest valid frame there is.
|
||||
# Preserved in scratch/fake_unit.py, captured from UM12947 on 2026-09-24.
|
||||
# Its payload[8:10] is 0x0030, the data length POLL then asks for.
|
||||
RSP_POLL_PROBE = bytes.fromhex("0200c5a400000000000030000000000000009903")
|
||||
|
||||
# SUB 0x49 -> 0xB6, the cheap state read. Carries a literal 0x02, escaped.
|
||||
RSP_STATE = bytes.fromhex("0200c5b6000005000000000000000000001002e8000b007503")
|
||||
|
||||
# SUB 0x48 -> 0xB7, a path-addressed file read. offset_hi 0x04 arrives escaped,
|
||||
# so page_hi is only correct if the parser destuffs before indexing.
|
||||
RSP_FILE_READ = bytes.fromhex("0200c5b70000100400000000000000010000000000018203")
|
||||
|
||||
# An 0x5A download chunk, 138 B on the wire. THE checksum case: its payload
|
||||
# holds literal 0x10 bytes, so plain SUM8 (0xC1, correct) and the DLE-aware
|
||||
# variant (0x91) disagree. Also holds a literal 0x41, which is NOT escaped.
|
||||
RSP_CHUNK_WITH_DLE = bytes.fromhex(
|
||||
"0200c5a500007000003400000000000000e3fd1f10020f0f0e2e1e1fd4f2d4c3d2f00f000"
|
||||
"e003fe03fe200d6e2d12d2e1c4e101011d101f3d0f3e0f2efe23d2b2e3c0d3e0e2c4101e0"
|
||||
"d4d3e21f1d311e202fe2e2f1e3f100210d201002ee2e22d2f2f0f30fd3f0101000f0111f1"
|
||||
"ff01e011010011f0f1002b4d200e0f0302c4010020001fffedb2dc103"
|
||||
)
|
||||
|
||||
# ── Captured request frames (Thor -> unit) ────────────────────────────────────
|
||||
|
||||
REQ_POLL = bytes.fromhex("41021010005b000030000000000000000000009b03")
|
||||
REQ_SERIAL = bytes.fromhex("41021010001500000a000000000000000000002f03")
|
||||
REQ_STATE = bytes.fromhex("41021010004900ffff000000000000000000005703")
|
||||
REQ_COMPLIANCE = bytes.fromhex("41021010001a00ffff000000000000000000002803")
|
||||
REQ_ARM_EVENT = bytes.fromhex("41021010009300ffff00000000000000000000a103")
|
||||
|
||||
# ⚠ The two frames that break a 0x10-only escaper.
|
||||
# Scheduler enable: params[7] = 0x03, on the wire as `10 03`.
|
||||
REQ_SCHED_ON = bytes.fromhex("41021010004700ffff00000000000000100300005803")
|
||||
# Bulk download, first chunk of event 055d4a81: offset 0x0400 puts a literal
|
||||
# 0x04 in offset_hi, on the wire as `10 04`. EVERY download frame needs this.
|
||||
REQ_DOWNLOAD_CHUNK0 = bytes.fromhex(
|
||||
"41021010005a00100400055d4a810000000000009b03"
|
||||
)
|
||||
# A later chunk of the same event: params[2:4] = 0x1000, doubled to `10 10`.
|
||||
REQ_DOWNLOAD_CHUNK4 = bytes.fromhex(
|
||||
"41021010005a0010040000001010000000000000007e03"
|
||||
)
|
||||
|
||||
|
||||
# ── Stuffing ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_escape_set_is_exactly_four_bytes():
|
||||
"""0x02, 0x03, 0x04 and 0x10 -- and nothing else.
|
||||
|
||||
ACK (0x41) in particular is NOT escaped; assuming it was reproduced only
|
||||
196 of 251 captured responses.
|
||||
"""
|
||||
assert stuff(bytes([0x02, 0x03, 0x04, 0x10])) == bytes(
|
||||
[DLE, 0x02, DLE, 0x03, DLE, 0x04, DLE, 0x10]
|
||||
)
|
||||
for b in (0x00, 0x01, 0x05, 0x41, 0xC5, 0xFF):
|
||||
assert stuff(bytes([b])) == bytes([b]), f"0x{b:02x} must not be escaped"
|
||||
|
||||
|
||||
def test_unstuff_is_uniform_with_no_inner_frame_carve_out():
|
||||
"""`10 XX` -> `XX` for any XX -- the series-3 DLE+ETX exception is absent."""
|
||||
assert unstuff(bytes.fromhex("1003")) == b"\x03"
|
||||
assert unstuff(bytes.fromhex("1010")) == b"\x10"
|
||||
assert unstuff(bytes.fromhex("001002ff")) == bytes.fromhex("0002ff")
|
||||
|
||||
|
||||
def test_stuff_unstuff_round_trips_over_every_byte_value():
|
||||
data = bytes(range(256))
|
||||
assert unstuff(stuff(data)) == data
|
||||
|
||||
|
||||
def test_a_trailing_dle_is_held_not_dropped():
|
||||
"""A DLE as the last byte of a chunk must not consume nothing and vanish."""
|
||||
assert unstuff(b"\xff\x10") == b"\xff\x10"
|
||||
|
||||
|
||||
# ── Checksum ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_checksum_is_plain_sum8_over_destuffed_payload():
|
||||
"""⚠ Plain SUM8 -- do NOT exclude 0x10 bytes.
|
||||
|
||||
RSP_CHUNK_WITH_DLE is a real frame whose payload holds literal 0x10 bytes.
|
||||
The wire says 0xC1; plain SUM8 gives 0xC1 and the DLE-aware variant used by
|
||||
series-3 `5A`/write frames gives 0x91. 55 of 251 captured responses
|
||||
disagree the same way.
|
||||
"""
|
||||
payload = unstuff(RSP_CHUNK_WITH_DLE[1:-1])[:-1]
|
||||
chk_on_wire = unstuff(RSP_CHUNK_WITH_DLE[1:-1])[-1]
|
||||
|
||||
assert DLE in payload, "this frame is only interesting if it holds a 0x10"
|
||||
assert checksum(payload) == chk_on_wire == 0xC1
|
||||
assert (sum(b for b in payload if b != DLE) & 0xFF) == 0x91 # the wrong rule
|
||||
|
||||
|
||||
def test_every_captured_frame_validates():
|
||||
parser = MicromateFrameParser()
|
||||
frames = parser.feed(
|
||||
RSP_POLL_PROBE + RSP_STATE + RSP_FILE_READ + RSP_CHUNK_WITH_DLE
|
||||
)
|
||||
assert len(frames) == 4
|
||||
assert all(f.checksum_valid for f in frames)
|
||||
|
||||
|
||||
# ── Request builder ───────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"wire, sub, offset, params",
|
||||
[
|
||||
(REQ_POLL, 0x5B, 0x0030, bytes(10)),
|
||||
(REQ_SERIAL, 0x15, 0x000A, bytes(10)),
|
||||
(REQ_STATE, 0x49, 0xFFFF, bytes(10)),
|
||||
(REQ_COMPLIANCE, 0x1A, 0xFFFF, bytes(10)),
|
||||
(REQ_ARM_EVENT, 0x93, 0xFFFF, bytes(10)),
|
||||
(REQ_SCHED_ON, 0x47, 0xFFFF, bytes.fromhex("00000000000000030000")),
|
||||
(REQ_DOWNLOAD_CHUNK0, 0x5A, 0x0400, bytes.fromhex("055d4a81000000000000")),
|
||||
(REQ_DOWNLOAD_CHUNK4, 0x5A, 0x0400, bytes.fromhex("00001000000000000000")),
|
||||
],
|
||||
ids="poll serial state compliance arm sched_on dl_chunk0 dl_chunk4".split(),
|
||||
)
|
||||
def test_builder_matches_thor_byte_for_byte(wire, sub, offset, params):
|
||||
assert build_request(sub, offset, params) == wire
|
||||
|
||||
|
||||
def test_a_0x10_in_params_needs_no_special_handling():
|
||||
"""The spec's one open question. Thor sends it; the wire doubles it."""
|
||||
frame = build_request(0x5A, 0x0400, bytes.fromhex("00001000000000000000"))
|
||||
assert bytes([DLE, DLE]) in frame
|
||||
assert frame == REQ_DOWNLOAD_CHUNK4
|
||||
|
||||
|
||||
def test_builder_rejects_malformed_arguments():
|
||||
with pytest.raises(ValueError):
|
||||
build_request(0x5B, 0, bytes(9))
|
||||
with pytest.raises(ValueError):
|
||||
build_request(0x5B, 0x10000)
|
||||
with pytest.raises(ValueError):
|
||||
build_request(0x100)
|
||||
|
||||
|
||||
# ── Parsing ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def test_poll_probe_reply_fields():
|
||||
(f,) = MicromateFrameParser().feed(RSP_POLL_PROBE)
|
||||
assert f.sub == 0xA4
|
||||
assert f.request_sub == 0x5B
|
||||
assert f.flags == FLAGS_BLASTWARE
|
||||
assert f.firmware_line == "blastware"
|
||||
assert f.checksum_valid
|
||||
assert f.probe_length == 0x0030
|
||||
|
||||
|
||||
def test_probe_length_is_a_uint16_not_a_byte():
|
||||
"""⚠ Read as data[3] alone, SUB 0x1A's 0x082C (2092) reads as 44 -- 47x low.
|
||||
|
||||
SYNTHESISED: no probe response survives in the captures on disk (the
|
||||
9-24-26 session uses single-step reads at offset 0xFFFF throughout, so it
|
||||
contains no probes at all). The field position is taken from the captured
|
||||
POLL probe reply above, which does exercise it for real.
|
||||
"""
|
||||
payload = bytes([0x00, FLAGS_BLASTWARE, 0xE5, 0x00, 0x00]) + bytes(
|
||||
[0x00, 0x00, 0x00, 0x08, 0x2C]
|
||||
)
|
||||
synth = bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
|
||||
|
||||
(f,) = MicromateFrameParser().feed(synth)
|
||||
assert f.probe_length == 0x082C == 2092
|
||||
assert f.data[3] == 0x08, "the high byte is where a byte-wide read loses 2048"
|
||||
|
||||
|
||||
def test_escaped_bytes_land_in_the_right_field():
|
||||
"""RSP_FILE_READ's first data byte is 0x04, which arrives as `10 04`.
|
||||
|
||||
Without destuffing it reads as 0x10 and every field after it is one byte
|
||||
late -- the failure mode that put `SUB 0x02` in the log as `SUB_10` for an
|
||||
afternoon.
|
||||
"""
|
||||
(f,) = MicromateFrameParser().feed(RSP_FILE_READ)
|
||||
assert f.sub == 0xB7
|
||||
assert f.request_sub == 0x48
|
||||
assert (f.page_hi, f.page_lo) == (0x00, 0x00)
|
||||
assert f.data[0] == 0x04
|
||||
assert len(f.data) == 15, "one byte shorter than the wire suggests"
|
||||
assert f.checksum_valid
|
||||
|
||||
|
||||
# The real 11.0BD POLL data section, UM20147 over USB 2026-09-30. This
|
||||
# replaces a synthesised frame -- the Thor firmware line was inference-only
|
||||
# until this capture.
|
||||
_POLL_BD_DATA = bytes.fromhex(
|
||||
"30000000000000000000000000005649"
|
||||
"6e7374616e74656c000600c3f04a00e4"
|
||||
"194f0052024d4d2f495345452f530058"
|
||||
"f406001f60c0755760c075"
|
||||
)
|
||||
|
||||
|
||||
def test_thor_firmware_line_survives_destuffing():
|
||||
"""⚠ flags = 0x03 is ETX, so it arrives as `10 03`.
|
||||
|
||||
A parser that does not destuff ends the frame at byte 2 on half the fleet.
|
||||
|
||||
The data section is REAL (UM20147, 11.0BD); the frame wrapper is rebuilt,
|
||||
which is sound because the framing is verified 251/251 elsewhere in this
|
||||
file. scratch/mm_frame_parse.py reported payload=64 for this frame, and
|
||||
the assertion below pins that, so the reconstruction is checked rather than
|
||||
assumed.
|
||||
"""
|
||||
payload = bytes([0x00, FLAGS_THOR, 0xA4, 0x00, 0x00]) + _POLL_BD_DATA
|
||||
assert len(payload) == 64, "the parser reported payload=64 for this frame"
|
||||
wire = bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
|
||||
|
||||
assert bytes([DLE, ETX]) in wire, "flags 0x03 must be escaped on the wire"
|
||||
(f,) = MicromateFrameParser().feed(wire)
|
||||
assert f.flags == FLAGS_THOR
|
||||
assert f.firmware_line == "thor"
|
||||
assert f.sub == 0xA4
|
||||
assert f.request_sub == 0x5B
|
||||
assert f.checksum_valid
|
||||
assert b"MM/ISEE/S\x00" in f.data, "the shorter BD model string"
|
||||
|
||||
|
||||
def test_an_escaped_checksum_byte_is_read_correctly():
|
||||
"""SYNTHESISED, but the behaviour is real: three captured responses have a
|
||||
checksum of 0x02/0x03/0x04 and all three escape it on the wire. The
|
||||
shortest is 1,070 B (an 0x5A chunk in
|
||||
raw_s3_20260925_011403_Download_events_then_delete_1_event.bin, chk = 0x03),
|
||||
too long to embed for one byte's worth of assertion.
|
||||
"""
|
||||
payload = bytes([0x00, FLAGS_BLASTWARE, 0xA4, 0x00, 0x00, 0x03])
|
||||
assert checksum(payload) == 0x03 + FLAGS_BLASTWARE + 0xA4 & 0xFF
|
||||
body = payload + bytes([checksum(payload)])
|
||||
synth = bytes([STX]) + stuff(body) + bytes([ETX])
|
||||
|
||||
(f,) = MicromateFrameParser().feed(synth)
|
||||
assert f.chk_byte == checksum(payload)
|
||||
assert f.checksum_valid
|
||||
|
||||
|
||||
def test_a_corrupted_checksum_still_yields_a_frame():
|
||||
"""Flag it, do not swallow it -- a dropped frame looks like a dead unit."""
|
||||
broken = bytearray(RSP_POLL_PROBE)
|
||||
broken[-2] ^= 0xFF
|
||||
(f,) = MicromateFrameParser().feed(bytes(broken))
|
||||
assert f.sub == 0xA4
|
||||
assert not f.checksum_valid
|
||||
|
||||
|
||||
def test_a_truncated_frame_yields_nothing():
|
||||
parser = MicromateFrameParser()
|
||||
assert parser.feed(RSP_POLL_PROBE[:-1]) == []
|
||||
assert parser.frames == []
|
||||
assert parser.bytes_fed == len(RSP_POLL_PROBE) - 1
|
||||
|
||||
|
||||
def test_a_frame_too_short_to_hold_a_header_is_rejected():
|
||||
assert MicromateFrameParser().feed(bytes([STX, 0x00, 0xC5, 0xA4, ETX])) == []
|
||||
|
||||
|
||||
def test_request_frames_are_not_mistaken_for_responses():
|
||||
"""Feeding a bidirectional capture must yield only the unit's side."""
|
||||
parser = MicromateFrameParser()
|
||||
frames = parser.feed(REQ_POLL + RSP_POLL_PROBE + REQ_SERIAL)
|
||||
assert len(frames) == 1
|
||||
assert frames[0].request_sub == 0x5B
|
||||
|
||||
|
||||
def test_leading_noise_is_discarded():
|
||||
"""Cold-boot banners and the RV55's RING/CONNECT chatter precede frames."""
|
||||
noise = b"\r\nRING\r\n\r\nCONNECT\r\n" + bytes([ACK])
|
||||
(f,) = MicromateFrameParser().feed(noise + RSP_POLL_PROBE)
|
||||
assert f.sub == 0xA4
|
||||
assert f.checksum_valid
|
||||
|
||||
|
||||
def test_frames_split_across_feeds_reassemble():
|
||||
"""The transport hands over whatever the socket returned, DLE pairs and all."""
|
||||
whole = RSP_STATE + RSP_CHUNK_WITH_DLE
|
||||
for cut in (1, 2, 5, 17, 24, 25, 40, len(RSP_STATE), len(whole) - 1):
|
||||
parser = MicromateFrameParser()
|
||||
got = parser.feed(whole[:cut]) + parser.feed(whole[cut:])
|
||||
assert len(got) == 2, f"split at {cut} lost a frame"
|
||||
assert all(f.checksum_valid for f in got), f"split at {cut} broke a checksum"
|
||||
|
||||
|
||||
def test_reset_clears_partial_state():
|
||||
parser = MicromateFrameParser()
|
||||
parser.feed(RSP_POLL_PROBE[:6])
|
||||
parser.reset()
|
||||
assert parser.bytes_fed == 0
|
||||
(f,) = parser.feed(RSP_POLL_PROBE)
|
||||
assert f.checksum_valid
|
||||
|
||||
|
||||
def test_firmware_line_of_an_unknown_flags_byte():
|
||||
f = MicromateFrame(sub=0xA4, flags=0x99, page_hi=0, page_lo=0,
|
||||
data=b"", checksum_valid=True)
|
||||
assert f.firmware_line == "unknown"
|
||||
assert f.probe_length is None
|
||||
|
||||
|
||||
# ── Whole-session round trip ──────────────────────────────────────────────────
|
||||
|
||||
_CAPTURES = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "bridges" / "captures" / "9-24-26 - micromate2"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not _CAPTURES.is_dir(),
|
||||
reason="capture directory is gitignored; present only on a dev box",
|
||||
)
|
||||
def test_whole_captured_sessions_parse_with_no_bad_checksums():
|
||||
"""Belt-and-braces against the real bytes when they happen to be here.
|
||||
|
||||
scratch/mm_frame_parse.py reported zero bad checksums on these sessions
|
||||
only because it accepts a frame matching *either* checksum rule. This
|
||||
asserts the single rule holds across all of them.
|
||||
"""
|
||||
total = 0
|
||||
for path in sorted(_CAPTURES.rglob("raw_s3_*.bin")):
|
||||
parser = MicromateFrameParser()
|
||||
frames = parser.feed(path.read_bytes())
|
||||
assert frames, f"{path.name}: no frames parsed"
|
||||
bad = [f for f in frames if not f.checksum_valid]
|
||||
assert not bad, f"{path.name}: {len(bad)} bad checksums"
|
||||
total += len(frames)
|
||||
assert total == 251, f"expected 251 response frames across the corpus, got {total}"
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not _CAPTURES.is_dir(),
|
||||
reason="capture directory is gitignored; present only on a dev box",
|
||||
)
|
||||
def test_builder_reproduces_every_captured_read_frame():
|
||||
"""218/218. This is the test that would have caught the escape-set error."""
|
||||
checked = 0
|
||||
for path in sorted(_CAPTURES.rglob("raw_bw_*.bin")):
|
||||
blob = path.read_bytes()
|
||||
i = 0
|
||||
while i < len(blob):
|
||||
if not (blob[i] == ACK and i + 1 < len(blob) and blob[i + 1] == STX):
|
||||
i += 1
|
||||
continue
|
||||
j = i + 2
|
||||
body = bytearray()
|
||||
while j < len(blob):
|
||||
if blob[j] == DLE and j + 1 < len(blob):
|
||||
body.append(blob[j + 1])
|
||||
j += 2
|
||||
continue
|
||||
if blob[j] == ETX:
|
||||
break
|
||||
body.append(blob[j])
|
||||
j += 1
|
||||
payload = bytes(body[:-1])
|
||||
if len(payload) == 16: # a read frame; writes carry a data section
|
||||
sub = payload[2]
|
||||
offset = (payload[4] << 8) | payload[5]
|
||||
assert build_request(sub, offset, payload[6:16]) == blob[i:j + 1], (
|
||||
f"{path.name} @0x{i:04x} SUB=0x{sub:02x} offset=0x{offset:04x}"
|
||||
)
|
||||
checked += 1
|
||||
i = j + 1
|
||||
assert checked == 218, f"expected 218 read frames, checked {checked}"
|
||||
@@ -0,0 +1,402 @@
|
||||
"""Protocol-layer tests for the Micromate (series-4) live client.
|
||||
|
||||
The load-bearing assertion in here is not "our parser understands the device" —
|
||||
it is **"the bytes we put on the wire are the bytes THOR puts on the wire."**
|
||||
Every request constant below is lifted from
|
||||
``bridges/captures/9-24-26 - micromate2/`` (UM12947, firmware 11.0CB), so a
|
||||
passing test means a real unit has already answered exactly that frame.
|
||||
|
||||
Responses are replayed through a scripted transport. No hardware, no network.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from micromate import protocol as P
|
||||
from micromate.framing import ETX, STX, checksum, stuff
|
||||
from micromate.protocol import (
|
||||
ACK_DATA_LEN,
|
||||
ChecksumError,
|
||||
MicromateProtocol,
|
||||
ShortRead,
|
||||
UnexpectedResponse,
|
||||
chunk_params,
|
||||
key_lo_params,
|
||||
key_params,
|
||||
token_params,
|
||||
)
|
||||
|
||||
FLAGS_CB = 0xC5
|
||||
|
||||
|
||||
# ── Test doubles ──────────────────────────────────────────────────────────────
|
||||
|
||||
class ScriptedTransport:
|
||||
"""Hands back queued responses; records every byte written."""
|
||||
|
||||
def __init__(self, responses: list[bytes] | None = None) -> None:
|
||||
self.queue = list(responses or [])
|
||||
self.written: list[bytes] = []
|
||||
self._connected = True
|
||||
|
||||
# BaseTransport surface actually used by MicromateProtocol
|
||||
def connect(self) -> None:
|
||||
self._connected = True
|
||||
|
||||
def disconnect(self) -> None:
|
||||
self._connected = False
|
||||
|
||||
def is_connected(self) -> bool:
|
||||
return self._connected
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
self.written.append(data)
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
return self.queue.pop(0) if self.queue else b""
|
||||
|
||||
|
||||
def frame(rsp_sub: int, data: bytes, *, flags: int = FLAGS_CB, page: int = 0) -> bytes:
|
||||
"""Build a response frame the way a unit would."""
|
||||
payload = bytes([0x00, flags, rsp_sub, (page >> 8) & 0xFF, page & 0xFF]) + data
|
||||
return bytes([STX]) + stuff(payload + bytes([checksum(payload)])) + bytes([ETX])
|
||||
|
||||
|
||||
def ack(rsp_sub: int) -> bytes:
|
||||
return frame(rsp_sub, bytes(ACK_DATA_LEN))
|
||||
|
||||
|
||||
def proto(responses: list[bytes], **kw) -> tuple[MicromateProtocol, ScriptedTransport]:
|
||||
t = ScriptedTransport(responses)
|
||||
return MicromateProtocol(t, recv_timeout=0.5, **kw), t
|
||||
|
||||
|
||||
# ── Captured THOR request frames ──────────────────────────────────────────────
|
||||
|
||||
REQ = {
|
||||
"poll": bytes.fromhex("41021010005b000030000000000000000000009b03"),
|
||||
"serial": bytes.fromhex("41021010001500000a000000000000000000002f03"),
|
||||
"state": bytes.fromhex("41021010004900ffff000000000000000000005703"),
|
||||
"compliance": bytes.fromhex("41021010001a00ffff000000000000000000002803"),
|
||||
"arm": bytes.fromhex("41021010009300ffff00000000000000000000a103"),
|
||||
"setup_first": bytes.fromhex("41021010003f00ffff000000000000000000004d03"),
|
||||
"setup_next": bytes.fromhex("41021010004000ffff000000000000000000004e03"),
|
||||
}
|
||||
|
||||
# The complete 0x5A sequence for event 055d4a81 (4,076 bytes → 4 chunks), as
|
||||
# THOR sent it. Chunk 1's params hold a literal 0x04 and chunk 3's offset is
|
||||
# the exact remainder.
|
||||
REQ_CHUNKS_4A81 = [
|
||||
bytes.fromhex("41021010005a00100400055d4a810000000000009b03"),
|
||||
bytes.fromhex("41021010005a0010040000001004000000000000007203"),
|
||||
bytes.fromhex("41021010005a00100400000008000000000000007603"),
|
||||
bytes.fromhex("41021010005a001003ec00000c000000000000006503"),
|
||||
]
|
||||
SIZE_4A81 = 4076
|
||||
|
||||
|
||||
# ── Params builders ───────────────────────────────────────────────────────────
|
||||
|
||||
def test_event_token_sits_at_params_7():
|
||||
"""⚠ THOR sends 0xFE here; the protocol reference documents all-zero params.
|
||||
|
||||
That reference entry describes our own browse probing, not THOR's.
|
||||
"""
|
||||
assert token_params() == bytes.fromhex("00000000000000fe0000")
|
||||
|
||||
|
||||
def test_event_record_takes_the_full_key_at_params_4():
|
||||
assert key_params(bytes.fromhex("055d4a81")) == bytes.fromhex("00000000055d4a810000")
|
||||
|
||||
|
||||
def test_monitor_log_takes_only_the_low_half_of_the_key():
|
||||
"""⚠ Inferred from one key value -- see key_lo_params' docstring."""
|
||||
assert key_lo_params(bytes.fromhex("055d4a81")) == bytes.fromhex("0000000000004a810000")
|
||||
|
||||
|
||||
def test_chunk_params_switch_from_key_to_byte_offset():
|
||||
key = bytes.fromhex("055d4a81")
|
||||
assert chunk_params(key, 0) == bytes.fromhex("055d4a81000000000000")
|
||||
assert chunk_params(key, 1024) == bytes.fromhex("00000400000000000000")
|
||||
assert chunk_params(key, 13312) == bytes.fromhex("00003400000000000000")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", [b"", b"\x01\x02\x03", b"\x01\x02\x03\x04\x05"])
|
||||
def test_params_builders_reject_a_wrong_length_key(bad):
|
||||
for fn in (key_params, key_lo_params):
|
||||
with pytest.raises(ValueError):
|
||||
fn(bad)
|
||||
|
||||
|
||||
# ── Each read emits the frame THOR emits ──────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"name, method, rsp_sub, data_len",
|
||||
[
|
||||
("poll", "poll", 0xA4, 59),
|
||||
("serial", "read_serial", 0xEA, 21),
|
||||
("state", "read_state", 0xB6, 16),
|
||||
("compliance", "read_compliance_config", 0xE5, 2103),
|
||||
("setup_first", "read_first_setup", 0xC0, 266),
|
||||
("setup_next", "read_next_setup", 0xBF, 266),
|
||||
],
|
||||
)
|
||||
def test_reads_match_thors_wire_bytes(name, method, rsp_sub, data_len):
|
||||
p, t = proto([frame(rsp_sub, bytes(data_len))])
|
||||
getattr(p, method)()
|
||||
assert t.written == [REQ[name]]
|
||||
|
||||
|
||||
def test_arm_event_matches_thors_wire_bytes():
|
||||
p, t = proto([ack(0x6C)])
|
||||
p.arm_event()
|
||||
assert t.written == [REQ["arm"]]
|
||||
|
||||
|
||||
def test_poll_is_the_only_read_with_a_non_ffff_offset_besides_serial():
|
||||
"""Reads are single-step at 0xFFFF; POLL and SERIAL are the exceptions."""
|
||||
assert set(P._OFFSETS) == {P.SUB_POLL, P.SUB_SERIAL}
|
||||
assert P._OFFSETS[P.SUB_POLL] == 0x0030
|
||||
assert P._OFFSETS[P.SUB_SERIAL] == 0x000A
|
||||
|
||||
|
||||
# ── The chunk walk ────────────────────────────────────────────────────────────
|
||||
|
||||
def test_download_reproduces_thors_chunk_sequence_byte_for_byte():
|
||||
"""The whole point of step 2. Four chunks, 4,076 bytes, THOR's exact frames."""
|
||||
payload = bytes(range(256)) * 16 # 4096 B, we use the first 4076
|
||||
payload = payload[:SIZE_4A81]
|
||||
responses = []
|
||||
for i in range(4):
|
||||
want = min(P.CHUNK_SIZE, SIZE_4A81 - i * P.CHUNK_SIZE)
|
||||
body = payload[i * P.CHUNK_SIZE: i * P.CHUNK_SIZE + want]
|
||||
responses.append(frame(0xA5, bytes(11) + body, page=want // 256))
|
||||
|
||||
p, t = proto(responses)
|
||||
got = p.read_event_file(bytes.fromhex("055d4a81"), SIZE_4A81)
|
||||
|
||||
assert t.written == REQ_CHUNKS_4A81
|
||||
assert got == payload
|
||||
assert len(got) == SIZE_4A81
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"size, n_chunks, last_offset",
|
||||
[
|
||||
(4076, 4, 0x03EC), (11032, 11, 0x0318), (11502, 12, 0x00EE),
|
||||
(13424, 14, 0x0070), (8746, 9, 0x022A), (6092, 6, 0x03CC),
|
||||
(1024, 1, 0x0400), (1, 1, 0x0001), (1025, 2, 0x0001),
|
||||
],
|
||||
)
|
||||
def test_chunk_count_and_final_offset(size, n_chunks, last_offset):
|
||||
"""The first six rows are the six bench events, with THOR's real offsets."""
|
||||
responses = []
|
||||
for i in range(n_chunks):
|
||||
want = min(P.CHUNK_SIZE, size - i * P.CHUNK_SIZE)
|
||||
responses.append(frame(0xA5, bytes(11) + bytes(want)))
|
||||
|
||||
p, t = proto(responses)
|
||||
p.read_event_file(bytes.fromhex("055d4a81"), size)
|
||||
|
||||
assert len(t.written) == n_chunks
|
||||
# offset is payload[4:5] of the request; recover it from the built frame
|
||||
final = t.written[-1]
|
||||
assert final[7:9] in (
|
||||
bytes([last_offset >> 8, last_offset & 0xFF]),
|
||||
# a 0x02/0x03/0x04/0x10 high byte arrives escaped, shifting the pair
|
||||
bytes([0x10, last_offset >> 8]),
|
||||
)
|
||||
|
||||
|
||||
def test_a_short_chunk_raises_rather_than_truncating():
|
||||
"""A silently short event is the failure mode this codebase keeps hitting."""
|
||||
p, _ = proto([frame(0xA5, bytes(11) + bytes(900))]) # asked for 1024
|
||||
with pytest.raises(ShortRead, match="asked for 1024 B, got 900"):
|
||||
p.read_event_file(bytes.fromhex("055d4a81"), 1024)
|
||||
|
||||
|
||||
def test_a_chunk_too_short_to_hold_its_header_raises():
|
||||
p, _ = proto([frame(0xA5, bytes(4))])
|
||||
with pytest.raises(ShortRead, match="too short to hold a chunk header"):
|
||||
p.read_event_file(bytes.fromhex("055d4a81"), 1024)
|
||||
|
||||
|
||||
def test_download_rejects_a_nonsense_size():
|
||||
p, _ = proto([])
|
||||
with pytest.raises(ValueError):
|
||||
p.read_event_file(bytes.fromhex("055d4a81"), 0)
|
||||
|
||||
|
||||
# ── The monitor-log walk ──────────────────────────────────────────────────────
|
||||
|
||||
def test_monitor_log_walk_ends_on_a_short_response():
|
||||
"""⚠ Not a keyed read -- the same request repeated, device-side cursor.
|
||||
|
||||
Eight records then an 11-byte ack, which is what the capture shows.
|
||||
"""
|
||||
key = bytes.fromhex("055d4a81")
|
||||
responses = [frame(0xF5, bytes(297)) for _ in range(8)] + [ack(0xF5)]
|
||||
p, t = proto(responses)
|
||||
|
||||
records = []
|
||||
while (rec := p.read_monitor_log_next(key)) is not None:
|
||||
records.append(rec)
|
||||
|
||||
assert len(records) == 8
|
||||
assert len(t.written) == 9
|
||||
assert len(set(t.written)) == 1, "every request in the walk is identical"
|
||||
|
||||
|
||||
# ── Error handling ────────────────────────────────────────────────────────────
|
||||
|
||||
def test_a_bad_checksum_raises_by_default():
|
||||
"""⚠ Deliberately stricter than the Series III sibling.
|
||||
|
||||
That one logs and continues because its parser cannot always tell an
|
||||
inner-frame delimiter from a checksum byte. The Micromate rule is exact on
|
||||
251/251 captured frames, so a mismatch here means something real.
|
||||
"""
|
||||
bad = bytearray(frame(0xA4, bytes(59)))
|
||||
bad[-2] ^= 0xFF
|
||||
p, _ = proto([bytes(bad)])
|
||||
with pytest.raises(ChecksumError, match="checksum mismatch"):
|
||||
p.poll()
|
||||
|
||||
|
||||
def test_a_bad_checksum_can_be_downgraded_for_field_diagnosis():
|
||||
bad = bytearray(frame(0xA4, bytes(59)))
|
||||
bad[-2] ^= 0xFF
|
||||
p, _ = proto([bytes(bad)], strict_checksums=False)
|
||||
assert p.poll().sub == 0xA4
|
||||
|
||||
|
||||
def test_the_wrong_response_sub_raises():
|
||||
p, _ = proto([frame(0xE0, bytes(19))]) # 0xE0 answers 0x1F, not 0x1E
|
||||
with pytest.raises(UnexpectedResponse, match="expected SUB 0xE1"):
|
||||
p.read_event_first()
|
||||
|
||||
|
||||
def test_a_timeout_reports_how_many_bytes_arrived():
|
||||
"""Separates "nothing came back" from "bytes arrived but never framed".
|
||||
|
||||
Those have completely different causes -- and on a Micromate the second one
|
||||
is the signature of a modem forwarding a session it should not be.
|
||||
"""
|
||||
p, _ = proto([])
|
||||
with pytest.raises(P.TimeoutError, match="0 bytes were received"):
|
||||
p.poll()
|
||||
|
||||
unframed = b"\x02\x00\xc5\xa4garbage-no-terminator"
|
||||
p2, _ = proto([unframed])
|
||||
with pytest.raises(P.TimeoutError, match=f"{len(unframed)} bytes were received"):
|
||||
p2.poll()
|
||||
|
||||
|
||||
def test_a_leftover_frame_is_discarded_rather_than_answered_with():
|
||||
"""If a read returns two frames, the extra must not answer the NEXT request.
|
||||
|
||||
Every exchange resets the parser before sending, so anything already
|
||||
buffered is treated as stale. Delivering it would be the worse failure:
|
||||
`expected_sub` happens to catch a mismatched SUB, but a same-SUB leftover
|
||||
would sail through and return data for the wrong key.
|
||||
"""
|
||||
# Both frames arrive while answering arm_event(); the 0xE1 is left over.
|
||||
p, _ = proto([ack(0x6C) + frame(0xE1, b"\xaa" * 19)])
|
||||
p.arm_event()
|
||||
|
||||
# The next request gets no bytes of its own, so it must time out rather
|
||||
# than hand back the stale 0xE1.
|
||||
with pytest.raises(P.TimeoutError):
|
||||
p.read_event_first()
|
||||
|
||||
|
||||
# ── Against the real capture, when it happens to be present ───────────────────
|
||||
|
||||
_CAPTURES = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "bridges" / "captures" / "9-24-26 - micromate2"
|
||||
)
|
||||
_DOWNLOAD = "raw_bw_20260925_011403_Download_events_then_delete_1_event.bin"
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not (_CAPTURES / _DOWNLOAD).is_file(),
|
||||
reason="capture is gitignored; present only on a dev box",
|
||||
)
|
||||
def test_every_captured_download_frame_is_one_we_would_have_sent():
|
||||
"""Replay the real session: for each event, assert our chunk walk emits
|
||||
exactly the frames THOR emitted -- all 50-odd of them, six events."""
|
||||
from micromate.framing import ACK, DLE
|
||||
|
||||
def destuffed_frames(blob: bytes, is_req: bool):
|
||||
i, n = 0, len(blob)
|
||||
while i < n:
|
||||
if is_req:
|
||||
if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
|
||||
i += 1
|
||||
continue
|
||||
j = i + 2
|
||||
else:
|
||||
if blob[i] != STX:
|
||||
i += 1
|
||||
continue
|
||||
j = i + 1
|
||||
out = bytearray()
|
||||
while j < n:
|
||||
if blob[j] == DLE and j + 1 < n:
|
||||
out.append(blob[j + 1])
|
||||
j += 2
|
||||
continue
|
||||
if blob[j] == ETX:
|
||||
break
|
||||
out.append(blob[j])
|
||||
j += 1
|
||||
if len(out) >= 6:
|
||||
yield blob[i:j + 1], bytes(out[:-1])
|
||||
i = j + 1
|
||||
|
||||
bw = list(destuffed_frames((_CAPTURES / _DOWNLOAD).read_bytes(), True))
|
||||
s3 = list(
|
||||
destuffed_frames(
|
||||
(_CAPTURES / _DOWNLOAD.replace("raw_bw", "raw_s3")).read_bytes(), False
|
||||
)
|
||||
)
|
||||
|
||||
# Group THOR's 0x5A frames per event, taking each event's key+size from the
|
||||
# 1E/1F that preceded them.
|
||||
events, cur = [], None
|
||||
for (wire, req), (_, rsp) in zip(bw, s3):
|
||||
sub, data = req[2], rsp[5:]
|
||||
if sub in (0x1E, 0x1F) and len(data) >= 19:
|
||||
cur = {"key": data[11:15], "size": int.from_bytes(data[15:19], "big"),
|
||||
"reqs": [], "rsps": []}
|
||||
if cur["size"]:
|
||||
events.append(cur)
|
||||
elif sub == 0x5A and cur is not None:
|
||||
cur["reqs"].append(wire)
|
||||
cur["rsps"].append(rsp)
|
||||
|
||||
# The capture walks the chain twice (it deletes an event on the second
|
||||
# pass), so some 1E/1F hits carry a size but no download behind them.
|
||||
events = [e for e in events if e["reqs"]]
|
||||
assert len(events) == 6, f"expected 6 downloaded events, found {len(events)}"
|
||||
|
||||
total = 0
|
||||
for e in events:
|
||||
p, t = proto([bytes([STX]) + stuff(r + bytes([checksum(r)])) + bytes([ETX])
|
||||
for r in e["rsps"]])
|
||||
got = p.read_event_file(e["key"], e["size"])
|
||||
assert t.written == e["reqs"], (
|
||||
f"event {e['key'].hex()}: our {len(t.written)} frames differ from "
|
||||
f"THOR's {len(e['reqs'])}"
|
||||
)
|
||||
assert len(got) == e["size"]
|
||||
total += len(e["reqs"])
|
||||
|
||||
assert total == 56, f"expected 56 download frames across the 6 events, saw {total}"
|
||||
Reference in New Issue
Block a user