Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  serversdownandClaude Opus 5
|
0a96a34b02 |
+3
-68
@@ -46,50 +46,6 @@ All notable changes to seismo-relay are documented here.
|
||||
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
|
||||
files via the watcher path and never runs this walk.
|
||||
|
||||
- **The Micromate (Series IV) live wire protocol, reverse-engineered end to
|
||||
end** — `docs/micromate_protocol_reference.md`. Worked out against a bench
|
||||
UM12947 over USB and a recording relay, with THOR driving every write so that
|
||||
no command has ever been originated against a unit by this project. **A
|
||||
Micromate answers Series III command frames**, with three framing differences:
|
||||
responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
|
||||
or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
|
||||
**uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
|
||||
`SUB 0x1A` by 47x. Read path, event chain, and `SUB 0x5A` streaming the
|
||||
`.IDFW` file verbatim are all confirmed.
|
||||
- **Series IV setup management, fully mapped.** `0x41` reads the active setup
|
||||
name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
|
||||
write it back. **Setups are read-modify-write** — the written block is the
|
||||
read block, 91% byte-identical at a fixed 11-byte shift. `0xDA` **creates**
|
||||
files rather than only overwriting, confirmed on the unit's own screen, and an
|
||||
overwrite is protocol-identical to a create: no handshake, no warning, and no
|
||||
protection even over the *active* setup of a monitoring unit.
|
||||
- **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
|
||||
records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
|
||||
half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
|
||||
name. Verified entry-for-entry against the operator's own THOR screen.
|
||||
- **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
|
||||
`0x8D`/`0x8E` write. This **retracts** an earlier conclusion in the same
|
||||
document that no such command existed; that was inferred from absent firmware
|
||||
strings and was wrong.
|
||||
- **Monitoring control and per-event delete.** `0x96`/`0x97` start and stop as
|
||||
on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
|
||||
for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
|
||||
constant. Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
|
||||
which is safer than Series III's erase-everything. `SUB 0x1C` also carries the
|
||||
device clock.
|
||||
- **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
|
||||
identically as "disconnected": refused, connect timeout (the silent-drop
|
||||
signature of a trusted-IP whitelist), **connected but no reply** (the modem
|
||||
answered and the unit did not), and replied. Each verdict names what to try
|
||||
next. `--slots N` tests single-session modem behaviour.
|
||||
- **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
|
||||
decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
|
||||
`onewaydev`) driven by a control file. No pyserial; stdlib `termios` only.
|
||||
- **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
|
||||
— a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
|
||||
all, since it scans for `DLE+STX`), byte-exact capture recovery from a
|
||||
`socat -x` relay log, and a serial-port stand-in that answers as a unit.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Connecting to a unit no longer walks its event chain.** `/device/events`
|
||||
@@ -102,32 +58,11 @@ All notable changes to seismo-relay are documented here.
|
||||
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
|
||||
event it has stored.
|
||||
|
||||
- **Recorded what THOR actually does on the wire**, measured rather than assumed.
|
||||
A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
|
||||
MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
|
||||
`MONITOR_STATUS` check at 60 s. The **status interval is honoured; the
|
||||
connection interval is not** — it sets `(status / connection) - 1` checks per
|
||||
cycle, so equal values yield *zero* cheap checks and every connection becomes
|
||||
the expensive one.
|
||||
- **Two THOR defects reproduced with timestamps.** After a connection drops
|
||||
mid-download it retries **once**, stops polling entirely and **never resumes**,
|
||||
while displaying `Connected` for as long as it is left alone — and `Idle` for a
|
||||
unit that is actively recording. Separately, THOR's own log shows a
|
||||
**subscription leak**: one logical event dispatched to a growing number of
|
||||
handlers, **1 to 12 over ten hours** of uptime, consistent with the field
|
||||
report that only a restart recovers it.
|
||||
- **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
|
||||
either firmware line. A PL2303 cable (Benfei) leaves a unit with no working
|
||||
modem port; an FTDI cable (Sabrent) works. Both are in circulation and
|
||||
indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
|
||||
|
||||
### Migration
|
||||
|
||||
**None.** Frontend, documentation and bench tooling only — no codec,
|
||||
waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump. The
|
||||
webapp is served from the image, so its changes appear after the next `sfm`
|
||||
rebuild. The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only;
|
||||
nothing under `sfm/`, `minimateplus/` or `micromate/` was touched.
|
||||
**None.** Frontend and documentation only — no codec, waveform-store or DB
|
||||
change, no schema change, and no `TOOL_VERSION` bump. The webapp is served
|
||||
from the image, so the change appears after the next `sfm` rebuild.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,338 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_link.py — a "perfect modem" between THOR and a Micromate, with a readable
|
||||
log and deliberate fault injection.
|
||||
|
||||
Why
|
||||
---
|
||||
THOR gives almost no visibility into a connection: a refresh button, two poll
|
||||
intervals, and no way to see whether a check succeeded, timed out, or was never
|
||||
sent. When a unit "won't stay connected" there is nothing to look at.
|
||||
|
||||
This sits where the cellular modem would sit and answers the question directly:
|
||||
|
||||
* **What is THOR actually doing?** Every frame is decoded and timestamped —
|
||||
`POLL`, `MONITOR_STATUS`, `SETUP_NAME_READ` — not a hex dump.
|
||||
* **Is it even trying?** Silence is visible: the log shows gaps.
|
||||
* **How does it behave when the link misbehaves?** Faults can be injected on
|
||||
demand, which a real cell link will not do on cue.
|
||||
|
||||
Point THOR at this host and port exactly as if it were a modem (Communication:
|
||||
TCP, IP: <this host>, Port: <--listen>).
|
||||
|
||||
Fault injection
|
||||
---------------
|
||||
Write a mode into the control file (default `mm_link.ctl`) and it takes effect
|
||||
on the next byte:
|
||||
|
||||
echo pass > mm_link.ctl # normal relay
|
||||
echo blackhole > mm_link.ctl # TCP stays up, bytes are swallowed
|
||||
echo drop > mm_link.ctl # close the connection abruptly (RST-ish)
|
||||
echo delay:2.0 > mm_link.ctl # forward, but 2 s late in both directions
|
||||
echo onewaydev > mm_link.ctl # THOR->unit passes, unit->THOR is swallowed
|
||||
|
||||
**`blackhole` is the one that matters.** It reproduces the classic cellular
|
||||
failure: the socket is still open as far as both ends are concerned, but nothing
|
||||
crosses. A client that relies on TCP to tell it the peer is gone will sit there
|
||||
until the OS keepalive fires — which by default is about two hours.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python3 bridges/mm_link.py --serial /dev/ttyACM0 --baud 115200 \\
|
||||
--listen 12345 --logdir ~/mm-captures
|
||||
|
||||
Writes, per session:
|
||||
<logdir>/mmlink_<ts>/session.log decoded, timestamped, human-readable
|
||||
<logdir>/mmlink_<ts>/raw_bw.bin THOR -> unit, raw
|
||||
<logdir>/mmlink_<ts>/raw_s3.bin unit -> THOR, raw
|
||||
|
||||
The raw pair loads straight into `scratch/mm_frame_parse.py`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scratch"))
|
||||
try:
|
||||
from mm_frame_parse import SUBNAME, destuff # noqa: F401
|
||||
except Exception: # pragma: no cover
|
||||
SUBNAME = {}
|
||||
|
||||
import errno
|
||||
import select
|
||||
import termios
|
||||
|
||||
DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
|
||||
|
||||
_BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
|
||||
57600: termios.B57600, 115200: termios.B115200}
|
||||
|
||||
|
||||
class SerialPort:
|
||||
"""Minimal raw serial port on stdlib termios — no pyserial dependency.
|
||||
|
||||
The bench hosts are whatever is to hand; requiring a pip install on someone
|
||||
else's machine is a poor trade for the ~30 lines this saves.
|
||||
"""
|
||||
|
||||
def __init__(self, path: str, baud: int):
|
||||
if baud not in _BAUD:
|
||||
raise ValueError(f"unsupported baud {baud}; pick one of {sorted(_BAUD)}")
|
||||
self.fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
a = termios.tcgetattr(self.fd)
|
||||
a[0] = 0 # iflag: no translation
|
||||
a[1] = 0 # oflag: raw
|
||||
a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL # cflag: 8N1, ignore modem lines
|
||||
a[3] = 0 # lflag: non-canonical, no echo
|
||||
a[4] = a[5] = _BAUD[baud]
|
||||
a[6] = list(a[6])
|
||||
a[6][termios.VMIN] = 0
|
||||
a[6][termios.VTIME] = 0
|
||||
termios.tcsetattr(self.fd, termios.TCSANOW, a)
|
||||
termios.tcflush(self.fd, termios.TCIOFLUSH)
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
r, _, _ = select.select([self.fd], [], [], 0.2)
|
||||
if not r:
|
||||
return b""
|
||||
try:
|
||||
return os.read(self.fd, n)
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
return b""
|
||||
raise
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
while data:
|
||||
try:
|
||||
data = data[os.write(self.fd, data):]
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
select.select([], [self.fd], [], 0.2)
|
||||
continue
|
||||
raise
|
||||
|
||||
def close(self) -> None:
|
||||
try:
|
||||
os.close(self.fd)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def name_of(sub: int, is_request: bool) -> str:
|
||||
if is_request:
|
||||
return SUBNAME.get(sub, f"SUB_{sub:02X}")
|
||||
return "rsp " + SUBNAME.get(0xFF - sub, f"SUB_{0xFF - sub:02X}")
|
||||
|
||||
|
||||
class FrameSniffer:
|
||||
"""Accumulate bytes and report complete frames, without altering the stream."""
|
||||
|
||||
def __init__(self, is_request: bool):
|
||||
self.is_request = is_request
|
||||
self.buf = bytearray()
|
||||
|
||||
def feed(self, data: bytes):
|
||||
"""Yield (sub, payload_len) for each complete frame seen."""
|
||||
self.buf.extend(data)
|
||||
while True:
|
||||
start = -1
|
||||
for i, b in enumerate(self.buf):
|
||||
if self.is_request and b == ACK and i + 1 < len(self.buf) and self.buf[i + 1] == STX:
|
||||
start = i
|
||||
break
|
||||
if not self.is_request and b == STX:
|
||||
start = i
|
||||
break
|
||||
if start < 0:
|
||||
if len(self.buf) > 8192:
|
||||
del self.buf[:-16]
|
||||
return
|
||||
j = start + (2 if self.is_request else 1)
|
||||
end = -1
|
||||
while j < len(self.buf):
|
||||
if self.buf[j] == DLE and j + 1 < len(self.buf):
|
||||
j += 2
|
||||
continue
|
||||
if self.buf[j] == ETX:
|
||||
end = j
|
||||
break
|
||||
j += 1
|
||||
if end < 0:
|
||||
return # wait for more bytes
|
||||
body = self.buf[start:end + 1]
|
||||
del self.buf[:end + 1]
|
||||
# SUB sits at a fixed spot past the leading framing -- but it is
|
||||
# DLE-escaped when its own value is 0x02/0x03/0x04/0x10, so a raw
|
||||
# read reports 0x10 for those. SUB 0x02 was being logged as
|
||||
# "SUB_10" until this was handled.
|
||||
off = 5 if self.is_request else 3
|
||||
if len(body) > off:
|
||||
sub = body[off]
|
||||
if sub == DLE and len(body) > off + 1:
|
||||
sub = body[off + 1]
|
||||
yield sub, len(body)
|
||||
|
||||
|
||||
class Link:
|
||||
def __init__(self, args):
|
||||
self.args = args
|
||||
self.mode = "pass"
|
||||
self.delay = 0.0
|
||||
self.ctl = Path(args.control)
|
||||
self.session: Path | None = None
|
||||
self.log_fh = None
|
||||
self.raw = {}
|
||||
self.t0 = time.time()
|
||||
self.counts = {}
|
||||
|
||||
# ── logging ────────────────────────────────────────────────────────────
|
||||
def open_session(self):
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
self.session = Path(self.args.logdir) / f"mmlink_{ts}"
|
||||
self.session.mkdir(parents=True, exist_ok=True)
|
||||
self.log_fh = open(self.session / "session.log", "a", buffering=1)
|
||||
self.raw = {
|
||||
"bw": open(self.session / "raw_bw.bin", "ab"),
|
||||
"s3": open(self.session / "raw_s3.bin", "ab"),
|
||||
}
|
||||
self.say(f"=== session {ts} — serial {self.args.serial} @ {self.args.baud} ===")
|
||||
|
||||
def say(self, text: str):
|
||||
line = f"{datetime.datetime.now().strftime('%H:%M:%S.%f')[:-3]} {text}"
|
||||
print(line, flush=True)
|
||||
if self.log_fh:
|
||||
self.log_fh.write(line + "\n")
|
||||
|
||||
# ── control file ───────────────────────────────────────────────────────
|
||||
def poll_control(self):
|
||||
while True:
|
||||
try:
|
||||
if self.ctl.exists():
|
||||
want = self.ctl.read_text().strip().lower()
|
||||
if want.startswith("delay:"):
|
||||
d = float(want.split(":", 1)[1])
|
||||
if ("delay", d) != (self.mode, self.delay):
|
||||
self.mode, self.delay = "delay", d
|
||||
self.say(f"*** MODE -> delay {d}s ***")
|
||||
elif want and want != self.mode:
|
||||
self.mode, self.delay = want, 0.0
|
||||
self.say(f"*** MODE -> {want} ***")
|
||||
except Exception:
|
||||
pass
|
||||
time.sleep(0.25)
|
||||
|
||||
# ── the relay ──────────────────────────────────────────────────────────
|
||||
def pump(self, src, dst, tag: str, is_request: bool, stop: threading.Event):
|
||||
sniff = FrameSniffer(is_request)
|
||||
arrow = "THOR->unit" if is_request else "unit->THOR"
|
||||
last = time.time()
|
||||
while not stop.is_set():
|
||||
timed_out = False
|
||||
try:
|
||||
data = src.recv(4096) if isinstance(src, socket.socket) else src.read(4096)
|
||||
except TimeoutError:
|
||||
timed_out = True
|
||||
# socket.timeout subclasses OSError, so it MUST be caught first.
|
||||
# Treating it as a dead socket closes the connection after 200 ms
|
||||
# of quiet -- which is exactly what `blackhole` produces, so the
|
||||
# relay killed the link it was supposed to be faking a fault on.
|
||||
data = b""
|
||||
except OSError:
|
||||
break
|
||||
if isinstance(src, socket.socket) and data == b"" and not timed_out:
|
||||
self.say(f"{arrow}: peer closed the connection")
|
||||
break
|
||||
if not data:
|
||||
if time.time() - last > self.args.quiet_after and self.counts:
|
||||
self.say(f"--- {self.args.quiet_after:.0f}s with no traffic ---")
|
||||
last = time.time()
|
||||
continue
|
||||
last = time.time()
|
||||
|
||||
self.raw[tag].write(data)
|
||||
self.raw[tag].flush()
|
||||
for sub, ln in sniff.feed(data):
|
||||
label = name_of(sub, is_request)
|
||||
self.counts[label] = self.counts.get(label, 0) + 1
|
||||
self.say(f"{arrow} {label:<20} ({ln} B)"
|
||||
+ ("" if self.mode == "pass" else f" [mode={self.mode}]"))
|
||||
|
||||
mode = self.mode
|
||||
if mode == "drop":
|
||||
self.say(f"{arrow}: DROPPING the connection (fault injection)")
|
||||
stop.set()
|
||||
break
|
||||
if mode == "blackhole":
|
||||
continue # swallow, keep the socket open
|
||||
if mode == "onewaydev" and not is_request:
|
||||
continue # unit's replies never reach THOR
|
||||
if mode == "delay" and self.delay:
|
||||
time.sleep(self.delay)
|
||||
try:
|
||||
if isinstance(dst, socket.socket):
|
||||
dst.sendall(data)
|
||||
else:
|
||||
dst.write(data)
|
||||
except OSError:
|
||||
break
|
||||
stop.set()
|
||||
|
||||
def serve(self):
|
||||
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
srv.bind(("0.0.0.0", self.args.listen))
|
||||
srv.listen(5)
|
||||
self.open_session()
|
||||
self.say(f"listening on 0.0.0.0:{self.args.listen} control file: {self.ctl}")
|
||||
self.say("point THOR at this host/port as Communication=TCP")
|
||||
threading.Thread(target=self.poll_control, daemon=True).start()
|
||||
|
||||
while True:
|
||||
conn, addr = srv.accept()
|
||||
conn.settimeout(0.2)
|
||||
self.say(f"+++ THOR connected from {addr[0]}:{addr[1]} +++")
|
||||
try:
|
||||
ser = SerialPort(self.args.serial, self.args.baud)
|
||||
except OSError as e:
|
||||
self.say(f"!!! cannot open {self.args.serial}: {e}")
|
||||
conn.close()
|
||||
continue
|
||||
stop = threading.Event()
|
||||
ts = [
|
||||
threading.Thread(target=self.pump, args=(conn, ser, "bw", True, stop), daemon=True),
|
||||
threading.Thread(target=self.pump, args=(ser, conn, "s3", False, stop), daemon=True),
|
||||
]
|
||||
for t in ts:
|
||||
t.start()
|
||||
for t in ts:
|
||||
t.join()
|
||||
conn.close()
|
||||
ser.close()
|
||||
summary = ", ".join(f"{k}x{v}" for k, v in sorted(self.counts.items()))
|
||||
self.say(f"--- connection closed. frames this session: {summary or 'none'} ---")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--serial", default="/dev/ttyACM0")
|
||||
ap.add_argument("--baud", type=int, default=115200)
|
||||
ap.add_argument("--listen", type=int, default=12345)
|
||||
ap.add_argument("--logdir", default=os.path.expanduser("~/mm-captures"))
|
||||
ap.add_argument("--control", default="mm_link.ctl")
|
||||
ap.add_argument("--quiet-after", type=float, default=30.0,
|
||||
help="log a marker after this many seconds of silence")
|
||||
Link(ap.parse_args()).serve()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,226 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_probe.py — answer "why can't we reach this unit?" in one command.
|
||||
|
||||
THOR reports a failed connection as "disconnected" and nothing else. That single
|
||||
word covers at least four completely different faults with four different fixes,
|
||||
and telling them apart is the difference between a modem reboot and a site visit:
|
||||
|
||||
* **connection refused** something answered and said no — wrong port, or the
|
||||
modem is refusing a further session
|
||||
* **connect timed out** nothing answered at all — trusted-IP whitelist,
|
||||
firewall, or the modem is off the network
|
||||
* **connected, no reply** the MODEM answered but the unit did not. The TCP
|
||||
path is fine; the modem is not forwarding to serial.
|
||||
This is the signature of a wedged transparent-TCP
|
||||
session, and it is the one THOR cannot distinguish
|
||||
from any of the others
|
||||
* **replied** the unit is alive; the problem is upstream software
|
||||
|
||||
Read-only. It sends `POLL`, then optionally `SERIAL` and the state read — the
|
||||
same three commands THOR's own connection check uses — and never writes.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python3 bridges/mm_probe.py 63.45.161.30:9034
|
||||
python3 bridges/mm_probe.py 10.0.0.8:12345 --timeout 5
|
||||
python3 bridges/mm_probe.py <host:port> --slots 3
|
||||
|
||||
`--slots N` opens N connections at once and reports how many the far end accepts.
|
||||
A transparent-TCP modem typically serves **one** session; if the first succeeds
|
||||
and the rest are refused or hang, that confirms the single-slot behaviour and
|
||||
explains why a leaked session takes a unit offline until the slot frees.
|
||||
|
||||
Works for both series: a Series III reply opens `DLE STX`, a Micromate reply
|
||||
opens with a bare `STX`, so the probe also tells you which one answered.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from minimateplus.framing import build_bw_frame # noqa: E402
|
||||
|
||||
DLE, STX, ETX = 0x10, 0x02, 0x03
|
||||
|
||||
|
||||
def destuff(raw: bytes) -> bytes:
|
||||
"""Strip framing and DLE escapes; return the payload without its checksum."""
|
||||
i = 1 if raw and raw[0] == STX else (2 if len(raw) > 1 and raw[1] == STX else 0)
|
||||
out = bytearray()
|
||||
while i < len(raw):
|
||||
b = raw[i]
|
||||
if b == DLE and i + 1 < len(raw):
|
||||
out.append(raw[i + 1])
|
||||
i += 2
|
||||
continue
|
||||
if b == ETX:
|
||||
break
|
||||
out.append(b)
|
||||
i += 1
|
||||
return bytes(out[:-1]) if len(out) > 1 else b""
|
||||
|
||||
|
||||
# Reads are two-step on Series III: a probe at offset 0, then a data read at the
|
||||
# block's length. THOR sends these offsets, and they also work on a Micromate.
|
||||
OFFSETS = {0x5B: 0x0030, 0x15: 0x000A, 0x49: 0xFFFF}
|
||||
|
||||
|
||||
def exchange(sock: socket.socket, sub: int, timeout: float) -> tuple[bytes, float]:
|
||||
sock.sendall(build_bw_frame(sub, OFFSETS.get(sub, 0)))
|
||||
t0 = time.time()
|
||||
buf, deadline = b"", t0 + timeout
|
||||
sock.settimeout(0.3)
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
chunk = sock.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
if buf.endswith(bytes([ETX])) and len(buf) > 8:
|
||||
break
|
||||
except TimeoutError:
|
||||
continue
|
||||
except OSError:
|
||||
break
|
||||
return buf, time.time() - t0
|
||||
|
||||
|
||||
def step(n: int, label: str, result: str) -> None:
|
||||
print(f" [{n}] {label:.<28} {result}")
|
||||
|
||||
|
||||
def probe(host: str, port: int, timeout: float) -> int:
|
||||
print(f"\ntarget {host}:{port} (read-only: POLL, SERIAL, state)\n")
|
||||
|
||||
# ── 1. TCP ────────────────────────────────────────────────────────────
|
||||
t0 = time.time()
|
||||
try:
|
||||
sock = socket.create_connection((host, port), timeout=timeout)
|
||||
except ConnectionRefusedError:
|
||||
step(1, "TCP connect", f"REFUSED after {1000*(time.time()-t0):.0f} ms")
|
||||
print("\nverdict: something answered and actively refused.")
|
||||
print(" Not a silent firewall drop — the host is reachable.")
|
||||
print(" Wrong port, the service is down, or the modem is refusing")
|
||||
print(" an additional session because its one slot is in use.")
|
||||
return 2
|
||||
except (TimeoutError, socket.timeout):
|
||||
step(1, "TCP connect", f"TIMED OUT after {time.time()-t0:.1f} s")
|
||||
print("\nverdict: nothing answered at all.")
|
||||
print(" A silent drop, which is what a trusted-IP whitelist looks")
|
||||
print(" like — it discards rather than refuses. Check the modem's")
|
||||
print(" Trusted IPs (and note a VPN changes the IP you arrive from),")
|
||||
print(" the firewall, and whether the modem is on the network.")
|
||||
return 3
|
||||
except OSError as e:
|
||||
step(1, "TCP connect", f"FAILED: {e}")
|
||||
return 4
|
||||
step(1, "TCP connect", f"ok ({1000*(time.time()-t0):.0f} ms)")
|
||||
|
||||
# ── 2. POLL ───────────────────────────────────────────────────────────
|
||||
try:
|
||||
raw, dt = exchange(sock, 0x5B, timeout)
|
||||
except OSError as e:
|
||||
step(2, "POLL", f"send failed: {e}")
|
||||
sock.close()
|
||||
return 4
|
||||
|
||||
if not raw:
|
||||
step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
|
||||
print("\nverdict: the MODEM answered but the unit did not.")
|
||||
print(" TCP is fine end to end — something accepted the connection.")
|
||||
print(" What is missing is the serial side. Most likely the modem is")
|
||||
print(" not forwarding to its serial port, which is what a wedged")
|
||||
print(" transparent-TCP session looks like: the slot is held by a")
|
||||
print(" connection that never closed.")
|
||||
print("\n Try, in order:")
|
||||
print(" 1. ACEmanager -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot forever. 2 minutes is the value")
|
||||
print(" this project standardised on.")
|
||||
print(" 2. Reboot the modem. If that fixes it, the modem was")
|
||||
print(" holding state and the timeout is the permanent fix.")
|
||||
print(" 3. Check the unit's own screen — serial cable, power.")
|
||||
sock.close()
|
||||
return 5
|
||||
|
||||
series = "Series III (DLE STX)" if raw[0] == DLE else "Micromate (bare STX)"
|
||||
step(2, "POLL", f"reply {len(raw)} B in {1000*dt:.0f} ms")
|
||||
p = destuff(raw)
|
||||
ok = len(p) > 3 and p[2] == 0xFF - 0x5B
|
||||
step(3, "frame", f"{'valid' if ok else 'MALFORMED'}, {series}")
|
||||
if not ok:
|
||||
print("\nverdict: something replied, but not a seismograph.")
|
||||
print(" Another service is on this port, or the modem is in a mode")
|
||||
print(" that injects its own text (check Quiet Mode / AT echo).")
|
||||
print(f" first bytes: {raw[:16].hex(' ')}")
|
||||
sock.close()
|
||||
return 6
|
||||
|
||||
# ── 3. identity + state ───────────────────────────────────────────────
|
||||
for n, (sub, label) in enumerate(((0x15, "serial"), (0x49, "state")), start=4):
|
||||
try:
|
||||
r, dt = exchange(sock, sub, timeout)
|
||||
d = destuff(r)[5:]
|
||||
if sub == 0x15:
|
||||
# serial is a null-terminated run; a further field follows it
|
||||
serial = bytes(d[11:]).split(b"\x00")[0]
|
||||
step(n, label, serial.decode("ascii", "replace") or "(empty)")
|
||||
else:
|
||||
step(n, label, "MONITORING" if len(d) > 11 and d[11] else "idle")
|
||||
except OSError:
|
||||
step(n, label, "no reply")
|
||||
|
||||
sock.close()
|
||||
print("\nverdict: the unit is alive and answering.")
|
||||
print(" If THOR still shows it disconnected, the fault is in THOR, not")
|
||||
print(" the network or the device.")
|
||||
return 0
|
||||
|
||||
|
||||
def slots(host: str, port: int, n: int, timeout: float) -> None:
|
||||
print(f"\nopening {n} simultaneous connections to {host}:{port}\n")
|
||||
held = []
|
||||
for i in range(n):
|
||||
try:
|
||||
s = socket.create_connection((host, port), timeout=timeout)
|
||||
held.append(s)
|
||||
step(i + 1, f"connection {i+1}", "accepted")
|
||||
except ConnectionRefusedError:
|
||||
step(i + 1, f"connection {i+1}", "REFUSED")
|
||||
except (TimeoutError, socket.timeout):
|
||||
step(i + 1, f"connection {i+1}", "timed out")
|
||||
except OSError as e:
|
||||
step(i + 1, f"connection {i+1}", f"failed: {e}")
|
||||
print(f"\n{len(held)} of {n} accepted.")
|
||||
if len(held) == 1:
|
||||
print(" Single-slot behaviour confirmed — this far end serves ONE")
|
||||
print(" session at a time. A connection that is never closed takes")
|
||||
print(" the unit offline until the idle timeout frees the slot.")
|
||||
for s in held:
|
||||
s.close()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("target", help="host:port, e.g. 63.45.161.30:9034")
|
||||
ap.add_argument("--timeout", type=float, default=10.0)
|
||||
ap.add_argument("--slots", type=int, metavar="N",
|
||||
help="open N simultaneous connections to test single-slot behaviour")
|
||||
a = ap.parse_args()
|
||||
host, _, port = a.target.rpartition(":")
|
||||
if not host:
|
||||
ap.error("target must be host:port")
|
||||
if a.slots:
|
||||
slots(host, int(port), a.slots, a.timeout)
|
||||
return 0
|
||||
return probe(host, int(port), a.timeout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -934,6 +934,120 @@ which looks like a different failure mode wearing the same signature.
|
||||
|
||||
---
|
||||
|
||||
### 8f. BE12599 — CONFIRMED on the bench: the geophone assembly, not the recorder (2026-09-23)
|
||||
|
||||
**The first mechanism in this investigation established physically rather than
|
||||
inferred.** Brian pulled BE12599 from I-80 Abut 1 West on 2026-09-21 and bench-
|
||||
tested it. A geophone swap settles it.
|
||||
|
||||
#### The swap test
|
||||
|
||||
Same recorder, twenty minutes apart, sensor self-check:
|
||||
|
||||
| geophone fitted | Tran | **Vert** | Long |
|
||||
|---|---|---|---|
|
||||
| BE12599's own (2026-09-22 21:10) | 4.2 | **2026.0** | 4.8 |
|
||||
| BE9888's known-good (2026-09-22 22:48) | 3.6 | **3.4** | 3.6 |
|
||||
|
||||
Vert frequency likewise **2.2 Hz -> 7.4 Hz**. Fleet reference: overswing 3.3–5.1
|
||||
(median 3.8, max 5.1 across 42,979 passing records), frequency 7.4–7.8 Hz.
|
||||
|
||||
With the good geophone the unit shows **no offsets, no self-triggering, all three
|
||||
channels Passed**, and records a clean damped impulse with broadband 5–100 Hz
|
||||
content. **BE12599 the recorder is functional** — it is only out of calibration
|
||||
(due 2025-08-12).
|
||||
|
||||
**The fault is in the geophone assembly: element, cable, or its connector.**
|
||||
The swap does not separate those three, because the cable is permanently glanded
|
||||
into the geophone.
|
||||
|
||||
#### The bench evidence, with ground motion removed
|
||||
|
||||
17 events recorded on a workbench with nothing touching the sensor.
|
||||
**The control** — `N599LRY0.4B0`, trigger forced by slapping the *microphone*,
|
||||
geophone untouched:
|
||||
|
||||
| channel | peak in/s | frac < 0 | mean/peak |
|
||||
|---|---|---|---|
|
||||
| Tran | 0.070 | **0.00** | 0.87 |
|
||||
| **Vert** | **0.190** | **0.35** | 0.13 |
|
||||
| Long | 0.070 | **0.00** | 0.84 |
|
||||
|
||||
Vert produces 0.19 in/s of bipolar wander with **no mechanical input at all**.
|
||||
The fault is *continuous*, not episodic — every field event was simply the
|
||||
moment it crossed the trigger.
|
||||
|
||||
**All three channels are affected**, consistently across all 17 events:
|
||||
Tran and Long parked at a small positive DC (`frac<0 = 0.00`, mp 0.64–0.95);
|
||||
Vert wandering at 0.89–1.78 Hz, peaks 0.14–0.54. Yet the swing test calls Tran
|
||||
and Long **Passed** (4.2 / 4.8). So their *coils* are healthy and merely sitting
|
||||
on an offset, while Vert's circuit has lost damping entirely.
|
||||
|
||||
That reads as **leakage on several contacts plus one gone open** — which is what
|
||||
progressive corrosion at a shared connection does. Three independent geophone
|
||||
elements do not fail in sequence.
|
||||
|
||||
**It is still progressing.** Tran's DC offset was ~0.011 in/s on 2026-09-14 and
|
||||
~0.064 on 2026-09-22 — 6x in eight days. Vert's overswing went 7.5 -> 2026.0 over
|
||||
the same window.
|
||||
|
||||
#### ⚠ This explains the autozero's 10% success rate
|
||||
|
||||
The journal has framed this fault around the **recorder** throughout: Instantel's
|
||||
autozero, the X1/X8 gains, the 2027–2069 window, `SUB 0x0E` as the best lead.
|
||||
All of that adjusts the recorder's zero reference.
|
||||
|
||||
**An autozero cannot fix a geophone.** If most offsets are geophone-assembly
|
||||
faults, a re-zero would succeed only on the minority where the recorder really is
|
||||
at fault — which is exactly the ~10% Brian's father reported and which has sat
|
||||
unexplained in §5 since 2026-08-28.
|
||||
|
||||
**Procedure change: swap the geophone first.** It is a five-minute field test
|
||||
costing nothing. On this unit it would have answered the question in August.
|
||||
|
||||
⚠ Scoped to one unit. The other four confirmed units are the obvious check.
|
||||
|
||||
#### Correction to §8e
|
||||
|
||||
§8e is titled "a connector, not a geophone" and concluded the element was fine
|
||||
because the fault hopped channels. The channel-hopping observation stands and the
|
||||
swap test supports the general shape, but **that title overstated what was known**:
|
||||
the swap narrows the fault to the *assembly*, and element / cable / connector are
|
||||
still open within it. §8e's reasoning was sound; its confidence was not.
|
||||
|
||||
#### Cause — road salt now outranks the mice
|
||||
|
||||
The enclosure held a mouse nest on the battery ([[§8e]]), and that framed the
|
||||
2026-09-22 analysis. But **the geophone is buried in the ground, not in the
|
||||
enclosure** — they are two independent contamination paths, and the fault is in
|
||||
the buried half.
|
||||
|
||||
The unit sat **~6 feet from I-80 westbound**. Winter road salt spray and highway
|
||||
runoff put a buried geophone in chloride-rich water for months a year. Chloride
|
||||
drives exactly this failure: corrosion at terminations, leakage across
|
||||
insulation, eventually an open circuit.
|
||||
|
||||
The mice remain a real finding for the *enclosure* and the recorder end. They are
|
||||
probably not the cause of this fault.
|
||||
|
||||
**Testable consequence:** road salt is a property of the **site**, not the unit.
|
||||
Units deployed near salted highways should carry higher geophone-assembly failure
|
||||
rates than units on quiet sites. Nothing in the archive records proximity to a
|
||||
road, so this needs the deployment records rather than the event data.
|
||||
|
||||
#### Open — to be settled by opening the geophone
|
||||
|
||||
1. **Meter at the geophone plug** before disassembly: three coil pairs, and each
|
||||
to shell. Prediction: **Vert open or very high** (that is the 2026 overswing);
|
||||
Tran and Long matched and normal; **insulation to shell finite on all three**,
|
||||
Vert worst (that is the leakage putting DC on healthy coils).
|
||||
2. **Then open it.** Look for a water line, white/green corrosion at the coil
|
||||
terminations, the state of the damping shunt, and the cable gland from the
|
||||
inside. Measure the coils again **at the internal terminals** — if Vert reads
|
||||
fine there, the fault is in the internal wiring or the gland, not the element.
|
||||
|
||||
---
|
||||
|
||||
### ⚠ Serial prefixes — four of these units are BlastMates, not MiniMates
|
||||
|
||||
Corrected 2026-09-06, after Brian queried "BA10895?" against a report that
|
||||
@@ -985,3 +1099,7 @@ decoder lost the geo threshold along with the serial. See commit `9ceff65`.
|
||||
| 2026-09-06 | The spread gate is **blind to onsets by construction** — it rejects a moving floor, which is what an onset is. It rejected the very record in which the ramp is visible. |
|
||||
| 2026-09-07 | **The non-motion test** (Brian: "it doesn't cross zero"). `\|mean\|/peak` is bimodal with a 1.09% dead zone and returns exactly the 5 confirmed units from physics, not a threshold. Independent corroboration of the unit list. **BE11007 settled as NOT an offset** — a one-sided 9 in/s blast. |
|
||||
| 2026-09-07 | **BE12599 is a connector fault, not a geophone fault.** Unipolar spikes with a 26→118 ms RC tail progressing to a flat pedestal, and the fault MOVES between Long and Tran while the sensor self-check passes on every event. An element cannot hop channels; a connector can. Inspect before reseating. |
|
||||
| 2026-09-23 | **CONFIRMED: BE12599's fault is the geophone assembly, not the recorder.** Geophone swap on the same unit: Vert overswing **2026.0 -> 3.4**, frequency 2.2 -> 7.4 Hz, all three channels Passed, no self-triggering. First physically established mechanism in this investigation. |
|
||||
| 2026-09-23 | **This explains the autozero's ~10% success rate** (§5): an autozero adjusts the *recorder's* zero reference and cannot fix a geophone. **Procedure change: swap the geophone first.** |
|
||||
| 2026-09-23 | Bench control event (mic-slap trigger, geophone untouched) shows Vert wandering **0.19 in/s with no mechanical input** — the fault is continuous, not episodic. All three channels affected; Tran/Long parked on DC while their coils pass the swing test. |
|
||||
| 2026-09-23 | **Cause: road salt now outranks the mice.** The geophone is buried, not in the enclosure — two independent contamination paths, and the fault is in the buried half. The unit sat ~6 ft from I-80 WB. Salt is a *site* property, so this predicts higher failure rates near salted highways. |
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
"""Pretend to be a Micromate on a serial port: log what arrives, reply to POLL.
|
||||
|
||||
Proves the modem's return path (serial -> TCP) independently of the real unit.
|
||||
"""
|
||||
import os, select, sys, termios, time
|
||||
|
||||
path, baud = sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 115200
|
||||
B = {9600: termios.B9600, 38400: termios.B38400, 115200: termios.B115200}[baud]
|
||||
fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
a = termios.tcgetattr(fd)
|
||||
a[0] = a[1] = a[3] = 0
|
||||
a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL
|
||||
a[4] = a[5] = B
|
||||
a[6] = list(a[6]); a[6][termios.VMIN] = 0; a[6][termios.VTIME] = 0
|
||||
termios.tcsetattr(fd, termios.TCSANOW, a)
|
||||
termios.tcflush(fd, termios.TCIOFLUSH)
|
||||
|
||||
# A real POLL probe reply, captured from UM12947 on 2026-09-24.
|
||||
REPLY = bytes.fromhex("0200c5a4000000000000300000000000000099") + b"\x03"
|
||||
|
||||
print(f"fake unit on {path} @ {baud}; will answer any inbound frame", flush=True)
|
||||
while True:
|
||||
r, _, _ = select.select([fd], [], [], 1.0)
|
||||
if not r:
|
||||
continue
|
||||
data = os.read(fd, 4096)
|
||||
if not data:
|
||||
continue
|
||||
ts = time.strftime("%H:%M:%S")
|
||||
print(f"{ts} IN {len(data):3} B {data.hex(' ')}", flush=True)
|
||||
time.sleep(0.02)
|
||||
os.write(fd, REPLY)
|
||||
print(f"{ts} OUT {len(REPLY):3} B {REPLY.hex(' ')} <- canned POLL reply", flush=True)
|
||||
@@ -1,202 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
|
||||
raw capture pair.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
`minimateplus.framing.S3FrameParser` cannot see Micromate traffic. It locates
|
||||
frames by scanning for `DLE STX`, and a Micromate response has **no leading
|
||||
DLE** — it starts at a bare `STX`. It also expects `payload[1] == 0x10`, where
|
||||
the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
|
||||
|
||||
The practical consequence, seen on the 9-24-26 setup-push capture: the
|
||||
Blastware-side requests parse fine (Thor emits Series III request frames), but
|
||||
**every device response is silently dropped or mis-framed** — so a capture that
|
||||
actually contains 12 acked writes looks like 12 unanswered requests.
|
||||
|
||||
Destuffing
|
||||
----------
|
||||
One rule covers both directions: after the leading doubled `BW_CMD`, every
|
||||
`10 XX` pair on the wire destuffs to `XX`. That includes `10 03` — Thor
|
||||
escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
|
||||
exactly as Blastware does.
|
||||
|
||||
That rule was chosen by evidence, not assumption: of the four candidates tried
|
||||
against the 9-24-26 capture's four data-carrying write frames, it is the only
|
||||
one under which all four checksums validate. See
|
||||
`docs/micromate_protocol_reference.md` → *The write path*.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python scratch/mm_frame_parse.py <capture-dir>
|
||||
python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
|
||||
python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
|
||||
|
||||
# Request SUB -> short name. Series III names where they carry over; the
|
||||
# Series IV additions are marked.
|
||||
SUBNAME = {
|
||||
0x01: "DEVICE_INFO",
|
||||
0x06: "STORAGE_RANGE",
|
||||
0x08: "EVENT_INDEX",
|
||||
0x0A: "WAVEFORM_HDR",
|
||||
0x0C: "WAVEFORM_REC",
|
||||
0x15: "SERIAL",
|
||||
0x1A: "COMPLIANCE_CFG",
|
||||
0x1C: "MONITOR_STATUS",
|
||||
0x1E: "EVENT_HDR",
|
||||
0x2C: "CALLHOME_CFG",
|
||||
0x2E: "TRIGGER_CFG_READ", # Series IV
|
||||
0x3E: "OPERATOR",
|
||||
0x41: "SETUP_NAME_READ", # Series IV
|
||||
0x5A: "BULK_DOWNLOAD",
|
||||
0x5B: "POLL",
|
||||
0x68: "EVENT_INDEX_WRITE",
|
||||
0x69: "WAVEFORM_WRITE",
|
||||
0x71: "COMPLIANCE_WRITE",
|
||||
0x72: "CONFIRM_A",
|
||||
0x73: "CONFIRM_B",
|
||||
0x74: "CONFIRM_C",
|
||||
0x82: "TRIGGER_WRITE",
|
||||
0x83: "TRIGGER_CONFIRM",
|
||||
0xDA: "SETUP_FILE_DECL", # Series IV — names the target .MMB
|
||||
0xFE: "FULL_CFG",
|
||||
}
|
||||
|
||||
|
||||
def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
|
||||
"""Destuff one frame starting at `start`.
|
||||
|
||||
Returns (payload, checksum, index_of_terminating_ETX). `payload` excludes
|
||||
the trailing checksum byte. A request frame opens `ACK STX 10 10`; a
|
||||
response opens with a bare `STX`.
|
||||
"""
|
||||
i = start + (2 if is_request else 1)
|
||||
out = bytearray()
|
||||
if is_request:
|
||||
# The doubled BW_CMD is the one guaranteed stuffed byte.
|
||||
if blob[i : i + 2] != bytes([DLE, DLE]):
|
||||
raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
|
||||
out.append(DLE)
|
||||
i += 2
|
||||
while i < len(blob):
|
||||
b = blob[i]
|
||||
if b == DLE and i + 1 < len(blob):
|
||||
out.append(blob[i + 1])
|
||||
i += 2
|
||||
continue
|
||||
if b == ETX:
|
||||
break
|
||||
out.append(b)
|
||||
i += 1
|
||||
if len(out) < 2:
|
||||
raise ValueError(f"@0x{start:04x}: frame too short")
|
||||
return bytes(out[:-1]), out[-1], i
|
||||
|
||||
|
||||
def frames(blob: bytes, *, is_request: bool):
|
||||
"""Yield (offset, payload, chk, checksum_kind)."""
|
||||
i, n = 0, len(blob)
|
||||
while i < n:
|
||||
if is_request:
|
||||
if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
|
||||
i += 1
|
||||
continue
|
||||
elif blob[i] != STX:
|
||||
i += 1
|
||||
continue
|
||||
try:
|
||||
payload, chk, end = destuff(blob, i, is_request=is_request)
|
||||
except ValueError:
|
||||
i += 1
|
||||
continue
|
||||
sum8 = sum(payload) & 0xFF
|
||||
dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
|
||||
if sum8 == chk:
|
||||
kind = "SUM8"
|
||||
elif dle_aware == chk:
|
||||
kind = "DLE-aware"
|
||||
else:
|
||||
kind = "BAD"
|
||||
yield i, payload, chk, kind
|
||||
i = end + 1
|
||||
|
||||
|
||||
def describe(payload: bytes, is_request: bool) -> str:
|
||||
if len(payload) < 3:
|
||||
return "??"
|
||||
sub = payload[2]
|
||||
if is_request:
|
||||
return SUBNAME.get(sub, f"SUB_{sub:02X}")
|
||||
req = 0xFF - sub
|
||||
return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
|
||||
|
||||
|
||||
def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
|
||||
blob = path.read_bytes()
|
||||
side = "Thor" if is_request else "unit"
|
||||
print(f"== {side:4} {path.name} ({len(blob)} bytes)")
|
||||
n_bad = 0
|
||||
for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
|
||||
if kind == "BAD":
|
||||
n_bad += 1
|
||||
sub = p[2] if len(p) > 2 else -1
|
||||
flags = p[1] if len(p) > 1 else -1
|
||||
# Requests carry offset at payload[4:6]; responses page at [3:5].
|
||||
word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
|
||||
data = len(p) - 16 if is_request else max(len(p) - 5, 0)
|
||||
print(
|
||||
f" [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
|
||||
f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
|
||||
f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
|
||||
)
|
||||
if dump_sub is not None and sub == dump_sub:
|
||||
body = p[16:] if is_request else p[5:]
|
||||
print(f" ---- data ({len(body)} bytes) ----")
|
||||
for o in range(0, len(body), 16):
|
||||
chunk = body[o : o + 16]
|
||||
txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
|
||||
print(f" {o:06x} {chunk.hex(' '):<47} |{txt}|")
|
||||
print(f" -- {idx + 1} frames, {n_bad} bad checksum\n")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("paths", nargs="+",
|
||||
help="a capture directory, or raw_bw.bin and raw_s3.bin")
|
||||
ap.add_argument("--dump", default=None,
|
||||
help="hex-dump the data section of this SUB (e.g. 0x71)")
|
||||
args = ap.parse_args()
|
||||
|
||||
dump_sub = int(args.dump, 0) if args.dump else None
|
||||
|
||||
if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
|
||||
d = Path(args.paths[0])
|
||||
bw = sorted(d.glob("raw_bw_*.bin"))
|
||||
s3 = sorted(d.glob("raw_s3_*.bin"))
|
||||
if not bw or not s3:
|
||||
print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
|
||||
return 2
|
||||
pairs = [(bw[0], True), (s3[0], False)]
|
||||
elif len(args.paths) == 2:
|
||||
pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
|
||||
else:
|
||||
ap.error("pass a capture directory, or exactly two .bin files")
|
||||
|
||||
for path, is_request in pairs:
|
||||
report(path, is_request=is_request, dump_sub=dump_sub)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,119 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
socat_log_split.py — recover a capture pair from a `socat -x` relay log.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
The bench relay that puts Thor in front of a USB-attached Micromate is:
|
||||
|
||||
socat -d -d -x TCP-LISTEN:12345,reuseaddr,fork /dev/ttyACM0,raw,echo=0,b115200 \
|
||||
> ~/mm-captures/socat_<ts>.log 2>&1
|
||||
|
||||
`-x` makes socat hex-dump every byte it forwards, in both directions, with
|
||||
timestamps. That log is therefore a **complete second copy of every capture**
|
||||
taken through the relay — independent of whether seismo_lab was recording.
|
||||
|
||||
On 2026-09-25 that mattered: a capture's `.bin` files never made it off the
|
||||
Windows machine, and the session was rebuilt from this log instead. When the
|
||||
real bins turned up later, the reconstruction was **byte-for-byte identical in
|
||||
both directions** (3,595 and 4,004 bytes). So this is a validated fallback, not
|
||||
a lossy approximation.
|
||||
|
||||
Log format
|
||||
----------
|
||||
```
|
||||
> 2026/09/25 00:30:35.000276659 length=21 from=0 to=20
|
||||
41 02 10 10 00 5b 00 00 30 00 ...
|
||||
2026/09/25 00:30:35 socat[32190] N write(5, 0x..., 21) completed
|
||||
< 2026/09/25 00:30:35.000384100 length=64 from=0 to=63
|
||||
02 00 c5 a4 00 00 30 00 ...
|
||||
```
|
||||
|
||||
`>` is data heading toward the serial device (Thor → unit). `<` is data coming
|
||||
back (unit → Thor). Hex lines are space-separated and indented; socat's own
|
||||
status lines start with a date and carry no payload.
|
||||
|
||||
Usage
|
||||
-----
|
||||
# whole log
|
||||
python scratch/socat_log_split.py socat_20260924_181248.log --out-dir ./recovered
|
||||
|
||||
# one session — line numbers from the "accepting connection" markers
|
||||
grep -n "accepting connection" socat_*.log
|
||||
python scratch/socat_log_split.py socat_*.log --from-line 919 --out-dir ./recovered
|
||||
|
||||
Then parse the result as usual:
|
||||
|
||||
python scratch/mm_frame_parse.py recovered/raw_bw.bin recovered/raw_s3.bin
|
||||
|
||||
⚠ A log spanning several sessions concatenates them. Split by line number using
|
||||
the `accepting connection` markers, or the frame walk will run sessions together.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
_HEX = re.compile(r"\A[0-9a-f]{2}\Z")
|
||||
_SOCAT_STATUS = re.compile(r"\A\d{4}/\d{2}/\d{2}")
|
||||
|
||||
|
||||
def split(lines) -> tuple[bytes, bytes]:
|
||||
"""Return (to_device, from_device) byte streams."""
|
||||
to_dev, from_dev = bytearray(), bytearray()
|
||||
cur = None
|
||||
for line in lines:
|
||||
if line.startswith(">"):
|
||||
cur = to_dev
|
||||
continue
|
||||
if line.startswith("<"):
|
||||
cur = from_dev
|
||||
continue
|
||||
if _SOCAT_STATUS.match(line):
|
||||
# socat's own status line ends the current dump block.
|
||||
cur = None
|
||||
continue
|
||||
if cur is None or not line.startswith(" "):
|
||||
continue
|
||||
toks = line.split()
|
||||
if toks and all(_HEX.match(t) for t in toks):
|
||||
cur.extend(int(t, 16) for t in toks)
|
||||
return bytes(to_dev), bytes(from_dev)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(
|
||||
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
|
||||
)
|
||||
ap.add_argument("log", help="a socat -x log file")
|
||||
ap.add_argument("--out-dir", default=".", help="where to write the .bin pair")
|
||||
ap.add_argument("--from-line", type=int, default=1,
|
||||
help="first log line to read (1-based) — use the "
|
||||
"'accepting connection' marker of the session you want")
|
||||
ap.add_argument("--to-line", type=int, default=None,
|
||||
help="last log line to read (1-based, inclusive)")
|
||||
ap.add_argument("--prefix", default="raw", help="output basename prefix")
|
||||
args = ap.parse_args()
|
||||
|
||||
lines = Path(args.log).read_text(errors="replace").splitlines()
|
||||
lo = max(args.from_line - 1, 0)
|
||||
hi = args.to_line if args.to_line is not None else len(lines)
|
||||
to_dev, from_dev = split(lines[lo:hi])
|
||||
|
||||
out = Path(args.out_dir)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
bw = out / f"{args.prefix}_bw.bin"
|
||||
s3 = out / f"{args.prefix}_s3.bin"
|
||||
bw.write_bytes(to_dev)
|
||||
s3.write_bytes(from_dev)
|
||||
print(f"Thor -> unit {len(to_dev):>7} bytes {bw}")
|
||||
print(f"unit -> Thor {len(from_dev):>7} bytes {s3}")
|
||||
if not to_dev or not from_dev:
|
||||
print("⚠ one direction is empty — check --from-line / --to-line")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user