Compare commits
10
Commits
dev
..
d2258dad4e
| Author | SHA1 | Date | |
|---|---|---|---|
 serversdown
|
d2258dad4e | ||
|
serversdown
|
e050e602c2 | ||
|
serversdown
|
fc1c7c7936 | ||
|
serversdown
|
7d0d12079b | ||
|
serversdown
|
5203aab849 | ||
|
serversdown
|
5b65718b72 | ||
|
serversdown
|
483762607e | ||
|
serversdown
|
d0b66368d5 | ||
|
serversdown
|
2eb1d25028 | ||
 claude
|
cc821f9ee3 |
-127
@@ -4,133 +4,6 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
---
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Waveform event times were the monitoring-session start, not the trigger
|
||||
(~hours off).** `read_blastware_file` stamped events with footer `ts1`, which
|
||||
for a waveform is the session start a unit shares across every event that day
|
||||
(a unit arming at 06:00 stamped 06:00 on all of them — the modal and PDF both
|
||||
showed it, since it's the stored value). The event time is footer `ts2` (the
|
||||
recording stop), and Blastware's trigger = `ts2 - record time`. The record
|
||||
time is a big-endian float32 in the recording-setup config block (30 bytes
|
||||
before the `Standard Recording Setup` marker), so the **exact trigger is now
|
||||
recovered from the binary alone** — all 7 BE12844 oracle events decode to
|
||||
their exact Blastware time (e.g. N844LQHB 10:33:29), no paired `.TXT` needed.
|
||||
Histograms keep `ts1` (the ~24 h window start). A paired report's
|
||||
`event_datetime` stays authoritative (unit-clock drift).
|
||||
⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
|
||||
|
||||
### Added
|
||||
|
||||
- **Diagnostics tab in the SFM standalone webapp.** Surfaces the device
|
||||
endpoints that previously existed only as `curl`: `events/storage_range` and
|
||||
`events/index` alongside `monitor/status`, then stop monitoring, disable ACH
|
||||
(`rescue?erase=false`, so stored events survive), and erase. The wedged-unit
|
||||
ladder — slow drip and blind stop — sits under its own heading pointing at
|
||||
`docs/runbooks/wedged_unit_recovery.md`, with the reminder that `slow_drip`'s
|
||||
success signal is `bytes_received > 0` and not a clean duration. Erase is
|
||||
guarded by typing the unit's serial: auth answers *who*, not *did you mean
|
||||
it*, and Swagger's try-it-out button on `/device/events/erase` is live on
|
||||
`:8200/docs`.
|
||||
|
||||
- **`docs/sfm_tool_status.md`** — an honest per-capability maturity assessment:
|
||||
what is production-grade (the codec library, the data side), what is
|
||||
emergency-grade (the device side), what is a research artifact, the
|
||||
known-issues table, and the gap to a real tool. Also records the **5A
|
||||
page-boundary bug** as known: `parse_strt_end_offset()` discards the key's
|
||||
page byte, so once a unit has recorded more than 64 KB since its last erase,
|
||||
an event spanning the boundary reads an `end_offset` *behind* its own start —
|
||||
the chunk loop fetches nothing and TERM packs a negative `offset_word`, which
|
||||
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
|
||||
files via the watcher path and never runs this walk.
|
||||
|
||||
- **The Micromate (Series IV) live wire protocol, reverse-engineered end to
|
||||
end** — `docs/micromate_protocol_reference.md`. Worked out against a bench
|
||||
UM12947 over USB and a recording relay, with THOR driving every write so that
|
||||
no command has ever been originated against a unit by this project. **A
|
||||
Micromate answers Series III command frames**, with three framing differences:
|
||||
responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
|
||||
or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
|
||||
**uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
|
||||
`SUB 0x1A` by 47x. Read path, event chain, and `SUB 0x5A` streaming the
|
||||
`.IDFW` file verbatim are all confirmed.
|
||||
- **Series IV setup management, fully mapped.** `0x41` reads the active setup
|
||||
name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
|
||||
write it back. **Setups are read-modify-write** — the written block is the
|
||||
read block, 91% byte-identical at a fixed 11-byte shift. `0xDA` **creates**
|
||||
files rather than only overwriting, confirmed on the unit's own screen, and an
|
||||
overwrite is protocol-identical to a create: no handshake, no warning, and no
|
||||
protection even over the *active* setup of a monitoring unit.
|
||||
- **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
|
||||
records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
|
||||
half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
|
||||
name. Verified entry-for-entry against the operator's own THOR screen.
|
||||
- **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
|
||||
`0x8D`/`0x8E` write. This **retracts** an earlier conclusion in the same
|
||||
document that no such command existed; that was inferred from absent firmware
|
||||
strings and was wrong.
|
||||
- **Monitoring control and per-event delete.** `0x96`/`0x97` start and stop as
|
||||
on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
|
||||
for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
|
||||
constant. Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
|
||||
which is safer than Series III's erase-everything. `SUB 0x1C` also carries the
|
||||
device clock.
|
||||
- **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
|
||||
identically as "disconnected": refused, connect timeout (the silent-drop
|
||||
signature of a trusted-IP whitelist), **connected but no reply** (the modem
|
||||
answered and the unit did not), and replied. Each verdict names what to try
|
||||
next. `--slots N` tests single-session modem behaviour.
|
||||
- **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
|
||||
decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
|
||||
`onewaydev`) driven by a control file. No pyserial; stdlib `termios` only.
|
||||
- **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
|
||||
— a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
|
||||
all, since it scans for `DLE+STX`), byte-exact capture recovery from a
|
||||
`socat -x` relay log, and a serial-port stand-in that answers as a unit.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Connecting to a unit no longer walks its event chain.** `/device/events`
|
||||
reads every event header over the cellular link; on a unit with a large or
|
||||
wrapped chain that takes minutes or fails outright, and it fired
|
||||
automatically on every connect. Connect now uses only ~2 s probes —
|
||||
`/device/info` (which already carried the compliance config the walk was
|
||||
re-reading) plus `events/storage_range` — and the Device tab gains an Event
|
||||
Chain card. The walk moved behind a **Load events** button in the Events
|
||||
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
|
||||
event it has stored.
|
||||
|
||||
- **Recorded what THOR actually does on the wire**, measured rather than assumed.
|
||||
A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
|
||||
MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
|
||||
`MONITOR_STATUS` check at 60 s. The **status interval is honoured; the
|
||||
connection interval is not** — it sets `(status / connection) - 1` checks per
|
||||
cycle, so equal values yield *zero* cheap checks and every connection becomes
|
||||
the expensive one.
|
||||
- **Two THOR defects reproduced with timestamps.** After a connection drops
|
||||
mid-download it retries **once**, stops polling entirely and **never resumes**,
|
||||
while displaying `Connected` for as long as it is left alone — and `Idle` for a
|
||||
unit that is actively recording. Separately, THOR's own log shows a
|
||||
**subscription leak**: one logical event dispatched to a growing number of
|
||||
handlers, **1 to 12 over ten hours** of uptime, consistent with the field
|
||||
report that only a restart recovers it.
|
||||
- **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
|
||||
either firmware line. A PL2303 cable (Benfei) leaves a unit with no working
|
||||
modem port; an FTDI cable (Sabrent) works. Both are in circulation and
|
||||
indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
|
||||
|
||||
### Migration
|
||||
|
||||
**None.** Frontend, documentation and bench tooling only — no codec,
|
||||
waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump. The
|
||||
webapp is served from the image, so its changes appear after the next `sfm`
|
||||
rebuild. The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only;
|
||||
nothing under `sfm/`, `minimateplus/` or `micromate/` was touched.
|
||||
|
||||
---
|
||||
|
||||
## v0.31.0 — 2026-09-18
|
||||
|
||||
**Report parity, and a second way to rescue a runaway unit.** Two threads.
|
||||
|
||||
@@ -61,24 +61,6 @@ Read this first when picking the project back up.
|
||||
4th-decimal tick and are Thor's own rounding — no single linear LSB can
|
||||
reproduce every printed value (the constraints are infeasible by 7e-5
|
||||
relative), so do NOT retune `_GEO_LSB_IPS`.
|
||||
- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
|
||||
`parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
|
||||
discarding the key's page byte. An event starting at `0x0111F2A2` and ending
|
||||
at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
|
||||
start. The chunk loop then exits before fetching anything and TERM computes
|
||||
a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
|
||||
`/device/events` walk 500s. Reproduced on BE12599 (2026-09-19), which had
|
||||
78 KB stored and had rolled into page `0x12`.
|
||||
**Why it hid so long:** every 5A capture the walk was verified against came
|
||||
from a freshly-erased BE11529 — all three confirmed TERM examples in
|
||||
`framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`. Prod is
|
||||
unaffected: it ingests complete files via BW ACH, never this walk.
|
||||
**Fixing it has two layers** — the arithmetic (`if end < start: end +=
|
||||
0x10000`) stops the crash and bounds the loop correctly; carrying the page
|
||||
byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
|
||||
over) needs a BW capture of a spanning event first. Do not ship layer one
|
||||
alone without a loud truncation warning — a silently short event is the
|
||||
failure mode this codec has been bitten by repeatedly.
|
||||
- **Open, not blocking:** 14 sensitive-range files show an exact 8x
|
||||
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
|
||||
inserts DB rows for store files that have none (one-time per store) and the
|
||||
|
||||
@@ -496,11 +496,6 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
|
||||
|
||||
## Roadmap (Future)
|
||||
|
||||
> **Where it stands *today*** — an honest per-capability maturity assessment,
|
||||
> what to rely on, known issues, and the gap to a real tool:
|
||||
> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md). This section covers
|
||||
> where it is *going*.
|
||||
|
||||
### Strategic direction — where this is going
|
||||
|
||||
seismo-relay is being built as a **suite of cooperating components**
|
||||
|
||||
@@ -1,338 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_link.py — a "perfect modem" between THOR and a Micromate, with a readable
|
||||
log and deliberate fault injection.
|
||||
|
||||
Why
|
||||
---
|
||||
THOR gives almost no visibility into a connection: a refresh button, two poll
|
||||
intervals, and no way to see whether a check succeeded, timed out, or was never
|
||||
sent. When a unit "won't stay connected" there is nothing to look at.
|
||||
|
||||
This sits where the cellular modem would sit and answers the question directly:
|
||||
|
||||
* **What is THOR actually doing?** Every frame is decoded and timestamped —
|
||||
`POLL`, `MONITOR_STATUS`, `SETUP_NAME_READ` — not a hex dump.
|
||||
* **Is it even trying?** Silence is visible: the log shows gaps.
|
||||
* **How does it behave when the link misbehaves?** Faults can be injected on
|
||||
demand, which a real cell link will not do on cue.
|
||||
|
||||
Point THOR at this host and port exactly as if it were a modem (Communication:
|
||||
TCP, IP: <this host>, Port: <--listen>).
|
||||
|
||||
Fault injection
|
||||
---------------
|
||||
Write a mode into the control file (default `mm_link.ctl`) and it takes effect
|
||||
on the next byte:
|
||||
|
||||
echo pass > mm_link.ctl # normal relay
|
||||
echo blackhole > mm_link.ctl # TCP stays up, bytes are swallowed
|
||||
echo drop > mm_link.ctl # close the connection abruptly (RST-ish)
|
||||
echo delay:2.0 > mm_link.ctl # forward, but 2 s late in both directions
|
||||
echo onewaydev > mm_link.ctl # THOR->unit passes, unit->THOR is swallowed
|
||||
|
||||
**`blackhole` is the one that matters.** It reproduces the classic cellular
|
||||
failure: the socket is still open as far as both ends are concerned, but nothing
|
||||
crosses. A client that relies on TCP to tell it the peer is gone will sit there
|
||||
until the OS keepalive fires — which by default is about two hours.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python3 bridges/mm_link.py --serial /dev/ttyACM0 --baud 115200 \\
|
||||
--listen 12345 --logdir ~/mm-captures
|
||||
|
||||
Writes, per session:
|
||||
<logdir>/mmlink_<ts>/session.log decoded, timestamped, human-readable
|
||||
<logdir>/mmlink_<ts>/raw_bw.bin THOR -> unit, raw
|
||||
<logdir>/mmlink_<ts>/raw_s3.bin unit -> THOR, raw
|
||||
|
||||
The raw pair loads straight into `scratch/mm_frame_parse.py`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "scratch"))
|
||||
try:
|
||||
from mm_frame_parse import SUBNAME, destuff # noqa: F401
|
||||
except Exception: # pragma: no cover
|
||||
SUBNAME = {}
|
||||
|
||||
import errno
|
||||
import select
|
||||
import termios
|
||||
|
||||
DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
|
||||
|
||||
_BAUD = {9600: termios.B9600, 19200: termios.B19200, 38400: termios.B38400,
|
||||
57600: termios.B57600, 115200: termios.B115200}
|
||||
|
||||
|
||||
class SerialPort:
|
||||
"""Minimal raw serial port on stdlib termios — no pyserial dependency.
|
||||
|
||||
The bench hosts are whatever is to hand; requiring a pip install on someone
|
||||
else's machine is a poor trade for the ~30 lines this saves.
|
||||
"""
|
||||
|
||||
def __init__(self, path: str, baud: int):
|
||||
if baud not in _BAUD:
|
||||
raise ValueError(f"unsupported baud {baud}; pick one of {sorted(_BAUD)}")
|
||||
self.fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
a = termios.tcgetattr(self.fd)
|
||||
a[0] = 0 # iflag: no translation
|
||||
a[1] = 0 # oflag: raw
|
||||
a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL # cflag: 8N1, ignore modem lines
|
||||
a[3] = 0 # lflag: non-canonical, no echo
|
||||
a[4] = a[5] = _BAUD[baud]
|
||||
a[6] = list(a[6])
|
||||
a[6][termios.VMIN] = 0
|
||||
a[6][termios.VTIME] = 0
|
||||
termios.tcsetattr(self.fd, termios.TCSANOW, a)
|
||||
termios.tcflush(self.fd, termios.TCIOFLUSH)
|
||||
|
||||
def read(self, n: int) -> bytes:
|
||||
r, _, _ = select.select([self.fd], [], [], 0.2)
|
||||
if not r:
|
||||
return b""
|
||||
try:
|
||||
return os.read(self.fd, n)
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
return b""
|
||||
raise
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
while data:
|
||||
try:
|
||||
data = data[os.write(self.fd, data):]
|
||||
except OSError as e:
|
||||
if e.errno in (errno.EAGAIN, errno.EWOULDBLOCK):
|
||||
select.select([], [self.fd], [], 0.2)
|
||||
continue
|
||||
raise
|
||||
|
||||
def close(self) -> None:
|
||||
try:
|
||||
os.close(self.fd)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def name_of(sub: int, is_request: bool) -> str:
|
||||
if is_request:
|
||||
return SUBNAME.get(sub, f"SUB_{sub:02X}")
|
||||
return "rsp " + SUBNAME.get(0xFF - sub, f"SUB_{0xFF - sub:02X}")
|
||||
|
||||
|
||||
class FrameSniffer:
|
||||
"""Accumulate bytes and report complete frames, without altering the stream."""
|
||||
|
||||
def __init__(self, is_request: bool):
|
||||
self.is_request = is_request
|
||||
self.buf = bytearray()
|
||||
|
||||
def feed(self, data: bytes):
|
||||
"""Yield (sub, payload_len) for each complete frame seen."""
|
||||
self.buf.extend(data)
|
||||
while True:
|
||||
start = -1
|
||||
for i, b in enumerate(self.buf):
|
||||
if self.is_request and b == ACK and i + 1 < len(self.buf) and self.buf[i + 1] == STX:
|
||||
start = i
|
||||
break
|
||||
if not self.is_request and b == STX:
|
||||
start = i
|
||||
break
|
||||
if start < 0:
|
||||
if len(self.buf) > 8192:
|
||||
del self.buf[:-16]
|
||||
return
|
||||
j = start + (2 if self.is_request else 1)
|
||||
end = -1
|
||||
while j < len(self.buf):
|
||||
if self.buf[j] == DLE and j + 1 < len(self.buf):
|
||||
j += 2
|
||||
continue
|
||||
if self.buf[j] == ETX:
|
||||
end = j
|
||||
break
|
||||
j += 1
|
||||
if end < 0:
|
||||
return # wait for more bytes
|
||||
body = self.buf[start:end + 1]
|
||||
del self.buf[:end + 1]
|
||||
# SUB sits at a fixed spot past the leading framing -- but it is
|
||||
# DLE-escaped when its own value is 0x02/0x03/0x04/0x10, so a raw
|
||||
# read reports 0x10 for those. SUB 0x02 was being logged as
|
||||
# "SUB_10" until this was handled.
|
||||
off = 5 if self.is_request else 3
|
||||
if len(body) > off:
|
||||
sub = body[off]
|
||||
if sub == DLE and len(body) > off + 1:
|
||||
sub = body[off + 1]
|
||||
yield sub, len(body)
|
||||
|
||||
|
||||
class Link:
|
||||
def __init__(self, args):
|
||||
self.args = args
|
||||
self.mode = "pass"
|
||||
self.delay = 0.0
|
||||
self.ctl = Path(args.control)
|
||||
self.session: Path | None = None
|
||||
self.log_fh = None
|
||||
self.raw = {}
|
||||
self.t0 = time.time()
|
||||
self.counts = {}
|
||||
|
||||
# ── logging ────────────────────────────────────────────────────────────
|
||||
def open_session(self):
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
self.session = Path(self.args.logdir) / f"mmlink_{ts}"
|
||||
self.session.mkdir(parents=True, exist_ok=True)
|
||||
self.log_fh = open(self.session / "session.log", "a", buffering=1)
|
||||
self.raw = {
|
||||
"bw": open(self.session / "raw_bw.bin", "ab"),
|
||||
"s3": open(self.session / "raw_s3.bin", "ab"),
|
||||
}
|
||||
self.say(f"=== session {ts} — serial {self.args.serial} @ {self.args.baud} ===")
|
||||
|
||||
def say(self, text: str):
|
||||
line = f"{datetime.datetime.now().strftime('%H:%M:%S.%f')[:-3]} {text}"
|
||||
print(line, flush=True)
|
||||
if self.log_fh:
|
||||
self.log_fh.write(line + "\n")
|
||||
|
||||
# ── control file ───────────────────────────────────────────────────────
|
||||
def poll_control(self):
|
||||
while True:
|
||||
try:
|
||||
if self.ctl.exists():
|
||||
want = self.ctl.read_text().strip().lower()
|
||||
if want.startswith("delay:"):
|
||||
d = float(want.split(":", 1)[1])
|
||||
if ("delay", d) != (self.mode, self.delay):
|
||||
self.mode, self.delay = "delay", d
|
||||
self.say(f"*** MODE -> delay {d}s ***")
|
||||
elif want and want != self.mode:
|
||||
self.mode, self.delay = want, 0.0
|
||||
self.say(f"*** MODE -> {want} ***")
|
||||
except Exception:
|
||||
pass
|
||||
time.sleep(0.25)
|
||||
|
||||
# ── the relay ──────────────────────────────────────────────────────────
|
||||
def pump(self, src, dst, tag: str, is_request: bool, stop: threading.Event):
|
||||
sniff = FrameSniffer(is_request)
|
||||
arrow = "THOR->unit" if is_request else "unit->THOR"
|
||||
last = time.time()
|
||||
while not stop.is_set():
|
||||
timed_out = False
|
||||
try:
|
||||
data = src.recv(4096) if isinstance(src, socket.socket) else src.read(4096)
|
||||
except TimeoutError:
|
||||
timed_out = True
|
||||
# socket.timeout subclasses OSError, so it MUST be caught first.
|
||||
# Treating it as a dead socket closes the connection after 200 ms
|
||||
# of quiet -- which is exactly what `blackhole` produces, so the
|
||||
# relay killed the link it was supposed to be faking a fault on.
|
||||
data = b""
|
||||
except OSError:
|
||||
break
|
||||
if isinstance(src, socket.socket) and data == b"" and not timed_out:
|
||||
self.say(f"{arrow}: peer closed the connection")
|
||||
break
|
||||
if not data:
|
||||
if time.time() - last > self.args.quiet_after and self.counts:
|
||||
self.say(f"--- {self.args.quiet_after:.0f}s with no traffic ---")
|
||||
last = time.time()
|
||||
continue
|
||||
last = time.time()
|
||||
|
||||
self.raw[tag].write(data)
|
||||
self.raw[tag].flush()
|
||||
for sub, ln in sniff.feed(data):
|
||||
label = name_of(sub, is_request)
|
||||
self.counts[label] = self.counts.get(label, 0) + 1
|
||||
self.say(f"{arrow} {label:<20} ({ln} B)"
|
||||
+ ("" if self.mode == "pass" else f" [mode={self.mode}]"))
|
||||
|
||||
mode = self.mode
|
||||
if mode == "drop":
|
||||
self.say(f"{arrow}: DROPPING the connection (fault injection)")
|
||||
stop.set()
|
||||
break
|
||||
if mode == "blackhole":
|
||||
continue # swallow, keep the socket open
|
||||
if mode == "onewaydev" and not is_request:
|
||||
continue # unit's replies never reach THOR
|
||||
if mode == "delay" and self.delay:
|
||||
time.sleep(self.delay)
|
||||
try:
|
||||
if isinstance(dst, socket.socket):
|
||||
dst.sendall(data)
|
||||
else:
|
||||
dst.write(data)
|
||||
except OSError:
|
||||
break
|
||||
stop.set()
|
||||
|
||||
def serve(self):
|
||||
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
srv.bind(("0.0.0.0", self.args.listen))
|
||||
srv.listen(5)
|
||||
self.open_session()
|
||||
self.say(f"listening on 0.0.0.0:{self.args.listen} control file: {self.ctl}")
|
||||
self.say("point THOR at this host/port as Communication=TCP")
|
||||
threading.Thread(target=self.poll_control, daemon=True).start()
|
||||
|
||||
while True:
|
||||
conn, addr = srv.accept()
|
||||
conn.settimeout(0.2)
|
||||
self.say(f"+++ THOR connected from {addr[0]}:{addr[1]} +++")
|
||||
try:
|
||||
ser = SerialPort(self.args.serial, self.args.baud)
|
||||
except OSError as e:
|
||||
self.say(f"!!! cannot open {self.args.serial}: {e}")
|
||||
conn.close()
|
||||
continue
|
||||
stop = threading.Event()
|
||||
ts = [
|
||||
threading.Thread(target=self.pump, args=(conn, ser, "bw", True, stop), daemon=True),
|
||||
threading.Thread(target=self.pump, args=(ser, conn, "s3", False, stop), daemon=True),
|
||||
]
|
||||
for t in ts:
|
||||
t.start()
|
||||
for t in ts:
|
||||
t.join()
|
||||
conn.close()
|
||||
ser.close()
|
||||
summary = ", ".join(f"{k}x{v}" for k, v in sorted(self.counts.items()))
|
||||
self.say(f"--- connection closed. frames this session: {summary or 'none'} ---")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--serial", default="/dev/ttyACM0")
|
||||
ap.add_argument("--baud", type=int, default=115200)
|
||||
ap.add_argument("--listen", type=int, default=12345)
|
||||
ap.add_argument("--logdir", default=os.path.expanduser("~/mm-captures"))
|
||||
ap.add_argument("--control", default="mm_link.ctl")
|
||||
ap.add_argument("--quiet-after", type=float, default=30.0,
|
||||
help="log a marker after this many seconds of silence")
|
||||
Link(ap.parse_args()).serve()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,226 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_probe.py — answer "why can't we reach this unit?" in one command.
|
||||
|
||||
THOR reports a failed connection as "disconnected" and nothing else. That single
|
||||
word covers at least four completely different faults with four different fixes,
|
||||
and telling them apart is the difference between a modem reboot and a site visit:
|
||||
|
||||
* **connection refused** something answered and said no — wrong port, or the
|
||||
modem is refusing a further session
|
||||
* **connect timed out** nothing answered at all — trusted-IP whitelist,
|
||||
firewall, or the modem is off the network
|
||||
* **connected, no reply** the MODEM answered but the unit did not. The TCP
|
||||
path is fine; the modem is not forwarding to serial.
|
||||
This is the signature of a wedged transparent-TCP
|
||||
session, and it is the one THOR cannot distinguish
|
||||
from any of the others
|
||||
* **replied** the unit is alive; the problem is upstream software
|
||||
|
||||
Read-only. It sends `POLL`, then optionally `SERIAL` and the state read — the
|
||||
same three commands THOR's own connection check uses — and never writes.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python3 bridges/mm_probe.py 63.45.161.30:9034
|
||||
python3 bridges/mm_probe.py 10.0.0.8:12345 --timeout 5
|
||||
python3 bridges/mm_probe.py <host:port> --slots 3
|
||||
|
||||
`--slots N` opens N connections at once and reports how many the far end accepts.
|
||||
A transparent-TCP modem typically serves **one** session; if the first succeeds
|
||||
and the rest are refused or hang, that confirms the single-slot behaviour and
|
||||
explains why a leaked session takes a unit offline until the slot frees.
|
||||
|
||||
Works for both series: a Series III reply opens `DLE STX`, a Micromate reply
|
||||
opens with a bare `STX`, so the probe also tells you which one answered.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||
from minimateplus.framing import build_bw_frame # noqa: E402
|
||||
|
||||
DLE, STX, ETX = 0x10, 0x02, 0x03
|
||||
|
||||
|
||||
def destuff(raw: bytes) -> bytes:
|
||||
"""Strip framing and DLE escapes; return the payload without its checksum."""
|
||||
i = 1 if raw and raw[0] == STX else (2 if len(raw) > 1 and raw[1] == STX else 0)
|
||||
out = bytearray()
|
||||
while i < len(raw):
|
||||
b = raw[i]
|
||||
if b == DLE and i + 1 < len(raw):
|
||||
out.append(raw[i + 1])
|
||||
i += 2
|
||||
continue
|
||||
if b == ETX:
|
||||
break
|
||||
out.append(b)
|
||||
i += 1
|
||||
return bytes(out[:-1]) if len(out) > 1 else b""
|
||||
|
||||
|
||||
# Reads are two-step on Series III: a probe at offset 0, then a data read at the
|
||||
# block's length. THOR sends these offsets, and they also work on a Micromate.
|
||||
OFFSETS = {0x5B: 0x0030, 0x15: 0x000A, 0x49: 0xFFFF}
|
||||
|
||||
|
||||
def exchange(sock: socket.socket, sub: int, timeout: float) -> tuple[bytes, float]:
|
||||
sock.sendall(build_bw_frame(sub, OFFSETS.get(sub, 0)))
|
||||
t0 = time.time()
|
||||
buf, deadline = b"", t0 + timeout
|
||||
sock.settimeout(0.3)
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
chunk = sock.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
if buf.endswith(bytes([ETX])) and len(buf) > 8:
|
||||
break
|
||||
except TimeoutError:
|
||||
continue
|
||||
except OSError:
|
||||
break
|
||||
return buf, time.time() - t0
|
||||
|
||||
|
||||
def step(n: int, label: str, result: str) -> None:
|
||||
print(f" [{n}] {label:.<28} {result}")
|
||||
|
||||
|
||||
def probe(host: str, port: int, timeout: float) -> int:
|
||||
print(f"\ntarget {host}:{port} (read-only: POLL, SERIAL, state)\n")
|
||||
|
||||
# ── 1. TCP ────────────────────────────────────────────────────────────
|
||||
t0 = time.time()
|
||||
try:
|
||||
sock = socket.create_connection((host, port), timeout=timeout)
|
||||
except ConnectionRefusedError:
|
||||
step(1, "TCP connect", f"REFUSED after {1000*(time.time()-t0):.0f} ms")
|
||||
print("\nverdict: something answered and actively refused.")
|
||||
print(" Not a silent firewall drop — the host is reachable.")
|
||||
print(" Wrong port, the service is down, or the modem is refusing")
|
||||
print(" an additional session because its one slot is in use.")
|
||||
return 2
|
||||
except (TimeoutError, socket.timeout):
|
||||
step(1, "TCP connect", f"TIMED OUT after {time.time()-t0:.1f} s")
|
||||
print("\nverdict: nothing answered at all.")
|
||||
print(" A silent drop, which is what a trusted-IP whitelist looks")
|
||||
print(" like — it discards rather than refuses. Check the modem's")
|
||||
print(" Trusted IPs (and note a VPN changes the IP you arrive from),")
|
||||
print(" the firewall, and whether the modem is on the network.")
|
||||
return 3
|
||||
except OSError as e:
|
||||
step(1, "TCP connect", f"FAILED: {e}")
|
||||
return 4
|
||||
step(1, "TCP connect", f"ok ({1000*(time.time()-t0):.0f} ms)")
|
||||
|
||||
# ── 2. POLL ───────────────────────────────────────────────────────────
|
||||
try:
|
||||
raw, dt = exchange(sock, 0x5B, timeout)
|
||||
except OSError as e:
|
||||
step(2, "POLL", f"send failed: {e}")
|
||||
sock.close()
|
||||
return 4
|
||||
|
||||
if not raw:
|
||||
step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
|
||||
print("\nverdict: the MODEM answered but the unit did not.")
|
||||
print(" TCP is fine end to end — something accepted the connection.")
|
||||
print(" What is missing is the serial side. Most likely the modem is")
|
||||
print(" not forwarding to its serial port, which is what a wedged")
|
||||
print(" transparent-TCP session looks like: the slot is held by a")
|
||||
print(" connection that never closed.")
|
||||
print("\n Try, in order:")
|
||||
print(" 1. ACEmanager -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot forever. 2 minutes is the value")
|
||||
print(" this project standardised on.")
|
||||
print(" 2. Reboot the modem. If that fixes it, the modem was")
|
||||
print(" holding state and the timeout is the permanent fix.")
|
||||
print(" 3. Check the unit's own screen — serial cable, power.")
|
||||
sock.close()
|
||||
return 5
|
||||
|
||||
series = "Series III (DLE STX)" if raw[0] == DLE else "Micromate (bare STX)"
|
||||
step(2, "POLL", f"reply {len(raw)} B in {1000*dt:.0f} ms")
|
||||
p = destuff(raw)
|
||||
ok = len(p) > 3 and p[2] == 0xFF - 0x5B
|
||||
step(3, "frame", f"{'valid' if ok else 'MALFORMED'}, {series}")
|
||||
if not ok:
|
||||
print("\nverdict: something replied, but not a seismograph.")
|
||||
print(" Another service is on this port, or the modem is in a mode")
|
||||
print(" that injects its own text (check Quiet Mode / AT echo).")
|
||||
print(f" first bytes: {raw[:16].hex(' ')}")
|
||||
sock.close()
|
||||
return 6
|
||||
|
||||
# ── 3. identity + state ───────────────────────────────────────────────
|
||||
for n, (sub, label) in enumerate(((0x15, "serial"), (0x49, "state")), start=4):
|
||||
try:
|
||||
r, dt = exchange(sock, sub, timeout)
|
||||
d = destuff(r)[5:]
|
||||
if sub == 0x15:
|
||||
# serial is a null-terminated run; a further field follows it
|
||||
serial = bytes(d[11:]).split(b"\x00")[0]
|
||||
step(n, label, serial.decode("ascii", "replace") or "(empty)")
|
||||
else:
|
||||
step(n, label, "MONITORING" if len(d) > 11 and d[11] else "idle")
|
||||
except OSError:
|
||||
step(n, label, "no reply")
|
||||
|
||||
sock.close()
|
||||
print("\nverdict: the unit is alive and answering.")
|
||||
print(" If THOR still shows it disconnected, the fault is in THOR, not")
|
||||
print(" the network or the device.")
|
||||
return 0
|
||||
|
||||
|
||||
def slots(host: str, port: int, n: int, timeout: float) -> None:
|
||||
print(f"\nopening {n} simultaneous connections to {host}:{port}\n")
|
||||
held = []
|
||||
for i in range(n):
|
||||
try:
|
||||
s = socket.create_connection((host, port), timeout=timeout)
|
||||
held.append(s)
|
||||
step(i + 1, f"connection {i+1}", "accepted")
|
||||
except ConnectionRefusedError:
|
||||
step(i + 1, f"connection {i+1}", "REFUSED")
|
||||
except (TimeoutError, socket.timeout):
|
||||
step(i + 1, f"connection {i+1}", "timed out")
|
||||
except OSError as e:
|
||||
step(i + 1, f"connection {i+1}", f"failed: {e}")
|
||||
print(f"\n{len(held)} of {n} accepted.")
|
||||
if len(held) == 1:
|
||||
print(" Single-slot behaviour confirmed — this far end serves ONE")
|
||||
print(" session at a time. A connection that is never closed takes")
|
||||
print(" the unit offline until the idle timeout frees the slot.")
|
||||
for s in held:
|
||||
s.close()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("target", help="host:port, e.g. 63.45.161.30:9034")
|
||||
ap.add_argument("--timeout", type=float, default=10.0)
|
||||
ap.add_argument("--slots", type=int, metavar="N",
|
||||
help="open N simultaneous connections to test single-slot behaviour")
|
||||
a = ap.parse_args()
|
||||
host, _, port = a.target.rpartition(":")
|
||||
if not host:
|
||||
ap.error("target must be host:port")
|
||||
if a.slots:
|
||||
slots(host, int(port), a.slots, a.timeout)
|
||||
return 0
|
||||
return probe(host, int(port), a.timeout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,150 +0,0 @@
|
||||
# SFM — where it actually stands as a tool
|
||||
|
||||
**Status as of 2026-09-20 (v0.31.0).** This is the honest assessment, not the
|
||||
roadmap — `README.md § Roadmap` covers where it is *going*. Expect this file to
|
||||
go stale; re-date it when you revise it.
|
||||
|
||||
---
|
||||
|
||||
## The framing
|
||||
|
||||
SFM is **three different things wearing one name**, at three very different
|
||||
levels of maturity:
|
||||
|
||||
| | what it is | maturity |
|
||||
|---|---|---|
|
||||
| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
|
||||
| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
|
||||
| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
|
||||
| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
|
||||
|
||||
Brian's own description — *"right now it's an emergency tool and a research
|
||||
project"* — is accurate, and it applies specifically to the **device side**.
|
||||
The data side is not an emergency tool; it has been carrying production for
|
||||
months.
|
||||
|
||||
Most confusion about "is SFM reliable?" comes from answering for the wrong
|
||||
tier.
|
||||
|
||||
---
|
||||
|
||||
## 1. What you can rely on
|
||||
|
||||
### Production-grade — trust it
|
||||
|
||||
- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
|
||||
preserved Blastware ASCII exports, 45 units, files back to 2018.
|
||||
- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
|
||||
Thor's own CSV exports; production IDFW 575/575 with zero truncations.
|
||||
- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
|
||||
842,442 per-interval frequency comparisons with zero mismatches.
|
||||
- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
|
||||
fed by the watchers — this is how prod actually gets its data, and it has
|
||||
been running unattended for months.
|
||||
- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
|
||||
listing, event detail and report.
|
||||
- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
|
||||
binaries, with operator review state preserved across regeneration.
|
||||
- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
|
||||
Proven in the field, including as a rescue tool (see the runbook).
|
||||
|
||||
### Emergency-grade — works, but you are the error handling
|
||||
|
||||
- **`/device/*` live endpoints.** They do what they say. But they are
|
||||
synchronous, unauthenticated, and a single cellular download can exceed the
|
||||
60 s timeouts that sit in front of them.
|
||||
- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
|
||||
Each has worked in a real incident — but each has been used a handful of
|
||||
times, by one person, with the runbook open.
|
||||
- **The standalone webapp.** Perfectly usable, and as of v0.31.0 the cheap
|
||||
probes and rescue actions are reachable without curl. No auth of any kind.
|
||||
|
||||
### Research artifacts — useful, not products
|
||||
|
||||
- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
|
||||
Inspector). Desktop-only, single-user, no tests.
|
||||
- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
|
||||
`verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
|
||||
These produced the numbers the production claims rest on, so they matter —
|
||||
but they are analysis scripts, not maintained code.
|
||||
- **`docs/offset_investigation.md`** — an open investigation, not a feature.
|
||||
|
||||
---
|
||||
|
||||
## 2. What to use when
|
||||
|
||||
| you want to… | use | notes |
|
||||
|---|---|---|
|
||||
| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
|
||||
| Know whether ACH is on | `GET /device/call_home` | ~2 s. **Not** `/device/events`. |
|
||||
| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
|
||||
| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
|
||||
| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
|
||||
| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
|
||||
| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
|
||||
|
||||
The single most useful habit: **the cheap probes are cheap and the event walk
|
||||
is not.** Reaching for `/device/events` to answer a yes/no question about a
|
||||
unit is the mistake that motivated the v0.31.0 webapp changes.
|
||||
|
||||
---
|
||||
|
||||
## 3. Known issues
|
||||
|
||||
| issue | impact | status |
|
||||
|---|---|---|
|
||||
| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`. Needs a BW capture of a spanning event to fix properly. |
|
||||
| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
|
||||
| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial. `/docs` itself is unguarded. |
|
||||
| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug. Surfaced in the UI as "unreliable". |
|
||||
| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint. Must be POST-starts-job / GET-polls before any remote lab. |
|
||||
| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known. Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
|
||||
| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
|
||||
| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
|
||||
|
||||
---
|
||||
|
||||
## 4. What stands between this and a real tool
|
||||
|
||||
Roughly in dependency order — each unblocks the ones below it.
|
||||
|
||||
**1. Authentication.** Everything else is gated on this. SFM has none, and
|
||||
the modem IP whitelist gives zero protection because SFM *is* the whitelisted
|
||||
origin. The agreed design delegates rather than builds: Terra-View becomes the
|
||||
authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
|
||||
auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
|
||||
|
||||
**2. Async long operations.** POST starts a job, GET polls. Retrofitting this
|
||||
after building a remote lab on top of synchronous endpoints would be far worse
|
||||
than designing for it now.
|
||||
|
||||
**3. Confirm-guards on the remaining destructive endpoints.** Auth answers
|
||||
*who*, not *did you mean it*. The webapp's erase is guarded; the other seven
|
||||
destructive POSTs and `/docs` are not.
|
||||
|
||||
**4. The 5A page-boundary fix.** Until this lands, live event download is
|
||||
unreliable on exactly the units most likely to need attention — the ones that
|
||||
have been recording heavily. Wants a Blastware capture of an event spanning a
|
||||
page boundary before the chunk-addressing half is trustworthy.
|
||||
|
||||
**5. A live Thor / Micromate client.** The device side is MiniMate-only.
|
||||
Series-4 units can only be read from forwarded files, so half the fleet has no
|
||||
live path at all.
|
||||
|
||||
**6. Test coverage that runs from a clean checkout.** 15 of 16 current
|
||||
failures are missing fixture bundles. A test suite that cannot go green on a
|
||||
fresh clone cannot gate anything.
|
||||
|
||||
**7. The SDM rename.** Cosmetic relative to the above, but the longer `sfm/`
|
||||
holds the data-side code the more the tiers blur. ~30–50 files here, ~10–15 in
|
||||
Terra-View, plus a Docker volume migration. Do it when the codebase is quiet.
|
||||
|
||||
---
|
||||
|
||||
## The short version
|
||||
|
||||
The **data side is a real tool already**. The **device side is a set of sharp
|
||||
instruments** that work in the hands of the person who wrote them, with the
|
||||
runbook open. The gap between those two states is mostly **auth, async, and
|
||||
guardrails** — not protocol work. The protocol is the part that is actually
|
||||
finished.
|
||||
@@ -296,16 +296,6 @@ def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
|
||||
event.sample_rate = report.sample_rate_sps
|
||||
if report.record_time_s is not None:
|
||||
event.rectime_seconds = report.record_time_s
|
||||
# The report's event_datetime is Blastware's exact trigger time (parsed
|
||||
# from Event Time + Event Date). Prefer it over the binary footer's stop
|
||||
# time so a report-paired import matches BW to the second.
|
||||
edt = report.event_datetime
|
||||
if edt is not None:
|
||||
event.timestamp = Timestamp(
|
||||
raw=b"", flag=0x10,
|
||||
year=edt.year, unknown_byte=0, month=edt.month, day=edt.day,
|
||||
hour=edt.hour, minute=edt.minute, second=edt.second,
|
||||
)
|
||||
|
||||
|
||||
def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
|
||||
@@ -818,30 +808,6 @@ def derive_record_type_from_filename(filename, default: str = "Waveform") -> str
|
||||
return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
|
||||
|
||||
|
||||
# Marker for the recording-setup config block, and the offset of the record-time
|
||||
# float32 within it. The configured post-trigger record time (seconds) is a
|
||||
# big-endian float32 exactly 30 bytes before the "Standard Recording Setup"
|
||||
# label. Verified across the corpus reading 1.0 / 2.0 / 3.0 s on different
|
||||
# setups — and ts2 - record_time reproduces Blastware's trigger to the second
|
||||
# (N844LQHB: stop 10:33:32 - 3.0 = 10:33:29).
|
||||
_RECSETUP_MARKER = b"Standard Recording Setup"
|
||||
_RECTIME_OFFSET_BEFORE_MARKER = 30
|
||||
|
||||
|
||||
def _parse_record_time_seconds(raw: bytes) -> Optional[float]:
|
||||
"""The configured post-trigger record time in seconds, from the recording-
|
||||
setup config block, or None when absent / implausible."""
|
||||
a = raw.find(_RECSETUP_MARKER)
|
||||
if a < _RECTIME_OFFSET_BEFORE_MARKER:
|
||||
return None
|
||||
off = a - _RECTIME_OFFSET_BEFORE_MARKER
|
||||
try:
|
||||
rt = struct.unpack(">f", raw[off:off + 4])[0]
|
||||
except struct.error:
|
||||
return None
|
||||
return rt if 0.05 <= rt <= 600.0 else None
|
||||
|
||||
|
||||
def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
"""
|
||||
Parse a Blastware waveform file into an Event.
|
||||
@@ -951,10 +917,6 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
# rest of the event (timestamp, waveform_key, project strings) is
|
||||
# still recoverable and useful.
|
||||
decoded = decode_waveform_v2(body)
|
||||
# Discriminator for the timestamp logic below: a waveform (trigger) event
|
||||
# vs a histogram window. Keyed on the codec, not the filename — the
|
||||
# save_imported_bw path passes a tmp ".bw" name whose extension lies.
|
||||
is_waveform_body = decoded is not None
|
||||
if decoded is None:
|
||||
decoded = decode_histogram_body(body)
|
||||
if decoded is None:
|
||||
@@ -986,31 +948,7 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
ev.total_samples = strt_fields.get("total_samples")
|
||||
ev.pretrig_samples = strt_fields.get("pretrig_samples")
|
||||
|
||||
# Event timestamp. The footer's two timestamps mean different things by
|
||||
# record type:
|
||||
# * Waveform: ts1 = the monitoring-SESSION start (shared across every
|
||||
# event that day — a unit arming at 06:00 stamps 06:00 on all of them),
|
||||
# ts2 = THIS event's recording STOP. Blastware's Date/Time is the
|
||||
# TRIGGER = ts2 - record time, and the record time is a float32 in the
|
||||
# recording-setup config block (see _parse_record_time_seconds), so the
|
||||
# exact trigger is recoverable from the binary alone. Falls back to ts2
|
||||
# (the stop, within the record duration) if the config block is absent.
|
||||
# (Stamping ts1 showed the session start, hours off.)
|
||||
# * Histogram / undecodable: ts1 = the window start, which IS the event
|
||||
# time — keep it.
|
||||
# Discriminate by ``is_waveform_body`` (the codec), not the filename.
|
||||
if is_waveform_body and ts2 is not None:
|
||||
_stop = datetime.datetime(ts2.year, ts2.month, ts2.day,
|
||||
ts2.hour, ts2.minute, ts2.second)
|
||||
_rt = _parse_record_time_seconds(raw)
|
||||
_trig = _stop - datetime.timedelta(seconds=_rt) if _rt is not None else _stop
|
||||
ev.timestamp = Timestamp(
|
||||
raw=footer[10:18],
|
||||
flag=0x10,
|
||||
year=_trig.year, unknown_byte=0, month=_trig.month, day=_trig.day,
|
||||
hour=_trig.hour, minute=_trig.minute, second=_trig.second,
|
||||
)
|
||||
elif ts1 is not None:
|
||||
if ts1 is not None:
|
||||
ev.timestamp = Timestamp(
|
||||
raw=footer[2:10],
|
||||
flag=0x10,
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
"""Pretend to be a Micromate on a serial port: log what arrives, reply to POLL.
|
||||
|
||||
Proves the modem's return path (serial -> TCP) independently of the real unit.
|
||||
"""
|
||||
import os, select, sys, termios, time
|
||||
|
||||
path, baud = sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 115200
|
||||
B = {9600: termios.B9600, 38400: termios.B38400, 115200: termios.B115200}[baud]
|
||||
fd = os.open(path, os.O_RDWR | os.O_NOCTTY | os.O_NONBLOCK)
|
||||
a = termios.tcgetattr(fd)
|
||||
a[0] = a[1] = a[3] = 0
|
||||
a[2] = termios.CS8 | termios.CREAD | termios.CLOCAL
|
||||
a[4] = a[5] = B
|
||||
a[6] = list(a[6]); a[6][termios.VMIN] = 0; a[6][termios.VTIME] = 0
|
||||
termios.tcsetattr(fd, termios.TCSANOW, a)
|
||||
termios.tcflush(fd, termios.TCIOFLUSH)
|
||||
|
||||
# A real POLL probe reply, captured from UM12947 on 2026-09-24.
|
||||
REPLY = bytes.fromhex("0200c5a4000000000000300000000000000099") + b"\x03"
|
||||
|
||||
print(f"fake unit on {path} @ {baud}; will answer any inbound frame", flush=True)
|
||||
while True:
|
||||
r, _, _ = select.select([fd], [], [], 1.0)
|
||||
if not r:
|
||||
continue
|
||||
data = os.read(fd, 4096)
|
||||
if not data:
|
||||
continue
|
||||
ts = time.strftime("%H:%M:%S")
|
||||
print(f"{ts} IN {len(data):3} B {data.hex(' ')}", flush=True)
|
||||
time.sleep(0.02)
|
||||
os.write(fd, REPLY)
|
||||
print(f"{ts} OUT {len(REPLY):3} B {REPLY.hex(' ')} <- canned POLL reply", flush=True)
|
||||
@@ -1,202 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
|
||||
raw capture pair.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
`minimateplus.framing.S3FrameParser` cannot see Micromate traffic. It locates
|
||||
frames by scanning for `DLE STX`, and a Micromate response has **no leading
|
||||
DLE** — it starts at a bare `STX`. It also expects `payload[1] == 0x10`, where
|
||||
the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
|
||||
|
||||
The practical consequence, seen on the 9-24-26 setup-push capture: the
|
||||
Blastware-side requests parse fine (Thor emits Series III request frames), but
|
||||
**every device response is silently dropped or mis-framed** — so a capture that
|
||||
actually contains 12 acked writes looks like 12 unanswered requests.
|
||||
|
||||
Destuffing
|
||||
----------
|
||||
One rule covers both directions: after the leading doubled `BW_CMD`, every
|
||||
`10 XX` pair on the wire destuffs to `XX`. That includes `10 03` — Thor
|
||||
escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
|
||||
exactly as Blastware does.
|
||||
|
||||
That rule was chosen by evidence, not assumption: of the four candidates tried
|
||||
against the 9-24-26 capture's four data-carrying write frames, it is the only
|
||||
one under which all four checksums validate. See
|
||||
`docs/micromate_protocol_reference.md` → *The write path*.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python scratch/mm_frame_parse.py <capture-dir>
|
||||
python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
|
||||
python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
|
||||
|
||||
# Request SUB -> short name. Series III names where they carry over; the
|
||||
# Series IV additions are marked.
|
||||
SUBNAME = {
|
||||
0x01: "DEVICE_INFO",
|
||||
0x06: "STORAGE_RANGE",
|
||||
0x08: "EVENT_INDEX",
|
||||
0x0A: "WAVEFORM_HDR",
|
||||
0x0C: "WAVEFORM_REC",
|
||||
0x15: "SERIAL",
|
||||
0x1A: "COMPLIANCE_CFG",
|
||||
0x1C: "MONITOR_STATUS",
|
||||
0x1E: "EVENT_HDR",
|
||||
0x2C: "CALLHOME_CFG",
|
||||
0x2E: "TRIGGER_CFG_READ", # Series IV
|
||||
0x3E: "OPERATOR",
|
||||
0x41: "SETUP_NAME_READ", # Series IV
|
||||
0x5A: "BULK_DOWNLOAD",
|
||||
0x5B: "POLL",
|
||||
0x68: "EVENT_INDEX_WRITE",
|
||||
0x69: "WAVEFORM_WRITE",
|
||||
0x71: "COMPLIANCE_WRITE",
|
||||
0x72: "CONFIRM_A",
|
||||
0x73: "CONFIRM_B",
|
||||
0x74: "CONFIRM_C",
|
||||
0x82: "TRIGGER_WRITE",
|
||||
0x83: "TRIGGER_CONFIRM",
|
||||
0xDA: "SETUP_FILE_DECL", # Series IV — names the target .MMB
|
||||
0xFE: "FULL_CFG",
|
||||
}
|
||||
|
||||
|
||||
def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
|
||||
"""Destuff one frame starting at `start`.
|
||||
|
||||
Returns (payload, checksum, index_of_terminating_ETX). `payload` excludes
|
||||
the trailing checksum byte. A request frame opens `ACK STX 10 10`; a
|
||||
response opens with a bare `STX`.
|
||||
"""
|
||||
i = start + (2 if is_request else 1)
|
||||
out = bytearray()
|
||||
if is_request:
|
||||
# The doubled BW_CMD is the one guaranteed stuffed byte.
|
||||
if blob[i : i + 2] != bytes([DLE, DLE]):
|
||||
raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
|
||||
out.append(DLE)
|
||||
i += 2
|
||||
while i < len(blob):
|
||||
b = blob[i]
|
||||
if b == DLE and i + 1 < len(blob):
|
||||
out.append(blob[i + 1])
|
||||
i += 2
|
||||
continue
|
||||
if b == ETX:
|
||||
break
|
||||
out.append(b)
|
||||
i += 1
|
||||
if len(out) < 2:
|
||||
raise ValueError(f"@0x{start:04x}: frame too short")
|
||||
return bytes(out[:-1]), out[-1], i
|
||||
|
||||
|
||||
def frames(blob: bytes, *, is_request: bool):
|
||||
"""Yield (offset, payload, chk, checksum_kind)."""
|
||||
i, n = 0, len(blob)
|
||||
while i < n:
|
||||
if is_request:
|
||||
if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
|
||||
i += 1
|
||||
continue
|
||||
elif blob[i] != STX:
|
||||
i += 1
|
||||
continue
|
||||
try:
|
||||
payload, chk, end = destuff(blob, i, is_request=is_request)
|
||||
except ValueError:
|
||||
i += 1
|
||||
continue
|
||||
sum8 = sum(payload) & 0xFF
|
||||
dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
|
||||
if sum8 == chk:
|
||||
kind = "SUM8"
|
||||
elif dle_aware == chk:
|
||||
kind = "DLE-aware"
|
||||
else:
|
||||
kind = "BAD"
|
||||
yield i, payload, chk, kind
|
||||
i = end + 1
|
||||
|
||||
|
||||
def describe(payload: bytes, is_request: bool) -> str:
|
||||
if len(payload) < 3:
|
||||
return "??"
|
||||
sub = payload[2]
|
||||
if is_request:
|
||||
return SUBNAME.get(sub, f"SUB_{sub:02X}")
|
||||
req = 0xFF - sub
|
||||
return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
|
||||
|
||||
|
||||
def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
|
||||
blob = path.read_bytes()
|
||||
side = "Thor" if is_request else "unit"
|
||||
print(f"== {side:4} {path.name} ({len(blob)} bytes)")
|
||||
n_bad = 0
|
||||
for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
|
||||
if kind == "BAD":
|
||||
n_bad += 1
|
||||
sub = p[2] if len(p) > 2 else -1
|
||||
flags = p[1] if len(p) > 1 else -1
|
||||
# Requests carry offset at payload[4:6]; responses page at [3:5].
|
||||
word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
|
||||
data = len(p) - 16 if is_request else max(len(p) - 5, 0)
|
||||
print(
|
||||
f" [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
|
||||
f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
|
||||
f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
|
||||
)
|
||||
if dump_sub is not None and sub == dump_sub:
|
||||
body = p[16:] if is_request else p[5:]
|
||||
print(f" ---- data ({len(body)} bytes) ----")
|
||||
for o in range(0, len(body), 16):
|
||||
chunk = body[o : o + 16]
|
||||
txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
|
||||
print(f" {o:06x} {chunk.hex(' '):<47} |{txt}|")
|
||||
print(f" -- {idx + 1} frames, {n_bad} bad checksum\n")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("paths", nargs="+",
|
||||
help="a capture directory, or raw_bw.bin and raw_s3.bin")
|
||||
ap.add_argument("--dump", default=None,
|
||||
help="hex-dump the data section of this SUB (e.g. 0x71)")
|
||||
args = ap.parse_args()
|
||||
|
||||
dump_sub = int(args.dump, 0) if args.dump else None
|
||||
|
||||
if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
|
||||
d = Path(args.paths[0])
|
||||
bw = sorted(d.glob("raw_bw_*.bin"))
|
||||
s3 = sorted(d.glob("raw_s3_*.bin"))
|
||||
if not bw or not s3:
|
||||
print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
|
||||
return 2
|
||||
pairs = [(bw[0], True), (s3[0], False)]
|
||||
elif len(args.paths) == 2:
|
||||
pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
|
||||
else:
|
||||
ap.error("pass a capture directory, or exactly two .bin files")
|
||||
|
||||
for path, is_request in pairs:
|
||||
report(path, is_request=is_request, dump_sub=dump_sub)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,119 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
socat_log_split.py — recover a capture pair from a `socat -x` relay log.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
The bench relay that puts Thor in front of a USB-attached Micromate is:
|
||||
|
||||
socat -d -d -x TCP-LISTEN:12345,reuseaddr,fork /dev/ttyACM0,raw,echo=0,b115200 \
|
||||
> ~/mm-captures/socat_<ts>.log 2>&1
|
||||
|
||||
`-x` makes socat hex-dump every byte it forwards, in both directions, with
|
||||
timestamps. That log is therefore a **complete second copy of every capture**
|
||||
taken through the relay — independent of whether seismo_lab was recording.
|
||||
|
||||
On 2026-09-25 that mattered: a capture's `.bin` files never made it off the
|
||||
Windows machine, and the session was rebuilt from this log instead. When the
|
||||
real bins turned up later, the reconstruction was **byte-for-byte identical in
|
||||
both directions** (3,595 and 4,004 bytes). So this is a validated fallback, not
|
||||
a lossy approximation.
|
||||
|
||||
Log format
|
||||
----------
|
||||
```
|
||||
> 2026/09/25 00:30:35.000276659 length=21 from=0 to=20
|
||||
41 02 10 10 00 5b 00 00 30 00 ...
|
||||
2026/09/25 00:30:35 socat[32190] N write(5, 0x..., 21) completed
|
||||
< 2026/09/25 00:30:35.000384100 length=64 from=0 to=63
|
||||
02 00 c5 a4 00 00 30 00 ...
|
||||
```
|
||||
|
||||
`>` is data heading toward the serial device (Thor → unit). `<` is data coming
|
||||
back (unit → Thor). Hex lines are space-separated and indented; socat's own
|
||||
status lines start with a date and carry no payload.
|
||||
|
||||
Usage
|
||||
-----
|
||||
# whole log
|
||||
python scratch/socat_log_split.py socat_20260924_181248.log --out-dir ./recovered
|
||||
|
||||
# one session — line numbers from the "accepting connection" markers
|
||||
grep -n "accepting connection" socat_*.log
|
||||
python scratch/socat_log_split.py socat_*.log --from-line 919 --out-dir ./recovered
|
||||
|
||||
Then parse the result as usual:
|
||||
|
||||
python scratch/mm_frame_parse.py recovered/raw_bw.bin recovered/raw_s3.bin
|
||||
|
||||
⚠ A log spanning several sessions concatenates them. Split by line number using
|
||||
the `accepting connection` markers, or the frame walk will run sessions together.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
_HEX = re.compile(r"\A[0-9a-f]{2}\Z")
|
||||
_SOCAT_STATUS = re.compile(r"\A\d{4}/\d{2}/\d{2}")
|
||||
|
||||
|
||||
def split(lines) -> tuple[bytes, bytes]:
|
||||
"""Return (to_device, from_device) byte streams."""
|
||||
to_dev, from_dev = bytearray(), bytearray()
|
||||
cur = None
|
||||
for line in lines:
|
||||
if line.startswith(">"):
|
||||
cur = to_dev
|
||||
continue
|
||||
if line.startswith("<"):
|
||||
cur = from_dev
|
||||
continue
|
||||
if _SOCAT_STATUS.match(line):
|
||||
# socat's own status line ends the current dump block.
|
||||
cur = None
|
||||
continue
|
||||
if cur is None or not line.startswith(" "):
|
||||
continue
|
||||
toks = line.split()
|
||||
if toks and all(_HEX.match(t) for t in toks):
|
||||
cur.extend(int(t, 16) for t in toks)
|
||||
return bytes(to_dev), bytes(from_dev)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(
|
||||
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
|
||||
)
|
||||
ap.add_argument("log", help="a socat -x log file")
|
||||
ap.add_argument("--out-dir", default=".", help="where to write the .bin pair")
|
||||
ap.add_argument("--from-line", type=int, default=1,
|
||||
help="first log line to read (1-based) — use the "
|
||||
"'accepting connection' marker of the session you want")
|
||||
ap.add_argument("--to-line", type=int, default=None,
|
||||
help="last log line to read (1-based, inclusive)")
|
||||
ap.add_argument("--prefix", default="raw", help="output basename prefix")
|
||||
args = ap.parse_args()
|
||||
|
||||
lines = Path(args.log).read_text(errors="replace").splitlines()
|
||||
lo = max(args.from_line - 1, 0)
|
||||
hi = args.to_line if args.to_line is not None else len(lines)
|
||||
to_dev, from_dev = split(lines[lo:hi])
|
||||
|
||||
out = Path(args.out_dir)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
bw = out / f"{args.prefix}_bw.bin"
|
||||
s3 = out / f"{args.prefix}_s3.bin"
|
||||
bw.write_bytes(to_dev)
|
||||
s3.write_bytes(from_dev)
|
||||
print(f"Thor -> unit {len(to_dev):>7} bytes {bw}")
|
||||
print(f"unit -> Thor {len(from_dev):>7} bytes {s3}")
|
||||
if not to_dev or not from_dev:
|
||||
print("⚠ one direction is empty — check --from-line / --to-line")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
+20
-295
@@ -108,12 +108,6 @@
|
||||
color: var(--text);
|
||||
}
|
||||
.btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
|
||||
.btn-danger { background: var(--red); color: #fff; }
|
||||
.btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
|
||||
.diag-result { display:block; margin-top:6px; font-size:12px; opacity:.85;
|
||||
white-space:pre-wrap; word-break:break-word; }
|
||||
.diag-result.ok { color: var(--green); }
|
||||
.diag-result.error { color: var(--red); }
|
||||
.btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
|
||||
|
||||
/* #connect-btn styles moved to #live-connect-bar block */
|
||||
@@ -916,7 +910,6 @@
|
||||
<button class="tab-btn" data-tab="events" onclick="switchTab('events')">Events</button>
|
||||
<button class="tab-btn" data-tab="config" onclick="switchTab('config')">Config</button>
|
||||
<button class="tab-btn" data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
|
||||
<button class="tab-btn" data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
@@ -945,10 +938,6 @@
|
||||
<div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
|
||||
|
||||
<div class="event-toolbar">
|
||||
<button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
|
||||
title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
|
||||
⟳ Load events
|
||||
</button>
|
||||
<button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
|
||||
<button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
|
||||
title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
|
||||
@@ -1216,77 +1205,6 @@
|
||||
|
||||
</div><!-- end #tab-call-home -->
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: Diagnostics
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-diagnostics" class="tab-pane">
|
||||
|
||||
<div class="cfg-grid">
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Device State</div>
|
||||
<div class="hint" style="margin-bottom:10px">
|
||||
Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
|
||||
</div>
|
||||
<div class="dev-table" id="diag-table"></div>
|
||||
<div class="cfg-actions" style="margin-top:12px">
|
||||
<button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
|
||||
<span id="diag-status"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Actions</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Stop Monitoring</label>
|
||||
<button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
|
||||
<div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
|
||||
<span class="diag-result" id="diag-stop-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Disable Auto Call Home</label>
|
||||
<button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
|
||||
<div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
|
||||
<span class="diag-result" id="diag-ach-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Erase All Events</label>
|
||||
<input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
|
||||
oninput="diagCheckEraseConfirm()" autocomplete="off" />
|
||||
<button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
|
||||
<div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
|
||||
<span class="diag-result" id="diag-erase-result"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Unresponsive Unit</div>
|
||||
<div class="hint" style="margin-bottom:10px">
|
||||
The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
|
||||
to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
|
||||
<code>ach_server</code> and answer its call — before racing it with these.
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
|
||||
<button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
|
||||
<div class="hint">Success is <code>bytes_received > 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
|
||||
<span class="diag-result" id="diag-drip-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
|
||||
<button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
|
||||
<span class="diag-result" id="diag-blind-result"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div><!-- end #tab-diagnostics -->
|
||||
|
||||
</div><!-- end #section-live -->
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
@@ -1443,8 +1361,6 @@
|
||||
// ── State ──────────────────────────────────────────────────────────────────────
|
||||
let unitInfo = null;
|
||||
let eventList = [];
|
||||
let storageInfo = null; // /device/events/storage_range — cheap, read on connect
|
||||
let eventsLoaded = false; // the event chain walk is opt-in; see loadEventList()
|
||||
let currentEvent = 0;
|
||||
let charts = {};
|
||||
let geoAdcScale = 6.206;
|
||||
@@ -1542,7 +1458,6 @@ function switchTab(name) {
|
||||
if (name === 'units') { if (!unitsLoaded) loadUnits(); }
|
||||
if (name === 'monlog') { if (!monlogLoaded) loadMonitorLog(); }
|
||||
if (name === 'sessions') { if (!sessLoaded) loadSessions(); }
|
||||
if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
|
||||
}
|
||||
|
||||
// ── Connect ────────────────────────────────────────────────────────────────────
|
||||
@@ -1563,13 +1478,18 @@ async function connectUnit() {
|
||||
btn.disabled = false; btn.textContent = 'Connect'; return;
|
||||
}
|
||||
|
||||
// Connecting deliberately does NOT walk the event chain. That walk reads
|
||||
// every event header over the cellular link and can take minutes — or fail
|
||||
// outright on a unit whose buffer has wrapped past 0xFFFF. Use the ~2 s
|
||||
// probes instead; the event list is opt-in via loadEventList().
|
||||
eventList = []; eventsLoaded = false;
|
||||
setStatus('Reading device state…', 'loading');
|
||||
storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
|
||||
setStatus('Fetching event list…', 'loading');
|
||||
try {
|
||||
const r = await fetch(`${api()}/device/events?${deviceParams()}`);
|
||||
if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
|
||||
const evData = await r.json();
|
||||
eventList = evData.events || [];
|
||||
// Merge compliance from /device/events response (it re-reads it)
|
||||
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
|
||||
} catch (e) {
|
||||
setStatus(`Event fetch failed: ${e.message}`, 'error');
|
||||
btn.disabled = false; btn.textContent = 'Reconnect'; return;
|
||||
}
|
||||
|
||||
populateDeviceBar();
|
||||
populateDeviceTab();
|
||||
@@ -1578,9 +1498,11 @@ async function connectUnit() {
|
||||
|
||||
document.getElementById('device-bar').style.display = 'flex';
|
||||
document.getElementById('monitor-panel').style.display = 'flex';
|
||||
setEventButtonsEnabled();
|
||||
document.getElementById('load-events-btn').disabled = false;
|
||||
setDiagButtonsEnabled(true);
|
||||
document.getElementById('load-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('save-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('download-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('prev-btn').disabled = true;
|
||||
document.getElementById('next-btn').disabled = eventList.length <= 1;
|
||||
document.getElementById('cfg-read-btn').disabled = false;
|
||||
document.getElementById('cfg-write-btn').disabled = false;
|
||||
document.getElementById('ch-read-btn').disabled = false;
|
||||
@@ -1588,9 +1510,7 @@ async function connectUnit() {
|
||||
|
||||
btn.disabled = false; btn.textContent = 'Reconnect';
|
||||
|
||||
setStatus(storageInfo && storageInfo.is_empty
|
||||
? 'Connected — no events stored.'
|
||||
: 'Connected. Event list not loaded (Events → Load events).', 'ok');
|
||||
setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
|
||||
|
||||
// Fetch monitor status in background (non-blocking)
|
||||
refreshMonitorStatus().catch(() => {});
|
||||
@@ -1602,48 +1522,6 @@ async function connectUnit() {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Shared fetch helper ────────────────────────────────────────────────────────
|
||||
async function fetchJson(path, opts) {
|
||||
const sep = path.includes('?') ? '&' : '?';
|
||||
const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
|
||||
const body = await r.json().catch(() => ({}));
|
||||
if (!r.ok) throw new Error(body.detail || r.statusText);
|
||||
return body;
|
||||
}
|
||||
|
||||
function setEventButtonsEnabled() {
|
||||
const n = eventList.length;
|
||||
document.getElementById('load-btn').disabled = n === 0;
|
||||
document.getElementById('save-btn').disabled = n === 0;
|
||||
document.getElementById('download-btn').disabled = n === 0;
|
||||
document.getElementById('prev-btn').disabled = true;
|
||||
document.getElementById('next-btn').disabled = n <= 1;
|
||||
}
|
||||
|
||||
// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
|
||||
async function loadEventList() {
|
||||
if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
|
||||
const btn = document.getElementById('load-events-btn');
|
||||
btn.disabled = true;
|
||||
setStatus('Walking the event chain — this can take a while…', 'loading');
|
||||
try {
|
||||
const evData = await fetchJson('/device/events');
|
||||
eventList = evData.events || [];
|
||||
eventsLoaded = true;
|
||||
// /device/events re-reads compliance; fold it in.
|
||||
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
|
||||
} catch (e) {
|
||||
setStatus(`Event fetch failed: ${e.message}`, 'error');
|
||||
btn.disabled = false; return;
|
||||
}
|
||||
populateDeviceBar();
|
||||
populateDeviceTab();
|
||||
populateEventChips();
|
||||
setEventButtonsEnabled();
|
||||
btn.disabled = false;
|
||||
setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
|
||||
}
|
||||
|
||||
// ── Device bar ─────────────────────────────────────────────────────────────────
|
||||
function populateDeviceBar() {
|
||||
qs('di-serial').textContent = unitInfo.serial || '—';
|
||||
@@ -1652,7 +1530,7 @@ function populateDeviceBar() {
|
||||
qs('di-sr').textContent = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
|
||||
qs('di-rt').textContent = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
|
||||
qs('di-trig').textContent = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
|
||||
qs('di-count').textContent = eventsLoaded ? eventList.length : '—';
|
||||
qs('di-count').textContent = eventList.length;
|
||||
qs('di-project').textContent = cc.project || '—';
|
||||
qs('di-client').textContent = cc.client || '—';
|
||||
qs('di-operator').textContent = cc.operator || '—';
|
||||
@@ -1782,8 +1660,7 @@ function populateDeviceTab() {
|
||||
{ label:'DSP', value: unitInfo.dsp_version || '—' },
|
||||
{ label:'Model', value: unitInfo.model || '—' },
|
||||
{ label:'Manufacturer', value: unitInfo.manufacturer || '—' },
|
||||
{ label:'Stored Events', value: eventsLoaded ? eventList.length : 'not loaded' },
|
||||
{ label:'Storage Used', value: storageUsedLabel() },
|
||||
{ label:'Stored Events', value: eventList.length },
|
||||
];
|
||||
for (const {label, value} of cardData) {
|
||||
const c = document.createElement('div');
|
||||
@@ -1830,158 +1707,6 @@ function renderTable(id, rows) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Diagnostics ────────────────────────────────────────────────────────────────
|
||||
// Everything here is a cheap probe (POLL + one read) or a single write. None of
|
||||
// it walks the event chain. See docs/runbooks/wedged_unit_recovery.md.
|
||||
|
||||
function storageUsedLabel() {
|
||||
if (!storageInfo) return '—';
|
||||
if (storageInfo.is_empty) return 'empty';
|
||||
const f = storageInfo.first_key, l = storageInfo.last_key;
|
||||
return (f && l) ? `${f} → ${l}` : '—';
|
||||
}
|
||||
|
||||
function setDiagButtonsEnabled(on) {
|
||||
for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
|
||||
'diag-drip-btn','diag-blind-btn']) {
|
||||
const el = document.getElementById(id);
|
||||
if (el) el.disabled = !on;
|
||||
}
|
||||
diagCheckEraseConfirm();
|
||||
}
|
||||
|
||||
// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
|
||||
function diagCheckEraseConfirm() {
|
||||
const box = document.getElementById('diag-erase-confirm');
|
||||
const btn = document.getElementById('diag-erase-btn');
|
||||
if (!box || !btn) return;
|
||||
const serial = (unitInfo && unitInfo.serial) || '';
|
||||
btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
|
||||
}
|
||||
|
||||
function diagResult(id, text, cls) {
|
||||
const el = document.getElementById(id);
|
||||
if (!el) return;
|
||||
el.textContent = text;
|
||||
el.className = 'diag-result' + (cls ? ' ' + cls : '');
|
||||
}
|
||||
|
||||
async function refreshDiagnostics() {
|
||||
if (!devHost()) return;
|
||||
const st = document.getElementById('diag-status');
|
||||
if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
|
||||
|
||||
const [mon, store, idx] = await Promise.all([
|
||||
fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
|
||||
fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
|
||||
fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
|
||||
]);
|
||||
|
||||
if (!store._err) storageInfo = store;
|
||||
|
||||
const err = v => `<span style="color:var(--red)">${v}</span>`;
|
||||
const rows = [];
|
||||
|
||||
rows.push(['Monitoring', mon._err ? err(mon._err)
|
||||
: (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
|
||||
if (!mon._err) {
|
||||
rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
|
||||
if (mon.memory_total_bytes) {
|
||||
const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
|
||||
const pct = (used / mon.memory_total_bytes * 100).toFixed(1);
|
||||
rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
|
||||
}
|
||||
}
|
||||
|
||||
rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
|
||||
if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
|
||||
|
||||
// SUB 0x08. Known to report 0 on units with years of history — suspected
|
||||
// field-offset bug in the decode, so show it but do not trust it.
|
||||
rows.push(['Lifetime events', idx._err ? err(idx._err)
|
||||
: `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
|
||||
|
||||
renderTable('diag-table', rows);
|
||||
populateDeviceTab();
|
||||
if (st) { st.textContent = ''; st.className = ''; }
|
||||
}
|
||||
|
||||
async function diagStopMonitoring() {
|
||||
const btn = document.getElementById('diag-stop-btn');
|
||||
btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
|
||||
try {
|
||||
await fetchJson('/device/monitor/stop', { method: 'POST' });
|
||||
diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
|
||||
refreshDiagnostics();
|
||||
} catch (e) {
|
||||
diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagDisableAch() {
|
||||
const btn = document.getElementById('diag-ach-btn');
|
||||
btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
|
||||
try {
|
||||
const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
|
||||
const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
|
||||
diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
|
||||
} catch (e) {
|
||||
diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagEraseEvents() {
|
||||
const serial = (unitInfo && unitInfo.serial) || 'this unit';
|
||||
if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
|
||||
const btn = document.getElementById('diag-erase-btn');
|
||||
btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
|
||||
try {
|
||||
await fetchJson('/device/events/erase', { method: 'POST' });
|
||||
diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
|
||||
document.getElementById('diag-erase-confirm').value = '';
|
||||
eventList = []; eventsLoaded = false;
|
||||
setEventButtonsEnabled(); populateEventChips();
|
||||
refreshDiagnostics();
|
||||
} catch (e) {
|
||||
diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
diagCheckEraseConfirm();
|
||||
}
|
||||
|
||||
async function diagSlowDrip() {
|
||||
const btn = document.getElementById('diag-drip-btn');
|
||||
btn.disabled = true;
|
||||
diagResult('diag-drip-result', 'Holding a session for 120 s…');
|
||||
try {
|
||||
const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
|
||||
{ method: 'POST' });
|
||||
const good = (r.bytes_received || 0) > 0;
|
||||
diagResult('diag-drip-result',
|
||||
`drips ${r.drips_sent} · held ${r.duration_s}s · bytes back ${r.bytes_received}` +
|
||||
(r.send_error ? ` · ${r.send_error}` : '') +
|
||||
(good ? ' → device responded' : ' → no response; the modem may not be bridging'),
|
||||
good ? 'ok' : 'error');
|
||||
} catch (e) {
|
||||
diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagBlindStop() {
|
||||
const btn = document.getElementById('diag-blind-btn');
|
||||
btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
|
||||
try {
|
||||
const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
|
||||
diagResult('diag-blind-result',
|
||||
`Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
|
||||
} catch (e) {
|
||||
diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
// ── Config form ────────────────────────────────────────────────────────────────
|
||||
function populateConfigFromDeviceInfo() {
|
||||
if (!unitInfo) return;
|
||||
|
||||
Vendored
BIN
Binary file not shown.
@@ -1,54 +0,0 @@
|
||||
"""Event timestamp decode — waveform trigger/stop vs histogram window start.
|
||||
|
||||
The Blastware footer holds two timestamps: ts1 = footer[2:10], ts2 = footer[10:18].
|
||||
Their meaning depends on record type:
|
||||
|
||||
* Waveform: ts1 is the monitoring-SESSION start (e.g. 06:00 for a unit that
|
||||
arms at 06:00 daily — shared across every event that day), and ts2 is THIS
|
||||
event's recording STOP. read_blastware_file used to stamp events with ts1 →
|
||||
every waveform showed the session start (~4.5 h off). Binary-only, the best
|
||||
estimate is ts2 (the stop); the exact trigger BW displays (= ts2 - record
|
||||
duration) comes from the paired report's event_datetime, since the binary
|
||||
STRT record-time byte is a misparsed record-type marker.
|
||||
* Histogram: ts1/ts2 are the ~24 h window [start, stop]; the event time is the
|
||||
window start = ts1 (unchanged).
|
||||
"""
|
||||
import datetime
|
||||
from pathlib import Path
|
||||
|
||||
from minimateplus.event_file_io import read_blastware_file, apply_report_to_event
|
||||
from minimateplus.bw_ascii_report import BwAsciiReport
|
||||
from minimateplus.models import Event
|
||||
|
||||
FIX = Path(__file__).parent / "fixtures"
|
||||
WAVEFORM = FIX / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W" # footer ts2 = 2026-08-25 10:33:32
|
||||
HISTOGRAM = FIX / "ts-fix" / "K441LKZU.C30H" # window start 2026-05-10 19:04:50
|
||||
|
||||
|
||||
def _tuple(ts):
|
||||
return (ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second)
|
||||
|
||||
|
||||
def test_waveform_timestamp_is_exact_trigger_from_binary():
|
||||
ev = read_blastware_file(WAVEFORM)
|
||||
# The EXACT Blastware trigger, from the binary alone: ts2 (stop 10:33:32)
|
||||
# minus the config record time (3.0 s) = 10:33:29 — NOT the 06:00:13
|
||||
# monitoring-session start the old decode used.
|
||||
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29), _tuple(ev.timestamp)
|
||||
|
||||
|
||||
def test_histogram_timestamp_is_window_start_unchanged():
|
||||
ev = read_blastware_file(HISTOGRAM)
|
||||
# Histogram event time = the window start (ts1); must NOT get the waveform
|
||||
# ts2 treatment (that would land ~24 h off).
|
||||
assert _tuple(ev.timestamp) == (2026, 5, 10, 19, 4, 50), _tuple(ev.timestamp)
|
||||
|
||||
|
||||
def test_report_event_datetime_is_authoritative_over_binary():
|
||||
# The binary already yields the exact trigger, but a paired report stays
|
||||
# authoritative (e.g. if the unit clock had drifted) — applying it wins.
|
||||
ev = read_blastware_file(WAVEFORM)
|
||||
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29) # exact, from binary
|
||||
apply_report_to_event(ev, BwAsciiReport(
|
||||
event_datetime=datetime.datetime(2026, 8, 25, 10, 35, 0)))
|
||||
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 35, 0) # report wins
|
||||
Reference in New Issue
Block a user