serversdown/seismo-relay
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(micromate): protocol layer -- reads only, verified against Thor's frames

Browse Source 
Step 2 of docs/micromate_client_spec.md: micromate/protocol.py plus 35
offline tests.  Reads only; nothing here writes, erases or changes
monitoring state.

The tests replay Thor's captured responses through a scripted transport and
assert the bytes we emit are the bytes Thor emits -- including a full replay
of the six-event download session, all 56 0x5A frames byte-for-byte.  A
passing test therefore means a real unit has already answered exactly that
frame.

Measuring the spec's command table against the captures found three more
errors in it, on top of the three the framing work found:

1. SUB 0x0A is the MONITOR-LOG WALK, not a keyed "event header, 30 B list
   record" read.  The same request repeated returns successive 297-byte
   records -- serial, mode, thresholds -- until an 11-byte ack ends the
   list.  The device holds the cursor; nothing in the request selects a
   record, and all nine captured frames carry identical params.

   Structural divergence worth noting: Series III reaches the same data via
   a record-type discriminator on its event chain, so partials and events
   share one walk.  Here the monitor log has its own cursor and the event
   chain never sees it.

2. 0x1E/0x1F carry token 0xFE at params[7].  The protocol reference
   documents all-zero params -- that was our own browse probing, which also
   worked.  Thor sends 0xFE on browse and download alike.

3. SUB 0x01 (device info) is never read by Thor in any captured session.
   Its 0xFFFF offset comes from our own probes, so it is the one read in
   the table with no Thor precedent.  Flagged in the docstring.

Two useful negatives, both from absence rather than presence:

- No SESSION_RESET (41 03).  Series III needs that 2-byte signal or a
  monitoring unit will not answer POLL over TCP.  Zero occurrences across
  all 8 sessions, including 40 frames exchanged with a unit that WAS
  monitoring.
- No universal preamble.  The only invariant is that a session opens with
  POLL; POLL -> SERIAL -> 0x49 -> POLL is Thor's connection check and
  appears in 3 of 8 sessions.  Setup pushes and scheduler reads open
  differently.

Two deliberate divergences from the Series III sibling:

- strict_checksums defaults True and RAISES.  minimateplus logs and
  continues because its parser cannot always tell an inner-frame delimiter
  from a checksum byte; that does not apply here, where the rule is exact
  on 251/251 frames.  The lenient default is instructive -- it hid a wrong
  checksum rule for two days.
- read_event_file() raises ShortRead rather than returning a truncated
  event.  The expected length is known up front, so the check is free, and
  a silently short event is the failure mode this codebase keeps hitting.

Every exchange resets the parser before sending, so a leftover frame is
discarded rather than answered with -- expected_sub catches a mismatched
SUB, but a same-SUB leftover would sail through with data for the wrong key.

File transfer (0x94/0x48) is deliberately out of scope: it needs a
data-carrying request frame, which is the frame type writes use, and that
boundary is worth keeping crisp in a read-only pass.

Full suite unchanged at 16 pre-existing failures (missing gitignored
fixtures); 419 passed, up 35.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
serversdown
2026-09-28 19:58:22 -04:00
co-authored by Claude Opus 5
parent 5fe99568a2
commit a7e3a8f20a
4 changed files with 1035 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/micromate_client_spec.md
+42 -11
View File
@@ -173,7 +173,16 @@ timeout, and that distinction earned its keep during the Series III work).
---
## `micromate/protocol.py`
## `micromate/protocol.py` — ✅ BUILT 2026-09-27
Implemented, with `tests/test_micromate_protocol.py` (35 tests). Reads only;
nothing here writes, erases or changes monitoring state.
The tests replay Thor's captured responses through a scripted transport and
assert **the bytes we emit are the bytes Thor emits** — including a full replay
of the six-event download session, all 56 `0x5A` frames byte-for-byte. That is
a stronger guarantee than "our parser understands the device": a passing test
means a real unit has already answered exactly that frame.
One method per command, returning raw payload bytes. No interpretation — that
belongs in `client.py`.
@@ -198,9 +207,29 @@ taking its data length. Per-command offsets, all observed:
| arm event | `0x93` | `0x6C` | — | before every event |
| first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
| next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
| event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks |
| event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record |
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** |
| event record | `0x0C` | `0xF3` | `0xFFFF` | 221 B — project, location, peaks |
| ~~event header~~ **monitor log** | `0x0A` | `0xF5` | `0xFFFF` | ⚠ 297 B, a **walk** — see below |
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim**, 1024 B at a time |
⚠ **Corrected 2026-09-27, from Thor's frames.** Three rows of the table above
were wrong or incomplete, and the last one is a different command than labelled:
- **`0x0A` is the monitor-log walk**, not a keyed "30 B list record" read. The
*same request repeated* returns successive 297-byte records — serial, mode,
thresholds — until an 11-byte ack ends the list. The device holds the cursor;
nothing in the request selects a record. Series III reaches this data through
a record-type discriminator on its event chain; here it has its own cursor and
the event chain never sees it.
- **`0x1E`/`0x1F` carry token `0xFE` at `params[7]`.** The reference documents
all-zero params (our own probing, which also worked). Thor's form is the one
with mileage.
- **`0x01` has no Thor frame behind it** — it is never read in any captured
session. Its `0xFFFF` comes from our probes.
And one useful negative: **no `SESSION_RESET` (`41 03`)**. Series III needs that
2-byte signal or a monitoring unit will not answer `POLL` over TCP. Thor never
sends it — zero occurrences across 8 sessions, including 40 frames exchanged
with a unit that *was* monitoring.
⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
Parse **forward** from `declared_length`, never backward from the end — Series
@@ -328,19 +357,21 @@ then `download_event()` and assert the bytes decode and match a
## Order of work
1. ✅ `framing.py` + its tests — **done 2026-09-27**, 31 tests, offline
2. `protocol.py` — reads only, one method per row of the table above
2. ✅ `protocol.py` + its tests — **done 2026-09-27**, 35 tests, offline
3. `client.py` — `connect()`, `get_state()`, `list_setups()`
4. the event chain and `download_event()`
5. decode end-to-end and compare against a store event
Steps 1–2 need no hardware at all.
**Worth carrying forward from step 1:** every rule got checked against the
captures *before* being written, and two of the three the spec asserted turned
out wrong — the escape set (26% of frames) and the checksum (22%). Both fail
silently. The captures are on disk and a re-stuff-and-compare loop takes about
two minutes per rule, so do that for `protocol.py`'s per-command offsets too
rather than trusting the table above.
**Worth carrying forward.** Both steps began by measuring against the captures
rather than trusting this document, and both found errors in it — three in the
framing rules (the escape set, 26% of frames; the checksum, 22%; the `0x5A`
chunk model) and three more in the command table (`0x0A`'s meaning, the
`1E`/`1F` token, `0x01`'s provenance). All six fail quietly. The captures are on
disk and a measure-then-write loop costs about two minutes per rule, so keep
doing it for `client.py`'s field offsets — and treat this spec as a plan, not a
source.
---
docs/micromate_protocol_reference.md
+90
View File
@@ -1216,6 +1216,96 @@ the protocol's requirements*. Thor sends it before trivial reads too, so it may
be habit rather than handshake. Do not assume it is mandatory. Note `POLL` here carries `offset = 0x0030` (its data
length), not `0xFFFF` — `POLL` is the one read Thor still addresses by length.
> #### ⚠ Narrowed 2026-09-27 — there is no *universal* preamble
>
> Across all 8 captured sessions, the only invariant is that **the session opens
> with `POLL`**. What follows depends on the operation:
>
> | opening sequence | sessions | operation |
> |---|---|---|
> | `5b 15 49 5b …` | 3 | status refresh / monitoring / ACH change |
> | `5b 41 08 2e 1a da …` | 3 | setup push |
> | `5b 94 48 48 48 …` | 2 | scheduler read |
>
> So `POLL → SERIAL → 0x49 → POLL` is Thor's **connection check**, not a
> handshake the protocol demands — it appears where Thor wants to refresh what
> it displays. Treat `POLL` as the one thing to send first.
### 🔑 No `SESSION_RESET` — the Series III requirement does not carry over
Series III needs a bare `41 03` (ACK + ETX, no STX) to wake a unit that is
actively monitoring; without it the unit will not answer `POLL` over TCP, and
`protocol.startup()` sends it before and between the POLL frames.
**Thor never sends it to a Micromate.** Zero occurrences across all 8 sessions
— including `raw_bw_20260924_191214_turn_on_monitormode_…`, which exchanges 40
frames with a unit that *was* monitoring at the time.
### Measured offsets and response lengths (all read off Thor's frames)
`offset = 0xFFFF` for everything except two commands. Data lengths are from
UM12947 (`11.0CB`) and are **orientation, not assertions** — `0x1C` is 4 bytes
longer on the Thor line.
| SUB | rsp | offset | data | notes |
|---|---|---|---|---|
| `0x5B` POLL | `0xA4` | **`0x0030`** | 59 | the one length-addressed read |
| `0x15` serial | `0xEA` | **`0x000A`** | 21 | |
| `0x49` state | `0xB6` | `0xFFFF` | 16 | |
| `0x1C` monitor status | `0xE3` | `0xFFFF` | 55 | +4 on `11.0BD` |
| `0x06` storage range | `0xF9` | `0xFFFF` | 47 | |
| `0x08` event index | `0xF7` | `0xFFFF` | 101 | contents unmapped |
| `0x2E` trigger config | `0xD1` | `0xFFFF` | 39 | |
| `0x1A` compliance | `0xE5` | `0xFFFF` | 2103 | one frame, not Series III's four |
| `0x2C` call-home | `0xD3` | `0xFFFF` | 137 | |
| `0x3F`/`0x40`/`0x41` setups | `0xC0`/`0xBF`/`0xBE` | `0xFFFF` | 266 | |
| `0x93` arm | `0x6C` | `0xFFFF` | 11 | ack only |
| `0x1E`/`0x1F` chain | `0xE1`/`0xE0` | `0xFFFF` | 19 | ⚠ **token `0xFE` at `params[7]`** |
| `0x0C` event record | `0xF3` | `0xFFFF` | 221 | full key at `params[4:8]` |
| `0x0A` monitor log | `0xF5` | `0xFFFF` | 297 | ⚠ a **walk** — see below |
| `0x5A` download | `0xA5` | computed | offset+11 | 1024-byte chunk loop |
An acknowledgement is an **11-byte data section**, and that doubles as the
end-of-list signal on the walks.
#### ⚠ `0x1E`/`0x1F` carry token `0xFE`
`params = 00 00 00 00 00 00 00 fe 00 00` on all 7 captured chain reads — the
same `token_params(0xFE)` form Series III uses to arm its bulk stream. The
event-chain section above documents **all-zero params**; that was our own browse
probing, which also worked. Both evidently do, but Thor's form is the one with
mileage on it, and it is sent on browse and download alike.
#### 🔑 `SUB 0x0A` is the monitor-log walk, not a keyed read
The command table long described `0x0A` as a keyed "waveform header / partial
record" read, by analogy with Series III. What the bytes show is a **cursor
walk**: the *same request repeated*, the device advancing its own position.
```
0x93 → 1E → 0x0A ×8 (297 B each: "UM12947", "Histo…", "Ver…", " 0.49", " 28.4")
0x0A (11 B ack = end of list)
```
All nine frames carry identical params (`…00 00 4a 81 00 00`), so nothing in the
request selects the record. Terminate on a response of `ACK_DATA_LEN` (11).
⚠ The `4a 81` is the **low two bytes** of the event key then in play
(`055d4a81`). One key cannot distinguish "the key's low half" from "a cursor
handle that happened to equal it" — both produce those bytes. It does not
matter operationally, since the walk works with the params held constant.
This is a genuine structural divergence: Series III reaches the same data
through a record-type discriminator (`0x2C` partial vs `0x46` full) *on its
event walk*, so partial records and events share one chain. Here the monitor
log has its own cursor and the event chain never sees it.
#### `SUB 0x01` has no Thor frame behind it
Thor never reads device info in any captured session. `0xFFFF` for `0x01` comes
from our own 2026-09-23 probes — it answered correctly on both firmware lines,
but it is the only read in the table with no Thor precedent.
### `SUB 0x96` / `0x97` — start and stop monitoring ✅
Identical to Series III, including the acks: