docs(series4): the setup-write path, observed end to end

Thor pushed a setup named TEST1.mmb to UM12947 while seismo_lab's TCP bridge
recorded both directions.  We still have not originated a write frame -- the
wire format is now known, our encoder is not written.

Topology worth reusing: socat shares /dev/ttyACM0 on TCP from mint-mac,
seismo_lab relays Thor to it.  Thor is pointed at 127.0.0.1 as if the unit were
a field modem.  No modem, no SIM, production Thor box untouched.

The sequence is Series III's, plus one command:

    Thor:  5B | 41 | 08 | 2E | 1A | DA | 68->73 | 82->83 | 71->72
    unit:  A4 | BE | F7 | D1 | E5 | 25 | 97  8C | 7D  7C | 8E  8D

All 12 device responses checksum-validate and every write is acked.  Every
write response SUB matches the Series III table exactly.

New:
  * SUB 0xDA names the target .MMB file -- 256 bytes, filename null-padded,
    nothing else.  This is why no generic file-transfer command exists: Thor
    names the file, then writes the ordinary config block into it.
  * SUB 0x41 reads the active setup's filename; SUB 0x2E reads trigger config.
  * Reads are single-step -- Thor asks offset=0xFFFF and skips the probe.
  * 0x71 writes the whole 2090-byte block in ONE frame, not Series III's three
    chunks.  0x69/0x74 are absent.

Write-frame destuffing is `10 XX` -> `XX` uniformly, including `10 03`.  Chosen
by checksum, not assumption: of four candidate rules, only this one makes all
four data-carrying write frames validate.  0x71's data holds 4 literal 0x03
bytes escaped as `10 03`, so escaping is mandatory for any writer.

The write body IS the read body -- 0x71 and the 0xE5 response align at a fixed
11-byte shift with 1902/2090 bytes equal (91.0%).  Setups are read-modify-write.
The 12 differing regions are fully mapped: setup name, four 64-byte
[label:22][value:42] note entries, sensor location, and the three geo trigger
levels (0.3 -> 0.5 in/s) on a 48-byte channel stride.

Independent confirmation of the geo LSB: each channel block carries float32BE
3.10308 at label+24.  3.10308/10000 = 0.000310308 = _GEO_LSB_IPS to 8 figures,
and 10.0/3.10308*10000 = 32226.046 = the 32226.05 full scale.  That value was
derived statistically from 991,415 rounding constraints in v0.30.0; the unit
reports it directly.  It is exactly half Series III's 6.206053, so the ADC runs
10,000 counts per volt.  Do NOT retune _GEO_LSB_IPS -- this corroborates it.

The `offset` field is NOT a single length formula: two frames are len, two are
len+2, and Series III's data[1]+2 reproduces neither.  Recorded as observed
constants the device accepted; pinning the rule needs a capture with
differently-sized payloads.  This doc has been wrong once by inferring a length
field -- not inferring this one.

Also adds scratch/mm_frame_parse.py, because S3FrameParser cannot see Micromate
responses at all (it scans for DLE+STX; Micromate responses start at a bare
STX).  That is why the first pass at this capture looked like 12 unanswered
requests.  24/24 frames parse with 0 bad checksums.

Stale claims corrected: the "write half is not yet attempted" note, the
"empty unit" limitation (5 events since 2026-09-23), and the unsafe-until-agreed
list, which now distinguishes observed from exercised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
2026-09-24 18:35:58 -04:00
co-authored by Claude Opus 5
parent 701af47170
commit c8d972f685
2 changed files with 431 additions and 12 deletions
+229 -12
View File
@@ -428,6 +428,20 @@ memory 15,000,000 total and free (no events stored).
Blastware, `0x03` Thor — where Series III has a constant `0x10`.
5. **Modem serial rate is 115200**, not 38400 (per TMI provisioning practice;
not independently verified here).
6. **Reads are single-step** — Thor asks for `offset = 0xFFFF` and gets the
whole block; Series III always probes first. (2026-09-24)
7. **Setup writes are preceded by `SUB 0xDA`**, which names the target `.MMB`
file. Series III has no equivalent — it has one config, not named files.
8. **The compliance block is written in ONE `0x71` frame**, not Series III's
three chunks.
9. **`0x69` / `0x74` (waveform data write) are absent** from a setup push.
10. **Six channel blocks** (`Tran`/`Vert`/`Long`/`Mic`/`LMic`/`SMic`) on a
48-byte stride, against Series III's four.
11. **The notes block is four fixed-width `[label:22][value:42]` entries** on a
64-byte stride, with different labels — not Series III's `label: value`
scan targets.
12. **The geophone scale factor is `3.10308`**, exactly half Series III's
`6.206053`, with an ADC of 10,000 counts per volt.
---
@@ -649,8 +663,192 @@ Note `LMic` / `SMic` — linear and sound-level microphone variants that
Series III does not have.
This is the **read half of setup management**, and it means a setup can be
round-tripped: read the active config, modify, write it back. The write half
is not yet attempted.
round-tripped: read the active config, modify, write it back. ✅ **The write
half was observed on 2026-09-24** — see *The write path* below, which confirms
the round-trip: the written block is the read block, 91% byte-identical.
## The write path — observed end to end (2026-09-24)
⚠ **We have still never sent a write command to a unit.** Thor did every write
below; seismo_lab's TCP bridge sat between Thor and the unit and recorded both
directions. Capture: `bridges/captures/be12599-diag/9-24-26 - micromate2/`
(`raw_bw_*` = Thor, `raw_s3_*` = unit), UM12947 on the USB **PC** port, relayed
to TCP by `socat` on mint-mac. Operation: push a setup named `TEST1.mmb`.
**All 12 device responses checksum-validate and every write is acked.**
### The sequence
```
Thor: 5B │ 41 │ 08 │ 2E │ 1A │ DA │ 68 → 73 │ 82 → 83 │ 71 → 72
unit: A4 │ BE │ F7 │ D1 │ E5 │ 25 │ 97 8C │ 7D 7C │ 8E 8D
└──────── reads ────────┘ └──────────── writes ────────────┘
```
Series III, for comparison: `68→73 │ 71×3→72 │ 82→83 │ 69→74→72`.
Same write SUBs, and **every response SUB matches the Series III table
exactly** (`68`→`97`, `73`→`8C`, `82`→`7D`, `83`→`7C`, `71`→`8E`, `72`→`8D`).
Three differences:
- **`SUB 0xDA` is new** and comes first — see below.
- **`0x71` is a single write, not three chunks.** Series III splits the
compliance block into 1027 + 1055 + remainder; the Micromate takes all
2,090 bytes in one frame.
- **`0x69` / `0x74` (waveform data write) do not appear at all.**
Every write ack is a 16-byte zero-data frame, same shape as Series III's.
### Three new read commands
| SUB | rsp | payload | what it carries |
|---|---|---|---|
| `0x41` | `0xBE` | 271 B | **the active setup's file name**, e.g. `TEST1.mmb` |
| `0x2E` | `0xD1` | 44 B | trigger-config block; mirrors what `0x82` writes |
| `0xDA` | `0x25` | 272 B (write) | **declares the target setup file** — see below |
### `SUB 0xDA` — name the setup file you are about to write
Data is exactly **256 bytes: the file name, null-padded, nothing else.**
```
54 45 53 54 31 2e 6d 6d 62 00 00 … "TEST1.mmb" + 247 × 0x00
```
This is the missing link in *setups are files*: there is no generic
file-transfer command because there does not need to be one. Thor names the
target file, then writes the ordinary config block into it. The unit acks with
`0x25` before any config bytes are sent.
### Reads are single-step — no probe
Series III sends every read twice (probe at `offset=0x00` to learn the length,
then a data step). Thor **skips the probe** and asks for
**`offset = 0xFFFF`**, getting the whole block in one response:
```
41 02 10 10 00 1a 00 ff ff 00 … SUB 0x1A, offset 0xFFFF
→ 2,103-byte response
```
`POLL` is the exception — it uses `offset = 0x0030` (48), its data length.
This does not contradict the probe response documented above; the probe still
works and still reports its length at `payload[8:10]`. Thor simply does not
need it.
### Write-frame destuffing — `10 XX` → `XX`, uniformly
Only the leading `BW_CMD` is doubled (`10 10`); after that **every `10 XX` pair
on the wire destuffs to `XX`**, including `10 03`.
This was settled by checksum, not by assumption. Four candidate rules were
tested against all four data-carrying write frames; **only this one makes all
four checksums validate**:
| rule | `0xDA` | `0x68` | `0x82` | `0x71` |
|---|---|---|---|---|
| **`10 XX` → `XX`** | ok | **ok** | ok | **ok** |
| only `10 03` → `03` | ok | BAD | ok | BAD |
| `10 10`→`10`, `10 03`→`03` | ok | BAD | ok | BAD |
| nothing collapses | ok | BAD | ok | BAD |
`0x71`'s data contains **4 literal `0x03` bytes**, escaped as `10 03` on the
wire. A writer that does not escape `0x03` will emit a frame the device
terminates early — this is the same defensive ETX escaping Blastware does, and
it is mandatory, not optional.
Checksum is plain SUM8 of the destuffed payload; the DLE-aware form gives the
same answer once destuffing is correct, so it does not discriminate.
### The `offset` field is a per-command constant
| SUB | data bytes | `offset` | note |
|---|---|---|---|
| `0xDA` | 256 | `0x0100` = 256 | = the name-field size |
| `0x68` | 88 | `0x005A` = 90 | **identical to Series III's documented value** |
| `0x82` | 28 | `0x001C` = 28 | **identical to Series III's documented value** |
| `0x71` | 2090 | `0x082C` = 2092 | = the length `SUB 0x1A` reports |
⚠ **There is no single length formula** — two are `len`, two are `len + 2`, and
Series III's `data[1] + 2` rule does not reproduce either. Treat these as
observed constants the device accepted. Pinning the actual rule needs a second
capture whose payloads differ in size. (This document has already been wrong
once by inferring a length field; do not infer this one.)
### The write body **is** the read body
The `0x71` write payload and the `0xE5` read response align at a fixed
**11-byte shift** with **1902/2090 bytes equal (91.0%)** — the remaining 9% is
exactly what was edited.
**So a setup is read-modify-write**, the same shape as Series III, and a write
client does not need to synthesise a config block from scratch.
### Field map, from the read/write diff
Twelve differing regions, all accounted for. Offsets are into the `0x71`
**data** section (= the read response's data + 11).
| offset | size | field | this capture |
|---|---|---|---|
| `0x0000` | 3 | block length echo | `08 2a` (=2090) → zeroed on write |
| `0x0009` | 1 | unidentified | `3c` → `02` |
| `0x0013` | 1 | unidentified | `00` → `40` |
| `0x002A` | 44 | **setup file name** | → `TEST1.mmb` |
| `0x0090` | 42 | note 1 value — `Location` | → `Test Location 1 - 1234 electric boogaloo` |
| `0x00D0` | 42 | note 2 value — `Client` | → `TMI` |
| `0x0110` | 42 | note 3 value — `Company` | → `ServersDownLabs` |
| `0x0150` | 42 | note 4 value — `General Notes` | → `Hopefully this works!` |
| `0x0428` | 16 | sensor location | → `Test Config Send` |
| `0x06CA` | 4 | **Tran trigger level** float32 BE | `0.3` → `0.5` in/s |
| `0x06FA` | 4 | **Vert trigger level** | `0.3` → `0.5` in/s |
| `0x072A` | 4 | **Long trigger level** | `0.3` → `0.5` in/s |
**Notes block: four entries on a 64-byte stride**, each `[label: 22][value: 42]`,
labels at `0x008A + 64n`. The labels are `Location`, `Client`, `Company`,
`General Notes` — **not** Series III's `Project:` / `Client:` / `User Name:` /
`Seis Loc:`, and they are fixed-width fields, not `label: value` pairs.
**Channel blocks: six, 48 bytes each**, from `0x06BC` — `Tran`, `Vert`, `Long`,
`Mic`, `LMic`, `SMic`. Trigger level sits at **label + 30**.
### The geophone scale factor is in the config block — and it confirms our LSB
Each geo channel block carries a float32 BE at **label + 24**:
```
40 46 98 dd = 3.10308003…
```
Which closes a loop from the file-decode work:
```
3.10308003… / 10000 = 0.000310308 = _GEO_LSB_IPS, to 8 figures
10.0 / 3.10308003… × 10000 = 32226.046 = the 32226.05 full scale
```
`_GEO_LSB_IPS` was derived statistically — intersecting 991,415 rounding
constraints from Thor's own CSV exports (see `idf_protocol_reference.md`).
**The unit reports the constant directly**, and it agrees. That upgrades the
value from a fit to a reading, and explains the odd full-scale count: the
Micromate's ADC is **10,000 counts per volt**, and 3.10308 in/s per volt is
**exactly half** Series III's documented `6.206053` (ratio 1.99997).
Do **not** retune `_GEO_LSB_IPS` — this is corroboration, not a correction.
### Still unknown on the write path
- **What `0x68` and `0x82` actually contain.** Both were written with
near-zero payloads here and nothing in them changed, so no field is located.
Series III maps backlight/power-save/LCD-cycle into `0x68`; unverified here.
- **The three header bytes** at data `0x0000`/`0x0009`/`0x0013`.
- **Whether a *new* setup file can be created**, or only an existing one
overwritten. `0xDA` named a file that did not previously exist and the unit
accepted it — but we did not confirm on the unit's screen that `TEST1` now
exists as a selectable setup. Worth checking on the device.
- **Scheduler / call-home writes.** `callhome.MMB` is a file too, so it may go
through the same `0xDA` + block-write shape with a different target name.
## Static analysis of the firmware (2026-09-23, solo session)
@@ -789,10 +987,17 @@ its maximum should equal the loudest event in it.
## ⚠ Untested and unsafe-until-agreed
Nothing below has been sent to a unit, and nothing should be without an
explicit decision:
**Nothing below has been sent to a unit by us, and nothing should be without an
explicit decision.**
- **Writes** (`0x68`–`0x83`), **call-home write** (`0x7E`/`0x7F`)
Note the distinction introduced on 2026-09-24: the setup-write sequence
(`0xDA`, `0x68`/`0x73`, `0x82`/`0x83`, `0x71`/`0x72`) has now been **observed**,
because Thor performed it while we recorded. Observed is not the same as
exercised — **we have still never originated a write frame.** The wire format
is known; our encoder is unwritten and unproven.
- **Writes** (`0x68`–`0x83`) — format now known, never sent by us
- **Call-home write** (`0x7E` / `0x7F`) — not observed at all
- **Erase** (`0xA3` / `0xA2`)
- **Start / stop monitoring** (`0x96` / `0x97`)
- `0x1F` (advance event pointer) — non-destructive on Series III but it does
@@ -800,7 +1005,8 @@ explicit decision:
Also unknown:
- Whether `0x10` bytes inside request params need stuffing
- Whether `0x10` bytes inside **request params** need stuffing. (Write-frame
**data** stuffing *is* now settled — `10 XX` → `XX`; see *The write path*.)
- Everything about the **call-home session** — the device-initiated direction
has not been observed at all. Specifically: how a unit announces itself,
and **how it learns an event was accepted so it stops re-sending it.**
@@ -812,14 +1018,25 @@ Also unknown:
## Session provenance
Unit **UM12947**, firmware **`11.0CB`** (Blastware line), on the bench via
USB, **zero events stored** (memory free == total). Read commands only.
Every response in this document was checksum-validated.
USB. Every response in this document was checksum-validated.
Three sittings, all on the same unit:
| date | state | what was exercised |
|---|---|---|
| 2026-09-23 | zero events stored | read commands, direct from Python |
| 2026-09-23 | 5 events (4 waveform, 1 histogram) | the event chain + `0x5A` |
| 2026-09-24 | 5 events | **Thor pushing a setup**, via a recording relay |
The 2026-09-24 sitting used a different topology worth noting, because it is
reusable: `socat` on mint-mac shares `/dev/ttyACM0` on TCP, seismo_lab's TCP
bridge relays Thor to it and records both directions. Thor is configured with
the unit at `127.0.0.1:<bridge port>` exactly as if it were a field modem. No
modem, no SIM, and **nothing on the production Thor box is touched.**
⚠ Firmware is the single biggest caveat on this document. Every finding here
is from one unit on the Blastware build. A `11.0BD` unit has not been
touched.
An empty unit is a real limitation: `0x08`, `0x1E`, `0x0A` and `0x06` all have
event-dependent payloads that could not be exercised. Recording a couple of
events on the bench unit would unlock the entire event-walk half of the
protocol.
✅ The original "empty unit" limitation is gone — `0x08`, `0x1E`, `0x0A` and
`0x06` were all exercised against 5 stored events on 2026-09-23.
+202
View File
@@ -0,0 +1,202 @@
#!/usr/bin/env python3
"""
mm_frame_parse.py — parse Micromate (Series IV) frames out of a seismo_lab
raw capture pair.
Why this exists
---------------
`minimateplus.framing.S3FrameParser` cannot see Micromate traffic. It locates
frames by scanning for `DLE STX`, and a Micromate response has **no leading
DLE** — it starts at a bare `STX`. It also expects `payload[1] == 0x10`, where
the Micromate sends `0xC5` (Blastware firmware) or `0x03` (Thor firmware).
The practical consequence, seen on the 9-24-26 setup-push capture: the
Blastware-side requests parse fine (Thor emits Series III request frames), but
**every device response is silently dropped or mis-framed** — so a capture that
actually contains 12 acked writes looks like 12 unanswered requests.
Destuffing
----------
One rule covers both directions: after the leading doubled `BW_CMD`, every
`10 XX` pair on the wire destuffs to `XX`. That includes `10 03` — Thor
escapes literal `0x03` bytes in write data so they are not mistaken for ETX,
exactly as Blastware does.
That rule was chosen by evidence, not assumption: of the four candidates tried
against the 9-24-26 capture's four data-carrying write frames, it is the only
one under which all four checksums validate. See
`docs/micromate_protocol_reference.md` → *The write path*.
Usage
-----
python scratch/mm_frame_parse.py <capture-dir>
python scratch/mm_frame_parse.py <raw_bw.bin> <raw_s3.bin>
python scratch/mm_frame_parse.py <capture-dir> --dump 0x71
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
DLE, STX, ETX, ACK = 0x10, 0x02, 0x03, 0x41
# Request SUB -> short name. Series III names where they carry over; the
# Series IV additions are marked.
SUBNAME = {
0x01: "DEVICE_INFO",
0x06: "STORAGE_RANGE",
0x08: "EVENT_INDEX",
0x0A: "WAVEFORM_HDR",
0x0C: "WAVEFORM_REC",
0x15: "SERIAL",
0x1A: "COMPLIANCE_CFG",
0x1C: "MONITOR_STATUS",
0x1E: "EVENT_HDR",
0x2C: "CALLHOME_CFG",
0x2E: "TRIGGER_CFG_READ", # Series IV
0x3E: "OPERATOR",
0x41: "SETUP_NAME_READ", # Series IV
0x5A: "BULK_DOWNLOAD",
0x5B: "POLL",
0x68: "EVENT_INDEX_WRITE",
0x69: "WAVEFORM_WRITE",
0x71: "COMPLIANCE_WRITE",
0x72: "CONFIRM_A",
0x73: "CONFIRM_B",
0x74: "CONFIRM_C",
0x82: "TRIGGER_WRITE",
0x83: "TRIGGER_CONFIRM",
0xDA: "SETUP_FILE_DECL", # Series IV — names the target .MMB
0xFE: "FULL_CFG",
}
def destuff(blob: bytes, start: int, *, is_request: bool) -> tuple[bytes, int, int]:
"""Destuff one frame starting at `start`.
Returns (payload, checksum, index_of_terminating_ETX). `payload` excludes
the trailing checksum byte. A request frame opens `ACK STX 10 10`; a
response opens with a bare `STX`.
"""
i = start + (2 if is_request else 1)
out = bytearray()
if is_request:
# The doubled BW_CMD is the one guaranteed stuffed byte.
if blob[i : i + 2] != bytes([DLE, DLE]):
raise ValueError(f"@0x{start:04x}: request does not open with 10 10")
out.append(DLE)
i += 2
while i < len(blob):
b = blob[i]
if b == DLE and i + 1 < len(blob):
out.append(blob[i + 1])
i += 2
continue
if b == ETX:
break
out.append(b)
i += 1
if len(out) < 2:
raise ValueError(f"@0x{start:04x}: frame too short")
return bytes(out[:-1]), out[-1], i
def frames(blob: bytes, *, is_request: bool):
"""Yield (offset, payload, chk, checksum_kind)."""
i, n = 0, len(blob)
while i < n:
if is_request:
if not (blob[i] == ACK and i + 1 < n and blob[i + 1] == STX):
i += 1
continue
elif blob[i] != STX:
i += 1
continue
try:
payload, chk, end = destuff(blob, i, is_request=is_request)
except ValueError:
i += 1
continue
sum8 = sum(payload) & 0xFF
dle_aware = (sum(b for b in payload if b != DLE) & 0xFF)
if sum8 == chk:
kind = "SUM8"
elif dle_aware == chk:
kind = "DLE-aware"
else:
kind = "BAD"
yield i, payload, chk, kind
i = end + 1
def describe(payload: bytes, is_request: bool) -> str:
if len(payload) < 3:
return "??"
sub = payload[2]
if is_request:
return SUBNAME.get(sub, f"SUB_{sub:02X}")
req = 0xFF - sub
return "rsp<-" + SUBNAME.get(req, f"SUB_{req:02X}")
def report(path: Path, *, is_request: bool, dump_sub: int | None) -> None:
blob = path.read_bytes()
side = "Thor" if is_request else "unit"
print(f"== {side:4} {path.name} ({len(blob)} bytes)")
n_bad = 0
for idx, (off, p, chk, kind) in enumerate(frames(blob, is_request=is_request)):
if kind == "BAD":
n_bad += 1
sub = p[2] if len(p) > 2 else -1
flags = p[1] if len(p) > 1 else -1
# Requests carry offset at payload[4:6]; responses page at [3:5].
word = int.from_bytes(p[4:6] if is_request else p[3:5], "big")
data = len(p) - 16 if is_request else max(len(p) - 5, 0)
print(
f" [{idx:2}] @0x{off:04x} payload={len(p):5} data={data:5} "
f"flags=0x{flags:02x} SUB=0x{sub:02x} {describe(p, is_request):18} "
f"{'offset' if is_request else 'page'}=0x{word:04x} chk={kind}"
)
if dump_sub is not None and sub == dump_sub:
body = p[16:] if is_request else p[5:]
print(f" ---- data ({len(body)} bytes) ----")
for o in range(0, len(body), 16):
chunk = body[o : o + 16]
txt = "".join(chr(c) if 32 <= c < 127 else "." for c in chunk)
print(f" {o:06x} {chunk.hex(' '):<47} |{txt}|")
print(f" -- {idx + 1} frames, {n_bad} bad checksum\n")
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("paths", nargs="+",
help="a capture directory, or raw_bw.bin and raw_s3.bin")
ap.add_argument("--dump", default=None,
help="hex-dump the data section of this SUB (e.g. 0x71)")
args = ap.parse_args()
dump_sub = int(args.dump, 0) if args.dump else None
if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
d = Path(args.paths[0])
bw = sorted(d.glob("raw_bw_*.bin"))
s3 = sorted(d.glob("raw_s3_*.bin"))
if not bw or not s3:
print(f"{d}: need one raw_bw_*.bin and one raw_s3_*.bin", file=sys.stderr)
return 2
pairs = [(bw[0], True), (s3[0], False)]
elif len(args.paths) == 2:
pairs = [(Path(args.paths[0]), True), (Path(args.paths[1]), False)]
else:
ap.error("pass a capture directory, or exactly two .bin files")
for path, is_request in pairs:
report(path, is_request=is_request, dump_sub=dump_sub)
return 0
if __name__ == "__main__":
sys.exit(main())