From e95d8e3b8c97696d916bdb9b04fcc30620b1459a Mon Sep 17 00:00:00 2001 From: serversdown Date: Wed, 30 Sep 2026 14:33:10 -0400 Subject: [PATCH] docs(changelog): Series-4 live client, verified on hardware Folded into the existing Unreleased sections rather than adding new headings. Added: the three new micromate/ modules with 98 offline tests, the hardware verification across both transports and both firmware lines, and bridges/mm_client_check.py. Changed: the six spec rules the captures refuted before any code shipped; the 0x0C peak-vector-sum resolution and the decoder self-check it enables; the ~0.65 s per-round-trip cellular cost and what it implies for design; event keys colliding across units; and setup names being spelled three ways by three commands. Fixed: scratch/mm_frame_parse.py accepting either checksum rule, which is why it could not falsify either. Migration: None -- corrected the previous entry's claim that nothing under micromate/ was touched, which this merge makes false. The new modules are additive, micromate/idf_file.py (the codec) is untouched, no TOOL_VERSION bump, and no backfill is owed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL --- CHANGELOG.md | 98 ++++++++++++++++++++++++++-- docs/micromate_protocol_reference.md | 2 +- 2 files changed, 94 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 38f3215..3922e5c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,15 @@ All notable changes to seismo-relay are documented here. `event_datetime` stays authoritative (unit-clock drift). ⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps. + +- **`scratch/mm_frame_parse.py` accepted either checksum rule, so it could not + falsify either.** It tried plain SUM8 *and* a DLE-aware variant and reported + whichever matched, which is why it never flagged a bad frame and why the + protocol reference carried the wrong rule for two days — "zero bad checksums + across three sessions" was true and carried no information. It now validates + against SUM8 alone and names the DLE-aware result only as a near-miss, never as + a pass. Still zero bad frames across all captures, strictly tighter. + ### Added - **Diagnostics tab in the SFM standalone webapp.** Surfaces the device @@ -90,6 +99,34 @@ All notable changes to seismo-relay are documented here. all, since it scans for `DLE+STX`), byte-exact capture recovery from a `socat -x` relay log, and a serial-port stand-in that answers as a unit. + +- **A live client for Series IV — `micromate/{framing,protocol,client}.py`.** + `micromate/` was codec-only; it can now talk to a unit. Connect over TCP (a + cellular modem) or serial/USB, identify a unit, read its state, clock, battery, + memory and setups, walk the event chain and download events as `.IDFW`/`.IDFH` + bytes the existing codec already decodes. **98 offline tests**, every response + constant a real captured data section. `minimateplus/transport.py` is reused + as-is; `minimateplus/framing.py` deliberately is **not** — see *Changed*. + ⚠ **Read-only.** Setups, schedules, call-home config, monitoring start/stop + and per-event delete are all mapped and none are implemented. No command has + ever been originated against a unit by this project; every write in the + protocol reference was performed by THOR while we recorded. +- **Verified on real hardware, both transports, both firmware lines.** Identical + wire bytes over USB CDC-ACM and an RX55 in PAD mode — 11,580 B in and 761 B out + to the byte. Every `11.0BD` inference confirmed on UM20147, including the four + extra trailing bytes in `SUB 0x1C` that make Series III's from-the-end offsets + report a battery voltage of **577.92 V**. A monitoring unit answers reads with + no `SESSION_RESET`, which Series III requires. All six bench events decode + with the existing IDF codec — 4 waveforms at 12,288/12,288/12,288/8,192 samples + and 2 histograms — so `/db/import/idf_file` ingests a directly downloaded event + unchanged. +- **`bridges/mm_client_check.py`** — drives the read client against a unit and + reports per-command timings, read counts and byte totals, so two transports or + two firmware lines can be diffed. `--capture DIR` writes a `raw_bw_*`/`raw_s3_*` + pair in the layout `scratch/mm_frame_parse.py` reads, turning a field run into + a test fixture. No pyserial — stdlib `termios`, because `pip install` is + refused outright by PEP 668 on the distros the bench hosts run. + ### Changed - **Connecting to a unit no longer walks its event chain.** `/device/events` @@ -121,13 +158,64 @@ All notable changes to seismo-relay are documented here. modem port; an FTDI cable (Sabrent) works. Both are in circulation and indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`. + +- **Six rules in the Series-4 client spec were wrong, and measuring against the + captures caught all six before any code shipped.** Each fails *silently* — + a frame the unit ignores, or a checksum that reads as bad: + requests escape **four** byte values (`0x02 0x03 0x04 0x10`), not one, so the + Series III builder reproduces only **161 of THOR's 218** read frames and misses + *every* `SUB 0x5A` download; the response checksum is **plain SUM8** of the + de-stuffed payload, not the DLE-aware variant, which disagrees with the wire on + **55 of 251** frames; `SUB 0x5A` is a **1024-byte chunk loop**, not one request + per event; `SUB 0x0A` is the **monitor-log walk** (same request repeated, + device-side cursor), not a keyed event-header read; `1E`/`1F` carry **token + `0xFE`** at `params[7]`; and `SUB 0x01` has **no THOR precedent at all**. + Recorded in `docs/micromate_protocol_reference.md` as corrections in place. +- **`SUB 0x0C`'s peak float is the per-sample peak vector sum** — resolved after + being marked *do-not-rely-on*. Exact to **0.000%** on all four bench waveforms + against a PVS recomputed from decoded samples, so its offset (`Tran` label − 12) + is established rather than inferred. The earlier "not the vector sum" reading + compared against `sqrt(Σpeak²)` from the three *reported* peaks, which is an + upper bound: the channel maxima do not occur at the same instant. **This gives + the decoder a free self-check** — the device computed that number from the same + samples, independently of our codec, so a mismatch means the decode is wrong. + Worth having in a codebase whose channel truncations have historically been + silent. +- **Cellular costs ~0.65 s per round trip, independent of payload size.** + Measured on UM12947 over an RX55: `list_setups()` on a unit with 24 setups takes + **16.0 s**, against 0.46 s over USB, for 24 commands. A 1,024-byte chunk and a + 16-byte state read cost the same. **Over cellular, minimise round trips, not + bytes** — cache the setup list rather than refreshing it on a timer, and note a + 13 KB event is 14 chunks ≈ 8.4 s of latency against ~0.03 s of data. The modem + also needs **fewer** reads than USB, not more: it buffers ~1 s then forwards one + large segment where CDC-ACM delivers many small ones. +- **Event keys collide across units.** UM20147's first event and UM12947's first + event are both `055d4a81` — different sizes, different contents, identical key. + The counter starts from the same value on every unit, so **a key is meaningless + without its serial**. Anything that stores, deduplicates or addresses Series IV + events must key on `(serial, event_key)`; keyed on the event key alone, one + unit's event is silently treated as a duplicate of another's and simply never + ingested. Series III has the cousin of this — its counter resets after an + erase, so keys are reused *within* a unit, which is why `ach_state.json` already + tracks `max_downloaded_key` per serial. +- **Setup file names are spelled three ways by three commands.** In one session + on one unit: `0x41` reported `test2.MMB`, `0x40` reported `test2.mmb`, and `0x0C` + reported `test2`. **Compare setup names case-insensitively and without the + extension** — an exact-string test of "is the active setup one I know about?" + answers *no*. + ### Migration -**None.** Frontend, documentation and bench tooling only — no codec, -waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump. The -webapp is served from the image, so its changes appear after the next `sfm` -rebuild. The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only; -nothing under `sfm/`, `minimateplus/` or `micromate/` was touched. +**None.** No codec change, no waveform-store change, no DB or schema change, +and **no `TOOL_VERSION` bump** — so no `backfill_sidecars.py` / +`backfill_event_shape.py` run is owed. The webapp is served from the image, so +its changes appear after the next `sfm` rebuild. + +The Series-4 live client adds **new** modules under `micromate/` +(`framing.py`, `protocol.py`, `client.py`) and appends two dataclasses to +`micromate/models.py`; `micromate/idf_file.py` — the codec — is untouched, and +nothing under `sfm/` or `minimateplus/` changed. The new modules are additive +and nothing imports them yet, so an existing deployment behaves identically. --- diff --git a/docs/micromate_protocol_reference.md b/docs/micromate_protocol_reference.md index d93d0ba..06ba509 100644 --- a/docs/micromate_protocol_reference.md +++ b/docs/micromate_protocol_reference.md @@ -1817,7 +1817,7 @@ name the file correctly. sample by sample.** Exact to 0.000% on all four bench waveforms, against a PVS recomputed from the decoded samples: -| key | `0x0C` float | per-sample PVS | err | `sqrt(Σpeak²)` | `max(T,V,L)` | +| key | `0x0C` float | per-sample PVS | err | `sqrt(Σpeak²)` | `max(T,V,L)` | |---|---|---|---|---|---| | `…82` | 1.37198 | 1.37198 | **+0.000%** | 1.41746 | 1.37063 | | `…83` | 2.35414 | 2.35414 | **+0.000%** | 2.37824 | 2.22274 |