diff --git a/CHANGELOG.md b/CHANGELOG.md index 3922e5c..562861a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,29 @@ All notable changes to seismo-relay are documented here. against SUM8 alone and names the DLE-aware result only as a near-miss, never as a pass. Still zero bad frames across all captures, strictly tighter. + +- **A 64 KB download cap, one event away from biting.** The `0x5A` chunk offset + was written as a **uint16** at `params[2:4]`, because every offset THOR was + observed to send fits in two bytes (largest `0x3400` = 13,312) — which capped a + download at 65,536 B. UM20147 holds a **72,560-byte** event, so that cap was + not hypothetical: the event could not have been fetched at all. + `params[0:4]` is demonstrably one 4-byte field (chunk 0 puts the 4-byte event + key there), so writing the offset as a uint32 BE is **byte-identical below + 65,536** — the replay test still matches all 74 of THOR's frames — and reaches + past it. ✅ Confirmed on the unit: **71 chunks, exact size.** + ⚠ **This is the Series III 64 KB page-boundary bug wearing a different hat**, + and that one is *still open* — `parse_strt_end_offset()` discards the key's page + byte and the `5A` walk crashes once a unit's buffer crosses 64 KB. The + transferable lesson: *an address field whose high bytes are zero in every + capture is not a narrow field, it is an untested one.* Same mistake, found + twice, in code written years apart. +- **The probe's own control was circular.** `scratch/mm_stream_probe.py` compared + each candidate request size against a reference download that used the library + *default* chunk size — which the same change had just raised to 16,384. A fault + at 16 KB would have been shared by the control and every comparison would have + passed. Pinned to `THOR_CHUNK_SIZE`, the only size with THOR's captures behind + it. + ### Added - **Diagnostics tab in the SFM standalone webapp.** Surfaces the device @@ -127,6 +150,34 @@ All notable changes to seismo-relay are documented here. a test fixture. No pyserial — stdlib `termios`, because `pip install` is refused outright by PEP 668 on the distros the bench hosts run. + +- **The Series-4 event chain.** `MicromateEventRef` plus `list_events()`, + `iter_events()`, `download_event()`, `get_event()` and `decode_error()` on + `MicromateClient`. Walk a unit's events with full metadata — key, size, type, + timestamp, sensor location, setup name, per-channel peaks, peak vector sum — + then download and decode any of them, waveform or histogram. + `ref.filename` generates the name THOR would have used + (`UM12947_20260923163319.IDFW`), returning None rather than guessing when the + record type is unknown, because `read_idf_file()` dispatches on exactly that + suffix. + The load-bearing test replays THOR's captured six-event session and asserts + **every byte we emit matches its 74 frames**, decoding all six events along the + way. Read-only, like the layers under it. +- **A per-event decode self-check.** `get_event(verify=True)` recomputes the peak + vector sum from the decoded samples and compares it against the float the + *device* put in the `0x0C` record — two independent computations over the same + samples, so a disagreement means our decode is wrong. Agreement on the bench + events is **0.000%**. Worth having in a codebase whose decode failures have + historically been silent: unhandled block tags shorten a channel and nothing + raises. ⚠ It is a decode-correctness check, **not** a truncation detector — a + channel cut after its peak still yields the right PVS. +- **`SUB 0x06` content[0:4] is the event count.** Three for three across both + firmware lines (0 events→0, 6→6, 5→5), and THOR reads it *before* the chain walk + then stops without ever reading the sentinel. `list_events()` still walks to + the sentinel — correct on evidence, one round trip slower — but this lets a + caller ask "is there anything here?" in one command. `content[4:8]` reads **9** + on both units regardless of count, so it is a constant, not a count. + ### Changed - **Connecting to a unit no longer walks its event chain.** `/device/events` @@ -204,6 +255,56 @@ All notable changes to seismo-relay are documented here. extension** — an exact-string test of "is the active setup one I know about?" answers *no*. + +- **16,384 bytes per `0x5A` request, up from THOR's 1024.** Measured on UM20147 + with every size checked byte-for-byte against a 1024 B control: 1 K / 2 K / 4 K + / 8 K / 16 K all served in full, and anything larger is **silently clamped to + 16,384** — correct bytes, short length, no error. + ⚠ **That clamp is why the download loop now tracks its offset by bytes + received** rather than striding by chunk index. A fixed stride would either + fail on a clamp or skip the bytes it never collected; tracking what arrived + makes a clamp cost one extra request and makes the loop self-correcting against + any short response. Consequence worth stating because it reverses an earlier + decision: **a short chunk is no longer an error.** The device returns short + legitimately; the protection is the offset arithmetic plus the final + total-length check, which still raises on a genuinely truncated event. + `THOR_CHUNK_SIZE = 1024` remains available and the replay tests pin it. + Confirmed over a cellular PAD as well as USB: 14,176 B in one frame, intact. +- **⚠ Corrected: cellular cost is NOT independent of payload.** This changelog + and the protocol reference both previously said **"~0.65 s per round trip, + regardless of payload"**, and the design advice drawn from it — *minimise round + trips, not bytes* — was too strong. Every measurement behind that claim had a + payload under 1 KB (59 B status, 266 B setup records, 1024 B chunks), so the + byte term was small and similar across all of them and per-command cost looked + constant. It was a narrow-range fit extrapolated past its evidence. + Measured against a 14,176 B single response over an RX55: + **`t ≈ 0.21 s + bytes / 2,350`**, fitting within ±11% across a 14× size range. + The old figure is that model evaluated at 1 KB. + **The advice splits in two:** *status* work is round-trip bound (a 24-setup walk + really is 16 s, so cache it), while a *download* is **throughput bound** — the + 16 KB chunk size saves ~30% on a large event, **not 14×**, because most of the + time is bytes on the wire and batching does not touch it. A 72,560-byte event + is ~46 s at 1024 B and ~32 s at 16,384 B. +- **⚠ Retracted: there is no `SUB 0x5A` streaming mode.** The reference + hypothesised that `offset_hi = 0x10` meant "stream until done", reconciling + THOR's chunk loop with a 2026-09-23 note about a single request returning an + entire 11 KB event. Measured directly: **`offset` is simply a byte count** and + the device returns exactly `offset + 11` bytes in one frame. `0x1000` is not a + marker, it is part of the number. The probe tried **both escapings** of + `offset_hi`, which is what makes the negative meaningful — a malformed frame + would have produced the same silence, and Series III requires that byte raw in + this very command. +- **Verified on real hardware across both firmware lines and both transports.** + Identical wire bytes over USB CDC-ACM and an RX55; every `11.0BD` inference + confirmed on UM20147 — including the four extra trailing bytes in `SUB 0x1C` + that make Series III's from-the-end offsets report **577.92 V**; a monitoring + unit answers reads with no `SESSION_RESET`; and `0x0C` content[11] gives the + record type (`0x07` waveform / `0x08` histogram) on 7 events across both lines. + ⚠ **Event keys collide across units** — UM12947 and UM20147 both have an event + `055d4a81`, different sizes, different contents. Anything that stores or + deduplicates must key on `(serial, event_key)`; `MicromateEventRef.uid` is that + pair. + ### Migration **None.** No codec change, no waveform-store change, no DB or schema change, @@ -212,11 +313,16 @@ and **no `TOOL_VERSION` bump** — so no `backfill_sidecars.py` / its changes appear after the next `sfm` rebuild. The Series-4 live client adds **new** modules under `micromate/` -(`framing.py`, `protocol.py`, `client.py`) and appends two dataclasses to +(`framing.py`, `protocol.py`, `client.py`) and appends three dataclasses to `micromate/models.py`; `micromate/idf_file.py` — the codec — is untouched, and nothing under `sfm/` or `minimateplus/` changed. The new modules are additive and nothing imports them yet, so an existing deployment behaves identically. +⚠ The chunk-size and short-chunk changes are **internal to +`micromate/protocol.py`'s download loop**, which no shipped code path calls — the +only callers are the new client and the bench tooling. Nothing in `sfm/` or the +ingest path is affected. + --- ## v0.31.0 — 2026-09-18