merge full s3 codec decoded #23

Merged
serversdown merged 18 commits from codec-re into main 2026-05-20 13:45:33 -04:00
Showing only changes of commit 5bf5329369 - Show all commits
+57
View File
@@ -17,6 +17,8 @@ minimateplus/ ← Python client library (primary focus)
protocol.py ← MiniMateProtocol — wire-level read/write methods protocol.py ← MiniMateProtocol — wire-level read/write methods
client.py ← MiniMateClient — high-level API (connect, get_events, …) client.py ← MiniMateClient — high-level API (connect, get_events, …)
models.py ← DeviceInfo, EventRecord, ComplianceConfig, … models.py ← DeviceInfo, EventRecord, ComplianceConfig, …
waveform_codec.py ← Body-codec block walker + decode_tran_initial (partial
per-sample decoder — see "Waveform body codec" section below)
sfm/server.py ← FastAPI REST server exposing device data over HTTP sfm/server.py ← FastAPI REST server exposing device data over HTTP
seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs) seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs)
@@ -57,6 +59,61 @@ Full read pipeline + write pipeline + erase pipeline + monitor log + call home c
--- ---
## Waveform body codec — PARTIAL (2026-05-11)
The **per-byte decoding** of the Blastware waveform-file body (between the
21-byte STRT record and the 26-byte footer) was historically claimed to be
"raw int16 LE, 8 bytes per sample-set." That was wrong — see the
retraction in `docs/instantel_protocol_reference.md §7.6.1`. The body
is actually a tagged-block stream with a custom delta+RLE codec.
### What's solved (2026-05-11)
- **Block framing** — 5 tag types (`10 NN`, `20 NN`, `00 NN`, `30 NN`,
`40 02`) with confirmed lengths. Implementation: `walk_body()` in
`minimateplus/waveform_codec.py`.
- **Tran channel segment 0** — preamble bytes [3:7] = `Tran[0]`, `Tran[1]`
as int16 BE in **16-count units** (LSB = 0.005 in/s). Then `10 NN`
(4-bit nibble deltas), `20 NN` (int8 deltas), and `00 NN` (RLE zero
deltas) carry Tran deltas from sample 2 onward. Verified byte-perfect
across 4 of 5 fixture events (510 samples each). Implementation:
`decode_tran_initial()`.
- **Segment header** — `40 02` is a 20-byte block. Payload bytes [0:2]
are the T_delta at the start of the new segment (int16 BE). Bytes
[6:8] are the byte length to the next segment header. Bytes [8:12]
are a monotonic uint32 LE counter. Bytes [12:14] are constant `02 00`.
### What's NOT solved
- **Tran past segment 0** — multi-segment Tran continuation has been
attempted but every hypothesis tested breaks at sample ~512. Likely
channels rotate across segments (e.g. segment 0 = Tran, segment 1 = Vert,
…) but this is unverified.
- **Vert / Long / Mic channels** — no per-channel decoder yet. These
almost certainly live in later segments but the segment-to-channel
mapping is open.
- **The `30 NN` block content** — appears in loud-from-start events
(SS0, SV0) and breaks the simple Tran walk there. Probably a channel-
switch or alternative-encoding marker for high-amplitude regions.
### Production-code status
`client.py:_decode_a5_waveform` still uses the old (broken) int16 LE
decoder. Until the multi-channel decoder lands, the `.h5` sidecars
produced by SFM contain WRONG samples — keep treating them as
"unverified" downstream. `decode_waveform_v2()` returns `None` as a
placeholder.
### Test fixtures
`tests/fixtures/decode-re-5-8-26/` and `tests/fixtures/5-11-26/`
seven BW binary + ASCII pairs captured from a live BE11529. The
5-11-26 high-amplitude bundle (PPV 67 in/s) is what cracked the Tran
codec; the V70 (mic-heavy) + JQ0 (Vert-heavy) pair cracked the `00 NN`
RLE rule.
---
## Protocol fundamentals ## Protocol fundamentals
### DLE framing ### DLE framing