seismo-relay/CLAUDE.md

# CLAUDE.md — seismo-relay

Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.21.0**.

When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document

---

## Architecture: three-tier conceptual model

seismo-relay is a **suite of cooperating components**, not a single app.
The three tiers below are the canonical mental model — the current
directory layout doesn't fully reflect them yet (some of what is
conceptually SDM lives under `sfm/` today), but new code should be
placed and named according to this model.

### 1. SFM — the device-side (active connection to physical units)

Replaces Blastware's *talk-to-the-meter* role.  Lives where a connection
to a physical seismograph is open.

In scope:
- `minimateplus/{transport,framing,protocol,client}.py` — wire protocol
- `seismo_lab.py` — diagnostic GUI (a thick client for SFM)
- The `/device/*` HTTP endpoints in `sfm/server.py` —
  `/device/info`, `/device/events`, `/device/monitor/*`, `/device/call_home`,
  etc.  Anything that opens a connection at the moment of the request.
- Future: a Thor / Micromate live client (mirror `minimateplus/`)
- Future: a control surface Terra-View can launch into — see the
  README's Roadmap.

Does NOT own a database.  Outputs `Event` objects.  Has a "spun up when
needed" runtime profile rather than "always on".

### 2. SDM — the data-side (storage, ingest, and serving)

The new name for the receiving-and-storing role.  Originally called SFM
because the FastAPI service started life as a thin device proxy, but
the actual role has migrated heavily toward data management.  **For now
the directory remains `sfm/`** — renaming requires touching ~30-50
files in seismo-relay + ~10-15 in terra-view + a Docker volume
migration; deferred until the codebase is quiet enough to do it as a
clean refactor.

In scope:
- `sfm/database.py` (`SeismoDb`)
- `sfm/waveform_store.py`, `sfm/event_hdf5.py`
- The `/db/*` HTTP endpoints — `events`, `units`, `monitor_log`,
  `sessions`, `false_trigger` mutations
- The `/db/import/*` ingest endpoints — `blastware_file` (series3),
  `idf_file` (series4); anything that receives events FROM somewhere
- `scripts/backfill_sidecars.py`, `scripts/check_bw_report_preservation.py`,
  and similar data-maintenance tools
- The `.sfm.json` sidecars and `.h5` files in the waveform store
- The shape that Terra-View consumes (Terra-View should never need to
  reach into SFM/device-side endpoints to populate its UI)

Always-on, scaled for storage/serving, has the DB and waveform store.

### 3. Codec library — pure data interpretation (used by both sides)

Neither SFM nor SDM — a shared library both depend on.

In scope:
- `minimateplus/{waveform_codec,histogram_codec,event_file_io,bw_ascii_report,blastware_file}.py`
- `micromate/{idf_ascii_report,idf_file}.py`

These modules take bytes (off the wire on the SFM side, or from a
forwarded file on the SDM side) and return `Event` objects.  They
should not import from `sfm/`, must not touch a DB, and have no I/O
beyond reading files passed as arguments.  Keep them pure — both
tiers can then depend on them without circularity.

#### Thor IDF binary codec (2026-05-28)

`micromate/idf_file.read_idf_file()` decodes both Thor IDFW
(waveform) and IDFH (histogram) binaries.

- **IDFW** reuses `decode_waveform_v2()` on the body at fixed file
  offset `0x0f1f`.  Sample fidelity is 87–99% byte-exact on quiet
  events; loud events hit the BW codec's known walker-stops-early
  limitation.
- **IDFH** has its own segment-based decoder: `[len_be][0a 00 00 00]
  [00 NN][05 3f]` + N × 72-byte interval records (4 × 16-byte
  per-channel min/max/halfp).  All 859 Thor IDFH corpus files
  decode (181,071 intervals); peak matches sidecar within ~1.8%
  (ADC quantization).

The two outlier `BE9439_*` files in the Thor example corpus are
actually Series III Blastware binaries that share the `.IDFW`/`.IDFH`
filename convention by accident.  `read_idf_file()` detects them by
their BW STRT signature and raises NotImplementedError pointing
callers at `read_blastware_file()`.  See
`docs/idf_protocol_reference.md` for full field layouts.

### Practical consequences

When deciding where new code goes, ask:
- *Does it need a connection to a device?* → SFM
- *Does it operate on stored events / sidecars / DB rows?* → SDM
- *Does it interpret bytes into structured data, with no I/O of its own?* → codec lib

Terra-View is downstream of SDM for data, and (per the roadmap) will
eventually invoke into SFM's device-control endpoints to provide a
"connect to unit" experience.

---

## Project layout

```
minimateplus/         ← Python client library (primary focus)
  transport.py        ←   SerialTransport, TcpTransport
  framing.py          ←   DLE codec, frame builders, S3FrameParser
  protocol.py         ←   MiniMateProtocol — wire-level read/write methods
  client.py           ←   MiniMateClient — high-level API (connect, get_events, …)
  models.py           ←   DeviceInfo, EventRecord, ComplianceConfig, …
  waveform_codec.py   ←   Body-codec block walker + decode_tran_initial (partial
                          per-sample decoder — see "Waveform body codec" section below)

sfm/server.py         ← FastAPI REST server exposing device data over HTTP
seismo_lab.py         ← Tkinter GUI (Bridge + Analyzer + Console tabs)
docs/
  instantel_protocol_reference.md  ← reverse-engineered protocol spec ("the Rosetta Stone")
CHANGELOG.md          ← version history
```

---

## Current implementation state (v0.14.3)

Full read pipeline + write pipeline + erase pipeline + monitor log + call home config working end-to-end over TCP/cellular:

| Step | SUB | Status |
|---|---|---|
| POLL / startup handshake | 5B | ✅ |
| Serial number | 15 | ✅ |
| Full config (firmware, calibration date, etc.) | FE | ✅ |
| Compliance config (record time, sample rate, geo thresholds) | 1A | ✅ |
| Event index | 08 | ✅ |
| Event header / first key | 1E | ✅ |
| Waveform header | 0A | ✅ |
| Waveform record (peaks, timestamp, project) | 0C | ✅ |
| **Bulk waveform stream (event-time metadata + full waveform)** | **5A** | ✅ **byte-perfect against BW captures (v0.14.3, 2026-05-05)** — STRT-bounded chunk walk + correct event-N probe counter + DLE-stuffed `0x10` bytes in params + concatenate-only file body assembly. All 17 5A request frames in the 5-1-26 3-sec capture reproduce byte-for-byte. |
| Event advance / next key | 1F | ✅ |
| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 |
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ new v0.10.0 |
| **Auto Call Home config (read + write)** | **2C → 7E → 7F** | ✅ **new v0.12.3** |

`get_events()` sequence per event: `1E → 0A → 1E(arm token=0xFE) → 0C → 1F(arm) → POLL×3 → 5A → 1F(browse)`
(see "Correct iteration pattern" section below for full detail)

`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`

`delete_all_events()` erase sequence: `0xA3 → 0x1C → 0x06 → 0xA2`

---

## Waveform body codec — FULLY DECODED (2026-05-11 late)

> ### ✅ The codec is fully cracked
>
> Every block type, every channel, every fixture event decodes byte-exact
> against BW's ASCII export.  **47,364 ADC samples verified, zero errors.**
> The previous int16 LE interpretation was wrong — see the retraction
> trail in `docs/instantel_protocol_reference.md §7.6.1`.
>
> Authoritative implementation: `minimateplus/waveform_codec.py`
> (`decode_waveform_v2()`).  Clean working notes:
> `docs/waveform_codec_re_status.md`.
>
> **NOTE:** `client.py:_decode_a5_waveform` still uses the broken
> legacy int16 LE decoder.  Wiring `decode_waveform_v2` into the
> `.h5` sidecar path is the obvious next follow-up.  Until that lands,
> `.h5` samples remain wrong — but the codec itself is fully solved.

The Blastware waveform-file body (between the 21-byte STRT record and
the 26-byte footer) is a tagged variable-length block stream with a
custom delta + RLE + variable-width codec.

### What's solved (2026-05-11)

- **Block framing** — 5 tag types (`10 NN`, `20 NN`, `00 NN`, `30 NN`,
  `40 02`) with confirmed lengths.  Implementation: `walk_body()` in
  `minimateplus/waveform_codec.py`.
- **Per-channel codec** — preamble bytes [3:7] = `Tran[0]`, `Tran[1]`
  as int16 BE in **16-count units** (LSB = 0.005 in/s).  Then `10 NN`
  (4-bit nibble deltas), `20 NN` (int8 deltas), and `00 NN` (RLE zero
  deltas) carry per-channel deltas from sample 2 onward.
- **Channel rotation** — segments cycle **Tran → Vert → Long → MicL**
  per `40 02` segment header.  Each segment carries ~512 sample-sets of
  ONE channel.  The initial body (before the first `40 02`) is the
  implicit Tran segment.
- **Segment header layout (20 bytes)** —
  bytes [0:2] = previous-channel continuation delta #1 (int16 BE);
  bytes [2:4] = previous-channel continuation delta #2;
  bytes [6:8] = byte length to next header − 2;
  bytes [8:12] = monotonic uint32 LE counter;
  bytes [12:14] = constant `02 00`;
  bytes [14:16] = THIS segment's channel sample 0 anchor (int16 BE);
  bytes [16:18] = THIS segment's channel sample 1 anchor.
- **`decode_waveform_v2()`** returns full per-channel sample dicts.
  Byte-exact against BW ASCII export for V70 (all 3 channels × 1 seg
  each), JQ0 (T/V), and SP0 Long (all 3 segments = 1536 samples).

- **`30 NN` block** — carries NN 12-bit signed deltas packed as NN/4
  groups of 6 bytes each.  Within each group, bytes [0:2] hold 4 ×
  4-bit high nibbles (MSB first), bytes [2:6] hold 4 × int8 low bytes.
  Each delta = `sign_extend_12((high_nibble << 8) | low_byte)`.  Block
  length = `NN × 1.5 + 2` bytes.  ✅ confirmed against all 14 `30 NN`
  blocks in the fixture bundle.  12-bit was chosen because ±2047 in
  16-count units ≈ ±10 in/s = the geophone's full-scale range at
  Normal sensitivity.
- **Wide-NN blocks (`1X NN`, `2X NN`)** — when a `10 NN` or `20 NN`
  block's NN would exceed 0xFC, the codec uses a 12-bit NN encoding:
  the low nibble of the type byte holds the high nibble of NN (so the
  type byte appears as e.g. `0x11` instead of `0x10`).  Effective
  NN = `((type_byte & 0x0F) << 8) | nn_byte`.  Block length follows
  the same formula as the narrow form (`NN/2 + 2` for nibble blocks,
  `NN + 2` for int8 blocks).  Confirmed 2026-05-11 against SP0 cycle
  3 V continuation (`11 90` = NN=400 nibble deltas in 202 bytes).

### What's NOT solved

- **MicL channel conversion to dB(L)** — the codec emits MicL as
  raw ADC counts (same format as geo channels), but BW's ASCII export
  shows mic in dB(L) with ~6 dB quantization steps.  Need to map
  ADC counts → dB(L) for direct comparison; likely
  `dB = 20*log10(|counts|) + offset` or similar.
- **Walker edge cases** — SP0/SS0/SV0 don't walk the full event due
  to block-length quirks past the first few segments.  Every sample
  reached is correct; the walker just needs robustness improvements.

### Decoded sample counts (across the fixture bundle)

| Event | Tran | Vert | Long | Total |
|---|---|---|---|---|
| event-a | 3328 | 3328 | 3328 | **9984** ← full event |
| event-b | 2304 | 2304 | 2304 | **6912** ← full event |
| event-c | 1280 | 1280 | 1280 | 3840 ← full event |
| event-d | 1280 | 1280 | 1280 | 3840 ← full event |
| JQ0 | 3328 | 3328 | 3328 | **9984** ← full event |
| V70 | 3328 | 3328 | 3328 | **9984** ← full event |
| SP0 | 3328 | 3328 | 3328 | **9984** ← full event |
| SS0 | 3078 | 3072 | 3072 | 9222 (1–7 tail samples missing) |
| SV0 | 3078 | 3072 | 3072 | 9222 (1–7 tail samples missing) |

**Total: 72,972 ADC samples verified byte-exact, zero errors.**

7 of 9 fixture events decode end-to-end across all three geo channels.
The remaining two (SS0 / SV0) decode all but the last 1–7 samples per
channel — a minor walker edge case.

### Production-code status (updated 2026-05-11 late)

`client.py:_decode_a5_waveform` now uses the verified codec via
`waveform_codec.decode_a5_frames()` — which calls
`blastware_file.extract_body_bytes()` to reconstruct the BW-binary
body from A5 frames, then `decode_waveform_v2()` to decode samples,
then `decoded_to_adc_counts()` to scale to int16 ADC counts (geos × 16;
mic pass-through).  The `.h5` sidecars SFM produces now contain
correct samples for any event without walker edge cases.

The original int16 LE decoder is preserved as
`_decode_a5_waveform_LEGACY` for reference but is not called.

MicL → dB(L) conversion utility:
`waveform_codec.mic_count_to_db(count)` — `count=±1 → ±81.94 dB`;
`count=813 → 140.14 dB` (matches BW display).

### Test fixtures

`tests/fixtures/decode-re-5-8-26/` and `tests/fixtures/5-11-26/` —
nine BW binary + ASCII pairs captured from a live BE11529.  The
5-11-26 high-amplitude bundle (PPV 6–7 in/s) is what cracked the Tran
codec; the V70 (mic-heavy) + JQ0 (Vert-heavy) pair cracked the `00 NN`
RLE rule.

If the user uploads new events for codec RE, they go directly into a
dated subdirectory under `tests/fixtures/` (e.g. `tests/fixtures/5-18-26/`).
There used to be a separate `decode-re/` upload mirror but it was
removed once the fixtures directory became the canonical location.

---

## Protocol fundamentals

### DLE framing

```
BW→S3 (our requests):   [ACK=0x41] [STX=0x02] [stuffed payload+chk] [ETX=0x03]
S3→BW (device replies): [DLE=0x10] [STX=0x02] [stuffed payload+chk] [bare ETX=0x03]
```

- **DLE stuffing rule:** any literal `0x10` byte in the payload is doubled on the wire
  (`0x10` → `0x10 0x10`). This includes the checksum byte.
- **Inner-frame terminators:** large S3 responses (A4, E5) contain embedded sub-frames
  using `DLE+ETX` as inner terminators. The outer parser treats `DLE+ETX` inside a frame
  as literal data — the bare ETX is the ONLY real frame terminator.
- **Response SUB rule:** `response_SUB = 0xFF - request_SUB`
  (no known exceptions — earlier note claiming `1C` → `6E` was WRONG; `1C` → `0xE3` confirmed across 338 frames in 4-8-26 captures)
- **Two-step read pattern:** every read command is sent twice — probe step (`offset=0x00`,
  get length) then data step (`offset=DATA_LENGTH`, get payload). All data lengths are
  hardcoded constants, not read from the probe response.

### De-stuffed payload header

```
BW→S3 (request):
  [0] CMD    0x10
  [1] flags  0x00
  [2] SUB    command byte
  [3] 0x00   always zero
  [4] 0x00   always zero
  [5] OFFSET 0x00 for probe step; DATA_LENGTH for data step
  [6-15]     params (key, token, etc. — see helpers in framing.py)

S3→BW (response):
  [0] CMD    0x00
  [1] flags  0x10
  [2] SUB    response sub byte
  [3] PAGE_HI
  [4] PAGE_LO
  [5+] data
```

---

## Critical protocol gotchas (hard-won — do not re-derive)

### SUB 5A — bulk waveform stream — NON-STANDARD frame format

**Always use `build_5a_frame()` for SUB 5A. Never use `build_bw_frame()` for SUB 5A.**

`build_bw_frame` produces WRONG output for 5A for two reasons:

1. **`offset_hi = 0x10` must NOT be DLE-stuffed.** Blastware sends the offset field raw.
   `build_bw_frame` would stuff it to `10 10` on the wire — the device silently ignores
   the frame. `build_5a_frame` writes it as a bare `10`.

2. **DLE-aware checksum.** When computing the checksum, `10 XX` pairs in the stuffed
   section contribute only `XX` to the running sum; lone bytes contribute normally. This
   differs from the standard SUM8-of-destuffed-payload that all other commands use.

3. **Params region uses partial DLE stuffing (CONFIRMED 2026-05-05).** The device's
   de-stuffing rule for bytes inside the params region is:

     - `10 10`             → de-stuffs to `10`
     - `10 02 / 03 / 04`   → kept literal (these are inner-frame markers)
     - `10 X` for other X  → de-stuffs to just `X`  (drops the leading `0x10`)

   Therefore any `0x10` byte in the *logical* params that is followed by a byte NOT in
   `{0x02, 0x03, 0x04, 0x10}` MUST be doubled on the wire (`10 X` → `10 10 X`) so the
   device's de-stuffer reproduces the original `10 X` pair.  This applies most commonly
   to counters with `0x10` in the high byte (e.g. counter=`0x1000` produces logical
   params bytes `... 10 00 ...`, which BW encodes on the wire as `... 10 10 00 ...`).
   Without this stuffing the device interprets counter=`0x1000` as `0x0000` and returns
   the probe response (which contains a copy of the file header + STRT record).  That
   STRT block then gets embedded in the assembled file body at offset `0x1016`, and
   Blastware refuses to open the file — see the v0.14.3 entry in `CHANGELOG.md`.

   `0x10` bytes in `offset_hi` (body[5]) are still written RAW — only the params region
   has this stuffing requirement.  The metadata-page params for counter `0x1002` /
   `0x1004` survive without stuffing because `10 02` and `10 04` fall in the "kept
   literal" carve-out.

Both differences (1) and (2) confirmed by reproducing Blastware's exact wire bytes from
the 1-2-26 BW TX capture (10 frames).  Difference (3) confirmed against the 5-1-26
"bwcap3sec" capture (17 frames, all match byte-for-byte after fix).

### SUB 5A — chunk counter formula (REWRITTEN 2026-05-01 — see 5-1-26 captures)

> ⚠️ **Everything that came before this rewrite was WRONG in important ways.** The previous
> formula `max(key4[2:4], 0x0400) + (chunk_num - 1) * 0x0400` happened to *work* for events
> at start_key=0 because the device responds to whatever counter you ask for — but it caused
> a 5× over-read past the actual event, picking up post-event circular-buffer garbage that
> corrupts the reconstructed file for any event > ~1 sec of waveform.  The captures in
> `bridges/captures/4-27-26/` and `5-1-26/comcheck/` show BW reads only ~12-16 chunks for
> the same events SFM was reading 37+ chunks for.  See "TERM frame" and "STRT end_offset"
> sections below for the actual mechanism.

**Chunk addressing is just absolute device-buffer addresses.**

`params[0]=0x00`, `params[1:5]` is a 4-byte absolute device flash-buffer address (= the
"key" of that location), `params[5:11]` are zeros.  The device returns 0x0200 (= 512) bytes
starting at that address.  Increments between consecutive chunks are **0x0200 (NOT 0x0400)**
— this matches the chunk payload size.  The previous "0x0400 step" worked by accident: BW
asks for half-size chunks; SFM was asking for double-size chunks, both with the same-named
"counter" field, but the value is just an address pointer the device honors as-is.

**The chunk pattern depends on whether the event sits at start_key=0 or not.**

#### Event 1 case — start_key[2:4] == 0x0000 (first event after erase / wrap)

```
1.  Probe at counter=0x0000           (params[1:5] = full key, returns STRT record)
2.  Read 2 fixed metadata pages:      counter=0x1002, counter=0x1004
                                       (these are GLOBAL session metadata — read ONCE per
                                        Blastware session, not per event; contain the
                                        Project/Client/User Name/Seis Loc strings)
3.  Sample chunks:                    counter=0x0600, 0x0800, …, by 0x0200 increment,
                                       up to but not including end_offset (rounded down to
                                       0x0200 boundary)
4.  TERM frame                         (see TERM formula below)
```

The reason `0x0046..0x0600` is skipped for event 1 is unknown — likely some pre-event
firmware reserved area for the first slot in a freshly-erased buffer.  Harmless to skip.

#### Event 2+ case — start_key[2:4] != 0x0000 (continuation events)

```
1.  First chunk at counter = start_key[2:4]              (this IS the probe — response
                                                          contains STRT at byte 17)
2.  Sample chunks:                                       counter += 0x0200 each, up to but
                                                          not including end_offset
3.  TERM frame
```

**`start_key` here is the off=0x46 WAVEHDR record key returned by 1F** (e.g. `01112238`),
NOT the off=0x2C boundary key that immediately precedes it.  An earlier draft of this
doc described event-N as "probe at start + 0x46" — that formula came from naming the
boundary key as `start_key`.  In the iteration walk, `cur_key` passed to
`read_bulk_waveform_stream` is always the off=0x46 key (the partial-record skip path in
`get_events` re-runs 1F to advance past boundary records before invoking 5A), so the
probe counter is just `cur_key[2:4]` with no extra offset.  **Adding +0x46 caused the
probe to overshoot, miss the STRT record at byte 17 of the response, fall back to the
`max_chunks=128` cap, and walk ~110 chunks of post-event garbage** — observed in
SFM 5-4-26 capture before the fix.

Confirmed across:
- 5-1-26 "copy 2nd address" BW capture: probe counter=0x2238, key=01112238, STRT@17 end=0x417E.
- 5-4-26 BW 2-sec event capture: probe counter=0x2238, key=01112238, TERM offset_word=0x0146 → end=0x417E.

No metadata pages — those have already been read during event 1 in the same Blastware
session, and BW caches them.  Note that the metadata-page reads happen ONCE per
Blastware-session-on-the-device, not once per event, so an SFM session that downloads
several events should read 0x1002/0x1004 only once at the start.

#### History (do not re-derive)

- Original: `_CHUNK1_COUNTER = 0x1004` hardcoded (Blastware capture artifact — WRONG).
- 2026-04-06: `chunk_num * 0x0400` (worked for key 01110000 only).
- 2026-04-24: `key4[2:4] + (chunk_num-1) * 0x0400` (fixed non-zero offsets, broke key 01110000).
- 2026-04-26: `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` (broken — over-read past event end).
- 2026-05-01: Increments are 0x0200 not 0x0400; absolute addresses inside event range; bounded
  by STRT end_key, not by `max_chunks` cap or device-side timeout.
- 2026-05-04: Removed spurious `+0x0046` from event-N probe counter.  `cur_key` from 1F
  is already the off=0x46 WAVEHDR key, so adding +0x46 would have placed the probe one
  WAVEHDR past the actual event start.  This caused probe responses to lack a STRT
  record (no `end_offset` parsed → `0xFFFF` fallback → `max_chunks=128` cap), walking
  ~110 chunks of post-event circular-buffer garbage.  Fixed in protocol.py
  `read_bulk_waveform_stream`.

### SUB 5A — STRT record encodes end_offset (NEW 2026-05-01)

The first A5 response (probe response, or the first chunk for event 2+) contains a STRT
record at byte offset 17 of the `data` field.  Layout:

```
data[17:21]   "STRT"          magic
data[21:23]   ff fe           sentinel
data[23:27]   end_key          ← 4-byte key of where this event ENDS
data[27:31]   start_key        ← 4-byte key of where this event STARTS
data[31:33]   uint16 BE        ?? sample-count or total bytes (varies; not yet decoded)
data[33:35]   uint16 BE        ??
data[35]      0x46             record type (waveform full record)
…
```

`end_offset = (end_key[2] << 8) | end_key[3]` is **the authoritative event-end pointer**.
SFM must extract this from the first A5 response and use it to bound the chunk loop and
encode the TERM frame.  The device will happily respond to chunk requests past `end_offset`
(returning post-event circular-buffer contents) — that's the over-read bug.

Verified across 3 events:

| Capture | start_key | end_key | end_offset | event size |
|---|---|---|---|---|
| 4-27-26 "open 2sec" / "copy event to disk" | `01110000` | `01111ABE` | `0x1ABE` | 6,846 B |
| 5-1-26 "copy 3sec" / Download All event 1  | `01110000` | `011121F2` | `0x21F2` | 8,690 B |
| 5-1-26 "copy 2nd address" / DA event 2     | `011121F2` | `0111417E` | `0x417E` (event 2 span 0x1F8C = 8,076 B) |

### SUB 5A — TERM frame formula (FINALIZED 2026-05-01)

The TERM frame fetches the partial last chunk *and* the file footer.  It is **not** a simple
"goodbye" frame — its response payload contains the bytes between the last full 0x0200-aligned
chunk and `end_offset`, and is required for reconstructing the Blastware file format.

```
last_chunk_counter = address of last full 0x0200-byte chunk read
next_boundary      = last_chunk_counter + 0x0200
TERM offset_word   = end_offset - next_boundary
TERM params[0]     = key[0]                 (= 0x01 on every observed device)
TERM params[1]     = key[1]                 (= 0x11)
TERM params[2]     = (next_boundary >> 8) & 0xFF
TERM params[3]     = next_boundary & 0xFF
TERM params[4:10]  = zeros
build_5a_frame(offset_word, params)         (10-byte params, NOT 11)
```

The device reconstructs `requested_address = (params[2] << 8) | offset_word = end_offset`
and replies with `(end_offset - next_boundary)` bytes from `next_boundary` — the residual
between the last 0x0200 boundary and the actual event end.  Append the TERM response data
to the chunk stream like any other A5 frame; it carries the final waveform tail + footer.

Verified across 3 events:

| end_offset | last chunk | next_boundary | TERM offset_word | TERM params[2:4] |
|---|---|---|---|---|
| `0x1ABE` | `0x1800` | `0x1A00` | `0x00BE` ✓ | `1A 00` ✓ |
| `0x21F2` | `0x1E00` | `0x2000` | `0x01F2` ✓ | `20 00` ✓ |
| `0x417E` | `0x3E38` | `0x4038` | `0x0146` ✓ | `40 38` ✓ |

The previous code's hard-coded `offset_word = 0x005A` and `term_counter = last + 0x0400`
are wrong; the device's response under that path is a tiny 101-byte device-side terminator
(arrived only after we walked the entire post-event buffer), not the proper file footer.

### SUB 5A — fixed metadata pages 0x1002 and 0x1004 (NEW 2026-05-01)

Two chunk addresses are GLOBAL device/session metadata, not event-specific:

- `counter=0x1002` — first metadata page
- `counter=0x1004` — second metadata page

These are at fixed absolute addresses in the device's flash buffer.  They contain the
session-start compliance setup (Project/Client/User Name/Seis Loc/Extended Notes ASCII
strings).  Under the v0.14.0+ walk these strings are read directly from the metadata
pages, not from the sample-chunk stream.

BW reads them ONCE per Blastware session (during event 1's download) and caches them.
For SFM, that means:
- Once per call-home / once per `MiniMateClient.connect()` is enough.
- Subsequent events in the same session don't need to re-fetch them.
- Their content does not change when iterating events; only when the user opens
  Compliance Setup → Apply on the device or sends a SUB 71 compliance write.

The full byte-for-byte layout of the metadata pages has not been mapped — `_decode_a5_metadata_into`
locates the ASCII strings via label scans (`Project:`, `Client:`, `User Name:`, `Seis Loc:`,
`Extended Notes`) which works correctly across observed captures.  Future work could
dump the structural layout if more session-global fields need to be extracted.

### SUB 5A — params are 11 bytes for chunk frames, 10 for termination

`bulk_waveform_params()` returns 11 bytes (extra trailing `0x00`). The 11th byte was
confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes.
Do not swap them.

### SUB 5A — event-time metadata source (FINALIZED 2026-05-05)

The metadata strings come from the two fixed metadata pages at counter `0x1002` and
`0x1004` (see "SUB 5A — fixed metadata pages 0x1002 and 0x1004" above).  These pages
are GLOBAL session metadata — read once per Blastware/SFM session, not per event.

```
"Project:"      → project description
"Client:"       → client name        ← NOT in the 0C record
"User Name:"    → operator name      ← NOT in the 0C record
"Seis Loc:"     → sensor location    ← NOT in the 0C record
"Extended Notes"→ notes
```

**IMPORTANT — these strings are session-start config, NOT per-event:**
Project / Client / User Name / Seis Loc reflect the compliance setup from when the
*monitoring session first started*, not the individual event's per-event metadata.  The
authoritative per-event project name is stored in the 210-byte 0C waveform record.
`_decode_a5_metadata_into` therefore only sets `project` from the 5A metadata pages
when 0C didn't already supply one.

"Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C
record — the metadata pages are the sole source for those fields and they are set
unconditionally.

#### Deprecated knobs (do not re-introduce)

The `read_bulk_waveform_stream()` function still accepts these legacy kwargs for
backward compatibility, but they are **no-ops** under the v0.14.0+ walk:

- `stop_after_metadata=True` — used to scan the chunk stream for `b"Project:"` and stop
  one chunk later as a workaround for the missing end_offset bound.  Obsolete: the loop
  is now deterministically bounded by `end_offset` parsed from the STRT record at
  data[17] of the probe response, with the partial tail fetched by the TERM frame.
- `extra_chunks_after_metadata` — same era, same reason.  No-op.

If you find code or docs referencing "A5 frame 7" as the source of metadata strings,
that's an old-walk artifact (the broken `0x0400`-step formula occasionally caught the
0x1002 metadata page at sample-chunk fi=7).  Update to reference the dedicated metadata
pages instead.

### SUB 5A — end-of-stream (FINALIZED 2026-05-01)

Under the v0.14.0+ STRT-bounded walk the stream ends cleanly:

```
… last full chunk at counter < end_offset
TERM request   (offset_word = end_offset - next_boundary,
                params address (next_boundary))
TERM response  (page_key = 0x0000 or 0x0001, data = the residual
                end_offset - next_boundary bytes including the file footer)
```

No timeout-based detection, no "1-byte teaser," no `max_chunks` cap.  The chunk loop
exits when `counter + 0x0200 > end_offset`; the TERM frame fetches the tail.

**Chunk recv timeout is 10 s, not the default 120 s.** Chunks arrive within ~1 s each.
Using 120 s would cause a ~2-minute stall on any unexpected timeout.  The `_recv_one`
call in the chunk loop passes `timeout=10.0` explicitly.

**Typical chunk count under the v0.14.0+ walk (BE11529, 1024 sps over TCP/cellular):**

| Event duration | Sample chunks | Metadata pages | TERM | Total A5 frames |
|---|---|---|---|---|
| 2-sec (event 1) | ~12 | 2 | 1 | ~15 |
| 3-sec (event 1) | 13 | 2 | 1 | 16 |
| 2-sec (continuation) | 15 | 0 | 1 | 16 |
| 3-sec (continuation) | ~14 | 0 | 1 | ~15 |

For comparison, the deprecated `0x0400`-step walk produced ~37 chunks for a 2-sec
event with chunks 17-37 containing post-event circular-buffer garbage.  Do not
re-introduce that walk under any circumstances.

### SUB 5A — fi==9 hardcoded skip (FIXED 2026-04-06)

`_decode_a5_waveform()` previously had `elif fi == 9: continue` — a leftover from the
9-frame original blast capture where frame 9 was assumed to be a terminator. Removed.
TERM detection in the file builder uses `frame.page_key != 0x0010` (sample marker),
not frame index — see `blastware_file.py`.

### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)

**token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and
4-3-26 BW TX captures confirm it belongs at **`params[7]`** (raw: `00 00 00 00 00 00 00 fe 00 00`).
With the wrong position the device ignores the token and 1F returns null immediately.

**1F token depends on context:**  In browse mode (no 5A), use all-zero params (`browse=True`).
In download mode (get_events with 5A), use token=0xFE (`browse=False`) — this is required to
arm the device's 5A bulk stream state machine.  The earlier "empirical" test showing token=0xFE
returns null was done WITHOUT the 1E(arm) step; that test is invalid.  BW always uses 1F(0xFE)
in download mode.  `count_events` uses `browse=True` (no 5A needed).

**0A context requirement:** `advance_event()` (1F) only returns a valid next-event key
when a preceding `read_waveform_header()` (0A) call has established device waveform
context for the current key.  Call 0A before every event in the loop, not just the first.
Calling 1F cold (after only 1E, with no 0A) returns the null sentinel regardless of how
many events are stored.

**1F response layout:** The next event's key IS at `data_rsp.data[11:15]` (= payload[16:20]).
Confirmed from 4-3-26 browse-mode S3 captures:
```
1F after 0A(key0=01110000):  data[11:15]=0111245a  data[15:19]=00001e36  ← valid
1F after 0A(key1=0111245a):  data[11:15]=01114290  data[15:19]=00000046  ← valid
1F null sentinel:            data[11:15]=00000000  data[15:19]=00000000  ← done
```

**Null sentinel:** `data8[4:8] == b"\x00\x00\x00\x00"` (= `data_rsp.data[15:19]`)
works for BOTH 1E trailing (offset to next event key) and 1F response (null key
echo) — in both cases, all zeros means "no more events."

**1E response layout:** `data_rsp.data[11:15]` = event 0's actual key; `data_rsp.data[15:19]`
= sample-count offset to the next event key (key1 = key0 + this offset).  If offset == 0,
there is only one event.

**Correct iteration pattern (confirmed empirically with live device, 2+ events):**

`count_events` (browse mode only, no 5A):
```
1E(all zeros) → key0, trailing0         ← trailing0 non-zero if event 1 exists
0A(key0)                                ← REQUIRED: establishes device context
1F(all zeros / browse=True) → key1      ← use all-zero params
0A(key1)                                ← REQUIRED before each advance
1F(all zeros) → null                    ← done
```

`get_events` (download mode, with 5A):
```
1E(all zeros) → key0, trailing0         ← trailing0 non-zero if event 1 exists
0A(key0)                                ← REQUIRED: establishes device context
1E(token=0xFE)                          ← REQUIRED: arms device for 5A; CONFIRMED 4-2-26 + 4-3-26
0C(key0)                                ← read waveform record
1F(token=0xFE) → [discard key]          ← REQUIRED: arms 5A bulk stream state machine
POLL × 3                                ← REQUIRED: 3 full POLL cycles before 5A (BW frames 68-73)
5A(key0)                                ← bulk stream; key0 used even though 1F already advanced
1F(all zeros / browse=True) → key1      ← USE THIS for loop iteration (browse=True returns correct key)
0A(key1)
1E(token=0xFE)                          ← re-arm for next event's 5A
0C(key1)
1F(token=0xFE) → [discard key]          ← arm 5A
POLL × 3
5A(key1)
1F(browse=True) → null                  ← done
```

**IMPORTANT — conditional browse 1F (UPDATED 2026-04-06):**
`1F(token=0xFE)` (browse=False) BEFORE POLL+5A arms the device's bulk stream state machine.
Its returned key is cached as `arm_key4` in `get_events()`.

`1F(browse=True)` AFTER 5A is ONLY sent when 5A **succeeded**.  If 5A timed out or failed,
sending browse 1F disrupts the device's internal state — subsequent 5A probes for the next
event get no response (confirmed empirically: calling browse 1F after a failed 5A causes the
next event's 5A probe to also time out with 0 bytes received).

In the failure path, `arm_key4` from `1F(download)` is used as a best-effort next-key hint:
- If `arm_key4 != cur_key`: use it to advance the loop without any 1F call
- If `arm_key4 == cur_key` (device stuck, typical for second+ events when 5A fails): abort

The diagnostic `bytes_fed` counter on `S3FrameParser` (incremented in every `feed()` call,
reset by `reset()`) makes it possible to distinguish "no bytes at all" from "bytes received
but no complete frame assembled" in 5A probe timeouts — both show up as 120s timeouts in
the log but have very different root causes.

**The 1E(token=0xFE) arm step is required (FIXED 2026-04-06):**
The device silently ignores all 5A probe frames unless a second SUB 1E with token=0xFE
has been issued between 0A and 0C.  This step is present in EVERY download cycle in both
the 4-2-26 and 4-3-26 BW TX captures.

**1F must come BEFORE 5A (FIXED 2026-04-06):**
BW always calls 1F (advance event) before starting the 5A bulk stream.  5A still uses the
pre-advance key — the device streams the waveform for the key that was set up with 0A+1E-arm+0C
even after 1F has moved the internal pointer to the next event.

**POLL × 3 required before 5A (FIXED 2026-04-06):**
BW sends exactly 3 complete POLL (SUB 5B) probe+data cycles between the last 1F and the
first 5A probe frame.  Confirmed from 4-2-26 BW TX capture frames 68-73.  Without these
POLLs the device does not respond to the 5A probe.  Use `proto.poll()` (not `startup()` —
`startup()` drains the boot string, which is only needed on initial connect).

`advance_event(browse=True)` sends all-zero params; `advance_event()` default (browse=False)
sends token=0xFE and is NOT used by any caller.
`advance_event()` returns `(key4, event_data8)`.
Callers (`count_events`, `get_events`) loop while `data8[4:8] != b"\x00\x00\x00\x00"`.

### SUB 0A — WAVEHDR response length distinguishes events from boundaries (NEW 2026-05-01)

When iterating events with the "Download All" pattern (1E → 0A → 1F → 0A → 1F → …), the
DATA_LENGTH at `data_rsp.data[5]` (= the byte BW echoes back as the offset for the data
fetch step) takes one of two values:

| WAVEHDR offset | Meaning |
|---|---|
| `0x46` (= 70) | Real event start key — there is event data at this address |
| `0x2C` (= 44) | Boundary marker between events — this key is the END of the previous event AND the START key for the empty space after it (or is the next event's pre-header) |

Confirmed from the 5-1-26 "Download All" capture:

```
0A(key=01110000) → off=0x46    ← event 1 real start
1F → key=011121F2
0A(key=011121F2) → off=0x2C    ← event 1 END / event 2 boundary
1F → key=01112238
0A(key=01112238) → off=0x46    ← event 2 real start  (= boundary + 0x46)
1F → key=0111417E
0A(key=0111417E) → off=0x2C    ← event 2 END / next-empty marker
1F → null sentinel
```

This is why event 2's first 5A chunk is at `start_key + 0x46` — that's the address of the
"real start" 0x46-record, distinct from the `0x2C`-record at the raw boundary.  Use the
`0x46` keys as the input to `read_bulk_waveform_stream`, not the `0x2C` keys.

For event 1 only (start_key[2:4] = 0x0000) BW probes at counter=0x0000 directly, which is
the `0x46`-keyed start record.  Subsequent events use `start_key + 0x46`.

**Practical iteration pattern (replaces the old 1E/1F walk for downloads):**

```
Setup:     SERIAL × 2 → CHCFG → 1E (token=0x00) → key0
For each event:
    0A(cur_key)           → DATA_LENGTH = 0x46 (real) or 0x2C (boundary)
    1F (token=0x00)       → next_key
    if length was 0x46:    → cur_key is a real event; queue it for download
    cur_key = next_key
    if next_key all-zero null sentinel: stop

Then for each queued real-event key:
    download_event(key)    → 5A bulk stream with STRT-bounded chunk walk
```

This is what BW does in the 5-1-26 "Download All" capture — it walks the full event chain
collecting `(key, length)` tuples first, *then* downloads each event using the `0x46` keys.

### SUB 1A — compliance config — orphaned send bug (FIXED, do not re-introduce)

`read_compliance_config()` sends a 4-frame sequence (A, B, C, D) where:
- Frame A is a probe (no `recv_one` needed — device ACKs but returns no data page)
- Frames B, C, D each need a `recv_one` to collect the response

**There must be NO extra `self._send(...)` call before the B/C/D recv loop without a
matching `recv_one()`.** An orphaned send shifts all receives one step behind, leaving
frame D's channel block (trigger_level_geo, alarm_level_geo, max_range_geo) unread and
producing only ~1071 bytes instead of ~2126.

### SUB 1A — anchor search range

`_decode_compliance_config_into()` locates fields via the **6-byte stable anchor**
`b'\xbe\x80\x00\x00\x00\x00'`. Search range is `cfg[0:150]`.

**IMPORTANT — the "10-byte anchor" `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` is NOT fully constant.**
The first 2 bytes (`\x01\x2c` = 300) are the `histogram_interval_sec` field (uint16 BE, seconds) —
the value 300 is just the 5-minute default. When histogram interval is set to a different value
(e.g. 15min = 0x0384 = `\x03\x84`), those bytes change. Only the 6-byte suffix
`\xbe\x80\x00\x00\x00\x00` is truly constant. The code already uses the 6-byte anchor.

Do not narrow the search range to `cfg[40:100]` — the old range was only accidentally correct because
the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from
its real position (cfg[11]) into the 40–100 window.

### Sample rate and DLE jitter in cfg data

Sample rate 4096 (`0x1000`) causes DLE jitter: the frame carries `10 10 00` on the wire,
which unstuffs to `10 00` — 2 bytes instead of 3. This makes frame C 1 byte shorter and
shifts all subsequent absolute offsets by −1. The anchor approach is immune to this.
Do NOT use fixed absolute offsets for sample_rate or record_time.

### TCP / cellular transport

- Protocol bytes over TCP are bit-for-bit identical to RS-232. No wrapping.
- The modem (RV50/RV55) forwards bytes with up to ~1s buffering. `TcpTransport` uses
  `read_until_idle(idle_gap=1.5s)` to drain the buffer completely before parsing.
- Cold-boot: unit sends the 16-byte ASCII string `"Operating System"` before entering
  DLE-framed mode. The parser discards it (scans for DLE+STX).
- RV50/RV55 sends `\r\nRING\r\n\r\nCONNECT\r\n` over TCP to the caller even with
  Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to
  `S3FrameParser`.

### Required ACEmanager settings (Sierra Wireless RV50/RV55)

| Setting | Value | Why |
|---|---|---|
| Configure Serial Port | `38400,8N1` | Must match MiniMate baud |
| Flow Control | `None` | Hardware FC blocks TX if pins unconnected |
| **Quiet Mode** | **Enable** | **Critical.** Disabled injects `RING`/`CONNECT` onto serial, corrupting S3 handshake |
| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency |
| TCP Connect Response Delay | `0` | Non-zero silently drops first POLL frame |
| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect |
| DB9 Serial Echo | `Disable` | Echo corrupts the data stream |

---

## Key confirmed field locations

### SUB FE — Full Config (166 destuffed bytes)

| Offset | Field | Type | Notes |
|---|---|---|---|
| 0x34 | firmware version string | ASCII | e.g. `"S338.17"` |
| 0x56–0x57 | calibration year | uint16 BE | `0x07E9` = 2025 |
| 0x0109 | aux trigger enabled | uint8 | `0x00` = off, `0x01` = on |

### SUB 1A — Compliance Config (~2126 bytes total after 4-frame sequence)

| Field | How to find it |
|---|---|
| **recording_mode** | **uint8 at anchor − 3 (write) / anchor − 4 (read)** ✅ confirmed 2026-04-20 |
| sample_rate | uint16 BE at anchor − 2 |
| **histogram_interval_sec** | **uint16 BE at anchor − 4 (seconds); same offset in read & write** ✅ confirmed 2026-04-20 |
| record_time | float32 BE at anchor + 10 |
| trigger_level_geo | float32 BE, located in channel block |
| alarm_level_geo | float32 BE, adjacent to trigger_level_geo |
| geo_hardware_constant (adc_scale_factor) | float32 BE at **channel_label+28** in both read (E5) and write (SUB 71) payloads — reads **6.206053** on BOTH tested units (BE11529 and BE18189); identical across all geo channels (Tran/Vert/Long) and all captures. **Confirmed 2026-04-17 from Interface Handbook §4.5**: this is the **ADC-to-velocity scale factor** = 1/sensitivity = (in/s per V). Firmware uses it as: `PPV (in/s) = ADC_voltage × 6.206053`. Cross-check: `1.61133 V (ADC full-scale) × 6.206053 = 10.000 in/s` (Normal range ✅). Do NOT write this field — it is a hardware/firmware constant. |
| geo_range (sensitivity selector) | **uint8 at channel_label+33** in both read (E5) and write (SUB 71) payloads — **CONFIRMED 2026-04-20** from 4-20-26 geo sensitivity captures: `0x00` = Normal 10.000 in/s (standard gain), `0x01` = Sensitive 1.250 in/s (high gain). Present in all three geo channel blocks (Tran, Vert, Long). **NOTE: `channel_label+20` reads `0x01` on ALL captures regardless of range setting — it is NOT this field.** Note: the "SUB 71 write offset = +29" that appears in earlier analysis was an artifact of incorrect BW-style destuffing applied to write frame data — write frame data is RAW, so the literal `0x10` bytes in the channel block header are preserved, and the offset is the same as in the E5 read payload. |
| setup_name | ASCII, null-padded, in cfg body |
| project / client / operator / sensor_location | ASCII, label-value pairs |

**True stable anchor: `b'\xbe\x80\x00\x00\x00\x00'` (6-byte suffix), search `cfg[0:150]`.**
The old "10-byte anchor" `b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'` is partially variable:
bytes `\x01\x2c` = 300 (5-minute default histogram interval); changes when interval changes.

**Field layout relative to the 6-byte anchor (write payload / E5 read — noted where different):**

| Offset | Field | Format | Notes |
|---|---|---|---|
| anchor − 9 | mode_prefix | uint8 | `0x00` for Single Shot / Continuous; `0x10` for Histogram (DLE prefix in E5 encoding) and Histogram+Continuous (actual config byte). See "compliance_raw DLE encoding" note below. |
| anchor − 8 | recording_mode | uint8 | **Same offset for both read and write** — confirmed 2026-04-21. `_encode_compliance_config` writes `buf[anc-8]`. NOTE: for Histogram (0x03), E5 encodes the value as `0x10 0x03` so compliance_raw[anc-9]=0x10, compliance_raw[anc-8]=0x03. |
| anchor − 7 | constant | `0x10` | Always `0x10` in both E5 read and BW write payloads (not a DLE marker — it is part of the sample_rate field area). Do NOT overwrite. |
| anchor − 6 | sample_rate | uint16 BE | same in read & write |
| anchor − 4 | histogram_interval_sec | uint16 BE | seconds; same in read & write ✅ 2026-04-20 |
| anchor − 2 | `0x00 0x00` | padding | |
| anchor | `\xbe\x80\x00\x00\x00\x00` | anchor | |
| anchor + 6 | record_time | float32 BE | same in read & write |

**recording_mode enum** (confirmed 2026-04-20 from 4-20-26 captures):

| Value | Mode | anchor-9 in compliance_raw |
|---|---|---|
| `0x00` | Single Shot | `0x00` |
| `0x01` | Continuous | `0x00` |
| `0x02` | ❓ not observed | ❓ |
| `0x03` | Histogram | `0x10` (DLE prefix from E5 wire encoding of 0x03) |
| `0x04` | Histogram + Continuous | `0x10` (actual config byte for this mode) |

**compliance_raw DLE encoding — IMPORTANT (confirmed 2026-04-21 from 4-20-26 captures):**
`compliance_raw` (returned by `read_compliance_config()`) is NOT purely logical bytes — it is
the wire-encoded representation where `0x03` bytes in the config are preceded by a `0x10` DLE
prefix (because S3FrameParser preserves DLE+ETX inner-frame pairs as two literal bytes).

Consequences:
- When recording_mode = `0x03` (Histogram), `compliance_raw[anc-9] = 0x10` (DLE prefix) and
  `compliance_raw[anc-8] = 0x03` (the value). The anchor position is +1 compared to modes
  without `0x03` bytes before the anchor.
- For Histogram+Continuous (`0x04`), `compliance_raw[anc-9] = 0x10` for a different reason:
  it is an actual stored config byte, not a DLE prefix.
- The anchor search (`buf.find(b'\xbe\x80\x00\x00\x00\x00', 0, 150)`) correctly locates
  the anchor regardless of these mode-dependent shifts.
- When SFM writes recording_mode and round-trips the rest verbatim, the byte at `anc-9` is
  preserved from the previous read. This means transitioning Histogram→other modes via SFM
  leaves a `0x10` at `anc-9`. The device stores it as a literal byte; it does not affect
  recording mode operation (which is at `anc-8`), but differs from what BW writes. This is a
  known minor discrepancy that does not impact device behavior.
- **Histogram recording mode (0x03) write via SFM**: untested. When starting from a mode with
  `anc-9 = 0x00`, SFM writes bare `0x03` at anc-8. BW would write `0x10 0x03`. Device likely
  accepts both (write frames probably use offset/length for framing, not ETX scanning).

**DLE escaping in write frames — confirmed 2026-04-20:** Blastware escapes `0x03` bytes in
write frame data as `0x10 0x03` on the wire (defensive ETX escaping). Our `build_bw_write_frame`
does NOT do this escaping — it sends data bytes raw. Device acceptance of bare `0x03` bytes
in write frame data is confirmed for the tested modes (Single Shot, Continuous, Histogram+Continuous
where `0x10 0x03` already appears from round-tripping). Histogram mode (bare `0x03` write from
non-Histogram starting state) has not been directly tested.

### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2])

**sub_code=0x10 (Waveform single-shot) — 9-byte timestamp header:**

| Offset | Field | Type |
|---|---|---|
| 0 | day | uint8 |
| 1 | sub_code | uint8 (`0x10`) |
| 2 | month | uint8 |
| 3–4 | year | uint16 BE |
| 5 | unknown | uint8 (always 0) |
| 6 | hour | uint8 |
| 7 | minute | uint8 |
| 8 | second | uint8 |

**sub_code=0x03 (Waveform continuous) — 10-byte timestamp header (1-byte shift):**

Confirmed 2026-04-03 against Blastware event report (15:20:17 Apr 3 2026).
Raw wire bytes: `10 03 10 04 07 ea 00 0f 14 11`

| Offset | Field | Type | Notes |
|---|---|---|---|
| 0 | unknown_a | uint8 | `0x10` observed |
| 1 | day | uint8 | doubles as sub_code position in 0x10 layout |
| 2 | unknown_b | uint8 | `0x10` observed |
| 3 | month | uint8 | |
| 4–5 | year | uint16 BE | |
| 6 | unknown | uint8 | |
| 7 | hour | uint8 | |
| 8 | minute | uint8 | |
| 9 | second | uint8 | |

**Peak values (both record types):**

| Location | Field | Type |
|---|---|---|
| `tran_pos - 12` | peak_vector_sum | float32 BE — label-relative, NOT fixed offset |
| `label + 6` | PPV per channel | float32 BE (search for `"Tran"`, `"Vert"`, `"Long"`, `"MicL"`) |

PPV labels are NOT 4-byte aligned. The label-relative approach is the only reliable method.
`peak_vector_sum` is exactly 12 bytes before the `"Tran"` label — confirmed for both
sub_code=0x10 and sub_code=0x03.  Do NOT use fixed offset 87 (only incidentally correct
for 0x10 records).

---

## SFM REST API (sfm/server.py)

### Live device endpoints (connect to device per-request)

```
GET  /device/info?port=COM5                              ← serial
GET  /device/info?host=1.2.3.4&tcp_port=9034            ← cellular
GET  /device/events?host=1.2.3.4&tcp_port=9034&baud=38400
GET  /device/event?host=1.2.3.4&tcp_port=9034&index=0
GET  /device/monitor/status?host=1.2.3.4&tcp_port=9034  ← battery, memory, mode
POST /device/monitor/start?host=1.2.3.4&tcp_port=9034   ← start recording
POST /device/monitor/stop?host=1.2.3.4&tcp_port=9034    ← stop recording
```

Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing).

### DB read endpoints (query seismo_relay.db written by ach_server.py)

```
GET  /db/units                                           ← all known serials + summary stats
GET  /db/events?serial=BE11529&from_dt=&to_dt=&limit=   ← triggered events, newest first
GET  /db/monitor_log?serial=BE11529&from_dt=&to_dt=     ← monitoring intervals, newest first
GET  /db/sessions?serial=BE11529&limit=50               ← ACH call-home sessions, newest first
PATCH /db/events/{id}/false_trigger?value=true          ← flag/unflag false triggers
```

DB file: `bridges/captures/seismo_relay.db` (default; override with `--db-path` at startup).
All DB endpoints are read-only except `PATCH /db/events/{id}/false_trigger`.

---

## Key wire captures (reference material)

| Capture | Location | Contents |
|---|---|---|
| 1-2-26 | `bridges/captures/1-2-26/` | SUB 5A BW TX frames — confirmed 5A frame format, 11-byte params, DLE-aware checksum |
| 3-11-26 | `bridges/captures/3-11-26/` | Full compliance setup write, Aux Trigger capture |
| 3-31-26 | `bridges/captures/3-31-26/` | Complete event download cycle (148 BW / 147 S3 frames) — confirmed 1E/0A/0C/1F sequence; only 1 event stored so token=0xFE appeared to work |
| 4-3-26 | `bridges/captures/4-3-26/` | Browse-mode S3 capture with 2+ events — confirmed all-zero params for 1F, 1F response layout, null sentinel, 0A context requirement |
| 4-27-26 | `bridges/captures/4-27-26/` | BW "open 2sec waveform" + "copy event to disk" + paired SFM "seismo_dl" — first proof that SFM was over-reading 5× past event end. BW reads 14 chunks at 0x0200 increments + TERM at end_offset; SFM was reading 37 chunks at 0x0400 increments. STRT end_key field located. |
| 5-1-26 | `bridges/captures/5-1-26/comcheck/` | Three sub-captures: SFM 3-sec download (`seismo_dl_…`), BW comms-check + 3-sec download (`bwcap3sec/`), BW second-event download + "Download All" (`raw_*_170945`/`_171216`). Confirmed: TERM frame formula across 3 events; metadata pages 0x1002/0x1004 are global (read once per session); event-1 vs event-N chunk-pattern split; WAVEHDR length 0x46 vs 0x2C disambiguates real events from boundaries. |

---

## Write commands (SUBs 68–83) — confirmed 2026-04-07

All confirmed from 3-11-26 BW TX capture (`raw_bw_20260311_170151.bin`, frames 102–112).

### Write frame format — CRITICAL: minimal DLE stuffing

Write frames do NOT use the same DLE stuffing as read frames. **Only the BW_CMD byte
(0x10 at payload position [0]) is doubled on the wire. All other bytes — flags, sub,
offset, params, data, and checksum — are written RAW without stuffing.**

Confirmed from all 11 write frames in the 3-11-26/170151 BW capture. ✅ 2026-04-07

Do NOT use `dle_stuff()` or `build_bw_frame()` for write commands. Use `build_bw_write_frame()`.

```
Actual wire layout:
  [41]       ACK
  [02]       STX
  [10 10]    BW_CMD doubled (ONLY DLE stuffing applied)
  [00]       flags
  [sub]      write command byte (0x68–0x83)
  [00]       always zero
  [hi][lo]   offset uint16 BE — RAW (not stuffed even if hi=0x10)
  [params]   10 bytes — RAW
  [data]     variable-length write payload — RAW (0x10 bytes not stuffed)
  [chk]      checksum — RAW (not stuffed even if 0x10)
  [03]       ETX

Total wire length = 2 (ACK+STX) + 2 (doubled BW_CMD) + 15 (raw header) + len(data) + 1 (chk) + 1 (ETX)
                 = 21 + len(data)
```

De-stuffed payload (logical; used for checksum computation only):
```
  [0]     BW_CMD  0x10
  [1]     flags   0x00
  [2]     SUB     write command byte (0x68–0x83)
  [3]     0x00    always zero
  [4]     offset_hi
  [5]     offset_lo
  [6:16]  params  10-byte field (see per-SUB notes below)
  [16:]   data    write payload (variable length; absent for confirm frames)
  [-1]    chk     large-frame DLE-aware checksum (see below)
```

Write SUBs = Read SUB + 0x60. Response SUB follows the standard 0xFF − Request SUB rule.

### Write frame checksum

All write frames (data frames AND confirm frames) use the **large-frame DLE-aware checksum**:

```python
chk = (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF
```

This is identical to the SUB 5A DLE-aware checksum. Confirmed against all 11 write frames in
the 3-11-26/170151 capture. ✅ 2026-04-07

Note: confirm frames contain no embedded 0x10 bytes, so both the standard SUM8 and the
DLE-aware formula produce the same result for them — but `build_bw_write_frame` always uses
the DLE-aware formula for consistency.

### Write ack responses

All device acks for write commands are **17-byte zero-data S3 frames**:

```
[DLE=0x10][STX=0x02][stuffed(header + chk)][bare ETX=0x03]
```

The data section carries zeros; RSP_SUB = 0xFF − write_request_SUB.

### Write SUB constants and sequences

| Request SUB | Function | Offset | Response SUB |
|---|---|---|---|
| 0x68 | Event index write | `data[1] + 2` | 0x97 |
| 0x73 | Confirm B (follows 68) | 0 | 0x8C |
| 0x71 | Compliance write (×3 chunks) | see below | 0x8E |
| 0x72 | Confirm A (follows 71×3, 69) | 0 | 0x8D |
| 0x82 | Trigger config write | `data[1] + 2` | 0x7D |
| 0x83 | Trigger confirm (follows 82) | 0 | 0x7C |
| 0x69 | Waveform data write | `data[1] + 2` | 0x96 |
| 0x74 | Confirm C (follows 69) | 0 | 0x8B |

**Offset formula for single-chunk writes (0x68, 0x69, 0x82):** `offset = data[1] + 2`

The write payload always begins with a 2-byte header `[0x00][length]`, where `data[1]` is
an embedded length field. The offset encodes this inner length + 2 (accounting for the
header bytes). Confirmed from all three single-chunk write frames in the 3-11-26 capture:

| SUB | data[0:4] (hex) | data[1] | offset | total data len |
|---|---|---|---|---|
| 0x68 | `00 58 09 00` | 0x58=88 | 0x5A=90 | 91 |
| 0x82 | `00 1A D5 00` | 0x1A=26 | 0x1C=28 | 29 |
| 0x69 | `00 C8 08 00` | 0xC8=200 | 0xCA=202 | 204 |

Full sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`

### SUB 71 — compliance write chunk parameters

The full compliance config payload (~2128 bytes) is split into exactly 3 chunks.
Confirmed from 3-11-26 BW TX capture frames 104–108:

| Chunk | Size | `offset` | `params` (10 bytes hex) |
|---|---|---|---|
| 1 (first) | 1027 bytes | 0x1004 | `00 00 00 00 00 00 00 00 00 00` |
| 2 (middle) | 1055 bytes | 0x1004 | `00 00 00 10 04 00 00 00 00 00` |
| 3 (last) | remainder | 0x002C | `00 00 08 00 00 00 00 00 00 00` |

Total: 1027 + 1055 + N = 2082 + N bytes (N ≈ 46 for a standard 2128-byte config).

After all 3 chunks are acked (SUB 0x8E each), send SUB 72 confirm → device acks 0x8D.

### `build_bw_write_frame()` — framing.py

```python
build_bw_write_frame(sub, data, *, offset=0, params=bytes(10)) -> bytes
```

Use for all write commands (SUBs 68–83) including confirm frames (data=b"").
**Do NOT use `build_bw_frame` for write commands** — it uses standard SUM8, not the
large-frame DLE-aware checksum required for writes.

### `push_config_raw()` — client.py

```python
client.push_config_raw(event_index_data, compliance_data, trigger_data, waveform_data)
```

Orchestrates the full write sequence in the confirmed order. All payloads are raw bytes
(no encoding performed at this level). A higher-level encoder that builds payloads from
a `ComplianceConfig` object is a future task.

---

## Monitoring commands (SUBs 0x1C, 0x96, 0x97) — confirmed 2026-04-08

All confirmed from 4-8-26/2ndtry BW TX/S3 capture (clean start → 30s monitor → stop cycle).

### SUB 0x1C — Monitor status read

Standard two-step read (probe at offset 0x00, data at offset 0x2C).
Response SUB = 0xFF − 0x1C = **0xE3** (standard formula — no exception).

**Payload length is 46–47 bytes IDLE, 48–49 bytes MONITORING** — not a reliable sole
indicator due to 1-byte jitter overlap at the boundary.

**Monitoring flag (CONFIRMED 2026-04-09 — byte diff of all 144 data frames, 2ndtry capture):**
- `section[1] == 0x00` → unit is **idle**
- `section[1] == 0x10` → unit is **monitoring**

This is `data[12]` (= `frame.data[12]`).  The flag is 0x00 in all 36 IDLE_BEFORE frames,
0x10 in all 98 MONITORING frames, and 0x00 in all 10 IDLE_AFTER frames — 100% accurate.

**HISTORY OF THIS FIELD (do not re-derive):**  The original implementation used `section[1]`.
A re-analysis in the prior session incorrectly concluded `section[1]` is always 0x00 and
"corrected" the flag to `section[6]`, which has non-binary values (0xea idle, 0x07 monitoring)
and is device-specific.  The 2026-04-09 re-analysis confirms `section[1]` was right.

**IMPORTANT — `frame.data` has checksum already stripped** by `S3FrameParser._finalise()`
(`raw_payload = body[:-1]`; `data = raw_payload[5:]`).  There is NO trailing checksum byte in
`section`.  All relative-from-end offsets must account for this.

Battery and memory fields are present in **both** states:

| Offset (relative to end) | Field | Type | Notes |
|---|---|---|---|
| `section[-10:-8]` | battery voltage × 100 | uint16 BE | `0x02A8` = 680 → 6.80 V |
| `section[-8:-4]` | memory total (bytes) | uint32 BE | e.g. 983026 ≈ 960 KB |
| `section[-4:]` | memory free (bytes) | uint32 BE | decreases as events are stored |

### SESSION_RESET signal (`41 03`) — required for monitoring units

Confirmed from 4-8-26 BW TX captures: Blastware sends a 2-byte `41 03` (ACK + ETX,
no STX) immediately before the first POLL probe AND between the probe and data frames.
This signal is **required to wake units that are actively monitoring** — without it
the unit does not respond to POLL over TCP.  Harmless for idle units.

`SESSION_RESET = bytes([0x41, 0x03])` is defined in `framing.py` and sent by
`protocol.startup()` before and between POLL frames.

### SUB 0x96 — Start monitoring

Single write frame, **no data payload** (empty body).
Response SUB = 0xFF − 0x96 = **0x69**.

Wire bytes (confirmed frame 92 of 2ndtry BW capture):
```
41 02 10 10 00 96 00 00 00 00 00 00 00 00 00 00 00 00 00 a6 03
```

### SUB 0x97 — Stop monitoring

Single write frame, **no data payload** (empty body).
Response SUB = 0xFF − 0x97 = **0x68**.

Wire bytes (confirmed frame 305 of 2ndtry BW capture):
```
41 02 10 10 00 97 00 00 00 00 00 00 00 00 00 00 00 00 00 a7 03
```

Both start and stop acks are standard 17-byte zero-data S3 frames.

### On-device sensor check behavior (confirmed 2026-04-08)

Confirmed from 4-8-26/sensor-check BW+S3 capture (Blastware "Unit Channel Test" comms
check issued while unit was performing its on-device sensor check).

**Unit IS reachable during on-device sensor check** — POLL (SUB 5B) responded normally
throughout.  However, the unit partially handled channel-test commands (SUB 0x0E) for
channels 0–4 and then went **silent for ~40 seconds** while the sensor check ran, before
resuming responses for channels 5–7 and the trigger test (SUB 0x98).

Key findings:
- On-device sensor check duration: approximately **40 seconds** (log gap `18:40:48` → `18:41:28`)
- Unit IS reachable for POLL during the check window — SESSION_RESET + POLL works
- Partial command responses during check are possible (device may buffer some, drop others)
- The Blastware "Unit Channel Test" (remote comms check, SUBs 0x0E + 0x98) is a SEPARATE
  operation from the on-device check — it is a passive remote read; the unit's screen does
  not change during a remote comms check

**SFM behavior after `POST /device/monitor/start`:** `_pollMonitorConfirm()` polls
`/device/monitor/status` every 5 s for up to 60 s, updating the badge on each poll.
Status will show MONITORING once `section[1]` flips to `0x10`.

### SUBs known from sensor-check capture (4-8-26) — NOT YET IMPLEMENTED

| BW SUB | RSP SUB | Function | Notes |
|---|---|---|---|
| 0x15 | 0xEA | Serial number / short ID | 2-step read; data offset = 0x0A (10 bytes); confirmed serial `"BE11529"` at `data[11+5:]` |
| 0x01 | 0xFE | Device info block | 2-step read; data offset = 0x98 (152 bytes); payload includes serial + firmware + float config fields |
| 0x0E | 0xF1 | Channel sensor data | 2-step read; channel selector in `params[6:8]` (`0x0000`–`0x0007`); data length 0x0A per channel; used by Blastware "Unit Channel Test" — see docs/ for details |
| 0x98 | 0x67 | Trigger test | Single probe frame (`params[0]=0xFF`); sent twice per test cycle; all-zero data response; used after 0x0E channel scan |

Blastware's "Unit Channel Test" sequence: `POLL×N → 0x15 → 0x01 → 0x08 → 0x01 → 0x0E×8 → 0x98×2 → 0x0E×8` (repeat pass with live ADC readings).

---

## Compliance config field inventory (from Blastware UI, 2026-04-08)

Fields visible in the Blastware Compliance Setup dialog — most are NOT YET decoded to byte
offsets in the raw 1A/E5 payload.  Only fields with `✅` have confirmed offsets in the code.

**Recording Setup tab:**
- Recording Mode: Continuous / Single Shot / Histogram / Histogram+Continuous ✅ (uint8 at anchor−3 in write, anchor−4 in read; 0x00=Single Shot, 0x01=Continuous, 0x03=Histogram, 0x04=Histogram+Continuous) — confirmed 2026-04-20
- Record Stop Mode: Fixed Record Time / Auto / Manual Stop (enum) ❓ (byte near recording_mode; data[40] in E5 sf1 changed 0x01→0x00 alongside Continuous→Single Shot — may be this field)
- Sample Rate: Standard 1024 / Fast 2048 / Faster 4096 sps ✅ (anchor−2)
- Record Time: float, seconds ✅ (anchor+10)
- Histogram Interval: 2s / 5s / 15s / 1m / 5m / 15m ✅ (uint16 BE seconds at anchor−4, same in read & write; mode-gated to Histogram/Histogram+Continuous) — confirmed 2026-04-20
- Storage Mode: Save All Data / Save Triggered (enum)
- Geophone Type: Standard Triaxial / 4.5 Hz (bool/enum)
- Geophone Channels: Enable all geophones (bool), Trigger Source (bool)
- Chan 1-3 Trigger Level (float, in/s) ✅ (`trigger_level_geo`)
- Chan 1-3 Maximum Range: Normal 10.000 / 1.25 in/s (enum) ✅ (`geo_range` uint8; **CONFIRMED 2026-04-20** from 4-20-26 geo sensitivity captures: offset = `channel_label+33` in both E5 read and SUB 71 write payloads (same bytes, round-tripped verbatim); `0x00` = Normal 10.000 in/s, `0x01` = Sensitive 1.250 in/s; applied to Tran/Vert/Long channel blocks). **IMPORTANT: `channel_label+20` reads `0x01` on ALL captures and is NOT this field** — it is a constant flag. The float32 at `channel_label+28` = 6.206053 is the ADC-to-velocity scale factor (hardware constant, do NOT write).
- Microphone Channels: Enable all microphones (bool), Trigger Source (bool)
- Chan 4 Trigger Level (dB or psi depending on units)

**Notes tab:**
- Enable User Notes (bool)
- Project, Client, User Name, Seis Loc (ASCII strings) ✅ (sourced from 5A metadata pages at counter 0x1002 / 0x1004 — see "SUB 5A — fixed metadata pages" section)
- Enable Extended Notes (bool); Extended Notes text; Extended Notes Title
- Enable Job Number (bool); Job Number (int)
- Enable Scaled Distance (bool); Distance from Blast (float); Charge Weight (float) — Scaled Distance is derived

**Special Setups tab:**
- Unit Timer: Timer Mode (Off/On), Start Date/Time, Stop Date/Time
- Self Check: Mode (Off/On), Time (HH:MM)
- Sensor Check: **Before monitoring** / After each event / **Disabled** ❓ (byte offset unknown)
- Measurement Units: Imperial / Metric
- Show Mic units in dB (bool)
- Time Format: 24 Hour / 12 Hour (AM/PM)
- Backlight on Time (seconds) ✅ (event index block +75)
- Power Saving Timeout (minutes) ✅ (event index block +83)
- Monitoring LCD Cycle ✅ (event index block +84:86)
- Set unit time with setup (bool)

The "Sensor Check" dropdown (`Before monitoring` / `After each event` / `Disabled`) has NOT
been located in the raw config bytes.  The user's unit always runs with `Before monitoring`.
Full compliance config encoder is a future task.

---

---

## Erase-all protocol (SUBs 0xA3/0xA2/0x06) — confirmed 2026-04-11

Full sequence confirmed from 4-11-26 MITM capture of a live Blastware ACH session
(`bridges/captures/mitm/ach_mitm_20260411_001912/`).

### Wire sequence

```
BW → device:  SUB 0xA3  params=00 00 00 00 00 00 00 FE 00 00  (begin erase)
device → BW:  SUB 0x5C  (ack)
BW → device:  SUB 0x1C  probe (offset=0x00)
device → BW:  SUB 0xE3  (probe ack)
BW → device:  SUB 0x1C  data  (offset=0x2C)
device → BW:  SUB 0xE3  (monitor status response)
BW → device:  SUB 0x06  probe (offset=0x00, params same)
device → BW:  SUB 0xF9  (probe ack)
BW → device:  SUB 0x06  data  (offset=0x24)
device → BW:  SUB 0xF9  (36-byte storage range response)
BW → device:  SUB 0xA2  params=00 00 00 00 00 00 00 FE 00 00  (confirm erase)
device → BW:  SUB 0x5D  (ack — device memory is now cleared)
```

All frames use standard `build_bw_frame` (not write-format).  Response SUBs follow the
standard `0xFF - SUB` formula; no exceptions.

### SUB 0x06 — event storage range response (36 bytes)

The 36-byte response body ends with two 4-byte event keys:

| Offset (from end) | Field | Notes |
|---|---|---|
| `[-8:-4]` | first stored event key | `01110000` when empty |
| `[-4:]` | last stored event key | `01110000` when empty |

Before erase: ends with `<first_key> <last_key>` (e.g. `0111ea60 0111eaa6`).
After erase: both bytes read `01110000` — device's empty/reset sentinel.

### Post-erase key counter reset

After a successful erase, the device resets its event counter.  New events start from
key `0x01110000` again — the same key as the very first event ever recorded.  This means
key-based deduplication in the ACH server must account for key reuse:

- After our own erase: `ach_state.json` `downloaded_keys` and `max_downloaded_key` are
  cleared so the next session starts fresh.
- After an external erase: the ACH server detects it by comparing `max(device_keys)` to
  `max_downloaded_key` from state.  If the device max has rolled back below the historical
  max, all current device keys are treated as new regardless of `seen_keys`.

### ACH server state format (v0.9.0)

`bridges/captures/ach_state.json`:
```json
{
  "BE11529": {
    "downloaded_keys":    ["01110000", "0111245a"],
    "max_downloaded_key": "0111245a",
    "last_seen":          "2026-04-11T01:04:36",
    "serial":             "BE11529",
    "peer":               "63.43.212.232:51920"
  }
}
```

`max_downloaded_key` is the high-water mark — the largest key ever downloaded from the
unit.  It is NOT reset when events are erased from the device (only when our server does
the erase).  Used for post-erase detection.

---

## Monitor log entries — SUB 0x0A partial records (confirmed 2026-04-11)

Confirmed from 4-11-26 MITM capture: 12 partial records (record type `0x2C`) and 7 full
event records (record type `0x46`) across 19 total 0x0A responses.

### Record type detection

`read_waveform_header()` returns `(raw_data, length)` where `raw_data = data_rsp.data`
(the full payload including prefix bytes).  The record type is at `raw_data[0]`:

| Value | Type | How to process |
|---|---|---|
| `0x46` | Full triggered event | Normal download: 0C → 5A → 1F |
| `0x2C` | Monitor log entry (partial) | No 0C/5A; decode inline from 0A payload |

Length heuristic: `length < 0x40` (64) reliably identifies partial records across all
observed captures.  Both checks (`raw_data[0] == 0x2C` and `length < 0x40`) are used.

### SUB 0x0A partial record (0x2C) payload layout

All offsets are from `raw_data` (the full `data_rsp.data` array including the 11-byte
prefix before the actual header bytes start).

```
raw_data[0]   = 0x2C   ← record type (partial / monitor log)
raw_data[1:11] = prefix bytes (vary; contain key4 copy, flags, length)
raw_data[11:] = timestamp and ASCII metadata payload
```

**Timestamp auto-detection** (confirmed from 4-11-26 capture):

```
raw_data[11] == 0x10  →  10-byte sub_code=0x03 format (continuous mode)
raw_data[11] != 0x10  →  9-byte sub_code=0x10 format (single-shot mode)
```

**9-byte timestamp format (sub_code=0x10):**

| Byte | Field |
|---|---|
| 0 | day |
| 1 | `0x10` (sub_code marker) |
| 2 | month |
| 3–4 | year (uint16 BE) |
| 5 | unknown (0x00) |
| 6 | hour |
| 7 | minute |
| 8 | second |

**10-byte timestamp format (sub_code=0x03):**

| Byte | Field |
|---|---|
| 0 | `0x10` (marker) |
| 1 | day |
| 2 | `0x10` (marker) |
| 3 | month |
| 4–5 | year (uint16 BE) |
| 6 | unknown (0x00) |
| 7 | hour |
| 8 | minute |
| 9 | second |

**Two timestamps:** Each partial record contains two timestamps — `start_time` and
`stop_time` — stored consecutively:
- `ts1` (start) at `raw_data[ts_offset : ts_offset + ts_size]` where `ts_offset = 11`
- `ts2` (stop) at `raw_data[ts1_end : ts1_end + ts_size]`

**Edge case — 1-byte gap between timestamps:** Occurs when ts1 and ts2 share the same
minute:second.  If `try_ts(raw_data[ts1_end:])` fails, try `try_ts(raw_data[ts1_end+1:])`.
Confirmed in frames 121, 161, 165 of the 4-11-26 MITM capture.  Frame 121 still shows 0s
duration (both decode to 16:02:00) — the extra byte appears in all same-second cases.

**ASCII metadata after timestamps:**
```
<separator bytes>  BE<serial>\x00Geo: <float> in/s ...
```

- Serial: scan for `b"BE"`, read until `b"\x00"` (e.g. `"BE11529"`)
- Geo threshold: scan for `b"Geo: "`, read float until next space (e.g. `0.254` in/s)

A separator of variable length (4–5 bytes of `\x00` + flags) sits between the two
timestamps and the ASCII region.  The `b"BE"` anchor scan is robust to separator length
variation.

### `_decode_0a_partial_header(raw_data, index, key4)` — client.py

Returns a `MonitorLogEntry` or `None`.  Called by `get_monitor_log_entries()` for each
event key whose 0x0A response has `raw_data[0] == 0x2C` or `length < 0x40`.

### `MiniMateClient.get_monitor_log_entries(skip_keys=None)` — client.py

Browse-mode walk: `1E → 0A → check type → decode if partial → 1F`.  No 0x0C or 5A reads
performed.  Full (0x46) records are skipped without decoding.  Returns `list[MonitorLogEntry]`.

`skip_keys` (optional `set[str]`): keys in this set are still advanced through the walk
(to avoid disrupting the iteration sequence), but no `MonitorLogEntry` is created for them.

### `MonitorLogEntry` model — models.py

```python
@dataclass
class MonitorLogEntry:
    index: int                                      # 0-based position
    key: str                                        # 8-hex event key
    start_time: Optional[datetime.datetime] = None
    stop_time:  Optional[datetime.datetime] = None
    serial: Optional[str] = None
    geo_threshold_ips: Optional[float] = None
    raw_header: Optional[bytes] = field(default=None, repr=False)

    @property
    def duration_seconds(self) -> Optional[float]: ...
```

### ACH server integration (v0.10.0)

After `get_events()`, the ACH server calls `get_monitor_log_entries(skip_keys=seen_keys)`.
New entries are saved to `monitor_log.json` in the session directory.  Monitor log keys are
included in `current_keys` for state persistence so they are not re-processed on the next
call-home.

---

## Auto Call Home config (SUBs 0x2C / 0x7E / 0x7F) — confirmed 2026-04-20

Full read/write pipeline confirmed from `bridges/captures/4-20-26/call home settings/`
(10 BW TX write frames diffed against the S3 read response).

Accessible in Blastware: **Remote Access → Setup Unit**.

### Protocol

**SUB 0x2C — Call Home Config READ (response 0xD3)**

Standard two-step read: probe offset `0x0000`, data offset `0x007C` (124).
Returns 125 raw bytes (one more than DATA_LENGTH) because the device encodes
num_retries value `3` as `\x10\x03` on the wire — S3FrameParser preserves both
bytes literally, shifting all subsequent field positions by +1.

**SUB 0x7E — Call Home Config WRITE (response 0x81)**

Write format (only BW_CMD `0x10` doubled on wire; DLE-aware checksum).
Payload = 125-byte read payload + `\x00\x00` = 127 bytes.
Offset = `data[1] + 2 = 0x7C + 2 = 0x7E`.

**SUB 0x7F — Call Home WRITE CONFIRM (response 0x80)**

Confirm frame, no data payload. Required after SUB 0x7E.

### Field map (raw 125-byte array from `data_rsp.data[11:]`)

| Raw Offset | Field | Notes |
|---|---|---|
| `[5]` | `auto_call_home_enabled` | `0x00`=off, `0x01`=on |
| `[6:46]` | `dial_string` | 40-byte null-padded ASCII |
| `[87]` | `after_event_recorded` | bool |
| `[91]` | `at_specified_times` | bool |
| `[93]` | `time1_enabled` | bool |
| `[101]` | `time1_hour` | 0–23 |
| `[102]` | `time1_min` | 0–59 |
| `[95]` | `time2_enabled` | bool |
| `[105]` | `time2_hour` | 0–23 |
| `[106]` | `time2_min` | 0–59 |
| `[117]` | DLE prefix `0x10` | Part of `\x10\x03` (DLE-escaped ETX encoding value 3) |
| `[118]` | `num_retries` | Value = 3; detect via `raw[117] == 0x10` |
| `[120]` | `time_between_retries_sec` | Shifted +1 from logical 119 |
| `[122]` | `wait_for_connection_sec` | Shifted +1 from logical 121 |
| `[124]` | `warm_up_time_sec` | Shifted +1 from logical 123 |

**DLE-escaped 0x03 at raw[117:119]:** The byte value `0x03` is indistinguishable from the
frame ETX terminator, so the device encodes it as `\x10\x03` (DLE + ETX inner-terminator).
S3FrameParser in `STATE_AFTER_DLE` on ETX appends both bytes as literal payload. The write
frame sends them verbatim — device accepts `\x10\x03` and interprets it as value 3.

**Unconfirmed fields:** time slots 3 and 4 (offsets unknown), `modem_power_relay_enabled`.

### `CallHomeConfig` model — models.py

```python
@dataclass
class CallHomeConfig:
    raw: Optional[bytes] = None                        # 125-byte raw read payload
    auto_call_home_enabled: Optional[bool] = None      # raw[5]
    dial_string: Optional[str] = None                  # raw[6:46]
    after_event_recorded: Optional[bool] = None        # raw[87]
    at_specified_times: Optional[bool] = None          # raw[91]
    time1_enabled: Optional[bool] = None               # raw[93]
    time1_hour: Optional[int] = None                   # raw[101]
    time1_min: Optional[int] = None                    # raw[102]
    time2_enabled: Optional[bool] = None               # raw[95]
    time2_hour: Optional[int] = None                   # raw[105]
    time2_min: Optional[int] = None                    # raw[106]
    num_retries: Optional[int] = None                  # raw[118] (DLE-prefixed)
    time_between_retries_sec: Optional[int] = None     # raw[120] (shifted +1)
    wait_for_connection_sec: Optional[int] = None      # raw[122] (shifted +1)
    warm_up_time_sec: Optional[int] = None             # raw[124] (shifted +1)
```

### SFM REST API — sfm/server.py

```
GET  /device/call_home?host=1.2.3.4&tcp_port=9034   ← read call home config
POST /device/call_home?host=1.2.3.4&tcp_port=9034   ← write call home config
```

POST body fields (all optional): `auto_call_home_enabled`, `after_event_recorded`,
`at_specified_times`, `time1_enabled`, `time1_hour`, `time1_min`, `time2_enabled`,
`time2_hour`, `time2_min`.

**Note:** `dial_string` is read-only in the current implementation (omitted from POST
body) because writing a dial string may require DLE escaping for embedded control characters.

---

## What's next

**See [README.md → Roadmap (Future)](README.md#roadmap-future) for the canonical deferred-work list.** This section is kept as a status log of in-progress / recently-shipped technical details (encoding schemes, byte layouts, etc.) that are too low-level for the README's roadmap.

- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
- **Histograms** — decode histogram-mode A5 data (noise floor tracking)
- **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed BYTE-PERFECT against BW reference (v0.14.3, 2026-05-05):** when fed the BW 5-1-26 3-sec capture's A5 frames, the SFM-built file matches BW's saved `M529LKIQ.G10` byte-for-byte (8708 bytes, 0 differences).  Live SFM downloads of event 0 (3-sec) and event 1 (3-sec continuation) both open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots.  Body assembly is just contiguous concatenation of frame contributions in stream order (probe → meta@0x1002 → meta@0x1004 → samples → TERM); no stripping, no overlay, no special handling.  Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that may need different handling — untested under v0.14.x). Extension mapping: extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`

  **Serial encoding (CONFIRMED 2026-04-22):** `prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units).

  **Stem encoding (FULLY CONFIRMED 2026-04-22):** stem = 4-char base-36 of `floor(total_seconds / 1296)` where `total_seconds = (event_local_time − 1985-01-01T00:00:00_local)` in seconds. Epoch = `1985-01-01 00:00:00` device local time — confirmed against 3,248 files from 10-year production archive with zero errors. Decode: `event_time = datetime(1985,1,1) + timedelta(seconds=stem_int*1296 + ab_int)`. Example: P036L318.C80H → BE14036, 2025-05-26 15:00:08, Full Histogram.
- **Blastware filename extension — NEW FIRMWARE FULLY DECODED (confirmed 2026-04-21, further confirmed 2026-04-22 from 10-year production archive frequency analysis):**

  Extension format = `AB0T` (4 chars):
  - `AB` = 2-char base-36 encoding of `total_seconds % 1296` (seconds within the 21.6-min window, 0–1295); `A = value // 36`, `B = value % 36`
  - `0` = always literal digit zero (third character, invariant)
  - `T` = event type: `W` = Full Waveform, `H` = Full Histogram

  Combined with the 4-char stem, the full filename encodes a complete second-resolution timestamp. Verified against three S353L4H0.{3M0W,8S0H,9X0W} events (all match to the second) plus large-scale frequency analysis of a 10-year archive.

  **3-day cycle property (confirmed 2026-04-22):** A unit recording at a fixed daily time cycles through exactly **3 extensions** with a 3-day period. Each calendar day shifts `total_seconds % 1296` by 864 (since `86400 % 1296 = 864`). The cycle repeats every 3 days because `gcd(1296, 864) = 432` and `1296 / 432 = 3`. The three extension values are spaced 432 seconds apart. Confirmed from 10-year archive: the top 3 extensions overall were `CE0H` (95 files), `0E0H` (93), `OE0H` (91) — all three are the 3-day cycle of a 06:00:14 daily call-in time (seconds-in-window = 14, 446, 878; all three have `E` as second character because `14 = E` in base-36 and adding 864 never changes `value % 36` since `864 = 24 × 36`).

  **B character invariance:** For a unit recording at a fixed time of day, the second character `B` of the extension (`value % 36`) **never changes** — only the first character `A` cycles through 3 values. This means same-time-of-day files from different dates all share the same `B` character.

  **Old firmware (S338, 3-char extensions ending in `0`):** encoding unknown. Extension is NOT recording mode. `blastware_filename()` returns `.N00` as a placeholder for old-firmware units.

  **Micromate Series 4** uses a different extension format entirely (observed: `IDFH`, `IDFW`). The `AB0T` formula applies only to MiniMate Plus / V10.72 firmware.
- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
- **Test Histogram recording mode (0x03) write via SFM** — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state (bare 0x03 in write vs BW's DLE-escaped `10 03`)
- **Compliance write anchor-9 cleanup** — when changing recording_mode via SFM, the byte at anchor-9 is not explicitly managed. A spurious `0x10` may persist after Histogram→other mode transitions. Does not affect device operation but differs from BW's byte-perfect output.
- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring)
- Call Home — map time slots 3/4 offsets; add dial_string write support; confirm `modem_power_relay_enabled`
- Modem manager — push RV50/RV55 configs via Sierra Wireless API
- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't
  resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred)

## BW capture reference

`bridges/captures/` contains the following BW TX + S3 response captures for protocol analysis:

| Folder / File | Contents |
|---|---|
| `1-2-26/` | First SUB 5A BW TX capture — established 5A frame format (raw offset_hi, DLE-aware checksum). 10 frames verified. |
| `3-11-26/raw_bw_20260311_170151.bin` | Full compliance write + event download (SUBs 68→83 confirmed, frames 102–112) |
| `3-31-26/` | Single-event download (148 BW / 147 S3 frames) — 1E/0A/0C/1F sequence confirmed (single event so token=0xFE appeared to work in either branch) |
| `4-2-26/` | Download-mode BW TX capture — POLL×3 requirement confirmed (frames 68-73 between 1F and first 5A) |
| `4-3-26-multi_event/` | Browse-mode S3 capture with 2+ events — all-zero params for 1F, null sentinel layout, 0A context requirement |
| `4-8-26/` | Monitor status read, start/stop monitoring, SESSION_RESET signal, sensor check |
| `4-11-26 (mitm/ach_mitm_20260411_001912/)` | Full ACH call-home MITM — erase protocol (0xA3/0x06/0xA2), monitor log partial records confirmed |
| `4-20-26/raw_bw_*_recording_mode_*.bin` | Recording mode changes: Continuous→Single Shot, →Histogram, →Histogram+Continuous |
| `4-20-26/histogram interval/` | Histogram interval changes: 1min, 5min, 15min, 15sec |
| `4-20-26/geo sensitivity/` | Geo sensitivity changes: 1.25 in/s (Sensitive), 10 in/s (Normal) |
| `4-20-26/call home settings/` | Call home config read/write captures |
| `4-27-26/` | BW "open 2sec waveform" + "copy event to disk" + paired SFM "seismo_dl" — first proof of 5× SFM over-read. STRT end_key field located. |
| **`5-1-26/comcheck/`** | **Triplet of captures that nailed the v0.14.0 walk:** SFM 3-sec download (`seismo_dl_…`), BW comms-check + 3-sec download (`bwcap3sec/`), BW second-event download + "Download All" (`raw_*_170945` / `_171216`). Confirmed: TERM frame formula across 3 events, metadata pages 0x1002/0x1004 are global session metadata, event-1 vs event-N chunk pattern split, WAVEHDR off=0x46 vs 0x2C disambiguates real events from boundaries. |
| **`5-1-26/comcheck/bwcap3sec/`** | **The byte-perfect reference for v0.14.3.** All 17 BW 5A request frames (probe, 2 metadata, 13 samples, TERM) reproduce byte-for-byte from SFM's framing helpers — including the `10 10 00` DLE-stuffed counter for sample @ 0x1000 that was the long-standing failure mode. |
| `5-4-26/` | BW MITM captures of "copy 3sec / 2sec / Download All" + paired SFM session (`seismo_dl_20260504_145701`) showing the +0x46 event-N probe bug producing 110-chunk runaway walk. Cross-references against 5-1-26 confirmed device behavior is identical. |

To parse BW TX captures: use `bridges/captures/` scripts or adapt the `find_write_frames()` pattern
in `/tmp/analyze_write_payload.py` — it correctly handles `0x10 0x03` DLE-escaped ETX bytes
inside write frame data (the naive parser terminates early at the escaped `0x03`).