From 257c8ad18694e9c4b7d08e3c4c3eebdfba45ded5 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Wed, 15 Apr 2026 01:47:53 -0400 Subject: [PATCH] doc: update protocl ref --- CLAUDE.md | 35 +++++++++++++++++++++++----- docs/instantel_protocol_reference.md | 31 +++++++++++++++--------- 2 files changed, 49 insertions(+), 17 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 93668bb..36e8365 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -137,10 +137,10 @@ counter and streams data for any valid 5A request; using `chunk_num * 0x0400` is confirmed from the BW wire capture. `bulk_waveform_term_params()` returns 10 bytes. Do not swap them. -### SUB 5A — event-time metadata lives in A5 frame 7 +### SUB 5A — event-time metadata lives in a middle A5 frame (position variable) -The bulk stream sends 9+ A5 response frames. Frame 7 (0-indexed) contains the compliance -setup as it existed when the event was recorded: +The bulk stream sends 9+ A5 response frames. One of the middle frames contains the +compliance setup as it existed when the event was recorded: ``` "Project:" → project description @@ -148,10 +148,14 @@ setup as it existed when the event was recorded: "User Name:" → operator name ← NOT in the 0C record "Seis Loc:" → sensor location ← NOT in the 0C record "Extended Notes"→ notes +"Geo: " → geo threshold (in/s) — always present ``` +**Metadata frame position is variable:** fi==7 for blast events (4-2-26 capture), +fi==6 for desk-thump events (2026-04-14 confirmed). + **IMPORTANT — 5A "Project:" is session-start config, NOT per-event (confirmed 2026-04-05):** -The "Project:" string in the A5 frame 7 payload reflects the compliance setup from when +The "Project:" string in the metadata frame reflects the compliance setup from when the *monitoring session first started*, not the individual event's project name. The per- event project name is correctly stored in the 210-byte 0C waveform record and must be used as the authoritative source. `_decode_a5_metadata_into` therefore only sets @@ -160,8 +164,27 @@ used as the authoritative source. `_decode_a5_metadata_into` therefore only set "Client:", "User Name:", "Seis Loc:", and "Extended Notes" are **NOT** present in the 0C record — 5A remains the sole source for those fields and they are set unconditionally. -`stop_after_metadata=True` (default) stops the 5A loop as soon as `b"Project:"` appears, -then sends the termination frame. +**Metadata frame detection uses `_METADATA_FRAME_NEEDLES` (CORRECTED 2026-04-15):** +`stop_after_metadata=True` stops the 5A loop as soon as any needle in +`_METADATA_FRAME_NEEDLES` is found in the frame, then sends the termination frame. +`_decode_a5_waveform` uses the same tuple to skip the frame instead of decoding it as ADC +samples. The tuple is: + +```python +_METADATA_FRAME_NEEDLES = ( + b"Project:", b"Client:", b"User Name:", b"Seis Loc:", + b"Extended Notes", b"Geo: ", +) +``` + +**Do NOT check only `b"Project:"`.** The old single-needle check caused a critical bug +(confirmed 2026-04-15): the metadata frame was decoded as waveform ADC samples, making +the serial number bytes `"BE11529\0"` appear at sample ~929 followed by `0xFF 0xFF` fill, +truncating the second half of every waveform. Root cause unknown (may be that the +Project/Client etc. label strings appear differently under certain conditions), but +`b"Geo: "` is the reliable universal anchor — monitoring cannot operate without a geo +threshold configured. The check is applied against the full `db` (= `rsp.data`) bytes, +not `db[7:]`, to eliminate any off-by-7 edge case. ### SUB 5A — STRT record layout and rectime_seconds (CORRECTED 2026-04-14) diff --git a/docs/instantel_protocol_reference.md b/docs/instantel_protocol_reference.md index 356cb07..aa93090 100644 --- a/docs/instantel_protocol_reference.md +++ b/docs/instantel_protocol_reference.md @@ -103,6 +103,7 @@ | 2026-04-11 | §5.1 | **CONFIRMED — SUB 0x06 (CHANNEL CONFIG READ) now confirmed as event storage range.** Two-step read, data offset = 0x24 (36 bytes). Token=0xFE at params[7]. Last 8 bytes of response: first stored event key (bytes −8:−4) and last stored event key (bytes −4:). Both equal `01110000` when device memory is empty. Used by Blastware to verify erase completion. | | 2026-04-11 | §7.11 (NEW) | **NEW — §7.11 Erase-All Protocol added.** Full wire sequence, SUB 0x06 storage range payload layout, post-erase key counter reset (resets to `0x01110000`). Confirmed from 4-11-26 MITM capture of live Blastware ACH session. | | 2026-04-11 | §14.6 | **RESOLVED — ACH Session Lifecycle is no longer "Future".** `bridges/ach_server.py` fully implements inbound ACH: POLL handshake, device info, event download. State tracked via `ach_state.json` (key-based, with `max_downloaded_key` for post-erase detection). `--clear-after-download` flag added for the standard delete-after-upload workflow. | +| 2026-04-15 | §7.8.3 | **CORRECTED — Metadata frame detection must use multi-needle tuple, not `b"Project:"` alone.** Using only `b"Project:"` caused the metadata frame to be decoded as waveform ADC samples under certain conditions (serial number bytes `"BE11529\0"` appeared at sample ~929, followed by `0xFF 0xFF` fill). Root cause not fully characterised but `b"Project:"` alone is unreliable. Fix: check `_METADATA_FRAME_NEEDLES = (b"Project:", b"Client:", b"User Name:", b"Seis Loc:", b"Extended Notes", b"Geo: ")` against the full `rsp.data` bytes. `b"Geo: "` is the universal fallback — monitoring cannot operate without a configured geo threshold. Applied to both `_decode_a5_waveform` (client.py) and `read_bulk_waveform_stream` (protocol.py). | --- @@ -823,9 +824,9 @@ Waveform: starts at byte 8 of db[7:] | Frame index | Contents | |---|---| | A5[0] | Probe response: STRT record + first waveform chunk | -| A5[7] | Event-time metadata strings only (no waveform data) | -| A5[9] | Terminator frame (page_key=0x0000) — ignored | -| A5[1..6,8] | Waveform chunks | +| A5[fi] | Event-time metadata strings — position variable (fi==7 blast capture, fi==6 desk-thump; detect via `_METADATA_FRAME_NEEDLES`) | +| A5[last] | Terminator frame (page_key=0x0000) — ignored | +| All others | Waveform chunks | **Confirmed from 4-2-26 blast capture (total_samples=9306, pretrig=298, rate=1024 Hz):** @@ -843,7 +844,7 @@ Total: 7633B → 954 naive sample-sets, 948 alignment-corrected ``` Only 948 of 9306 sample-sets captured (10%) — `stop_after_metadata=True` terminated -download after A5[7] was received. +download after the metadata frame (A5[7] in this capture) was received. **Channel identification note:** The 4-2-26 blast saturated all four geophone channels to near-maximum ADC output (~32000–32617 counts). Channel ordering [Tran, Vert, Long, Mic] @@ -1187,15 +1188,22 @@ Two critical differences from `build_bw_frame`: > this equals `n * 0x0400` since `key4[2:4] = 0x0000`. The monotonic formula is correct > for all keys encountered on this device. -The `stop_after_metadata=True` flag causes the loop to stop as soon as `b"Project:"` is -found in the accumulated A5 frame data, typically after 7–9 chunks. A termination frame -is always sent before returning. +The `stop_after_metadata=True` flag causes the loop to stop as soon as any needle in +`_METADATA_FRAME_NEEDLES` is found in the A5 frame data, typically after 7–9 chunks. +A termination frame is always sent before returning. + +**⚠️ CORRECTED 2026-04-15 — do NOT check only `b"Project:"`.** The single-needle check +caused a critical waveform corruption bug: the metadata frame was decoded as ADC samples, +making serial number bytes `"BE11529\0"` appear at sample ~929 followed by `0xFF 0xFF` +fill, truncating the second half of every waveform. Use `_METADATA_FRAME_NEEDLES` (see +`protocol.py`) which includes `b"Geo: "` as the reliable universal anchor. #### 7.8.3 A5 Frame Layout -Each A5 response frame contains a chunk of raw bulk data. Frame 7 of the stream carries the -compliance text block with all project-info label-value pairs. The `client` layer searches -for ASCII labels with a null-terminated value read: +Each A5 response frame contains a chunk of raw bulk data. One middle frame (position +variable: fi==7 for blast events, fi==6 for desk-thump events) carries the compliance +text block with project-info label-value pairs. The `client` layer searches for ASCII +labels with a null-terminated value read: ``` "Project:" → null-terminated project name @@ -1203,9 +1211,10 @@ for ASCII labels with a null-terminated value read: "User Name:" → null-terminated operator name "Seis Loc:" → null-terminated sensor location "Extended Notes" → null-terminated notes +"Geo: " → geo threshold (float string, in/s) — ALWAYS present ``` -All five fields reflect the **setup at event-record time**, not the current device config. +All fields reflect the **setup at event-record time**, not the current device config. #### 7.8.4 End-of-Stream Behaviour and Chunk Timing