docs: update for v0.14.3 - Full continuous waveform download successful!
This commit is contained in:
@@ -111,6 +111,9 @@
|
||||
| 2026-04-20 | §7.6.2, §7.9, Appendix B | **CONFIRMED — Geophone maximum range / sensitivity selector byte location.** Two targeted captures (4-20-26, geo sensitivity folder): one at Normal 10.000 in/s, one at Sensitive 1.250 in/s. E5 read payload diff: exactly 3 bytes differ at channel_label+33 for Tran/Vert/Long. Values: `0x00`=Normal 10.000 in/s, `0x01`=Sensitive 1.250 in/s. Same offset applies to the SUB 71 write payload (which is the same 2126-byte E5-format buffer round-tripped verbatim). **`channel_label+20` reads `0x01` in ALL captures regardless of range setting — it is NOT this field.** Previous hypothesis (uint8 at Tran+20, 0x01=Normal) was WRONG. Stored as `geo_range` in `ComplianceConfig`. Encoded to all three geo channel blocks (Tran/Vert/Long) at label+33. |
|
||||
| 2026-04-20 | §5.1, §5.3, §7.12 (NEW) | **NEW — Auto Call Home config protocol confirmed from 4-20-26 call home settings captures.** SUB 0x2C (Call Home Config READ, response 0xD3, data offset 0x7C=124) and SUB 0x7E/0x7F (WRITE + CONFIRM, response 0x81/0x80) confirmed. Write payload = read payload (125 bytes) + `\x00\x00` (127 bytes total). **DLE-escaped ETX at raw[117:119]:** the device returns logical value 0x03 (num_retries=3) as `\x10\x03` on the wire — S3FrameParser preserves both bytes as two literals, causing a +1 byte shift for all subsequent fields. Write frame sends these bytes verbatim (device interprets `\x10\x03` as literal value 3). Field map confirmed from 10-frame BW TX diff. See §7.12 for full layout. |
|
||||
| 2026-05-01 | §7.8.2, §7.8.5 (NEW), §7.8.6 (NEW), §7.8.7 (NEW) | **REWRITTEN — SUB 5A bulk waveform stream protocol.** Five BW MITM captures (4-27-26 "open 2sec waveform" + "copy event to disk", 5-1-26 BW 3-sec + 2nd-event + Download All) prove that the previous chunk-counter formula `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400` over-reads 5× past the actual event end. BW reads ~12-16 chunks per event at **0x0200 increments (NOT 0x0400)**, bounded by `end_offset` extracted from the STRT record at `data[23:27]` of the first A5 response. **TERM frame formula corrected:** `offset_word = end_offset - next_boundary`, `params[2:4] = next_boundary BE` where `next_boundary = last_chunk_counter + 0x0200`. Verified across 3 events (offsets 0x1ABE, 0x21F2, 0x417E). **Metadata pages 0x1002 / 0x1004** are global, fixed-address device pages containing Project/Client/User Name/Seis Loc/Extended Notes — read ONCE per Blastware session (not per event). **Event-1 vs event-N split:** events at start_key[2:4]=0 use probe@0x0000 + metadata pages + sample chunks at 0x0600 onward; continuation events skip metadata and start at start_key+0x0046. **WAVEHDR length 0x46 vs 0x2C disambiguates real events from boundary markers** — the "Download All" pattern walks 1E/0A/1F to map all event keys+lengths upfront, then downloads each `0x46`-keyed event in turn. Old `stop_after_metadata=True` knob is a workaround for the missing end_offset bound and becomes obsolete under the new walk. See new §7.8.5 / §7.8.6 / §7.8.7 for full details. |
|
||||
| 2026-05-04 | §7.8.5, §7.8.8 | **CORRECTED — Event-N probe counter is just `start_offset`, NOT `start_offset + 0x0046`.** The `+0x46` formula in the original §7.8.5 was based on calling the off=0x2C boundary key the "start_key", but in the iteration walk `cur_key` passed into `read_bulk_waveform_stream` is always the off=0x46 WAVEHDR record key from 1F (the partial-record skip path in `get_events` re-runs 1F to advance past 0x2C boundary records). Adding +0x46 placed the probe one WAVEHDR past the actual event start; the response no longer contained STRT at byte 17, `parse_strt_end_offset` returned None, and the chunk loop fell back to the `max_chunks=128` cap, walking ~110 chunks of post-event circular-buffer garbage. Confirmed against both the 5-1-26 "copy 2nd address" capture (probe at counter=0x2238 with key=01112238) and the 5-4-26 BW 2-sec event capture. Fixed in protocol.py `read_bulk_waveform_stream` v0.14.1. |
|
||||
| 2026-05-05 | §7.8.1 (rule #3 added) | **CONFIRMED — Partial DLE stuffing of `0x10` bytes in 5A params region.** The device's de-stuffing rule for the SUB 5A params region is: `10 10` → `10`, `10 02/03/04` → kept literal (inner-frame markers), `10 X` for any other X → de-stuffs to just `X` (drops the `0x10`). Therefore any `0x10` byte in the logical params followed by a byte NOT in {0x02, 0x03, 0x04, 0x10} MUST be doubled on the wire. This affects counters with `0x10` in the high byte — most importantly counter=`0x1000`, where logical params bytes `... 10 00 ...` were being sent raw and the device de-stuffed `10 00` to just `00`, returning the response for counter=0x0000 (= the file header + STRT). That STRT block then ended up embedded in the assembled file body at file offset `0x1016` and Blastware refused to open the file. This was the root cause of the long-standing ">1-sec event 0 won't open in BW" pattern (1-sec events worked because their `end_offset < 0x1000`, so no chunk request ever needed counter `0x10__`). All 17 5A request frames in the 5-1-26 bwcap3sec capture (probe + 2 meta + 13 samples + TERM) now match BW byte-for-byte after the fix. Fixed in framing.py `build_5a_frame` v0.14.3. |
|
||||
| 2026-05-05 | §7.8 / Blastware file format | **CONFIRMED — File body assembly is contiguous concatenation, no de-duplication.** The "duplicate header+STRT strip" hack from v0.13.x was actively destroying valid waveform data — sample chunks at counter `0x1000` and beyond often coincidentally contain the byte sequence `00 12 03 00 STRT` in their delta-encoded ADC stream, and the strip was zeroing 25 bytes per match. Removed in v0.14.2. The correct file body is: probe contribution + meta@0x1002 + meta@0x1004 + sample contributions in stream order + TERM contribution. Verified byte-perfect against BW reference `M529LKIQ.G10` (8708 bytes, 0 differences) when fed the same A5 frames as the BW capture. |
|
||||
|
||||
---
|
||||
|
||||
@@ -261,7 +264,7 @@ Step 4 — Device sends actual data payload:
|
||||
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 |
|
||||
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Sub_code at byte[1]: 0x10=Waveform (9-byte timestamp hdr), 0x03=Waveform-continuous (10-byte hdr, 1-byte shift). PPV floats at label+6 (search "Tran"/"Vert"/"Long"/"MicL"). Peak Vector Sum at tran_label−12 (NOT fixed offset). Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-03 |
|
||||
| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[7] (⚠️ NOT params[6]): 0x00=browse (all-zero params), 0xFE=download (arm 5A state machine). Returns next key at data[11:15]; null sentinel when data[15:19]=0x00000000. Requires preceding 0A to establish context. Browse 1F must ONLY be called after successful 5A — calling it after a failed 5A disrupts device state for the next event's 5A probe. | ✅ CONFIRMED 2026-04-06 |
|
||||
| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum. Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. A5[7] contains event-time metadata (Project:/Client:/User Name:/Seis Loc:). 9+ A5 frames for full waveform; stop_after_metadata=True exits after A5[7]. | ✅ CONFIRMED 2026-04-06 |
|
||||
| `5A` | **BULK WAVEFORM STREAM** | Bulk download of raw ADC sample data. Non-standard frame format: offset_hi=0x10 sent raw (not DLE-stuffed), DLE-aware checksum, **partial DLE stuffing of 0x10 in params** (`10 X` where X∉{02,03,04,10} must be doubled to `10 10 X` — see §7.8). Requires 1E-arm + 0C + 1F(0xFE) + POLL×3 before first probe. Walk: probe at counter=`start_offset` (event 1: 0x0000) → metadata pages 0x1002 + 0x1004 (event 1 only) → sample chunks at 0x0600, 0x0800, …, step 0x0200, bounded by `end_offset` parsed from STRT@data[17] of probe response → TERM frame at residual offset_word. Project:/Client:/User Name:/Seis Loc: live in the metadata pages, NOT in the sample-chunk stream. | ✅ CONFIRMED 2026-05-05 (BYTE-PERFECT vs BW capture) |
|
||||
| `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED |
|
||||
| `25` | **WAVEFORM PAGE B?** | Paged waveform read, possibly channel group B. | 🔶 INFERRED |
|
||||
| `09` | **UNKNOWN READ A** | Read command, response (`F6`) returns 0xCA (202) bytes. Purpose unknown. | 🔶 INFERRED |
|
||||
@@ -837,6 +840,20 @@ MicL: 39 64 1D AA = 0.0000875 psi
|
||||
|
||||
### 7.6 Bulk Waveform Stream (SUB A5) — Raw ADC Sample Records
|
||||
|
||||
> ⛔ **§7.6 below describes the deprecated `0x0400`-step walk and is RETAINED FOR HISTORICAL CONTEXT ONLY.**
|
||||
> The "A5[7] is metadata", "A5[9] is terminator", and chunk-counter frame-index claims in this section
|
||||
> are all artifacts of the broken walk that was overrunning past event end by ~5×.
|
||||
>
|
||||
> **For the corrected protocol (v0.14.0+), use:**
|
||||
> - **§7.8.5** — chunk addressing (probe at `start_offset`, samples step 0x0200, bounded by STRT `end_offset`)
|
||||
> - **§7.8.6** — TERM frame formula
|
||||
> - **§7.8.7** — fixed metadata pages 0x1002 / 0x1004 (this is where Project / Client / User Name / Seis Loc
|
||||
> strings actually live — NOT in any sample-chunk frame)
|
||||
> - **§7.8.8** — multi-event "Download All" sequence
|
||||
>
|
||||
> The waveform sample encoding (4-channel interleaved s16 LE, 8 bytes per sample-set) described in §7.6.1
|
||||
> below is still correct. Only the frame-indexing claims and metadata-source claims are wrong.
|
||||
|
||||
**Two distinct formats exist depending on recording mode. Both confirmed from captures.**
|
||||
|
||||
---
|
||||
@@ -1119,20 +1136,26 @@ Near-ambient: 0x3C75C28F = 0.015 in/s (histogram event, near-zero ambient)
|
||||
|
||||
**Project strings** — ASCII label-value pairs (search for label, read null-terminated value):
|
||||
```
|
||||
"Project:" → project description (in 0C record ✅)
|
||||
"Client:" → client name (in SUB 5A / A5 frame 7 ✅ — NOT in 0C)
|
||||
"User Name:" → operator / user (in SUB 5A / A5 frame 7 ✅ — NOT in 0C)
|
||||
"Seis Loc:" → sensor location (in SUB 5A / A5 frame 7 ✅ — NOT in 0C)
|
||||
"Extended Notes"→ notes field (in SUB 5A / A5 frame 7 ✅)
|
||||
"Project:" → project description (in 0C record ✅, also mirrored in metadata pages)
|
||||
"Client:" → client name (in SUB 5A metadata pages ✅ — NOT in 0C)
|
||||
"User Name:" → operator / user (in SUB 5A metadata pages ✅ — NOT in 0C)
|
||||
"Seis Loc:" → sensor location (in SUB 5A metadata pages ✅ — NOT in 0C)
|
||||
"Extended Notes"→ notes field (in SUB 5A metadata pages ✅)
|
||||
```
|
||||
|
||||
> ✅ **2026-04-02 — CONFIRMED:** `Client:`, `User Name:`, and `Seis Loc:` are sourced from
|
||||
> **SUB 5A (bulk waveform stream)**, specifically A5 frame 7 of the multi-frame response.
|
||||
> They are NOT present in the 210-byte SUB 0C waveform record. The strings reflect the
|
||||
> compliance setup that was active when the event was recorded on the device — making SUB 5A
|
||||
> the authoritative source for true event-time metadata. The `get_events()` client method
|
||||
> now issues a SUB 5A request after each 0C download (`stop_after_metadata=True`) and
|
||||
> overwrites `event.project_info` with the decoded fields.
|
||||
> ✅ **UPDATED 2026-05-05:** `Client:`, `User Name:`, and `Seis Loc:` come from the
|
||||
> dedicated **SUB 5A metadata pages at counter `0x1002` and `0x1004`** — see §7.8.7.
|
||||
> They are NOT present in the 210-byte SUB 0C waveform record.
|
||||
>
|
||||
> An earlier draft of this doc claimed they came from "A5 frame 7" of the bulk waveform
|
||||
> stream — that was an artifact of the deprecated `0x0400`-step walk where the broken
|
||||
> chunk counter formula happened to land sample-chunk fi=7 on top of the 0x1002 metadata
|
||||
> page. Under the corrected v0.14.0+ walk (§7.8.5), sample chunks at `0x1000` / `0x1200`
|
||||
> contain ordinary waveform data, and the metadata pages are read separately.
|
||||
>
|
||||
> The strings reflect the compliance setup that was active when the *monitoring session*
|
||||
> first started (not per-event). `get_events()` reads the metadata pages once at the start
|
||||
> of the SFM session and the decoded values are stamped onto every event in that session.
|
||||
|
||||
---
|
||||
|
||||
@@ -1166,7 +1189,9 @@ return events
|
||||
|
||||
### 7.7.7 Updated Download Loop with SUB 5A Metadata
|
||||
|
||||
> ✅ **Added 2026-04-02.** Confirmed working on BE11529 over TCP/cellular.
|
||||
> ⛔ **The loop in this subsection is DEPRECATED — it uses the broken `stop_after_metadata=True`
|
||||
> hack and the wrong sequence ordering.** See §7.8.5–§7.8.8 for the corrected protocol.
|
||||
> The pseudocode below is preserved as historical record only.
|
||||
|
||||
```python
|
||||
key4, _ = proto.read_event_first() # SUB 1E
|
||||
@@ -1201,13 +1226,25 @@ return events
|
||||
|
||||
### 7.8 SUB 5A — Bulk Waveform Stream (event-time metadata)
|
||||
|
||||
> ✅ **Added 2026-04-02.** Frame format confirmed by reproducing Blastware wire bytes
|
||||
> byte-for-byte from the 1-2-26 BW capture.
|
||||
> ✅ **§7.8.1 (frame format) — added 2026-04-02; v0.14.3 partial DLE stuffing finalized 2026-05-05.**
|
||||
> Frame format confirmed by reproducing Blastware wire bytes byte-for-byte across the 1-2-26
|
||||
> capture (10 frames) and the 5-1-26 bwcap3sec capture (17 frames, all match including the
|
||||
> DLE-stuffed `10 10 00` for counter=0x1000).
|
||||
|
||||
SUB 5A initiates a bulk transfer of the raw sample data for a stored event. The response is a
|
||||
sequence of A5 frames. Frame 7 (0-indexed) contains the full compliance setup as it existed
|
||||
when the event was recorded — including `Client:`, `User Name:`, `Seis Loc:`, and
|
||||
`Extended Notes` ASCII label-value pairs.
|
||||
SUB 5A initiates a bulk transfer of the raw sample data for a stored event. The response is
|
||||
a sequence of A5 frames. Project-info ASCII strings (`Project:`, `Client:`, `User Name:`,
|
||||
`Seis Loc:`, `Extended Notes`) live in the dedicated metadata pages at counter `0x1002`
|
||||
and `0x1004` (see §7.8.7), not in the sample-chunk stream.
|
||||
|
||||
**For the corrected protocol read in order:**
|
||||
- §7.8.1 — frame format (raw `offset_hi`, DLE-aware checksum, partial DLE stuffing of params)
|
||||
- §7.8.5 — chunk addressing (probe → metadata pages → samples → TERM, all bounded by `end_offset`)
|
||||
- §7.8.6 — TERM frame formula
|
||||
- §7.8.7 — fixed metadata pages 0x1002 / 0x1004
|
||||
- §7.8.8 — multi-event "Download All" sequence
|
||||
|
||||
§7.8.2–§7.8.4 are retained as historical record of earlier (incorrect) understandings —
|
||||
do not implement against them.
|
||||
|
||||
#### 7.8.1 Frame Format
|
||||
|
||||
@@ -1218,7 +1255,7 @@ SUB 5A uses a **non-standard frame layout** that differs from all other BW→S3
|
||||
41 02 10 10 00 5A 00 ^^raw^^ ^^raw^^ ^^stuffed^^
|
||||
```
|
||||
|
||||
Two critical differences from `build_bw_frame`:
|
||||
Three critical differences from `build_bw_frame`:
|
||||
|
||||
1. **`offset_hi` is sent raw, not DLE-stuffed.** When `offset_hi = 0x10`, the wire carries
|
||||
a bare `0x10` — NOT the stuffed `10 10` that `build_bw_frame` would produce. The device
|
||||
@@ -1227,6 +1264,31 @@ Two critical differences from `build_bw_frame`:
|
||||
2. **DLE-aware checksum.** Walking the full frame byte sequence: when a `10 XX` pair is seen,
|
||||
only `XX` is added to the running sum; lone bytes are added normally.
|
||||
|
||||
3. **Partial DLE stuffing of `0x10` bytes in the params region** (CONFIRMED 2026-05-05).
|
||||
The device's de-stuffing rule for the params region is:
|
||||
|
||||
- `10 10` → de-stuffs to `10`
|
||||
- `10 02 / 03 / 04` → kept literal (these are inner-frame markers)
|
||||
- `10 X` for other X → de-stuffs to just `X` (drops the leading `0x10`)
|
||||
|
||||
Therefore any `0x10` byte in the *logical* params that is followed by a byte NOT in
|
||||
`{0x02, 0x03, 0x04, 0x10}` MUST be doubled on the wire (`10 X` → `10 10 X`) so the
|
||||
device's de-stuffer reproduces the original `10 X` pair. This applies most commonly
|
||||
to counters with `0x10` in the high byte (e.g. counter=`0x1000` produces logical
|
||||
params bytes `... 10 00 ...`, which BW encodes on the wire as `... 10 10 00 ...`).
|
||||
Without this stuffing the device interprets counter=`0x1000` as `0x0000` and returns
|
||||
the probe response (= a copy of the file header + STRT record); that STRT block then
|
||||
ends up embedded in the assembled file body and Blastware refuses to open the file.
|
||||
|
||||
`0x10` bytes in `offset_hi` are still written RAW per (1) above — only the params
|
||||
region has this stuffing requirement. Metadata-page params for counter `0x1002` /
|
||||
`0x1004` survive without stuffing because `10 02` / `10 04` fall in the "kept literal"
|
||||
carve-out.
|
||||
|
||||
Verified against BW 5-1-26 bwcap3sec frame 20: params logical bytes
|
||||
`00 01 11 10 00 00 00 00 00 00 00` (counter=0x1000) are encoded on the wire as
|
||||
`00 01 11 10 10 00 00 00 00 00 00 00` (12 wire bytes for 11 logical bytes).
|
||||
|
||||
#### 7.8.2 Request Sequence — DEPRECATED 2026-05-01 (see §7.8.5–§7.8.7 for the corrected protocol)
|
||||
|
||||
> ⛔ **The 0x0400-step / max(key4[2:4], 0x0400) formula in this section is WRONG.** Five new
|
||||
@@ -1267,11 +1329,20 @@ when the broken 0x0400-step walk passed the global metadata pages at 0x1002/0x10
|
||||
the corrected walk, those strings come from explicit reads at counter=0x1002 and 0x1004,
|
||||
not from the sample-chunk stream — see §7.8.7.
|
||||
|
||||
#### 7.8.3 A5 Frame Layout
|
||||
#### 7.8.3 A5 Frame Layout — DEPRECATED 2026-05-01
|
||||
|
||||
Each A5 response frame contains a chunk of raw bulk data. Frame 7 of the stream carries the
|
||||
compliance text block with all project-info label-value pairs. The `client` layer searches
|
||||
for ASCII labels with a null-terminated value read:
|
||||
> ⛔ **The "Frame 7 carries the compliance text block" claim below is WRONG.** It was
|
||||
> an artifact of the deprecated `0x0400`-step walk where the broken counter formula
|
||||
> happened to land sample-chunk fi=7 on top of the 0x1002 metadata page in flash.
|
||||
> Under the corrected v0.14.0+ walk (§7.8.5), Frame 7 of the sample-chunk sequence is
|
||||
> just sample-chunk #5 (counter=0x1000), and contains either ordinary waveform data or —
|
||||
> critically when DLE-stuffing of params is wrong (§7.8.1.3) — a duplicate file header +
|
||||
> STRT block when the device misinterprets counter=0x1000 as 0x0000. See §7.8.7 for the
|
||||
> actual source of these strings.
|
||||
|
||||
Historical claim (NOT TO BE IMPLEMENTED): each A5 response frame contains a chunk of raw
|
||||
bulk data; Frame 7 of the stream carries the compliance text block with all project-info
|
||||
label-value pairs:
|
||||
|
||||
```
|
||||
"Project:" → null-terminated project name
|
||||
@@ -1281,7 +1352,9 @@ for ASCII labels with a null-terminated value read:
|
||||
"Extended Notes" → null-terminated notes
|
||||
```
|
||||
|
||||
All five fields reflect the **setup at event-record time**, not the current device config.
|
||||
All five fields do reflect the **setup at event-record time**, not the current device
|
||||
config. But the source is the metadata pages (§7.8.7), not "Frame 7" of the sample
|
||||
stream.
|
||||
|
||||
#### 7.8.4 End-of-Stream Behaviour and Chunk Timing — REINTERPRETED 2026-05-01
|
||||
|
||||
@@ -1560,10 +1633,10 @@ Fields visible in the Blastware "Compliance Setup" dialog. ✅ = byte offset co
|
||||
| Field | Values / Type | Status |
|
||||
|---|---|---|
|
||||
| Enable User Notes | bool | ❓ |
|
||||
| Project | ASCII string | ✅ (sourced from A5 frame 7 via SUB 5A) |
|
||||
| Client | ASCII string | ✅ (sourced from A5 frame 7) |
|
||||
| User Name | ASCII string | ✅ (sourced from A5 frame 7) |
|
||||
| Seis Loc | ASCII string | ✅ (sourced from A5 frame 7) |
|
||||
| Project | ASCII string | ✅ (sourced from SUB 5A metadata pages at counter `0x1002` / `0x1004` — see §7.8.7) |
|
||||
| Client | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
|
||||
| User Name | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
|
||||
| Seis Loc | ASCII string | ✅ (sourced from SUB 5A metadata pages — see §7.8.7) |
|
||||
| Enable Extended Notes | bool | ❓ |
|
||||
| Extended Notes | ASCII text | ❓ |
|
||||
| Extended Notes Title | ASCII string | ❓ |
|
||||
|
||||
Reference in New Issue
Block a user