docs: document 5A end-of-stream signal, chunk timing, fi==9 bug, ADC conversion
Adds §7.8.4 to protocol reference and corresponding CLAUDE.md sections: - End-of-stream: device sends exactly 1 raw byte after last chunk; handled via TimeoutError + bytes_fed>0 check → graceful break to termination - Chunk timing: ~1s per chunk, 35 chunks for a 9,306-sample event, safe timeout is 10s (not default 120s) - fi==9 decoder bug: hardcoded skip drops ~133 sample-sets per event; noted as known issue pending fix - ADC conversion: counts × (range/32767) → physical units (in/s for geo) Changelog entries added for all four items (2026-04-06). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -157,6 +157,31 @@ record — 5A remains the sole source for those fields and they are set uncondit
|
||||
`stop_after_metadata=True` (default) stops the 5A loop as soon as `b"Project:"` appears,
|
||||
then sends the termination frame.
|
||||
|
||||
### SUB 5A — end-of-stream signal (confirmed 2026-04-06)
|
||||
|
||||
After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to
|
||||
the next chunk request, then goes silent. This is the natural end-of-stream indicator — NOT
|
||||
a complete A5 frame. `S3FrameParser.bytes_fed` will be 1; no frame is assembled.
|
||||
|
||||
Handling: on `TimeoutError`, if `bytes_fed > 0` AND frames were already collected, treat as
|
||||
graceful end-of-stream, break the loop, and proceed to the termination frame. If `bytes_fed
|
||||
== 0` with no prior frames, it is a genuine transport failure — re-raise.
|
||||
|
||||
**Chunk recv timeout must be 10 s, not the default 120 s.** Chunks arrive within ~1 s each.
|
||||
Using 120 s causes a ~2-minute stall at every end-of-stream detection. The `_recv_one` call
|
||||
in the chunk loop passes `timeout=10.0` explicitly.
|
||||
|
||||
**Typical chunk count (BE11529, 1024 sps):** A 9,306-sample event produces 35 chunks before
|
||||
end-of-stream. Chunks with uniform 1,036-byte data are all-zero ADC samples (post-event
|
||||
silence). Only the initial variable-size chunks contain actual signal.
|
||||
|
||||
### SUB 5A — known decoder issue: fi==9 hardcoded skip (not yet fixed)
|
||||
|
||||
`_decode_a5_waveform()` in `client.py` has `elif fi == 9: continue` — a leftover from the
|
||||
9-frame original blast capture where frame 9 was assumed to be a terminator. For current
|
||||
35-frame streams, fi==9 is live waveform data (~133 sample-sets dropped). Terminator
|
||||
detection is via `page_key == 0x0000`, not frame index. This skip should be removed.
|
||||
|
||||
### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)
|
||||
|
||||
**token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and
|
||||
|
||||
@@ -89,6 +89,10 @@
|
||||
| 2026-04-06 | §6.1 | **CONFIRMED — browse 1F must be conditional.** Calling 1F(browse=True/all-zero) after a FAILED 5A disrupts device state and causes the next event's 5A probe to time out with 0 bytes received. Browse 1F is only called after a SUCCESSFUL 5A. Failure fallback: use the key returned by the prior 1F(arm/0xFE) call. |
|
||||
| 2026-04-06 | §7.8 | **ADDED — `bytes_fed` diagnostic counter on S3FrameParser.** Counts raw bytes fed to the parser since last `reset()`. Logged at WARNING when 5A probe times out — distinguishes "device sent no bytes at all" from "device responded but frame was malformed or had wrong SUB". |
|
||||
| 2026-04-06 | §7.8.2 | **CORRECTED — SUB 5A chunk counter is monotonic for ALL chunks.** Previous doc hard-coded chunk 1 counter as `0x1004` (from 4-2-26 BW TX capture). This was a Blastware artifact. Empirically confirmed: `counter = chunk_num * 0x0400` works (device responds immediately); counter=0x1004 for chunk 1 causes 120 s timeout. BW's true internal formula appears to be `key4[2:4] + n * 0x0400` — for event 1 (key `01110000`) this equals `n * 0x0400`. The device does not strictly validate counter values. |
|
||||
| 2026-04-06 | §7.8.4 | **NEW — 5A end-of-stream signalling confirmed.** After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to the next chunk request, then goes silent for the full recv timeout. This byte is NOT a complete DLE-framed A5 response — the frame parser accumulates it as `bytes_fed=1` and never assembles a frame. This is the device's natural end-of-stream signal. Handling: on TimeoutError, if `bytes_fed > 0` AND prior chunks were received, treat as graceful end and proceed to the termination frame. A `bytes_fed=0` timeout with no prior chunks is a genuine transport failure and must still raise. |
|
||||
| 2026-04-06 | §7.8.4 | **NEW — 5A chunk timing and count (empirical, BE11529 at 1024 sps).** Each chunk response arrives within ~1 second over TCP/cellular. A 9,306-sample event (≈9.1 s at 1024 sps) produces **35 chunks** before end-of-stream. Chunks 1–16 have varying data lengths (1036–1123 bytes); chunks 17–35 are uniformly 1036 bytes each (post-event silence, all-zero ADC samples). Safe recv timeout for chunk loop: **10 s** (10× typical response time). Default transport timeout (120 s) results in a ~2-minute stall per event at end-of-stream. |
|
||||
| 2026-04-06 | §7.8.3 | **KNOWN ISSUE — `_decode_a5_waveform` hardcoded fi==9 skip.** The decoder contains `elif fi == 9: continue` which was written for the 9-frame original blast capture where frame 9 was a device terminator. For streams with >9 frames (current device produces 35+), frame index 9 is live waveform data — this skip discards ~1,070 bytes (~133 sample-sets) per event. The terminator is now detected via `page_key == 0x0000`, not by frame index. The fi==9 skip should be removed. |
|
||||
| 2026-04-06 | §7.8 | **CONFIRMED — ADC count-to-physical-unit conversion.** Raw waveform samples are signed 16-bit integers (counts). Conversion: `value = counts × (range / 32767)`. For geo channels: range = 10.000 in/s (from the device's compliance config geo range field). For the mic channel: range is in psi (device-specific). Near-full-scale counts (≈32,700) on all four channels simultaneously indicate ADC saturation (clipping) from a high-amplitude event. |
|
||||
|
||||
---
|
||||
|
||||
@@ -1180,6 +1184,46 @@ for ASCII labels with a null-terminated value read:
|
||||
|
||||
All five fields reflect the **setup at event-record time**, not the current device config.
|
||||
|
||||
#### 7.8.4 End-of-Stream Behaviour and Chunk Timing
|
||||
|
||||
> ✅ **Confirmed 2026-04-06** — empirical observation on BE11529 (S338.17) over TCP/cellular.
|
||||
|
||||
**End-of-stream signal:** After sending all waveform chunks, the device sends exactly **1 raw byte** in response to the next chunk request, then goes silent. This byte is not a complete DLE-framed A5 response — `S3FrameParser.bytes_fed` reports 1 and no frame is ever assembled. This is the device's natural end-of-stream indicator.
|
||||
|
||||
Handling logic in `read_bulk_waveform_stream`:
|
||||
```
|
||||
TimeoutError caught:
|
||||
if bytes_fed > 0 AND frames already collected:
|
||||
→ graceful end-of-stream; break loop; proceed to termination frame
|
||||
else (bytes_fed == 0, no prior frames):
|
||||
→ genuine transport failure; re-raise
|
||||
```
|
||||
|
||||
**Chunk timing (BE11529, 1024 sps, TCP/cellular):**
|
||||
|
||||
| Metric | Observed value |
|
||||
|---|---|
|
||||
| Chunk response time | ~1 s per chunk |
|
||||
| Chunks for a 9,306-sample event | 35 chunks |
|
||||
| Data per chunk (active signal) | 1,036–1,123 bytes |
|
||||
| Data per chunk (post-event silence) | 1,036 bytes (uniform) |
|
||||
| Safe recv timeout per chunk | **10 s** (10× typical) |
|
||||
| Default transport timeout | 120 s → ~2-min stall at end-of-stream |
|
||||
|
||||
Chunks with uniform 1,036-byte payload (chunks 17–35 in the observed event) contain all-zero ADC samples — the device continues recording silence until the configured record time expires before terminating the stream.
|
||||
|
||||
**ADC count-to-physical conversion:**
|
||||
|
||||
Raw samples are signed 16-bit integers (−32,768 to +32,767). To convert to physical units:
|
||||
```
|
||||
value_in_s (in/s) = counts × (geo_range / 32767)
|
||||
```
|
||||
where `geo_range` is from the compliance config (typically 10.000 in/s). Mic channel uses psi units with its own range. Near-full-scale values on all channels simultaneously indicate ADC saturation (clipping).
|
||||
|
||||
**Known decoder issue — fi==9 hardcoded skip:**
|
||||
|
||||
`_decode_a5_waveform()` contains `elif fi == 9: continue` from an earlier assumption that frame index 9 is always the device terminator. For streams with more than 9 frames, frame 9 is live waveform data. The skip discards ~1,070 bytes (~133 sample-sets) per event. Terminator detection should use `page_key == 0x0000`, not frame index. This skip should be removed.
|
||||
|
||||
---
|
||||
|
||||
## 8. Timestamp Format
|
||||
|
||||
Reference in New Issue
Block a user