docs: document 5A end-of-stream signal, chunk timing, fi==9 bug, ADC conversion

Adds §7.8.4 to protocol reference and corresponding CLAUDE.md sections:

- End-of-stream: device sends exactly 1 raw byte after last chunk; handled
  via TimeoutError + bytes_fed>0 check → graceful break to termination
- Chunk timing: ~1s per chunk, 35 chunks for a 9,306-sample event, safe
  timeout is 10s (not default 120s)
- fi==9 decoder bug: hardcoded skip drops ~133 sample-sets per event;
  noted as known issue pending fix
- ADC conversion: counts × (range/32767) → physical units (in/s for geo)

Changelog entries added for all four items (2026-04-06).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-06 21:28:26 -04:00
parent 781d21f132
commit 9bef430451
2 changed files with 69 additions and 0 deletions
+25
View File
@@ -157,6 +157,31 @@ record — 5A remains the sole source for those fields and they are set uncondit
`stop_after_metadata=True` (default) stops the 5A loop as soon as `b"Project:"` appears,
then sends the termination frame.
### SUB 5A — end-of-stream signal (confirmed 2026-04-06)
After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to
the next chunk request, then goes silent. This is the natural end-of-stream indicator — NOT
a complete A5 frame. `S3FrameParser.bytes_fed` will be 1; no frame is assembled.
Handling: on `TimeoutError`, if `bytes_fed > 0` AND frames were already collected, treat as
graceful end-of-stream, break the loop, and proceed to the termination frame. If `bytes_fed
== 0` with no prior frames, it is a genuine transport failure — re-raise.
**Chunk recv timeout must be 10 s, not the default 120 s.** Chunks arrive within ~1 s each.
Using 120 s causes a ~2-minute stall at every end-of-stream detection. The `_recv_one` call
in the chunk loop passes `timeout=10.0` explicitly.
**Typical chunk count (BE11529, 1024 sps):** A 9,306-sample event produces 35 chunks before
end-of-stream. Chunks with uniform 1,036-byte data are all-zero ADC samples (post-event
silence). Only the initial variable-size chunks contain actual signal.
### SUB 5A — known decoder issue: fi==9 hardcoded skip (not yet fixed)
`_decode_a5_waveform()` in `client.py` has `elif fi == 9: continue` — a leftover from the
9-frame original blast capture where frame 9 was assumed to be a terminator. For current
35-frame streams, fi==9 is live waveform data (~133 sample-sets dropped). Terminator
detection is via `page_key == 0x0000`, not frame index. This skip should be removed.
### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)
**token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and
+44
View File
@@ -89,6 +89,10 @@
| 2026-04-06 | §6.1 | **CONFIRMED — browse 1F must be conditional.** Calling 1F(browse=True/all-zero) after a FAILED 5A disrupts device state and causes the next event's 5A probe to time out with 0 bytes received. Browse 1F is only called after a SUCCESSFUL 5A. Failure fallback: use the key returned by the prior 1F(arm/0xFE) call. |
| 2026-04-06 | §7.8 | **ADDED — `bytes_fed` diagnostic counter on S3FrameParser.** Counts raw bytes fed to the parser since last `reset()`. Logged at WARNING when 5A probe times out — distinguishes "device sent no bytes at all" from "device responded but frame was malformed or had wrong SUB". |
| 2026-04-06 | §7.8.2 | **CORRECTED — SUB 5A chunk counter is monotonic for ALL chunks.** Previous doc hard-coded chunk 1 counter as `0x1004` (from 4-2-26 BW TX capture). This was a Blastware artifact. Empirically confirmed: `counter = chunk_num * 0x0400` works (device responds immediately); counter=0x1004 for chunk 1 causes 120 s timeout. BW's true internal formula appears to be `key4[2:4] + n * 0x0400` — for event 1 (key `01110000`) this equals `n * 0x0400`. The device does not strictly validate counter values. |
| 2026-04-06 | §7.8.4 | **NEW — 5A end-of-stream signalling confirmed.** After streaming all waveform chunks, the device sends exactly **1 raw byte** in response to the next chunk request, then goes silent for the full recv timeout. This byte is NOT a complete DLE-framed A5 response — the frame parser accumulates it as `bytes_fed=1` and never assembles a frame. This is the device's natural end-of-stream signal. Handling: on TimeoutError, if `bytes_fed > 0` AND prior chunks were received, treat as graceful end and proceed to the termination frame. A `bytes_fed=0` timeout with no prior chunks is a genuine transport failure and must still raise. |
| 2026-04-06 | §7.8.4 | **NEW — 5A chunk timing and count (empirical, BE11529 at 1024 sps).** Each chunk response arrives within ~1 second over TCP/cellular. A 9,306-sample event (≈9.1 s at 1024 sps) produces **35 chunks** before end-of-stream. Chunks 116 have varying data lengths (10361123 bytes); chunks 1735 are uniformly 1036 bytes each (post-event silence, all-zero ADC samples). Safe recv timeout for chunk loop: **10 s** (10× typical response time). Default transport timeout (120 s) results in a ~2-minute stall per event at end-of-stream. |
| 2026-04-06 | §7.8.3 | **KNOWN ISSUE — `_decode_a5_waveform` hardcoded fi==9 skip.** The decoder contains `elif fi == 9: continue` which was written for the 9-frame original blast capture where frame 9 was a device terminator. For streams with >9 frames (current device produces 35+), frame index 9 is live waveform data — this skip discards ~1,070 bytes (~133 sample-sets) per event. The terminator is now detected via `page_key == 0x0000`, not by frame index. The fi==9 skip should be removed. |
| 2026-04-06 | §7.8 | **CONFIRMED — ADC count-to-physical-unit conversion.** Raw waveform samples are signed 16-bit integers (counts). Conversion: `value = counts × (range / 32767)`. For geo channels: range = 10.000 in/s (from the device's compliance config geo range field). For the mic channel: range is in psi (device-specific). Near-full-scale counts (≈32,700) on all four channels simultaneously indicate ADC saturation (clipping) from a high-amplitude event. |
---
@@ -1180,6 +1184,46 @@ for ASCII labels with a null-terminated value read:
All five fields reflect the **setup at event-record time**, not the current device config.
#### 7.8.4 End-of-Stream Behaviour and Chunk Timing
> ✅ **Confirmed 2026-04-06** — empirical observation on BE11529 (S338.17) over TCP/cellular.
**End-of-stream signal:** After sending all waveform chunks, the device sends exactly **1 raw byte** in response to the next chunk request, then goes silent. This byte is not a complete DLE-framed A5 response — `S3FrameParser.bytes_fed` reports 1 and no frame is ever assembled. This is the device's natural end-of-stream indicator.
Handling logic in `read_bulk_waveform_stream`:
```
TimeoutError caught:
if bytes_fed > 0 AND frames already collected:
→ graceful end-of-stream; break loop; proceed to termination frame
else (bytes_fed == 0, no prior frames):
→ genuine transport failure; re-raise
```
**Chunk timing (BE11529, 1024 sps, TCP/cellular):**
| Metric | Observed value |
|---|---|
| Chunk response time | ~1 s per chunk |
| Chunks for a 9,306-sample event | 35 chunks |
| Data per chunk (active signal) | 1,0361,123 bytes |
| Data per chunk (post-event silence) | 1,036 bytes (uniform) |
| Safe recv timeout per chunk | **10 s** (10× typical) |
| Default transport timeout | 120 s → ~2-min stall at end-of-stream |
Chunks with uniform 1,036-byte payload (chunks 1735 in the observed event) contain all-zero ADC samples — the device continues recording silence until the configured record time expires before terminating the stream.
**ADC count-to-physical conversion:**
Raw samples are signed 16-bit integers (32,768 to +32,767). To convert to physical units:
```
value_in_s (in/s) = counts × (geo_range / 32767)
```
where `geo_range` is from the compliance config (typically 10.000 in/s). Mic channel uses psi units with its own range. Near-full-scale values on all channels simultaneously indicate ADC saturation (clipping).
**Known decoder issue — fi==9 hardcoded skip:**
`_decode_a5_waveform()` contains `elif fi == 9: continue` from an earlier assumption that frame index 9 is always the device terminator. For streams with more than 9 frames, frame 9 is live waveform data. The skip discards ~1,070 bytes (~133 sample-sets) per event. Terminator detection should use `page_key == 0x0000`, not frame index. This skip should be removed.
---
## 8. Timestamp Format