fix: change db7 to 5byte frame header from 8byte header

2026-04-16 13:57:26 -04:00
parent a46961c124
commit 145074d8b5
2 changed files with 66 additions and 12 deletions
@@ -316,13 +316,14 @@ The device bulk-streams flash pages beyond the configured record window.  Un-wri
 pages read as 0xFF.  Decoded as int16 LE, `0xFF 0xFF = -1`, which maps to ~0 in/s after
 the geo scale factor is applied — producing a visible flat-line at the end of the waveform.

-**Confirmed from diagnostic output (desk-thump event, 1024 sps, record_time=3.0 s):**
+**Confirmed from raw capture analysis (desk-thump event key=01110000, 1024 sps,
+record_time=3.0 s, ach_inbound_20260412_231505):**
 - total_samples = 256 (pretrig) + 3072 (post) = 3328
- samples_decoded = 4417 (36 frames)
- Excess tail: 4417 − 3328 = 1089 samples (8.7 frames of 0xFF data)
- Flat-line onset: sample 1960, t=1664ms post-trigger (within the active signal window
-  — earlier than expected because fi=4 re-probe and fi=15 false-positive removed ~270
-  samples; once those bugs are fixed the real onset should be at or beyond total_samples)
+- A5 stream: 37 frames total; frames 8–35 are uniform 1036-byte 0xFF data (unwritten flash)
+- Actual ADC signal: frames 0 (probe) + 1–3 + 5 + 7 = 6 data chunks ≈ 815 sample-sets (0.80 s)
+- The desk-thump vibration was genuinely short; the remaining ~2.2 s of the record window
+  is unwritten flash (0xFF), not signal.  The flatline is physically correct.
+- After 0xFF tail clip: display = min(decoded, 3328) excludes the all-zeros post-record padding

 **Fix (two parts):**
 1. `_decode_a5_waveform` (Python): already returns `total_samples` alongside
@@ -333,6 +334,26 @@ the geo scale factor is applied — producing a visible flat-line at the end of
   plotting — `(channels[ch] || []).slice(0, display)`.  Without this, the chart rendered
   all `decoded` samples including the 0xFF tail.

+### SUB 5A — chunk frame header is 5 bytes, NOT 8 (CORRECTED 2026-04-15)
+
+**`_decode_a5_waveform` uses `wave = w[5:]` to strip the per-chunk frame header.**
+
+The header is 5 bytes: 2 counter bytes + 3 zero bytes.  The code previously used `w[8:]`
+(8-byte strip) — this was WRONG and discarded 3 bytes of real waveform data per frame,
+misaligning the channel decode for all frames after any non-multiple-of-8 frame.
+
+**Proof — "Standard Recording Setup" cross-frame continuity test (MITM capture 4-11-26):**
+The ASCII string `"Standard Recording Setup"` spans the A5[5]→A5[6] frame boundary:
+- A5[5].w last 2 bytes: `53 74` = `"St"` (end of "Standard")
+- A5[6].w[0:5] = `00 00 00 00 00` (5 null bytes = header)
+- A5[6].w[5:] begins with `61 6e 64` = `"and"` (start of "andard Recording Setup…")
+
+Only `header=5` produces the contiguous string `"Standard Recording Setup"`.
+`header=6/7/8` all produce broken text (tested exhaustively).
+
+**Also updated:** The probe frame (fi=0) strip of `w[10:]` is unchanged — the STRT magic
+itself is at `w[10]` inside the probe frame, which has a different 10-byte preamble.
+
 ### SUB 1E / 1F — event iteration null sentinel and token position (FIXED, do not re-introduce)

 **token_params bug (FIXED):** The token byte was at `params[6]` (wrong). Both 3-31-26 and
@@ -1079,6 +1100,32 @@ call-home.

 ---

+## Known broken / deferred
+
+### Waveform chart Y-axis scaling (see GitHub issue)
+
+**The waveform chart in the web viewer shows incorrect amplitudes — DO NOT trust the Y-axis values.**
+
+PPV values reported by the system (stored in the DB, shown in event summaries) come from
+the **0C waveform record** computed on-device and are correct.  The issue is purely in the
+ADC→in/s conversion used to draw the chart.
+
+What we know:
+- Decoded ADC peaks from 5A (~30703–32661 counts) imply ~9× larger PPV than the 0C record reports
+  (e.g. Vert 5.8 in/s decoded vs 0.690 in/s from 0C for the same event)
+- ALL four channels show near-full-scale peaks, which is implausible for a desk-thump event
+  where Vert should dominate
+- The `max_range_geo` value in the compliance config is 6.206053 in/s (not 10.000 or 1.25
+  as displayed in the Blastware UI) — unclear whether this is an ADC full-scale or something else
+- The empirical full-scale implied by 0C data is ~0.736 in/s, not 6.206
+- Tried: LE vs BE decode, header=5 vs 8, float32 format — all produce near-full-scale peaks
+- Root cause unknown; likely requires a Blastware-generated waveform export for the same
+  event to reverse-engineer the correct scaling formula
+
+Tracking: **GitHub issue #TODO** (create before resuming this work)
+
+---
+
 ## What's next

 - **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
@@ -1356,9 +1356,12 @@ def _decode_a5_waveform(
        they are derived from the compliance config (the correct permanent source).

    A5[1..N] (chunk responses):
-        db[7:]  = [8-byte per-frame header] [waveform bytes ...]
-        Header:   [ctr LE uint16, 0x00 × 6] — frame sequence counter
-        Waveform starts at byte 8 of db[7:].
+        db[7:]  = [5-byte per-frame header] [waveform bytes ...]
+        Header:   [ctr LE uint16, 0x00 × 3] — frame sequence counter + 3 null bytes
+        Waveform starts at byte 5 of db[7:].
+        NOTE: Previously documented as 8-byte header — INCORRECT.  Confirmed 5 bytes
+        via "Standard Recording Setup" cross-frame continuity test on MITM capture
+        (4-11-26): A5[5] ends "St", A5[6].w[0:5] = 5 nulls, w[5:]= "andard…" ✓.

    ── Cross-frame alignment ────────────────────────────────────────────────────
    Frame waveform chunk sizes are NOT multiples of 8.  Naive concatenation
@@ -1491,10 +1494,14 @@ def _decode_a5_waveform(
        # (read_bulk_waveform_stream returns early on page_key==0).
        # No hardcoded frame-index skip here — all non-metadata frames are data.
        else:
-            # Strip the 8-byte per-frame header (ctr + 6 zero bytes)
-            if len(w) < 8:
+            # Strip the 5-byte per-frame header (ctr[2] + 3 zero bytes + 1 flag byte).
+            # Confirmed 2026-04-15 via "Standard Recording Setup" cross-frame continuity
+            # test on MITM capture (4-11-26): A5[5] ends with "St", A5[6].w[0:5] are
+            # 5 null bytes, w[5:] begins with "andard Recording Setup…" — contiguous iff
+            # header=5.  Previously documented as 8 bytes — INCORRECT.
+            if len(w) < 5:
                continue
-            wave = w[8:]
+            wave = w[5:]

        if len(wave) < 2:
            continue