fix: update STRT record parsing to reflect confirmed offsets and derive total/pretrig_samples from compliance config
This commit is contained in:
@@ -188,15 +188,30 @@ the `b'STRT'` magic bytes:
|
|||||||
| +10..13 | prev_event_key | ✅ 3 events |
|
| +10..13 | prev_event_key | ✅ 3 events |
|
||||||
| +18 | mode byte: 0x46 ('F') = single-shot, 0x0E = continuous | ✅ |
|
| +18 | mode byte: 0x46 ('F') = single-shot, 0x0E = continuous | ✅ |
|
||||||
|
|
||||||
**UNCONFIRMED — total_samples and pretrig_samples locations unknown:**
|
**CONFIRMED (2026-04-14) — total_samples and pretrig_samples are NOT stored in the STRT record.**
|
||||||
The prior documented offsets (+14..15 for total_samples, +16..17 for pretrig_samples) were
|
The prior documented offsets (+14..15 for total_samples, +16..17 for pretrig_samples) were
|
||||||
WRONG — confirmed by cross-checking STRT-derived rectime against compliance record_time
|
WRONG — confirmed by cross-checking STRT-derived rectime against compliance record_time
|
||||||
(4-14-26): all 3 events give STRT-derived rectime of 21–61 s vs actual 3.0 s (ratio 7–20×).
|
(4-14-26): all 4 events give STRT-derived rectime of 21–61 s vs actual 3.0 s (ratio 7–20×).
|
||||||
The "confirmed 4-9-26" note in prior versions was incorrect.
|
Extending the STRT dump to 32 bytes confirmed that bytes 21+ are the start of the raw ADC
|
||||||
|
waveform samples, not more STRT fields. Blastware itself derives total_samples and
|
||||||
|
pretrig_samples from the compliance config — exactly what our fallback does.
|
||||||
|
|
||||||
The true offsets for total_samples and pretrig_samples within STRT have not been located.
|
**The compliance-config fallback IS the correct permanent solution, not a workaround.**
|
||||||
**Until they are found, `_decode_a5_waveform` relies on the compliance-config cross-check
|
`_decode_a5_waveform` uses:
|
||||||
fallback for all total_samples and pretrig_samples values.**
|
- `pretrig_samples = round(0.25 × sample_rate)` (compliance monitoring standard)
|
||||||
|
- `total_samples = pretrig_samples + round(record_time × sample_rate)`
|
||||||
|
|
||||||
|
**CONFIRMED (2026-04-14) — waveform starts at strt_pos + 21 (no preamble).**
|
||||||
|
The original `sp + 27` skip (STRT 21B + null-pad 2B + 0xFF-sentinel 4B) was WRONG.
|
||||||
|
The 6-byte "preamble" in the 4-2-26 blast capture (`00 00 ff ff ff ff`) was actually the
|
||||||
|
first ~0.75 sample-sets of quiet pre-trigger ADC data misread as padding. Desk-thump
|
||||||
|
events show different bytes at positions 21-26 (e.g. `00 10 02 00 ff fc`) — they are real
|
||||||
|
ADC readings, not a fixed preamble. The `sp + 27` skip discarded 6 bytes of real waveform
|
||||||
|
data and misaligned the channel decode for all subsequent frames. Fixed: `wave = w[sp+21:]`.
|
||||||
|
|
||||||
|
The +6..9 next_key and +10..13 prev_key fields are confirmed across 4 events including the
|
||||||
|
first-event-after-erase case (prev_key = self-reference `01110000`; next_key = device
|
||||||
|
pre-allocates the predicted next slot even before any second event exists).
|
||||||
|
|
||||||
**CRITICAL — strt[18] is a record-mode byte, NOT rectime_seconds (confirmed 2026-04-14):**
|
**CRITICAL — strt[18] is a record-mode byte, NOT rectime_seconds (confirmed 2026-04-14):**
|
||||||
Analysis of 15 distinct STRT records across the 4-9-26 ACH capture shows:
|
Analysis of 15 distinct STRT records across the 4-9-26 ACH capture shows:
|
||||||
|
|||||||
+84
-93
@@ -1338,15 +1338,22 @@ def _decode_a5_waveform(
|
|||||||
|
|
||||||
── Frame structure ──────────────────────────────────────────────────────────
|
── Frame structure ──────────────────────────────────────────────────────────
|
||||||
A5[0] (probe response):
|
A5[0] (probe response):
|
||||||
db[7:] = [11-byte header] [21-byte STRT record] [6-byte preamble] [waveform ...]
|
db[7:] = [11-byte header] [21-byte STRT record] [waveform ...]
|
||||||
STRT: b'STRT' at offset 11, total 21 bytes
|
STRT: b'STRT' at offset 11, total 21 bytes
|
||||||
+8 uint16 BE: total_samples (expected full-record sample-sets)
|
+4..5 0xFF 0xFE (single-shot) or 0xFF 0xFD (continuous)
|
||||||
+16 uint16 BE: pretrig_samples (pre-trigger sample count)
|
+6..9 next_event_key4 (device pre-allocates next slot)
|
||||||
+18 uint8: rectime_seconds (record duration)
|
+10..13 prev_event_key4
|
||||||
Preamble: 6 bytes after the STRT record (confirmed from 4-2-26 blast capture):
|
+14..15 UNKNOWN (not total_samples)
|
||||||
bytes 21-22: 0x00 0x00 (null padding)
|
+16..17 UNKNOWN (not pretrig_samples)
|
||||||
bytes 23-26: 0xFF × 4 (sync sentinel / alignment marker)
|
+18 mode byte: 0x46 single-shot, 0x0E continuous
|
||||||
Waveform starts at strt_pos + 27 within db[7:].
|
+19..20 0x00 0x00
|
||||||
|
Waveform starts immediately at strt_pos + 21 (no preamble).
|
||||||
|
NOTE: The original 4-2-26 blast capture appeared to show a 6-byte
|
||||||
|
"preamble" (00 00 ff ff ff ff) after the STRT record, but this was
|
||||||
|
actually the first ~0.75 sample-sets of quiet pre-trigger ADC data
|
||||||
|
misread as padding. Confirmed 2026-04-14: bytes 21+ are raw waveform.
|
||||||
|
total_samples and pretrig_samples are NOT stored in the STRT record;
|
||||||
|
they are derived from the compliance config (the correct permanent source).
|
||||||
|
|
||||||
A5[1..N] (chunk responses):
|
A5[1..N] (chunk responses):
|
||||||
db[7:] = [8-byte per-frame header] [waveform bytes ...]
|
db[7:] = [8-byte per-frame header] [waveform bytes ...]
|
||||||
@@ -1380,91 +1387,48 @@ def _decode_a5_waveform(
|
|||||||
|
|
||||||
# STRT record layout (21 bytes, offsets relative to b'STRT'):
|
# STRT record layout (21 bytes, offsets relative to b'STRT'):
|
||||||
# +0..3 magic b'STRT'
|
# +0..3 magic b'STRT'
|
||||||
# +4..5 0xFF 0xFE (flags)
|
# +4..5 0xFF 0xFE (single-shot) or 0xFF 0xFD (continuous)
|
||||||
# +6..9 key4 (4-byte event key)
|
# +6..9 next_event_key4 (NOT current key — device pre-allocates next)
|
||||||
# +10..13 prev_key4
|
# +10..13 prev_event_key4
|
||||||
# +14..15 uint16 BE total_samples (full-record expected sample-set count)
|
# +14..15 UNKNOWN — confirmed NOT total_samples (confirmed 2026-04-14)
|
||||||
# +16..17 uint16 BE pretrig_samples
|
# +16..17 UNKNOWN — confirmed NOT pretrig_samples (confirmed 2026-04-14)
|
||||||
# +18 uint8 rectime_seconds
|
# +18 mode byte: 0x46='F' single-shot, 0x0E continuous — NOT rectime
|
||||||
#
|
# +19..20 0x00 0x00
|
||||||
# NOTE: strt[8:10] is the LOWER 2 bytes of key4, NOT total_samples.
|
# Bytes 21+ are raw ADC waveform samples — no preamble.
|
||||||
# Confirmed from raw_rx capture (4-9-26): strt[14:16] = total_samples. ✅
|
# total_samples / pretrig_samples are NOT stored in STRT at all.
|
||||||
# Extract 32 bytes so we can see past the first 21 — the true total_samples
|
# The compliance config fallback is the correct permanent source.
|
||||||
# and pretrig_samples offsets have not yet been confirmed and may be > +20.
|
strt = w0[strt_pos : strt_pos + 21]
|
||||||
strt = w0[strt_pos : strt_pos + 32]
|
|
||||||
if len(strt) < 21:
|
if len(strt) < 21:
|
||||||
log.warning("_decode_a5_waveform: STRT record truncated (%dB)", len(strt))
|
log.warning("_decode_a5_waveform: STRT record truncated (%dB)", len(strt))
|
||||||
return
|
return
|
||||||
|
|
||||||
log.info(
|
log.info(
|
||||||
"_decode_a5_waveform: STRT raw[0:32]: %s",
|
"_decode_a5_waveform: STRT raw[0:21]: %s",
|
||||||
strt[:32].hex(' '),
|
strt.hex(' '),
|
||||||
)
|
)
|
||||||
|
|
||||||
total_samples = struct.unpack_from(">H", strt, 14)[0]
|
# STRT bytes +14..17 are unknown fields — confirmed NOT total/pretrig_samples
|
||||||
pretrig_samples = struct.unpack_from(">H", strt, 16)[0]
|
# (2026-04-14). total_samples and pretrig_samples are derived from the
|
||||||
|
# compliance config, which is the correct permanent source.
|
||||||
# ── STRT sanity checks ───────────────────────────────────────────────────
|
_strt_invalid = True
|
||||||
# Check 1: pretrig must be strictly less than total_samples.
|
|
||||||
# Check 2 (cross-check): if compliance_config is available, the STRT-derived
|
|
||||||
# post-trigger record time must be within 2× of the configured record_time.
|
|
||||||
# A 3-second record at 1024 sps = 3072 post-trig samples; if STRT gives
|
|
||||||
# e.g. 21 s, something is wrong with the byte offsets.
|
|
||||||
#
|
|
||||||
# Both failures trigger the same fallback: derive pretrig from the compliance
|
|
||||||
# monitoring standard (0.25 s) and total from compliance record_time.
|
|
||||||
_strt_invalid = False
|
|
||||||
_sample_rate_default = 1024
|
_sample_rate_default = 1024
|
||||||
|
total_samples = 0
|
||||||
if pretrig_samples >= total_samples:
|
pretrig_samples = 0
|
||||||
log.warning(
|
rectime_seconds = 0
|
||||||
"_decode_a5_waveform: STRT check1 FAIL — pretrig_samples=%d >= "
|
|
||||||
"total_samples=%d. Raw strt[0:21]: %s "
|
|
||||||
"Will derive pretrig from compliance config.",
|
|
||||||
pretrig_samples, total_samples, strt[0:21].hex(' '),
|
|
||||||
)
|
|
||||||
pretrig_samples = 0
|
|
||||||
total_samples = 0
|
|
||||||
_strt_invalid = True
|
|
||||||
elif compliance_config is not None and compliance_config.record_time:
|
|
||||||
cc_rt = compliance_config.record_time
|
|
||||||
cc_sr = compliance_config.sample_rate or _sample_rate_default
|
|
||||||
strt_post_trig = (total_samples - pretrig_samples) / cc_sr
|
|
||||||
# Allow up to 2× tolerance to account for zero-padding beyond the window.
|
|
||||||
# Anything more than that is a layout error, not zero-padding.
|
|
||||||
if strt_post_trig > cc_rt * 2.0 or strt_post_trig < cc_rt * 0.5:
|
|
||||||
log.warning(
|
|
||||||
"_decode_a5_waveform: STRT check2 FAIL — STRT-derived rectime "
|
|
||||||
"%.1fs is implausible vs compliance record_time=%.1fs (ratio=%.1f×). "
|
|
||||||
"Raw strt[0:21]: %s Falling back to compliance-config values.",
|
|
||||||
strt_post_trig, cc_rt, strt_post_trig / cc_rt,
|
|
||||||
strt[0:21].hex(' '),
|
|
||||||
)
|
|
||||||
pretrig_samples = 0
|
|
||||||
total_samples = 0
|
|
||||||
_strt_invalid = True
|
|
||||||
|
|
||||||
# strt[18] is a record-mode/type byte, NOT rectime in seconds.
|
|
||||||
# Confirmed from analysis of 4-9-26 ACH capture (15 distinct events):
|
|
||||||
# flags=0xFFFE (single-shot) → strt[18]=0x46 ('F') for all events regardless of duration
|
|
||||||
# flags=0xFFFD (continuous) → strt[18]=0x0E for all events regardless of duration
|
|
||||||
# The actual post-trigger record time must be derived from total_samples and pretrig_samples.
|
|
||||||
rectime_seconds = int(round(
|
|
||||||
max(0, total_samples - pretrig_samples) / _sample_rate_default
|
|
||||||
))
|
|
||||||
|
|
||||||
event.total_samples = total_samples
|
|
||||||
event.pretrig_samples = pretrig_samples
|
|
||||||
event.rectime_seconds = rectime_seconds
|
|
||||||
|
|
||||||
log.info(
|
log.info(
|
||||||
"_decode_a5_waveform: STRT total_samples=%d pretrig=%d "
|
"_decode_a5_waveform: STRT flags=0x%04X next_key=%s prev_key=%s "
|
||||||
"strt[18]=0x%02X (mode byte) computed_rectime=%ds "
|
"mode=0x%02X → using compliance-config for total/pretrig",
|
||||||
"raw strt[0:21]: %s",
|
struct.unpack_from(">H", strt, 4)[0],
|
||||||
total_samples, pretrig_samples, strt[18], rectime_seconds,
|
strt[6:10].hex(),
|
||||||
strt[0:21].hex(' '),
|
strt[10:14].hex(),
|
||||||
|
strt[18],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
event.total_samples = 0 # will be overwritten by compliance-config fallback below
|
||||||
|
event.pretrig_samples = 0
|
||||||
|
event.rectime_seconds = 0
|
||||||
|
|
||||||
# ── Collect per-frame waveform bytes with global offset tracking ─────────
|
# ── Collect per-frame waveform bytes with global offset tracking ─────────
|
||||||
# global_offset is the cumulative byte count across all frames, used to
|
# global_offset is the cumulative byte count across all frames, used to
|
||||||
# compute the channel alignment at each frame boundary.
|
# compute the channel alignment at each frame boundary.
|
||||||
@@ -1474,13 +1438,21 @@ def _decode_a5_waveform(
|
|||||||
for fi, db in enumerate(frames_data):
|
for fi, db in enumerate(frames_data):
|
||||||
w = db[7:]
|
w = db[7:]
|
||||||
|
|
||||||
# A5[0]: waveform begins after the 21-byte STRT record and 6-byte preamble.
|
# A5[0]: waveform begins immediately after the 21-byte STRT record.
|
||||||
# Layout: STRT(21B) + null-pad(2B) + 0xFF sentinel(4B) = 27 bytes total.
|
# Confirmed 2026-04-14: there is NO preamble after STRT — bytes 21+
|
||||||
|
# are raw ADC sample data. The earlier sp+27 skip was eating 6 bytes
|
||||||
|
# of real waveform, misaligning the channel decode for all subsequent
|
||||||
|
# frames.
|
||||||
if fi == 0:
|
if fi == 0:
|
||||||
sp = w.find(b"STRT")
|
sp = w.find(b"STRT")
|
||||||
if sp < 0:
|
if sp < 0:
|
||||||
continue
|
continue
|
||||||
wave = w[sp + 27 :]
|
wave = w[sp + 21 :]
|
||||||
|
log.info(
|
||||||
|
"_decode_a5_waveform: A5[0] waveform starts at sp+21; "
|
||||||
|
"first 24 wave bytes: %s",
|
||||||
|
wave[:24].hex(' '),
|
||||||
|
)
|
||||||
|
|
||||||
# Frame 7 carries event-time metadata strings ("Project:", "Client:", …)
|
# Frame 7 carries event-time metadata strings ("Project:", "Client:", …)
|
||||||
# and no waveform ADC data.
|
# and no waveform ADC data.
|
||||||
@@ -1556,11 +1528,29 @@ def _decode_a5_waveform(
|
|||||||
|
|
||||||
running_offset += len(wave)
|
running_offset += len(wave)
|
||||||
|
|
||||||
|
n_decoded_total = len(tran)
|
||||||
log.debug(
|
log.debug(
|
||||||
"_decode_a5_waveform: decoded %d alignment-corrected sample-sets "
|
"_decode_a5_waveform: decoded %d alignment-corrected sample-sets "
|
||||||
"(skipped %d due to frame boundary misalignment)",
|
"(skipped %d due to frame boundary misalignment)",
|
||||||
len(tran), n_sets - len(tran),
|
n_decoded_total, n_sets - n_decoded_total,
|
||||||
)
|
)
|
||||||
|
# Log first 16 and last 8 samples for every channel — essential for
|
||||||
|
# validating decoder output and diagnosing flatline / misalignment issues.
|
||||||
|
_N = min(16, n_decoded_total)
|
||||||
|
_L = max(0, n_decoded_total - 8)
|
||||||
|
log.info(
|
||||||
|
"_decode_a5_waveform: first %d samples — "
|
||||||
|
"Tran=%s Vert=%s Long=%s Mic=%s",
|
||||||
|
_N,
|
||||||
|
tran[:_N], vert[:_N], long_[:_N], mic[:_N],
|
||||||
|
)
|
||||||
|
if n_decoded_total > 16:
|
||||||
|
log.info(
|
||||||
|
"_decode_a5_waveform: last 8 samples (idx %d–%d) — "
|
||||||
|
"Tran=%s Vert=%s Long=%s Mic=%s",
|
||||||
|
_L, n_decoded_total - 1,
|
||||||
|
tran[_L:], vert[_L:], long_[_L:], mic[_L:],
|
||||||
|
)
|
||||||
|
|
||||||
event.raw_samples = {
|
event.raw_samples = {
|
||||||
"Tran": tran,
|
"Tran": tran,
|
||||||
@@ -1569,10 +1559,10 @@ def _decode_a5_waveform(
|
|||||||
"Mic": mic,
|
"Mic": mic,
|
||||||
}
|
}
|
||||||
|
|
||||||
# ── Correct STRT-invalid pretrig/total from decoded count + compliance config ──
|
# ── Derive pretrig/total from compliance config (always — STRT doesn't store them) ──
|
||||||
# When the STRT layout was suspect (pretrig >= total_samples), both values were
|
# total_samples and pretrig_samples are not stored in the STRT record (confirmed
|
||||||
# zeroed out earlier. Now that we know the actual decoded sample count, use it
|
# 2026-04-14). They are derived from compliance config record_time × sample_rate.
|
||||||
# together with the compliance config record_time to derive a meaningful pretrig.
|
# _strt_invalid is always True; this block always runs.
|
||||||
#
|
#
|
||||||
# Formula:
|
# Formula:
|
||||||
# post_trig = record_time (s) × sample_rate (sps)
|
# post_trig = record_time (s) × sample_rate (sps)
|
||||||
@@ -1604,11 +1594,12 @@ def _decode_a5_waveform(
|
|||||||
event.pretrig_samples = derived_pretrig
|
event.pretrig_samples = derived_pretrig
|
||||||
event.rectime_seconds = int(round(cc_rt))
|
event.rectime_seconds = int(round(cc_rt))
|
||||||
log.info(
|
log.info(
|
||||||
"_decode_a5_waveform: STRT was invalid — using pretrig=%d "
|
"_decode_a5_waveform: pretrig=%d (%.2fs compliance default) "
|
||||||
"(%.2fs default) from compliance standard; record_time=%.1fs "
|
"post_trig=%d total=%d record_time=%.1fs "
|
||||||
"sr=%d sps decoded=%d samples display_total=%d",
|
"sr=%d sps decoded=%d samples",
|
||||||
derived_pretrig, _PRETRIG_SECONDS_DEFAULT,
|
derived_pretrig, _PRETRIG_SECONDS_DEFAULT,
|
||||||
cc_rt, cc_sr, n_decoded, event.total_samples,
|
post_trig_samples, event.total_samples,
|
||||||
|
cc_rt, cc_sr, n_decoded,
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
event.total_samples = n_decoded
|
event.total_samples = n_decoded
|
||||||
|
|||||||
Reference in New Issue
Block a user