feat: implement raw ADC waveform decoding and download functionality
- Added `_decode_a5_waveform()` to parse SUB 5A frames into per-channel time-series data. - Introduced `download_waveform(event)` method in `MiniMateClient` to fetch full waveform data. - Updated `Event` model to include new fields: `total_samples`, `pretrig_samples`, `rectime_seconds`, and `_waveform_key`. - Enhanced documentation in `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect new features and confirmed protocol details.
This commit is contained in:
Brian Harrison
@@ -77,6 +77,9 @@
|
||||
| 2026-04-02 | §7.7.5 | **CONFIRMED — Event-time metadata source.** `Client:`, `User Name:`, and `Seis Loc:` strings are present in **A5 frame 7** of the SUB 5A bulk waveform stream — they are NOT in the 210-byte SUB 0C waveform record. They reflect the compliance setup active when the event was stored on the device (not the current setup). `get_events()` now issues SUB 5A after each 0C download. Sequence: `1E → 0A → 0C → 5A → 1F`. |
|
||||
| 2026-04-02 | §7.6.2 | **FIXED — Compliance config orphaned send bug.** An extra `self._send(SUB_COMPLIANCE / 0x2A / DATA_PARAMS)` before the B/C/D receive loop had no corresponding `recv_one()`. Every receive in the loop was consuming the previous send's response, leaving frame D's channel block unread. Bug removed. Total config bytes now ~2126 (was ~1071 due to truncation). `trigger_level_geo`, `alarm_level_geo`, `max_range_geo` are now correctly populated. |
|
||||
| 2026-04-02 | §7.6.1 | **CORRECTED — Anchor search range.** Previous doc stated anchor search range `cfg[40:100]`. With the orphaned-send bug fixed, the 44-byte header padding is gone and the anchor now appears at `cfg[11]`. Corrected to `cfg[0:150]`. |
|
||||
| 2026-04-03 | §7.6 | **CONFIRMED — Blast waveform format (4-2-26 capture).** Blast/waveform-mode SUB 5A stream uses 4-channel interleaved signed int16 LE, 8 bytes per sample-set [T,V,L,M]. NOT the 32-byte block format (which is noise/histogram mode only). Frame sizes are NOT multiples of 8 — cross-frame alignment correction required (track global byte offset mod 8; skip `(8-align)%8` bytes at each frame start). A5[0] STRT record confirmed: 21 bytes at db[7:]+11; waveform starts at strt_pos+27 (after 2-byte null pad + 4-byte 0xFF sentinel). Frame index 7 = metadata only, no ADC data. Full §7.6 rewritten. |
|
||||
| 2026-04-03 | §7.6 | **CONFIRMED — Noise block format details.** 32-byte blocks: LE uint16 type + LE uint16 ctr + 9×int16 LE samples + 10B metadata. Samples are little-endian (previous doc said big-endian — WRONG). Type: 0x0016=sync (appears at start of each A5 frame), 0x0000=data. Noise floor ≈ 9–11 counts. Metadata fixed pattern `00 01 43 [2B var] 00 [pretrig] [rectime] 00 00` confirmed. |
|
||||
| 2026-04-03 | client.py | **NEW — `_decode_a5_waveform()` and `download_waveform()` implemented.** `_decode_a5_waveform(frames_data, event)` decodes full A5 waveform stream into `event.raw_samples = {"Tran":[…], "Vert":[…], "Long":[…], "Mic":[…]}`. Populates `event.total_samples`, `event.pretrig_samples`, `event.rectime_seconds` from STRT record. Handles cross-frame alignment. `MiniMateClient.download_waveform(event)` calls `read_bulk_waveform_stream(stop_after_metadata=False)` then invokes the decoder. Waveform key stored on Event as `_waveform_key` during `get_events()`. |
|
||||
|
||||
---
|
||||
|
||||
@@ -722,20 +725,110 @@ MicL: 39 64 1D AA = 0.0000875 psi
|
||||
|
||||
### 7.6 Bulk Waveform Stream (SUB A5) — Raw ADC Sample Records
|
||||
|
||||
Each repeating record (🔶 INFERRED structure):
|
||||
**Two distinct formats exist depending on recording mode. Both confirmed from captures.**
|
||||
|
||||
---
|
||||
|
||||
#### 7.6.1 Blast / Waveform mode — ✅ CONFIRMED (4-2-26 capture)
|
||||
|
||||
4-channel interleaved signed 16-bit little-endian, 8 bytes per sample-set:
|
||||
|
||||
```
|
||||
[CH_ID] [S0_HI] [S0_LO] [S1_HI] [S1_LO] ... [S8_HI] [S8_LO] [00 00] [01] [PEAK × 3 bytes]
|
||||
01 00 0A 00 0B 43 xx xx
|
||||
[T_lo T_hi V_lo V_hi L_lo L_hi M_lo M_hi] × N sample-sets
|
||||
```
|
||||
|
||||
- `CH_ID` — Channel identifier. `01` consistently observed. Full mapping unknown. 🔶 INFERRED
|
||||
- 9× signed 16-bit big-endian ADC samples. Noise floor ≈ `0x000A`–`0x000B`
|
||||
- `00 00` — separator / padding
|
||||
- `01` — unknown flag byte
|
||||
- 3-byte partial IEEE 754 float — peak value for this sample window. `0x43` prefix = range 130–260
|
||||
- **T** = Transverse (Tran), **V** = Vertical (Vert), **L** = Longitudinal (Long), **M** = Microphone
|
||||
- Channel order follows the Blastware convention: Tran is always first (ch[0]).
|
||||
- Encoding: signed int16 little-endian. Full scale = ±32768 counts.
|
||||
- Sample rate: set by compliance config (typical: 1024 Hz for blast monitoring).
|
||||
- Each A5 frame chunk carries a different number of waveform bytes. Frame sizes
|
||||
are NOT multiples of 8, so naive concatenation scrambles channel assignments at
|
||||
frame boundaries. **Always track cumulative byte offset mod 8 to correct alignment.**
|
||||
|
||||
> ❓ SPECULATIVE: At 1024 sps, 9 samples ≈ 8.8ms per record. Sample rate unconfirmed from captured data alone.
|
||||
**A5[0] frame layout:**
|
||||
|
||||
```
|
||||
db[7:]: [11-byte header] [21-byte STRT record] [6-byte preamble] [waveform ...]
|
||||
STRT: offset 11 in db[7:]
|
||||
+0..3 b'STRT' magic
|
||||
+8..9 uint16 BE total_samples (full-record expected sample-set count)
|
||||
+16..17 uint16 BE pretrig_samples (pre-trigger window, in sample-sets)
|
||||
+18 uint8 rectime_seconds
|
||||
preamble: +19..20 0x00 0x00 null padding
|
||||
+21..24 0xFF × 4 synchronisation sentinel
|
||||
Waveform: starts at strt_pos + 27 within db[7:]
|
||||
```
|
||||
|
||||
**A5[1..N] frame layout (non-metadata frames):**
|
||||
|
||||
```
|
||||
db[7:]: [8-byte per-frame header] [waveform ...]
|
||||
Header: [counter LE uint16, 0x00 × 6] — frame sequence counter (0, 8, 12, 16, 20, …×0x400)
|
||||
Waveform: starts at byte 8 of db[7:]
|
||||
```
|
||||
|
||||
**Special frames:**
|
||||
|
||||
| Frame index | Contents |
|
||||
|---|---|
|
||||
| A5[0] | Probe response: STRT record + first waveform chunk |
|
||||
| A5[7] | Event-time metadata strings only (no waveform data) |
|
||||
| A5[9] | Terminator frame (page_key=0x0000) — ignored |
|
||||
| A5[1..6,8] | Waveform chunks |
|
||||
|
||||
**Confirmed from 4-2-26 blast capture (total_samples=9306, pretrig=298, rate=1024 Hz):**
|
||||
|
||||
```
|
||||
Frame Waveform bytes Cumulative Align(mod 8)
|
||||
A5[0] 933B 933B 0
|
||||
A5[1] 963B 1896B 5
|
||||
A5[2] 946B 2842B 0
|
||||
A5[3] 960B 3802B 2
|
||||
A5[4] 952B 4754B 2
|
||||
A5[5] 946B 5700B 2
|
||||
A5[6] 941B 6641B 4
|
||||
A5[8] 992B 7633B 1
|
||||
Total: 7633B → 954 naive sample-sets, 948 alignment-corrected
|
||||
```
|
||||
|
||||
Only 948 of 9306 sample-sets captured (10%) — `stop_after_metadata=True` terminated
|
||||
download after A5[7] was received.
|
||||
|
||||
**Channel identification note:** The 4-2-26 blast saturated all four geophone channels
|
||||
to near-maximum ADC output (~32000–32617 counts). Channel ordering [Tran, Vert, Long, Mic]
|
||||
= [ch0, ch1, ch2, ch3] is the Blastware convention and is consistent with per-channel PPV
|
||||
values (Tran=0.420, Vert=3.870, Long=0.495 in/s from 0C record), but cannot be
|
||||
independently confirmed from a fully-saturating event alone.
|
||||
|
||||
---
|
||||
|
||||
#### 7.6.2 Noise monitoring / Histogram mode — ✅ CONFIRMED (3-31-26 capture)
|
||||
|
||||
32-byte blocks with the following layout:
|
||||
|
||||
```
|
||||
Offset Size Type Description
|
||||
0 2 uint16 LE block type: 0x0016=sync, 0x0000=data
|
||||
2 2 uint16 LE block counter (ctr)
|
||||
4 18 int16 LE × 9 ADC samples
|
||||
22 10 bytes metadata: [00 01 43 VAR VAR 00 pretrig rectime 00 00]
|
||||
```
|
||||
|
||||
- Sync blocks (type=0x0016) appear at the start of each A5 frame; ctr=0 in sync blocks.
|
||||
- Data blocks (type=0x0000) carry actual sample data. First data block ctr=288 (empirical,
|
||||
not yet decoded — likely related to a pre-trigger sample offset).
|
||||
- Metadata fixed bytes: `00 01 43` then 2 variable bytes, then `00 [pretrig] [rectime] 00 00`.
|
||||
Pretrig byte = 0x1E (30) and rectime byte = 0x0A (10) for the 3-31-26 capture.
|
||||
- 9 samples per block (int16 LE, NOT big-endian). Noise floor ≈ 9–11 counts.
|
||||
- **This is a different recording mode** from waveform/blast — the device firmware uses
|
||||
32-byte blocks for histogram/noise monitoring and 4-channel continuous for waveform events.
|
||||
|
||||
> ❓ **Open:** The 9-sample-per-block structure does not divide evenly into 4 channels.
|
||||
> Whether these represent a single channel, all channels in rotation, or downsampled
|
||||
> aggregates is not yet determined. The first data block ctr=288 vs pretrig=30 is also
|
||||
> unexplained — possibly counting in units other than sample-sets.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user