serversdown/seismo-relay
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: decode waveform record timestamp, record type, and Peak Vector Sum

Browse Source 
Confirmed 2026-04-01 against Blastware event report for BE11529 thump
event ("00:28:12 April 1, 2026", PVS 3.906 in/s).

models.py:
- Timestamp.from_waveform_record(): decode 9-byte format from 0C record
  bytes[0-8]: [day][sub_code][month][year:2BE][?][hour][min][sec]
- Timestamp: add hour/minute/second optional fields; __str__ includes
  time when available
- PeakValues: add peak_vector_sum field (confirmed fixed offset 87)

client.py:
- _decode_waveform_record_into: add timestamp decode from bytes[0:9]
- _extract_record_type: decode byte[1] (sub_code), not ASCII string
  search; 0x10 → "Waveform", histogram TBD
- _extract_peak_floats: add PVS from offset 87 (IEEE 754 BE float32)
  = √(T²+V²+L²) at max instantaneous vector moment

sfm/server.py:
- _serialise_timestamp: add hour/minute/second/day fields to JSON
- _serialise_peak_values: add peak_vector_sum to JSON

docs: update §7.7.5 and §8 with confirmed 9-byte timestamp layout,
PVS field, and byte[1] record type encoding; update command table;
close resolved open questions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Brian Harrison
2026-04-01 00:53:34 -04:00
parent f74992f4e5
commit 4944974f6e
4 changed files with 257 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

128
docs/instantel_protocol_reference.md
View File

@@ -63,6 +63,10 @@
| 2026-03-31 | §14.2 | **CORRECTED — Sierra Wireless RV50/RV55 sends `RING`/`CONNECT` over TCP to caller even with Quiet Mode enabled.** Quiet Mode suppresses these only on the serial port (protecting the MiniMate). TCP client still receives `\r\nRING\r\n\r\nCONNECT\r\n` prefixed before the first S3 frame bytes. Parser handles correctly by scanning for DLE+STX (`0x10 0x02`) and discarding prefix bytes. Previous note "no CONNECT string" described Raven X ENQ-disable behaviour; RV50/RV55 differ. |
| 2026-03-31 | §7.3 | **NEW — Calibration date field confirmed** at Full Config (SUB FE) destuffed payload offsets 0x53–0x57. Two-unit comparison: BE18189 (calibrated 2023) has `07 E7` at 0x56–0x57; BE11529 (calibrated 2025) has `07 E9`. Bytes 0x56–0x57 = uint16 BE calibration year ✅ CONFIRMED. Adjacent bytes at 0x53–0x55 likely encode month/day (both units show `0x10` at offset 0x54 = BCD October; 0x53 and 0x55 differ between units). Full date layout 🔶 INFERRED — pending third-unit capture or recalibration diff. Resolves open question. |
| 2026-03-31 | §9 | **CONFIRMED via Console cold-start capture** — `"Operating System"` (16 B: `4f 70 65 72 61 74 69 6e 67 20 53 79 73 74 65 6d`) arrives as first TCP bytes on cold-connect before unit enters DLE-framed mode. `TcpTransport` + retry logic handles gracefully: first attempt times out waiting for SUB A4; second connect (after unit fully booted) succeeds. |
| 2026-04-01 | §7.7.5, §8 | **CONFIRMED — Full waveform record (0C) timestamp layout** cross-referenced against Blastware event report for BE11529 thump event ("00:28:12 April 1, 2026"). 9-byte format at bytes[0–8]: `[day][sub_code][month][year:2 BE][unknown][hour][min][sec]`. All fields verified. Sub_code `0x10` = Waveform (continuous/single-shot). **Previous 7-byte format doc was wrong** — replaced with confirmed 9-byte layout. |
| 2026-04-01 | §7.7.5 | **CONFIRMED — Record type** encoded in byte[1] (sub_code), not as ASCII string. `0x10` = Waveform ✅. Histogram sub_code not yet captured. ASCII string search approach removed. |
| 2026-04-01 | §7.7.5, §14 | **CONFIRMED — Per-channel PPV** at label+6 (✅ all four channels), cross-referenced vs Blastware: Tran=0.420, Vert=3.870, Long=0.495 in/s. **CONFIRMED — Peak Vector Sum** at fixed offset 87 = 3.906 in/s ✅ matches Blastware "Peak Vector Sum". Is √(Tran²+Vert²+Long²) at max instantaneous vector moment, not vector sum of per-channel peaks. Open question "offset 87 purpose" closed. |
| 2026-04-01 | §8 | **RESOLVED — §8 unknown byte at offset 3.** Field is confirmed absent in the 9-byte waveform record format (no such field). The 6-byte event-index format has a separator byte at [3] whose purpose remains ❓ but is no longer actively blocking anything. |
---
@@ -211,7 +215,7 @@ Step 4 — Device sends actual data payload:
| `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED |
| `1E` | **EVENT HEADER READ** | Gets the first waveform key (4-byte opaque record address). All-zero params; key returned at data[11:15]. | ✅ CONFIRMED 2026-03-31 |
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. | ✅ CONFIRMED 2026-03-31 |
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. Contains record type, PPV floats (at channel label+6), project strings, 7-byte timestamp. Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-03-31 |
| `0C` | **FULL WAVEFORM RECORD** | Downloads 210-byte waveform/histogram record. 9-byte timestamp at bytes[0–8]; record sub_code at byte[1] (0x10=Waveform); PPV floats at channel label+6; Peak Vector Sum float at offset 87; project strings. Key at params[4..7], DATA_LENGTH=0xD2. | ✅ CONFIRMED 2026-04-01 |
| `1F` | **EVENT ADVANCE** | Advances to next waveform key. Token byte at params[6]: 0x00=browse (one step), 0xFE=download (skip partial bins). Returns next key at data[11:15]; zeros = no more events. | ✅ CONFIRMED 2026-03-31 |
| `5A` | **BULK WAVEFORM STREAM** | Initiates bulk download of raw ADC sample data, keyed by waveform key. Large multi-page transfer. | ✅ CONFIRMED |
| `24` | **WAVEFORM PAGE A?** | Paged waveform read, possibly channel group A. | 🔶 INFERRED |
@@ -775,26 +779,46 @@ Actual data lengths:
#### 7.7.5 Waveform Record Layout (210 bytes, SUB F3 → response F3)
> ✅ **Updated 2026-04-01** — Full timestamp layout confirmed against Blastware
> event report (BE11529 thump event, "00:28:12 April 1, 2026"). Record type
> encoding corrected (byte[1], not ASCII string search). Peak Vector Sum field
> confirmed at fixed offset 87.
The 210-byte record (`data_rsp.data[11:11+0xD2]`) contains:
**Record type string** (search at variable offset):
- `"Histogram"` — histogram mode recording
- `"Waveform"` — single-shot waveform recording
**Timestamp** (7-byte format, confirmed from 3-31-26 capture):
**Header / Timestamp** (9 bytes at offsets 0–8, ✅ CONFIRMED 2026-04-01):
```
byte 0: 0x09 (magic/type marker)
bytes 1–2: year (uint16 big-endian)
byte 3: 0x00
byte 4: hour
byte 5: minute
byte 6: second
byte[0]: day (uint8)
byte[1]: sub_code 0x10 = Waveform (continuous/single-shot) ✅
histogram code not yet captured ❓
byte[2]: month (uint8)
bytes[3–4]: year (uint16 big-endian)
byte[5]: unknown (0x00 in all observed samples ❓)
byte[6]: hour (uint8)
byte[7]: minute (uint8)
byte[8]: second (uint8)
```
> ❓ Month and day are not present in the waveform record timestamp.
> Month/day may appear in the event index (SUB F7) or a separate header
> field not yet confirmed.
**Peak particle velocity floats** (✅ CONFIRMED 2026-03-31):
Thump event raw bytes (2026-04-01 00:28:12):
```
01 10 04 07 ea 00 00 1c 0c
↑ ↑ ↑ ↑──↑ ↑ ↑ ↑ ↑
d=1 sub m=4 y=2026 ? h=0 m=28 s=12
```
Cross-referenced against the `.MLG` file for the same event, which stores an
8-byte timestamp at two offsets (trigger time and end time):
```
MLG format: [day:1][month:1][year:2 LE][?:1][hour:1][min:1][sec:1]
01 04 ea 07 00 00 1c 0c → trigger at April 1, 2026 00:28:12
01 04 ea 07 00 00 1c 0f → end time April 1, 2026 00:28:15 (3.0 s record time ✅)
```
**Record type** — encoded in `byte[1]` (sub_code), NOT as an ASCII string:
- `0x10` → `"Waveform"` (continuous / single-shot mode) ✅
- histogram sub_code: not yet confirmed — capture a histogram event with `debug=true`
**Peak particle velocity floats** (✅ CONFIRMED 2026-03-31, re-confirmed 2026-04-01):
Channel labels `"Tran"`, `"Vert"`, `"Long"`, `"MicL"` are embedded as
ASCII strings at variable offsets within the record. The PPV float for
@@ -803,26 +827,42 @@ each channel is at `label_offset + 6` (IEEE 754 big-endian float32).
The floats are **NOT 4-byte aligned** — Tran, Long, and MicL all fall at
non-aligned offsets. The previous heuristic step-4 scanner missed all three.
Example from 3-31-26 capture:
Confirmed offsets from thump event (2026-04-01, cross-referenced vs Blastware):
```
"Tran" at offset N → float at N+6 = 0.0916 in/s
"Vert" at offset M → float at M+6 = 0.0907 in/s
"Long" at offset P → float at P+6 = 0.0605 in/s
"MicL" at offset Q → float at Q+6 = 0.000145 psi
"Tran" at offset 99 → float at 105 = 0x3ED70A2D = 0.420 in/s ✅ Blastware: 0.420
"Vert" at offset 114 → float at 120 = 0x4077AE01 = 3.870 in/s ✅ Blastware: 3.870
"Long" at offset 129 → float at 135 = 0x3EFD7090 = 0.495 in/s ✅ Blastware: 0.495
"MicL" at offset 144 → float at 150 = 0x3985114E = 0.000254 psi
```
Channel labels are separated by inner-frame bytes `10 03` (DLE ETX),
preserved as literal data by `S3FrameParser`.
**Peak Vector Sum** (✅ CONFIRMED 2026-04-01):
```
Offset 87: IEEE 754 big-endian float32
= √(Tran² + Vert² + Long²) at the sample instant of maximum
combined geo motion
NOT the vector sum of the three per-channel peaks (those may
occur at different sample times)
Thump event: 0x4079F6C5 = 3.906 in/s ✅ matches Blastware "Peak Vector Sum: 3.906 in/s"
Near-ambient: 0x3C75C28F = 0.015 in/s (histogram event, near-zero ambient)
```
**Project strings** — ASCII label-value pairs (search for label, read null-terminated value):
```
"Project:" → project description
"Client:" → client name ✅ offset confirmed
"User Name:" → operator / user
"Seis Loc:" → sensor location
"Project:" → project description (present in 0C record ✅)
"Client:" → client name (NOT in 0C; comes from compliance config SUB 1A/E5 ❓)
"User Name:" → operator / user (NOT confirmed in 0C ❓)
"Seis Loc:" → sensor location (NOT confirmed in 0C ❓)
"Extended Notes"→ notes field
```
> ❓ **Clarification needed:** Only "Project:" has been confirmed in the 210-byte 0C record.
> "Client:", "User Name:", and "Seis Loc:" appear in the Blastware event report but their
> source in the protocol (0C vs SUB 1A/E5 compliance config) is not yet confirmed.
---
#### 7.7.6 Complete Download Loop (Python pseudocode)
@@ -854,28 +894,54 @@ return events
---
## 8. Timestamp Format
Two timestamp wire formats are used:
### 8.1 6-byte format (event index / 1E header)
> 🔶 **Updated 2026-02-26** — Year field resolved. Confidence upgraded.
Timestamps are 6-byte sequences appearing in event headers and waveform keys.
Appears in event index blocks. Time-of-day fields (hour/min/sec) are absent.
**Observed example:**
```
01 07 CB 00 06 1E
```
**Decoded:**
| Byte(s) | Value | Meaning | Certainty |
|---|---|---|---|
| `01` | 1 | Record validity / type flag | 🔶 INFERRED |
| `07 CB` | 1995 | Year — 16-bit big-endian integer | ✅ CONFIRMED — 2026-02-26 |
| `00` | 0 | Unknown — possibly hours, minutes, or padding | ❓ SPECULATIVE |
| `00` | 0 | Unknown separator | ❓ |
| `06` | 6 | Month (June) | ✅ CONFIRMED |
| `1E` | 30 | Day (0x1E = 30 decimal) | ✅ CONFIRMED |
> ✅ **2026-02-26 — CONFIRMED:** The year 1995 is the **MiniMate Plus factory default RTC date**, which the device reverts to whenever the internal battery is disconnected or the real-time clock loses power. Any event timestamped around 1995 means the clock was not set. This is known device behavior, not an encoding anomaly.
> ❓ **Still unknown:** The `00` byte at offset 3. Likely encodes time-of-day (hours or minutes). Needs a capture with a precisely known event time to decode.
### 8.2 9-byte format (Full Waveform Record / SUB 0C, bytes 0–8)
> ✅ **CONFIRMED 2026-04-01** — Cross-referenced against Blastware event report
> for BE11529 thump event: "00:28:12 April 1, 2026".
Full date + time, including a sub_code byte that encodes the recording mode.
**Observed example (thump event, 2026-04-01):**
```
01 10 04 07 ea 00 00 1c 0c
```
| Byte(s) | Value | Meaning | Certainty |
|---|---|---|---|
| `01` | 1 | Day | ✅ |
| `10` | 0x10 | Sub_code: `0x10` = Waveform (continuous mode) | ✅ / histogram code ❓ |
| `04` | 4 | Month (April) | ✅ |
| `07 ea` | 2026 | Year — 16-bit big-endian integer | ✅ |
| `00` | 0 | Unknown separator | ❓ |
| `00` | 0 | Hour | ✅ |
| `1c` | 28 | Minute | ✅ |
| `0c` | 12 | Second | ✅ |
The `.MLG` file for the same event stores the timestamp in a different binary
representation (little-endian year, no sub_code byte), confirming the waveform
record and the saved file use distinct serialisation formats.
---
@@ -1258,7 +1324,7 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger
| Question | Priority | Added | Notes |
|---|---|---|---|
| Byte at timestamp offset 3 — hours, minutes, or padding? | MEDIUM | 2026-02-26 | |
| Timestamp 6-byte format byte[3] — purpose of the separator `0x00` byte | LOW | 2026-02-26 | Not blocking; 9-byte waveform record format (§8.2) fully confirmed without this byte. |
| `trail[0]` in serial number response — unit-specific byte, derivation unknown. `trail[1]` resolved as firmware minor version. | MEDIUM | 2026-02-26 | |
| Full channel ID mapping in SUB `5A` stream (01/02/03/04 → which sensor?) | MEDIUM | 2026-02-26 | |
| Exact byte boundaries of project string fields in SUB `71` write frame — padding rules unconfirmed | MEDIUM | 2026-02-26 | |