Frame 0 is always the probe; frames 1+ are always data (waveform ADC
chunks, compliance config, compliance continuation). Gating on
classify_frame() at fi>0 produces false positives: ADC binary data
can coincidentally contain b"STRT\xff\xfe", causing frames 1 and 5
to be silently dropped from the body (confirmed from live capture on
event key=01110000). Remove all type-based filtering; include every
frame unconditionally with the standard index-based skip amounts.
Add classify_frame() which categorises each A5 frame by content:
terminator — page_key == 0x0000
probe_or_strt — contains b"STRT"
metadata — contains compliance-config ASCII markers
(Project:, Client:, Standard Recording Setup, …)
waveform — binary-heavy (< 20% printable ASCII), i.e. raw ADC data
unknown — fallback
Update write_blastware_file() body loop: frame 0 (probe) is still
always processed; frames 1+ are only included when classify_frame
returns "waveform". Metadata frames (compliance config block with
Project:/Client:/etc.) and any stray STRT-bearing frames are skipped
with a warning/debug log. Terminator frame handling is unchanged.
Adds temporary print() diagnostics so each frame's classification is
visible in the server log to aid debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For 2-second events at 1024 sps the "Project:" metadata frame appears
beyond chunk 32 (the old default cap), causing the safety limit to be
hit and ~34 KB of waveform data to be downloaded instead of stopping
at the metadata frame. Raising max_chunks to 128 ensures
stop_after_metadata=True can locate the metadata frame for record
times up to ~4 seconds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Update `s3_bridge.py` to default raw capture file paths to "auto" for timestamped naming.
- Modify `gui_bridge.py` to pre-check raw capture options and streamline path handling.
- Extend `ach_server.py` to save both incoming and outgoing raw bytes for analysis.
- Revise `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect changes in recording mode handling and compliance data encoding.
- Added `CallHomeConfig` model to represent the Auto Call Home settings.
- Introduced methods in `MiniMateClient` for reading (`get_call_home_config`) and writing (`set_call_home_config`) the call home configuration.
- Updated `MiniMateProtocol` with new commands for call home operations (SUB 0x2C for read, SUB 0x7E for write, and SUB 0x7F for confirm).
- Created API endpoints for retrieving and updating call home settings in the server.
- Enhanced the web interface with a new "Call Home" tab for user interaction with call home settings.
- Implemented JavaScript functions for reading and writing call home configurations from the web app.
2026-04-20 18:23:48 -04:00
13 changed files with 3043 additions and 156 deletions
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.12.2**.
(Sierra Wireless RV50 / RV55). Current version: **v0.12.3**.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
@@ -27,9 +27,9 @@ CHANGELOG.md ← version history
---
## Current implementation state (v0.10.0)
## Current implementation state (v0.12.3)
Full read pipeline + write pipeline + erase pipeline + monitor log working end-to-end over TCP/cellular:
Full read pipeline + write pipeline + erase pipeline + monitor log + call home config working end-to-end over TCP/cellular:
| Step | SUB | Status |
|---|---|---|
@@ -45,7 +45,8 @@ Full read pipeline + write pipeline + erase pipeline + monitor log working end-t
| Event advance / next key | 1F | ✅ |
| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 |
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
@@ -385,7 +424,9 @@ bytes `\x01\x2c` = 300 (5-minute default histogram interval); changes when inter
| Offset | Field | Format | Notes |
|---|---|---|---|
| anchor −7 (write) / anchor − 8 (read) | recording_mode | uint8 | E5 read has extra `0x10` at anchor−7 |
| anchor −9 | mode_prefix | uint8 | `0x00` for Single Shot / Continuous; `0x10` for Histogram (DLE prefix in E5 encoding) and Histogram+Continuous (actual config byte). See "compliance_raw DLE encoding" note below. |
| anchor − 8 | recording_mode | uint8 | **Same offset for both read and write** — confirmed 2026-04-21. `_encode_compliance_config` writes `buf[anc-8]`. NOTE: for Histogram (0x03), E5 encodes the value as `0x10 0x03` so compliance_raw[anc-9]=0x10, compliance_raw[anc-8]=0x03. |
| anchor − 7 | constant | `0x10` | Always `0x10` in both E5 read and BW write payloads (not a DLE marker — it is part of the sample_rate field area). Do NOT overwrite. |
| anchor − 6 | sample_rate | uint16 BE | same in read & write |
| anchor − 4 | histogram_interval_sec | uint16 BE | seconds; same in read & write ✅ 2026-04-20 |
| anchor − 2 | `0x00 0x00` | padding | |
@@ -394,15 +435,42 @@ bytes `\x01\x2c` = 300 (5-minute default histogram interval); changes when inter
**recording_mode enum** (confirmed 2026-04-20 from 4-20-26 captures):
| Value | Mode |
|---|---|
| `0x00` | Single Shot |
| `0x01` | Continuous |
| `0x02` | ❓ not observed |
| `0x03` | Histogram |
| `0x04` | Histogram + Continuous |
| Value | Mode | anchor-9 in compliance_raw |
|---|---|---|
| `0x00` | Single Shot |`0x00` |
| `0x01` | Continuous |`0x00` |
| `0x02` | ❓ not observed | ❓ |
| `0x03` | Histogram |`0x10` (DLE prefix from E5 wire encoding of 0x03) |
| `0x04` | Histogram + Continuous |`0x10` (actual config byte for this mode) |
**DLE escaping in write frames — CONFIRMED 2026-04-20:** Write frame data payloads DO escape `0x03` (ETX) bytes with a `0x10` DLE prefix. For histogram_interval = 900 (0x0384), the wire carries `10 03 84` — the `0x03` high byte is preceded by a DLE escape. After DLE destuffing (`10 XX → XX`), the logical field value is correctly `03 84` = 900. The CLAUDE.md claim that write frame data is "written RAW" was incorrect; at minimum ETX (0x03) bytes are escaped. S3FrameParser handles this transparently so the decoded `compliance_raw` always contains logical (destuffed) bytes.
**compliance_raw DLE encoding — IMPORTANT (confirmed 2026-04-21 from 4-20-26 captures):**
`compliance_raw` (returned by `read_compliance_config()`) is NOT purely logical bytes — it is
the wire-encoded representation where `0x03` bytes in the config are preceded by a `0x10` DLE
prefix (because S3FrameParser preserves DLE+ETX inner-frame pairs as two literal bytes).
Consequences:
- When recording_mode = `0x03` (Histogram), `compliance_raw[anc-9] = 0x10` (DLE prefix) and
`compliance_raw[anc-8] = 0x03` (the value). The anchor position is +1 compared to modes
without `0x03` bytes before the anchor.
- For Histogram+Continuous (`0x04`), `compliance_raw[anc-9] = 0x10` for a different reason:
it is an actual stored config byte, not a DLE prefix.
- The anchor search (`buf.find(b'\xbe\x80\x00\x00\x00\x00', 0, 150)`) correctly locates
the anchor regardless of these mode-dependent shifts.
- When SFM writes recording_mode and round-trips the rest verbatim, the byte at `anc-9` is
preserved from the previous read. This means transitioning Histogram→other modes via SFM
leaves a `0x10` at `anc-9`. The device stores it as a literal byte; it does not affect
recording mode operation (which is at `anc-8`), but differs from what BW writes. This is a
known minor discrepancy that does not impact device behavior.
- **Histogram recording mode (0x03) write via SFM**: untested. When starting from a mode with
`anc-9 = 0x00`, SFM writes bare `0x03` at anc-8. BW would write `0x10 0x03`. Device likely
accepts both (write frames probably use offset/length for framing, not ETX scanning).
**DLE escaping in write frames — confirmed 2026-04-20:** Blastware escapes `0x03` bytes in
write frame data as `0x10 0x03` on the wire (defensive ETX escaping). Our `build_bw_write_frame`
does NOT do this escaping — it sends data bytes raw. Device acceptance of bare `0x03` bytes
in write frame data is confirmed for the tested modes (Single Shot, Continuous, Histogram+Continuous
where `0x10 0x03` already appears from round-tripping). Histogram mode (bare `0x03` write from
non-Histogram starting state) has not been directly tested.
### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2])
@@ -971,12 +1039,148 @@ call-home.
---
## Auto Call Home config (SUBs 0x2C / 0x7E / 0x7F) — confirmed 2026-04-20
Full read/write pipeline confirmed from `bridges/captures/4-20-26/call home settings/`
(10 BW TX write frames diffed against the S3 read response).
Accessible in Blastware: **Remote Access → Setup Unit**.
### Protocol
**SUB 0x2C — Call Home Config READ (response 0xD3)**
Standard two-step read: probe offset `0x0000`, data offset `0x007C` (124).
Returns 125 raw bytes (one more than DATA_LENGTH) because the device encodes
num_retries value `3` as `\x10\x03` on the wire — S3FrameParser preserves both
bytes literally, shifting all subsequent field positions by +1.
**SUB 0x7E — Call Home Config WRITE (response 0x81)**
Write format (only BW_CMD `0x10` doubled on wire; DLE-aware checksum).
**Note:**`dial_string` is read-only in the current implementation (omitted from POST
body) because writing a dial string may require DLE escaping for embedded control characters.
---
## What's next
- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
- **Histograms** — decode histogram-mode A5 data (noise floor tracking)
- **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed working for Continuous mode events (2026-04-23):** SFM-generated file opens in Blastware, shows correct PPV/waveform/timestamp. File is ~200 bytes shorter than BW (missing last ADC tail slice) — all measurements correct. Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that create spurious STRT markers in the body). Extension mapping: **CONFIRMED FALSE 2026-04-21** — extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`
**Serial encoding (CONFIRMED 2026-04-22):**`prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units).
**Stem encoding (FULLY CONFIRMED 2026-04-22):** stem = 4-char base-36 of `floor(total_seconds / 1296)` where `total_seconds = (event_local_time − 1985-01-01T00:00:00_local)` in seconds. Epoch = `1985-01-01 00:00:00` device local time — confirmed against 3,248 files from 10-year production archive with zero errors. Decode: `event_time = datetime(1985,1,1) + timedelta(seconds=stem_int*1296 + ab_int)`. Example: P036L318.C80H → BE14036, 2025-05-26 15:00:08, Full Histogram.
- **Blastware filename extension — NEW FIRMWARE FULLY DECODED (confirmed 2026-04-21, further confirmed 2026-04-22 from 10-year production archive frequency analysis):**
Extension format = `AB0T` (4 chars):
-`AB` = 2-char base-36 encoding of `total_seconds % 1296` (seconds within the 21.6-min window, 0–1295); `A = value // 36`, `B = value % 36`
-`0` = always literal digit zero (third character, invariant)
-`T` = event type: `W` = Full Waveform, `H` = Full Histogram
Combined with the 4-char stem, the full filename encodes a complete second-resolution timestamp. Verified against three S353L4H0.{3M0W,8S0H,9X0W} events (all match to the second) plus large-scale frequency analysis of a 10-year archive.
**3-day cycle property (confirmed 2026-04-22):** A unit recording at a fixed daily time cycles through exactly **3 extensions** with a 3-day period. Each calendar day shifts `total_seconds % 1296` by 864 (since `86400 % 1296 = 864`). The cycle repeats every 3 days because `gcd(1296, 864) = 432` and `1296 / 432 = 3`. The three extension values are spaced 432 seconds apart. Confirmed from 10-year archive: the top 3 extensions overall were `CE0H` (95 files), `0E0H` (93), `OE0H` (91) — all three are the 3-day cycle of a 06:00:14 daily call-in time (seconds-in-window = 14, 446, 878; all three have `E` as second character because `14 = E` in base-36 and adding 864 never changes `value % 36` since `864 = 24 × 36`).
**B character invariance:** For a unit recording at a fixed time of day, the second character `B` of the extension (`value % 36`) **never changes** — only the first character `A` cycles through 3 values. This means same-time-of-day files from different dates all share the same `B` character.
**Old firmware (S338, 3-char extensions ending in `0`):** encoding unknown. Extension is NOT recording mode. `blastware_filename()` returns `.N00` as a placeholder for old-firmware units.
**Micromate Series 4** uses a different extension format entirely (observed: `IDFH`, `IDFW`). The `AB0T` formula applies only to MiniMate Plus / V10.72 firmware.
- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
- **Test Histogram recording mode (0x03) write via SFM** — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state (bare 0x03 in write vs BW's DLE-escaped `10 03`)
- **Compliance write anchor-9 cleanup** — when changing recording_mode via SFM, the byte at anchor-9 is not explicitly managed. A spurious `0x10` may persist after Histogram→other mode transitions. Does not affect device operation but differs from BW's byte-perfect output.
- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring)
- Call Home — map time slots 3/4 offsets; add dial_string write support; confirm `modem_power_relay_enabled`
- Modem manager — push RV50/RV55 configs via Sierra Wireless API
- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't
resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred)