Compare commits
106 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e1a6fd5386 | |||
| 6b875e161b | |||
| f5c81f2cab | |||
| a7585cb5e0 | |||
| ae30a02898 | |||
| 2f084ed105 | |||
| 7976b544ed | |||
| 0415af19b4 | |||
| 35c3f4f945 | |||
| 43c8158493 | |||
| 242666f358 | |||
| 03540fdc00 | |||
| f83fd880c0 | |||
| ab2c11e9a9 | |||
| fa887b85d9 | |||
| ecd980d345 | |||
| bc9f16e503 | |||
| aa2b02535b | |||
| 2a2031c3a9 | |||
| 9e7e0bce2a | |||
| 5e2f3bf2a1 | |||
| 39ebd4bdaa | |||
| 84c87d0b57 | |||
| ec6362cb8e | |||
| 3eeafd24aa | |||
| 8cb8b86192 | |||
| 6dcca4da79 | |||
| c47e3a3af0 | |||
| dfbc9f29c5 | |||
| 4331215e23 | |||
| b3dcfe7239 | |||
| 9b5cdfd857 | |||
| 7129aae279 | |||
| 2186bc238b | |||
| 3fb24e1895 | |||
| 7bdd7c92f2 | |||
| b6ffdcfa87 | |||
| a7aec31915 | |||
| 34df9ec5fa | |||
| eec6c3dc6a | |||
| 702e06873e | |||
| 94767f5a9d | |||
| e04114fd6c | |||
| f10c5c1b86 | |||
| aa28495a43 | |||
| b23cf4bb50 | |||
| 969010b983 | |||
| 5fba9bcff8 | |||
| ec7be4d784 | |||
| b8ed237363 | |||
| 5866ecdb3e | |||
| ea9c69b7c9 | |||
| 71bcf71cf7 | |||
| 3e7de848bc | |||
| 72a4209cfd | |||
| 2b5574511e | |||
| ce2c859f11 | |||
| 7f322f9ff9 | |||
| 42b7a88c3d | |||
| c474db4f69 | |||
| 2765ee6ea7 | |||
| ef88240796 | |||
| 5591d345d9 | |||
| 7883a31aa7 | |||
| b241da970d | |||
| 6acb419ebd | |||
| f6a0846bab | |||
| 3d9db8b662 | |||
| c7e7d177e6 | |||
| a3b8d10fa8 | |||
| 4921b0489a | |||
| 8688d815a0 | |||
| 9b50ec9133 | |||
| cba8b1b401 | |||
| 41a14ca468 | |||
| 1bfc6e4258 | |||
| 574d40027f | |||
| 0358acb51d | |||
| cf7d838bf4 | |||
| 5e44cdc668 | |||
| 37d32077a4 | |||
| b384ba66d1 | |||
| 27d9823cc1 | |||
| 70c9528611 | |||
| e8bef1ac7c | |||
| 27db663579 | |||
| e5ea17388a | |||
| c0a5131c7d | |||
| 4ec2f33308 | |||
| 6282eacf8b | |||
| 034b3f044d | |||
| 48d7e94c02 | |||
| 03d224ccc3 | |||
| ef2c38e7db | |||
| b9a8e50b3c | |||
| 77d9c17680 | |||
| 8a1bd34551 | |||
| 09788b931a | |||
| e712d68505 | |||
| 8f5da918b5 | |||
| a03c77af09 | |||
| 87fa9c954f | |||
| 3f7b5c07b5 | |||
| 3d2ebfc057 | |||
| 9d9c14af79 | |||
| 2db565ff9c |
@@ -3,6 +3,11 @@
|
|||||||
|
|
||||||
/manuals/
|
/manuals/
|
||||||
|
|
||||||
|
# Python build artifacts
|
||||||
|
*.egg-info/
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
|
||||||
# Python bytecode
|
# Python bytecode
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.py[cod]
|
*.py[cod]
|
||||||
|
|||||||
+383
@@ -4,6 +4,389 @@ All notable changes to seismo-relay are documented here.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## v0.12.5 — 2026-04-21
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **`s3_bridge.py` — raw captures always-on by default** — `--raw-bw` and `--raw-s3` now
|
||||||
|
default to `"auto"` instead of `None`. Every bridge session automatically generates
|
||||||
|
timestamped `raw_bw_<ts>.bin` and `raw_s3_<ts>.bin` files alongside the `.bin`/`.log`
|
||||||
|
session files. Pass `--raw-bw ""` (explicit empty string) to disable if needed.
|
||||||
|
|
||||||
|
- **`gui_bridge.py` — raw capture checkboxes pre-checked** — Both "BW→S3 raw" and
|
||||||
|
"S3→BW raw" checkboxes start checked. Path fields are empty by default (bridge auto-names
|
||||||
|
the files). Unchecking a box passes `--raw-bw ""` to explicitly disable capture.
|
||||||
|
|
||||||
|
- **`ach_server.py` — TX capture added (`raw_tx_<ts>.bin`)** — Every ACH inbound session
|
||||||
|
now saves both directions: `raw_rx_<ts>.bin` (device → us, S3 side, as before) and
|
||||||
|
`raw_tx_<ts>.bin` (us → device, BW side). Both files are usable in the Analyzer.
|
||||||
|
TX bytes are buffered in memory until startup handshake succeeds (same as RX), preventing
|
||||||
|
scanner probes from creating empty files.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.12.4 — 2026-04-21 (protocol analysis / docs only — no code changes)
|
||||||
|
|
||||||
|
### Discovered
|
||||||
|
|
||||||
|
- **compliance_raw is wire-encoded, not logical bytes** — `read_compliance_config()` returns
|
||||||
|
bytes that include DLE prefix bytes (`0x10`) before any `0x03` values (because S3FrameParser
|
||||||
|
preserves DLE+ETX inner-frame pairs as two literal bytes). The previous CLAUDE.md claim that
|
||||||
|
"S3FrameParser handles this transparently so compliance_raw contains logical bytes" was wrong.
|
||||||
|
|
||||||
|
- **anchor-9 behavior per recording mode** (confirmed from 4-20-26 BW write captures):
|
||||||
|
- Single Shot (0x00) / Continuous (0x01): anchor-9 = `0x00`
|
||||||
|
- Histogram (0x03): anchor-9 = `0x10` — the E5 DLE prefix for the `0x03` recording_mode byte
|
||||||
|
- Histogram+Continuous (0x04): anchor-9 = `0x10` — an actual stored config byte for this mode
|
||||||
|
Anchor position shifts by ±1 when recording_mode = `0x03` due to the extra DLE byte; the
|
||||||
|
dynamic anchor search (`buf.find(ANCHOR, 0, 150)`) handles this correctly without code changes.
|
||||||
|
|
||||||
|
- **Write frame ETX escaping** — BW escapes `0x03` bytes in write frame data as `0x10 0x03`
|
||||||
|
on the wire. Our `build_bw_write_frame` sends data bytes raw without ETX escaping. Device
|
||||||
|
accepts our raw writes for all tested modes. Hypothesis: device write parser uses the
|
||||||
|
offset/length field for frame boundaries, not ETX scanning, making ETX escaping optional.
|
||||||
|
Histogram mode (recording_mode = 0x03) write via SFM from a non-Histogram starting state
|
||||||
|
not yet tested.
|
||||||
|
|
||||||
|
- **BW write payload vs E5 read payload are byte-identical** around the anchor region (confirmed
|
||||||
|
by comparing 3-11-26 BW TX and S3 captures). BW does NOT strip DLE prefix bytes before writing;
|
||||||
|
it round-trips the wire-encoded bytes verbatim with only the modified fields changed.
|
||||||
|
|
||||||
|
- **Capture folder content catalogued** — see CLAUDE.md "BW capture reference" table for a
|
||||||
|
summary of all available protocol captures and their contents.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.12.3 — 2026-04-20
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Auto Call Home config protocol** — Full read/write/decode/encode pipeline for the
|
||||||
|
device's Remote Access → Setup Unit ACH settings, confirmed from 4-20-26 call home
|
||||||
|
settings captures.
|
||||||
|
|
||||||
|
**Protocol (new):**
|
||||||
|
- `SUB 0x2C` — Call Home Config READ (response `0xD3`); two-step read; data offset
|
||||||
|
`0x7C` = 124; raw payload 125 bytes (1-byte longer than DATA_LENGTH due to DLE-escaped
|
||||||
|
`\x10\x03` at raw[117:119] representing num_retries = 3)
|
||||||
|
- `SUB 0x7E` — Call Home Config WRITE (response `0x81`); 127-byte payload (125-byte read
|
||||||
|
payload + `\x00\x00`); offset = `data[1]+2 = 0x7E`; write format (DLE-aware checksum)
|
||||||
|
- `SUB 0x7F` — Call Home WRITE CONFIRM (response `0x80`); no data
|
||||||
|
|
||||||
|
**Field map (confirmed from 10-frame BW TX diff):**
|
||||||
|
- `raw[5]` — auto_call_home_enabled (bool)
|
||||||
|
- `raw[6:46]` — dial_string (40-byte null-padded ASCII)
|
||||||
|
- `raw[87]` — after_event_recorded (bool)
|
||||||
|
- `raw[91]` — at_specified_times (bool)
|
||||||
|
- `raw[93]` — time1_enabled / `raw[101]` — time1_hour / `raw[102]` — time1_min
|
||||||
|
- `raw[95]` — time2_enabled / `raw[105]` — time2_hour / `raw[106]` — time2_min
|
||||||
|
- `raw[117:119]` — `\x10\x03` (DLE-escaped 0x03 = num_retries value 3)
|
||||||
|
- `raw[120]` — time_between_retries_sec / `raw[122]` — wait_for_connection_sec / `raw[124]` — warm_up_time_sec
|
||||||
|
|
||||||
|
**Library (`minimateplus/`):**
|
||||||
|
- `models.py` — `CallHomeConfig` dataclass (14 fields; `raw` bytes preserved for
|
||||||
|
round-trip writes)
|
||||||
|
- `protocol.py` — `SUB_CALL_HOME = 0x2C`, `SUB_CALL_HOME_WRITE = 0x7E`,
|
||||||
|
`SUB_CALL_HOME_CONFIRM = 0x7F`; `read_call_home_config()`, `write_call_home_config()`
|
||||||
|
- `client.py` — `get_call_home_config()`, `set_call_home_config()`,
|
||||||
|
`_decode_call_home_config()` (handles DLE prefix at raw[117]),
|
||||||
|
`_encode_call_home_config()` (patches in-place; raises `ValueError` if hour/min = 3)
|
||||||
|
|
||||||
|
**REST API (`sfm/server.py`):**
|
||||||
|
- `GET /device/call_home` — reads and decodes call home config from device
|
||||||
|
- `POST /device/call_home` — reads, patches specified fields, writes back to device
|
||||||
|
- `CallHomeConfigBody` Pydantic model with 9 optional writable fields
|
||||||
|
|
||||||
|
**Web UI (`sfm/sfm_webapp.html`):**
|
||||||
|
- New "Call Home" tab with enable flag, dial string (read-only), after-event trigger,
|
||||||
|
at-specified-times flag, two time slots (enable + HH:MM each), and read-only retry
|
||||||
|
settings (num_retries, time_between_retries_sec, wait_for_connection_sec,
|
||||||
|
warm_up_time_sec)
|
||||||
|
- "Read from Device", "Write to Device", "Clear Form" action buttons
|
||||||
|
- Client-side guard: rejects hour or minute value equal to 3 with a clear message
|
||||||
|
explaining the DLE-encoding limitation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.12.2 — 2026-04-20
|
||||||
|
|
||||||
|
### Added / Fixed
|
||||||
|
|
||||||
|
- **Geophone sensitivity / maximum range field confirmed** — 4-20-26 geo sensitivity
|
||||||
|
captures (1.25 in/s vs 10 in/s) diffed across all three SUB 71 write chunks and both
|
||||||
|
E5 read payloads. The `geo_range` uint8 field per channel is now fully confirmed:
|
||||||
|
- E5 read offset: `channel_label + 33`; SUB 71 write offset: `channel_label + 29`
|
||||||
|
- `0x00` = Normal 10.000 in/s (standard gain); `0x01` = Sensitive 1.250 in/s (high gain)
|
||||||
|
- **Correction:** previous hypothesis (`channel_label+20`, `0x01`=Normal) was wrong.
|
||||||
|
`channel_label+20` reads `0x01` on ALL captures regardless of range — not this field.
|
||||||
|
- `_decode_compliance_config_into`: read offset corrected from `tran_pos+20` → `tran_pos+33`
|
||||||
|
- `_encode_compliance_config`: added `geo_range` parameter; writes to Tran/Vert/Long at `+29`
|
||||||
|
- `apply_config`: added `geo_range` parameter
|
||||||
|
- `POST /device/config`: added `geo_range` to `DeviceConfigBody`
|
||||||
|
- Web UI Config tab: added "Maximum Range — Geo" select (Normal / Sensitive)
|
||||||
|
- Web UI Device tab: added "Max Range (geo)" row to compliance table
|
||||||
|
|
||||||
|
- **`recording_mode` + `histogram_interval_sec` confirmed and implemented** (4-20-26 captures)
|
||||||
|
- `recording_mode`: uint8 at anchor−8 (E5 read) / anchor−7 (write); enum: 0x00=Single Shot,
|
||||||
|
0x01=Continuous, 0x03=Histogram, 0x04=Histogram+Continuous
|
||||||
|
- `histogram_interval_sec`: uint16 BE seconds at anchor−4; same offset in read & write;
|
||||||
|
valid: 2, 5, 15, 60, 300, 900 (matching Blastware dropdown: 2s, 5s, 15s, 1m, 5m, 15m)
|
||||||
|
- Both fields added to `ComplianceConfig`, `_decode_compliance_config_into`,
|
||||||
|
`_encode_compliance_config`, `apply_config`, REST API body, and web UI
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.12.1 — 2026-04-16
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`sfm/server.py` — `_LiveCache`** — in-memory live device cache that eliminates
|
||||||
|
redundant TCP round-trips between web requests. Plain Python dict +
|
||||||
|
`threading.Lock`, no extra dependencies.
|
||||||
|
|
||||||
|
Cache strategy per endpoint:
|
||||||
|
|
||||||
|
| Endpoint | Strategy |
|
||||||
|
|---|---|
|
||||||
|
| `GET /device/info` | Indefinite; invalidated by `POST /device/config` |
|
||||||
|
| `GET /device/events` | Count-probe fast path — `poll()+count_events()` (~2 s); returns cached data if event count is unchanged; full download only when new events are detected |
|
||||||
|
| `GET /device/monitor/status` | 30-second TTL; invalidated immediately on monitor start/stop |
|
||||||
|
| `GET /device/event/{idx}/waveform` | Permanent per-index (waveforms are immutable once recorded) |
|
||||||
|
|
||||||
|
- **`?force=true`** query param on all cached endpoints — bypasses cache and forces
|
||||||
|
a fresh read from the device.
|
||||||
|
|
||||||
|
- **Cache invalidation hooks** — `POST /device/config` marks device info and events
|
||||||
|
stale; `POST /device/monitor/start` and `/stop` evict the monitor status entry
|
||||||
|
immediately so the next status poll reflects the actual device state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.12.0 — 2026-04-13
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`sfm/server.py` — `_LiveCache`** — in-memory live device cache, eliminating
|
||||||
|
redundant TCP round-trips between requests. No extra dependencies (plain Python
|
||||||
|
dict + threading.Lock). Replaces the SQLAlchemy-based `sfm/cache.py` experiment
|
||||||
|
from the `feature/intelligent-caching` branch.
|
||||||
|
|
||||||
|
Cache behaviour by endpoint:
|
||||||
|
|
||||||
|
| Endpoint | Cache strategy |
|
||||||
|
|---|---|
|
||||||
|
| `GET /device/info` | Indefinite; invalidated by `POST /device/config` |
|
||||||
|
| `GET /device/events` | Count-probe fast path: quick `poll()+count_events()` (~2s); return cache if count matches; full download only when new events detected |
|
||||||
|
| `GET /device/monitor/status` | 30-second TTL; invalidated by monitor start/stop |
|
||||||
|
| `GET /device/event/{idx}/waveform` | Permanent per-index (waveforms are immutable) |
|
||||||
|
|
||||||
|
- **`?force=true` param** on all four cached endpoints — bypasses cache and re-reads
|
||||||
|
from device.
|
||||||
|
|
||||||
|
- **`POST /device/config` cache invalidation** — marks device info + events dirty so
|
||||||
|
the next read reflects the new compliance config.
|
||||||
|
|
||||||
|
- **`POST /device/monitor/start` / `stop` cache invalidation** — evicts the monitor
|
||||||
|
status cache entry immediately so the next poll returns the updated state.
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- `sfm/cache.py` — SQLAlchemy-based cache from the experimental caching branch.
|
||||||
|
Its logic has been ported to the sqlite3-native `_LiveCache` class above.
|
||||||
|
`sqlalchemy` is no longer a dependency.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.11.0 — 2026-04-13
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`sfm/database.py` — SeismoDb** — SQLite persistence layer for all ACH data.
|
||||||
|
Three tables, all unit-keyed by serial number:
|
||||||
|
- `ach_sessions` — one row per inbound call-home: serial, timestamp, peer IP,
|
||||||
|
events_downloaded, monitor_entries, duration_seconds
|
||||||
|
- `events` — one row per triggered waveform event: serial, waveform_key (dedup),
|
||||||
|
timestamp, Tran/Vert/Long/VectorSum/Mic PPV, project/client/operator/sensor_location
|
||||||
|
strings, sample_rate, record_type, false_trigger flag
|
||||||
|
- `monitor_log` — one row per monitoring interval: serial, waveform_key (dedup),
|
||||||
|
start_time, stop_time, duration_seconds, geo_threshold_ips
|
||||||
|
- WAL mode, per-request connections — safe for the single-writer / occasional-reader
|
||||||
|
ACH server pattern
|
||||||
|
- Deduplication by `(serial, waveform_key)` UNIQUE constraint — re-runs and repeat
|
||||||
|
call-homes never produce duplicate rows
|
||||||
|
|
||||||
|
- **`ach_server.py` — DB integration** — after each successful call-home, writes new
|
||||||
|
events and monitor log entries to `seismo_relay.db` then records the session in
|
||||||
|
`ach_sessions`. DB write failures are logged as warnings and do not abort the session.
|
||||||
|
|
||||||
|
- **`sfm/server.py` — DB read endpoints**:
|
||||||
|
- `GET /db/units` — distinct serials with last_seen, total_events, total_monitor_entries
|
||||||
|
- `GET /db/events` — query events with serial / date range / false_trigger filters
|
||||||
|
- `GET /db/monitor_log` — query monitoring intervals
|
||||||
|
- `GET /db/sessions` — query ACH call-home sessions
|
||||||
|
- `PATCH /db/events/{id}/false_trigger` — flag/unflag false triggers (for review UI)
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
- seismo-relay DB is unit-keyed only — no project concepts. Project aggregation is
|
||||||
|
terra-view's responsibility via `UnitAssignment` / `DeploymentRecord` + date range
|
||||||
|
queries against the SFM DB endpoints.
|
||||||
|
- DB file lives at `bridges/captures/seismo_relay.db` by default.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.10.0 — 2026-04-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`MiniMateClient.get_monitor_log_entries(skip_keys=None)`** — browse-mode walk
|
||||||
|
(`1E → 0A → 1F`) that collects partial records (`0x2C` record type) from the device's
|
||||||
|
event list without triggering a full waveform download (no 0C or 5A). Returns
|
||||||
|
`list[MonitorLogEntry]`. Each entry represents one continuous monitoring interval where
|
||||||
|
no threshold was exceeded.
|
||||||
|
|
||||||
|
- **`_decode_0a_partial_header(raw_data, index, key4)`** in `client.py` — decodes a SUB
|
||||||
|
0x0A response payload whose record type is `0x2C`. Extracts:
|
||||||
|
- `start_time` / `stop_time` — two consecutive timestamps; auto-detects 9-byte
|
||||||
|
(sub_code=0x10, single-shot) vs 10-byte (sub_code=0x03, continuous) format from
|
||||||
|
`raw_data[11]`. Handles a 1-byte gap between the two timestamps that occurs when
|
||||||
|
ts1 and ts2 share the same minute:second.
|
||||||
|
- `serial` — device serial string found via `b"BE"` anchor scan.
|
||||||
|
- `geo_threshold_ips` — trigger level found via `b"Geo: "` anchor scan.
|
||||||
|
|
||||||
|
- **`MonitorLogEntry` dataclass** in `models.py` — new model for partial records:
|
||||||
|
`index`, `key`, `start_time`, `stop_time`, `serial`, `geo_threshold_ips`,
|
||||||
|
`raw_header`, and a `duration_seconds` property.
|
||||||
|
|
||||||
|
- **`read_waveform_header()` return value extended** — now returns `(data_rsp.data, length)`
|
||||||
|
(full payload) instead of `(data_rsp.data[11:11+length], length)`. Callers get the
|
||||||
|
complete payload including the record-type byte at position 0. Full records use
|
||||||
|
`raw_data[11:11+length]` as before; partial records are detected by `raw_data[0] == 0x2C`.
|
||||||
|
|
||||||
|
- **ACH server: monitor log collection** — after `get_events()`, calls
|
||||||
|
`get_monitor_log_entries(skip_keys=seen_keys)` and saves new entries to
|
||||||
|
`monitor_log.json` in the session directory. Monitor log keys are included in
|
||||||
|
`downloaded_keys` for state persistence (no re-processing on next call-home).
|
||||||
|
|
||||||
|
- **`_monitor_log_entry_to_dict()`** in `ach_server.py` — serialises a `MonitorLogEntry`
|
||||||
|
to a JSON-compatible dict with ISO-format timestamps.
|
||||||
|
|
||||||
|
### Protocol / Documentation
|
||||||
|
|
||||||
|
- **SUB 0x0A partial record (0x2C) format confirmed** (✅ 4-11-26 MITM capture, 12 frames):
|
||||||
|
- Record type `0x2C` at `raw_data[0]`; length < 64 bytes.
|
||||||
|
- Two timestamps at `raw_data[11:]` — start and stop of the monitoring interval.
|
||||||
|
- ASCII metadata region after timestamps: `BE<serial>\x00Geo: <float> in/s`.
|
||||||
|
- Edge case: 1-byte separator between timestamps when ts1 and ts2 share minute:second.
|
||||||
|
- 10-byte timestamp format (sub_code=0x03) signalled by `raw_data[11] == 0x10`.
|
||||||
|
|
||||||
|
- **Key reuse detection for monitor log entries** — monitor log keys are tracked alongside
|
||||||
|
event keys in `ach_state.json` so the ACH server does not re-process them after a
|
||||||
|
call-home cycle.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.9.0 — 2026-04-11
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`MiniMateClient.list_event_keys()`** — fast browse-mode walk (1E → 0A → 1F, no waveform
|
||||||
|
download) that returns the list of event key hex strings currently stored on the device.
|
||||||
|
Used by the ACH server as a cheap pre-check before deciding whether to call `get_events()`.
|
||||||
|
|
||||||
|
- **`get_events(skip_waveform_for_keys=set(...))`** — new optional parameter. For any key in
|
||||||
|
the set the function performs only 0A + 1F(browse) instead of the full
|
||||||
|
1E-arm → 0C → POLL×3 → 5A sequence. Eliminates redundant waveform downloads on repeat
|
||||||
|
call-homes when the device still holds previously downloaded events.
|
||||||
|
|
||||||
|
- **`MiniMateClient.delete_all_events()`** — erases all events from device memory using the
|
||||||
|
confirmed 4-step sequence:
|
||||||
|
- SUB 0xA3 `begin_erase_all` — initiate erase (token=0xFE) → ack 0x5C
|
||||||
|
- SUB 0x1C `read_monitor_status` — intermediate status read (Blastware-required)
|
||||||
|
- SUB 0x06 `read_event_storage_range` — verify storage state (token=0xFE) → 36-byte response
|
||||||
|
- SUB 0xA2 `confirm_erase_all` — commit erase (token=0xFE) → ack 0x5D
|
||||||
|
|
||||||
|
All four steps confirmed from 4-11-26 MITM capture of a live Blastware ACH session.
|
||||||
|
After a successful call, the device's event counter resets to `0x01110000`.
|
||||||
|
|
||||||
|
- **`MiniMateProtocol` erase methods**: `begin_erase_all()`, `confirm_erase_all()`,
|
||||||
|
`read_event_storage_range()` added to `protocol.py` with documented SUB constants
|
||||||
|
`SUB_ERASE_ALL_BEGIN = 0xA3` and `SUB_ERASE_ALL_CONFIRM = 0xA2`.
|
||||||
|
|
||||||
|
- **`bridges/ach_mitm.py`** — transparent TCP-to-TCP MITM proxy. Listens for inbound unit
|
||||||
|
connections, connects upstream to a real Blastware ACH server, and saves both directions
|
||||||
|
to `raw_bw_<ts>.bin` / `raw_s3_<ts>.bin` files matching the existing capture format.
|
||||||
|
Used to capture the 4-11-26 Blastware ACH session including event deletion.
|
||||||
|
Usage: `python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998`
|
||||||
|
|
||||||
|
- **ACH server: key-based state tracking** — `ach_state.json` now stores
|
||||||
|
`downloaded_keys: [hex_strings]` and `max_downloaded_key: hex_string` per unit instead of
|
||||||
|
`event_count: N`. This correctly handles the standard workflow where events are deleted
|
||||||
|
from the device after upload — a count-based approach would see `count=0` on the next
|
||||||
|
call-home and silently skip new events.
|
||||||
|
|
||||||
|
- **ACH server: `--clear-after-download` flag** — after a successful download (at least one
|
||||||
|
new event saved), erases all events from the device using `delete_all_events()`. Mirrors
|
||||||
|
the standard Blastware ACH workflow. On success, `downloaded_keys` and
|
||||||
|
`max_downloaded_key` are reset to empty so the next session starts fresh.
|
||||||
|
|
||||||
|
- **ACH server: post-erase key-reuse detection** — after an external erase (Blastware or
|
||||||
|
manual), device keys restart from `0x01110000`, colliding with previously downloaded keys.
|
||||||
|
On each browse walk, if `max(device_keys) < max_downloaded_key` (device counter rolled
|
||||||
|
back), all device keys are treated as new regardless of `seen_keys`. This also catches
|
||||||
|
erases performed by Blastware between our sessions.
|
||||||
|
|
||||||
|
### Protocol / Documentation
|
||||||
|
|
||||||
|
- **SUB 0xA3 / SUB 0xA2 — erase-all sequence confirmed** (✅ 4-11-26 MITM capture):
|
||||||
|
Both frames use `token=0xFE` at `params[7]` and are standard `build_bw_frame` requests
|
||||||
|
(not write-format). Response SUBs follow the standard formula: 0x5C and 0x5D.
|
||||||
|
The intermediate 0x1C + 0x06 reads between them are required by Blastware.
|
||||||
|
|
||||||
|
- **SUB 0x06 — event storage range read confirmed** (✅ 4-11-26 MITM capture):
|
||||||
|
Two-step read, data offset = 0x24 (36 bytes). The last 8 bytes of the response contain
|
||||||
|
the first and last stored event keys (4 bytes each). After a successful erase, both keys
|
||||||
|
read as `01110000` (device-empty state).
|
||||||
|
|
||||||
|
- **Event key counter resets to `0x01110000` after erase** — confirmed by observing key
|
||||||
|
`01110000` on the device immediately after the MITM erase session.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## v0.8.0 — 2026-04-07
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Write pipeline end-to-end** — `push_config_raw(event_index_data, compliance_data,
|
||||||
|
trigger_data, waveform_data)` on `MiniMateClient` orchestrates the full
|
||||||
|
`68→73 | 71×3→72 | 82→83 | 69→74→72` write sequence.
|
||||||
|
|
||||||
|
- **`build_bw_write_frame(sub, data, *, offset, params)`** in `framing.py` — dedicated frame
|
||||||
|
builder for write commands (SUBs 0x68–0x83). Doubles only the BW_CMD byte; all other
|
||||||
|
bytes including offset, params, data, and checksum are written raw. Uses the large-frame
|
||||||
|
DLE-aware checksum (`sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF`).
|
||||||
|
|
||||||
|
- **`MiniMateProtocol` write methods** — `write_event_index()`, `write_compliance()`,
|
||||||
|
`write_trigger_config()`, `write_waveform_data()`, `write_confirm()`,
|
||||||
|
`start_monitoring()`, `stop_monitoring()`.
|
||||||
|
|
||||||
|
- **`AchSession` inbound server** (`bridges/ach_server.py`) — accepts call-home TCP
|
||||||
|
connections, runs the full handshake + device-info + event-download sequence, saves
|
||||||
|
`device_info.json` + `events.json` per session.
|
||||||
|
|
||||||
|
### Protocol / Documentation
|
||||||
|
|
||||||
|
- **Write frame format confirmed** (✅ 3-11-26 BW TX capture, all 11 frames): only BW_CMD
|
||||||
|
byte `0x10` is doubled; all other bytes sent raw. Standard `build_bw_frame` DLE-stuffing
|
||||||
|
is incorrect for write commands.
|
||||||
|
- **Write ack responses** confirmed as 17-byte zero-data S3 frames.
|
||||||
|
- **Monitoring SUBs 0x96/0x97** confirmed from 4-8-26 capture.
|
||||||
|
- **SESSION_RESET signal** (`41 03`) required before POLL for monitoring units.
|
||||||
|
- **SUB 0x1C monitoring flag** at `section[1]`: `0x00` = idle, `0x10` = monitoring.
|
||||||
|
Confirmed by byte-diff of all 144 data frames in 4-8-26/2ndtry capture.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## v0.7.0 — 2026-04-03
|
## v0.7.0 — 2026-04-03
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
@@ -2,7 +2,9 @@
|
|||||||
|
|
||||||
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
|
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
|
||||||
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
|
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
|
||||||
(Sierra Wireless RV50 / RV55). Current version: **v0.8.0**.
|
(Sierra Wireless RV50 / RV55). Current version: **v0.12.3**.
|
||||||
|
|
||||||
|
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -25,9 +27,9 @@ CHANGELOG.md ← version history
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Current implementation state (v0.8.0)
|
## Current implementation state (v0.12.3)
|
||||||
|
|
||||||
Full read pipeline + write pipeline working end-to-end over TCP/cellular:
|
Full read pipeline + write pipeline + erase pipeline + monitor log + call home config working end-to-end over TCP/cellular:
|
||||||
|
|
||||||
| Step | SUB | Status |
|
| Step | SUB | Status |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
@@ -41,12 +43,17 @@ Full read pipeline + write pipeline working end-to-end over TCP/cellular:
|
|||||||
| Waveform record (peaks, timestamp, project) | 0C | ✅ |
|
| Waveform record (peaks, timestamp, project) | 0C | ✅ |
|
||||||
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 |
|
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 |
|
||||||
| Event advance / next key | 1F | ✅ |
|
| Event advance / next key | 1F | ✅ |
|
||||||
| **Write commands (push config to device)** | **68–83** | ✅ **new v0.8.0** |
|
| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 |
|
||||||
|
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
|
||||||
|
| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ new v0.10.0 |
|
||||||
|
| **Auto Call Home config (read + write)** | **2C → 7E → 7F** | ✅ **new v0.12.3** |
|
||||||
|
|
||||||
`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F`
|
`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F`
|
||||||
|
|
||||||
`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`
|
`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`
|
||||||
|
|
||||||
|
`delete_all_events()` erase sequence: `0xA3 → 0x1C → 0x06 → 0xA2`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Protocol fundamentals
|
## Protocol fundamentals
|
||||||
@@ -111,21 +118,29 @@ S3→BW (response):
|
|||||||
Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26
|
Both differences confirmed by reproducing Blastware's exact wire bytes from the 1-2-26
|
||||||
BW TX capture. All 10 frames verified.
|
BW TX capture. All 10 frames verified.
|
||||||
|
|
||||||
### SUB 5A — chunk counter is monotonic (CORRECTED 2026-04-06)
|
### SUB 5A — chunk counter formula (FINAL CORRECTION 2026-04-26)
|
||||||
|
|
||||||
**Chunk counters are `chunk_num * 0x0400` for ALL chunks including chunk 1.**
|
**Chunk counter = `max(key4[2:4], 0x0400) + (chunk_num - 1) * 0x0400` for ALL chunks.**
|
||||||
|
|
||||||
The 4-2-26 BW TX capture showed `counter=0x1004` for chunk 1 of event key `01110000`, which
|
where `key4[2:4] = (key4[2] << 8) | key4[3]` is the event's circular-buffer base offset.
|
||||||
led to `_CHUNK1_COUNTER = 0x1004` being hardcoded as a special case. This was a Blastware
|
|
||||||
artifact, not a protocol requirement. Empirical test 2026-04-06: with `counter=0x1004` for
|
|
||||||
chunk 1 the device times out (120 s); with `counter=0x0400` (= `1 * 0x0400`) it responds
|
|
||||||
immediately and streams all frames correctly.
|
|
||||||
|
|
||||||
The 4-3-26 capture confirms the pattern for a second event (key `0111245a`):
|
The `max(..., 0x0400)` guard is critical for events at the start of the circular buffer
|
||||||
chunk 1 = `0x245A`, chunk 2 = `0x285A`, chunk 3 = `0x2C5A` (each +0x0400). Blastware's
|
(key4[2:4] == 0x0000, e.g. key `01110000`). Without it, chunk 1 gets counter=0x0000, which
|
||||||
true formula is `key4[2:4] + n * 0x0400` — but since `key4[2:4]` of the first event is
|
is the same address as the probe frame — the device re-returns the STRT record data instead
|
||||||
`0x0000`, `n * 0x0400` produces the right result. The device does not strictly validate the
|
of waveform payload. With the guard, chunk 1 gets counter=0x0400, which is confirmed correct
|
||||||
counter and streams data for any valid 5A request; using `chunk_num * 0x0400` is correct.
|
from the empirical live-device test 2026-04-06 (`counter=0x0400 → responds immediately and
|
||||||
|
streams all frames correctly`).
|
||||||
|
|
||||||
|
The 4-3-26 capture confirms the pattern for a second event (key `0111245a`, key4[2:4]=0x245a):
|
||||||
|
chunk 1 = `0x245A`, chunk 2 = `0x285A`, chunk 3 = `0x2C5A` (each +0x0400).
|
||||||
|
`max(0x245a, 0x0400) = 0x245a` → formula works correctly for non-zero base offset too.
|
||||||
|
|
||||||
|
**History:**
|
||||||
|
- Original: `_CHUNK1_COUNTER = 0x1004` hardcoded (Blastware capture artifact — WRONG).
|
||||||
|
- 2026-04-06: Corrected to `chunk_num * 0x0400` (worked for key 01110000 only).
|
||||||
|
- 2026-04-24: Corrected to `key4[2:4] + (chunk_num-1) * 0x0400` (fixed non-zero offsets,
|
||||||
|
but accidentally broke key 01110000 — counter=0x0000 sends probe address again).
|
||||||
|
- 2026-04-26: Final formula: `max(key4[2:4], 0x0400) + (chunk_num-1) * 0x0400`.
|
||||||
|
|
||||||
### SUB 5A — params are 11 bytes for chunk frames, 10 for termination
|
### SUB 5A — params are 11 bytes for chunk frames, 10 for termination
|
||||||
|
|
||||||
@@ -301,10 +316,16 @@ producing only ~1071 bytes instead of ~2126.
|
|||||||
|
|
||||||
### SUB 1A — anchor search range
|
### SUB 1A — anchor search range
|
||||||
|
|
||||||
`_decode_compliance_config_into()` locates sample_rate and record_time via the anchor
|
`_decode_compliance_config_into()` locates fields via the **6-byte stable anchor**
|
||||||
`b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`. Search range is `cfg[0:150]`.
|
`b'\xbe\x80\x00\x00\x00\x00'`. Search range is `cfg[0:150]`.
|
||||||
|
|
||||||
Do not narrow this to `cfg[40:100]` — the old range was only accidentally correct because
|
**IMPORTANT — the "10-byte anchor" `\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00` is NOT fully constant.**
|
||||||
|
The first 2 bytes (`\x01\x2c` = 300) are the `histogram_interval_sec` field (uint16 BE, seconds) —
|
||||||
|
the value 300 is just the 5-minute default. When histogram interval is set to a different value
|
||||||
|
(e.g. 15min = 0x0384 = `\x03\x84`), those bytes change. Only the 6-byte suffix
|
||||||
|
`\xbe\x80\x00\x00\x00\x00` is truly constant. The code already uses the 6-byte anchor.
|
||||||
|
|
||||||
|
Do not narrow the search range to `cfg[40:100]` — the old range was only accidentally correct because
|
||||||
the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from
|
the orphaned-send bug was prepending a 44-byte spurious header, pushing the anchor from
|
||||||
its real position (cfg[11]) into the 40–100 window.
|
its real position (cfg[11]) into the 40–100 window.
|
||||||
|
|
||||||
@@ -326,6 +347,36 @@ Do NOT use fixed absolute offsets for sample_rate or record_time.
|
|||||||
Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to
|
Quiet Mode enabled. Parser handles this — do not strip it manually before feeding to
|
||||||
`S3FrameParser`.
|
`S3FrameParser`.
|
||||||
|
|
||||||
|
**SUB 5A (bulk waveform) TCP frame splitting — confirmed 2026-04-27:**
|
||||||
|
|
||||||
|
Over TCP via cellular modem, each 5A chunk request that produces a single ~1100-byte
|
||||||
|
A5 response over direct RS-232 may arrive as **two separate, complete S3 frames** of
|
||||||
|
~550 bytes each ("2-frame mode"). The modem's Data Forwarding Timeout (~100-150 ms)
|
||||||
|
can split the RS-232 response into two TCP segments, each parsed as a complete S3 frame.
|
||||||
|
Under different modem/timing conditions the full ~1100-byte response arrives as **one
|
||||||
|
S3 frame** ("1-frame mode").
|
||||||
|
|
||||||
|
**Both modes require `extra_chunks_after_metadata=1`** (the extra chunk at metadata_counter
|
||||||
|
+ 0x0400). The device's waveform footer data lives at circular-buffer address 0x1C00 for
|
||||||
|
this event; the terminator frame must be sent at 0x1C00 (not 0x1800) to receive it.
|
||||||
|
|
||||||
|
Example for a 2-second Continuous event (BE11529, key=01110000) via TCP:
|
||||||
|
- **2-frame mode:** 1 probe frame (554 B) + 5 chunks × 2 frames (556-573 B) + 1 extra chunk × 2 frames + 1 terminator (208 B) = **14 A5 frames** → 6864-byte file
|
||||||
|
- **1-frame mode:** 1 probe frame (~1097 B) + 5 chunks × 1 frame (~1079-1113 B) + 1 extra chunk × 1 frame (smaller, tail of event) + 1 terminator → **8 A5 frames** → 6864-byte file
|
||||||
|
- All frames contribute body data; using all of them gives the correct file.
|
||||||
|
|
||||||
|
**Fix (confirmed 2026-04-27):** `_recv_5a_batch()` in `protocol.py` collects ALL
|
||||||
|
A5 frames per chunk request before the next request is sent, using a 0.5 s batch
|
||||||
|
timeout after the first frame to catch the ~150 ms delayed second frame. `write_blastware_file()`
|
||||||
|
includes ALL body frames without skipping — the extra chunk's frames are part of the
|
||||||
|
body data, NOT padding to be discarded.
|
||||||
|
|
||||||
|
**WRONG earlier hypothesis (do not re-introduce):** An attempt was made to auto-detect
|
||||||
|
1-frame vs 2-frame mode from the probe frame size and skip the extra chunk when
|
||||||
|
`probe_data_len >= 700`. This was wrong — the extra chunk is always needed to advance
|
||||||
|
the device's internal state to the footer address. The `_probe_is_large` branch was
|
||||||
|
removed 2026-04-27.
|
||||||
|
|
||||||
### Required ACEmanager settings (Sierra Wireless RV50/RV55)
|
### Required ACEmanager settings (Sierra Wireless RV50/RV55)
|
||||||
|
|
||||||
| Setting | Value | Why |
|
| Setting | Value | Why |
|
||||||
@@ -354,15 +405,72 @@ Do NOT use fixed absolute offsets for sample_rate or record_time.
|
|||||||
|
|
||||||
| Field | How to find it |
|
| Field | How to find it |
|
||||||
|---|---|
|
|---|---|
|
||||||
|
| **recording_mode** | **uint8 at anchor − 3 (write) / anchor − 4 (read)** ✅ confirmed 2026-04-20 |
|
||||||
| sample_rate | uint16 BE at anchor − 2 |
|
| sample_rate | uint16 BE at anchor − 2 |
|
||||||
|
| **histogram_interval_sec** | **uint16 BE at anchor − 4 (seconds); same offset in read & write** ✅ confirmed 2026-04-20 |
|
||||||
| record_time | float32 BE at anchor + 10 |
|
| record_time | float32 BE at anchor + 10 |
|
||||||
| trigger_level_geo | float32 BE, located in channel block |
|
| trigger_level_geo | float32 BE, located in channel block |
|
||||||
| alarm_level_geo | float32 BE, adjacent to trigger_level_geo |
|
| alarm_level_geo | float32 BE, adjacent to trigger_level_geo |
|
||||||
| max_range_geo | float32 BE, adjacent to alarm_level_geo |
|
| geo_hardware_constant (adc_scale_factor) | float32 BE at **channel_label+28** in both read (E5) and write (SUB 71) payloads — reads **6.206053** on BOTH tested units (BE11529 and BE18189); identical across all geo channels (Tran/Vert/Long) and all captures. **Confirmed 2026-04-17 from Interface Handbook §4.5**: this is the **ADC-to-velocity scale factor** = 1/sensitivity = (in/s per V). Firmware uses it as: `PPV (in/s) = ADC_voltage × 6.206053`. Cross-check: `1.61133 V (ADC full-scale) × 6.206053 = 10.000 in/s` (Normal range ✅). Do NOT write this field — it is a hardware/firmware constant. |
|
||||||
|
| geo_range (sensitivity selector) | **uint8 at channel_label+33** in both read (E5) and write (SUB 71) payloads — **CONFIRMED 2026-04-20** from 4-20-26 geo sensitivity captures: `0x00` = Normal 10.000 in/s (standard gain), `0x01` = Sensitive 1.250 in/s (high gain). Present in all three geo channel blocks (Tran, Vert, Long). **NOTE: `channel_label+20` reads `0x01` on ALL captures regardless of range setting — it is NOT this field.** Note: the "SUB 71 write offset = +29" that appears in earlier analysis was an artifact of incorrect BW-style destuffing applied to write frame data — write frame data is RAW, so the literal `0x10` bytes in the channel block header are preserved, and the offset is the same as in the E5 read payload. |
|
||||||
| setup_name | ASCII, null-padded, in cfg body |
|
| setup_name | ASCII, null-padded, in cfg body |
|
||||||
| project / client / operator / sensor_location | ASCII, label-value pairs |
|
| project / client / operator / sensor_location | ASCII, label-value pairs |
|
||||||
|
|
||||||
Anchor: `b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'`, search `cfg[0:150]`
|
**True stable anchor: `b'\xbe\x80\x00\x00\x00\x00'` (6-byte suffix), search `cfg[0:150]`.**
|
||||||
|
The old "10-byte anchor" `b'\x01\x2c\x00\x00\xbe\x80\x00\x00\x00\x00'` is partially variable:
|
||||||
|
bytes `\x01\x2c` = 300 (5-minute default histogram interval); changes when interval changes.
|
||||||
|
|
||||||
|
**Field layout relative to the 6-byte anchor (write payload / E5 read — noted where different):**
|
||||||
|
|
||||||
|
| Offset | Field | Format | Notes |
|
||||||
|
|---|---|---|---|
|
||||||
|
| anchor − 9 | mode_prefix | uint8 | `0x00` for Single Shot / Continuous; `0x10` for Histogram (DLE prefix in E5 encoding) and Histogram+Continuous (actual config byte). See "compliance_raw DLE encoding" note below. |
|
||||||
|
| anchor − 8 | recording_mode | uint8 | **Same offset for both read and write** — confirmed 2026-04-21. `_encode_compliance_config` writes `buf[anc-8]`. NOTE: for Histogram (0x03), E5 encodes the value as `0x10 0x03` so compliance_raw[anc-9]=0x10, compliance_raw[anc-8]=0x03. |
|
||||||
|
| anchor − 7 | constant | `0x10` | Always `0x10` in both E5 read and BW write payloads (not a DLE marker — it is part of the sample_rate field area). Do NOT overwrite. |
|
||||||
|
| anchor − 6 | sample_rate | uint16 BE | same in read & write |
|
||||||
|
| anchor − 4 | histogram_interval_sec | uint16 BE | seconds; same in read & write ✅ 2026-04-20 |
|
||||||
|
| anchor − 2 | `0x00 0x00` | padding | |
|
||||||
|
| anchor | `\xbe\x80\x00\x00\x00\x00` | anchor | |
|
||||||
|
| anchor + 6 | record_time | float32 BE | same in read & write |
|
||||||
|
|
||||||
|
**recording_mode enum** (confirmed 2026-04-20 from 4-20-26 captures):
|
||||||
|
|
||||||
|
| Value | Mode | anchor-9 in compliance_raw |
|
||||||
|
|---|---|---|
|
||||||
|
| `0x00` | Single Shot | `0x00` |
|
||||||
|
| `0x01` | Continuous | `0x00` |
|
||||||
|
| `0x02` | ❓ not observed | ❓ |
|
||||||
|
| `0x03` | Histogram | `0x10` (DLE prefix from E5 wire encoding of 0x03) |
|
||||||
|
| `0x04` | Histogram + Continuous | `0x10` (actual config byte for this mode) |
|
||||||
|
|
||||||
|
**compliance_raw DLE encoding — IMPORTANT (confirmed 2026-04-21 from 4-20-26 captures):**
|
||||||
|
`compliance_raw` (returned by `read_compliance_config()`) is NOT purely logical bytes — it is
|
||||||
|
the wire-encoded representation where `0x03` bytes in the config are preceded by a `0x10` DLE
|
||||||
|
prefix (because S3FrameParser preserves DLE+ETX inner-frame pairs as two literal bytes).
|
||||||
|
|
||||||
|
Consequences:
|
||||||
|
- When recording_mode = `0x03` (Histogram), `compliance_raw[anc-9] = 0x10` (DLE prefix) and
|
||||||
|
`compliance_raw[anc-8] = 0x03` (the value). The anchor position is +1 compared to modes
|
||||||
|
without `0x03` bytes before the anchor.
|
||||||
|
- For Histogram+Continuous (`0x04`), `compliance_raw[anc-9] = 0x10` for a different reason:
|
||||||
|
it is an actual stored config byte, not a DLE prefix.
|
||||||
|
- The anchor search (`buf.find(b'\xbe\x80\x00\x00\x00\x00', 0, 150)`) correctly locates
|
||||||
|
the anchor regardless of these mode-dependent shifts.
|
||||||
|
- When SFM writes recording_mode and round-trips the rest verbatim, the byte at `anc-9` is
|
||||||
|
preserved from the previous read. This means transitioning Histogram→other modes via SFM
|
||||||
|
leaves a `0x10` at `anc-9`. The device stores it as a literal byte; it does not affect
|
||||||
|
recording mode operation (which is at `anc-8`), but differs from what BW writes. This is a
|
||||||
|
known minor discrepancy that does not impact device behavior.
|
||||||
|
- **Histogram recording mode (0x03) write via SFM**: untested. When starting from a mode with
|
||||||
|
`anc-9 = 0x00`, SFM writes bare `0x03` at anc-8. BW would write `0x10 0x03`. Device likely
|
||||||
|
accepts both (write frames probably use offset/length for framing, not ETX scanning).
|
||||||
|
|
||||||
|
**DLE escaping in write frames — confirmed 2026-04-20:** Blastware escapes `0x03` bytes in
|
||||||
|
write frame data as `0x10 0x03` on the wire (defensive ETX escaping). Our `build_bw_write_frame`
|
||||||
|
does NOT do this escaping — it sends data bytes raw. Device acceptance of bare `0x03` bytes
|
||||||
|
in write frame data is confirmed for the tested modes (Single Shot, Continuous, Histogram+Continuous
|
||||||
|
where `0x10 0x03` already appears from round-tripping). Histogram mode (bare `0x03` write from
|
||||||
|
non-Histogram starting state) has not been directly tested.
|
||||||
|
|
||||||
### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2])
|
### SUB 0C — Waveform Record (210 bytes = data[11:11+0xD2])
|
||||||
|
|
||||||
@@ -412,6 +520,8 @@ for 0x10 records).
|
|||||||
|
|
||||||
## SFM REST API (sfm/server.py)
|
## SFM REST API (sfm/server.py)
|
||||||
|
|
||||||
|
### Live device endpoints (connect to device per-request)
|
||||||
|
|
||||||
```
|
```
|
||||||
GET /device/info?port=COM5 ← serial
|
GET /device/info?port=COM5 ← serial
|
||||||
GET /device/info?host=1.2.3.4&tcp_port=9034 ← cellular
|
GET /device/info?host=1.2.3.4&tcp_port=9034 ← cellular
|
||||||
@@ -424,6 +534,19 @@ POST /device/monitor/stop?host=1.2.3.4&tcp_port=9034 ← stop recording
|
|||||||
|
|
||||||
Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing).
|
Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing).
|
||||||
|
|
||||||
|
### DB read endpoints (query seismo_relay.db written by ach_server.py)
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /db/units ← all known serials + summary stats
|
||||||
|
GET /db/events?serial=BE11529&from_dt=&to_dt=&limit= ← triggered events, newest first
|
||||||
|
GET /db/monitor_log?serial=BE11529&from_dt=&to_dt= ← monitoring intervals, newest first
|
||||||
|
GET /db/sessions?serial=BE11529&limit=50 ← ACH call-home sessions, newest first
|
||||||
|
PATCH /db/events/{id}/false_trigger?value=true ← flag/unflag false triggers
|
||||||
|
```
|
||||||
|
|
||||||
|
DB file: `bridges/captures/seismo_relay.db` (default; override with `--db-path` at startup).
|
||||||
|
All DB endpoints are read-only except `PATCH /db/events/{id}/false_trigger`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Key wire captures (reference material)
|
## Key wire captures (reference material)
|
||||||
@@ -682,16 +805,16 @@ Fields visible in the Blastware Compliance Setup dialog — most are NOT YET dec
|
|||||||
offsets in the raw 1A/E5 payload. Only fields with `✅` have confirmed offsets in the code.
|
offsets in the raw 1A/E5 payload. Only fields with `✅` have confirmed offsets in the code.
|
||||||
|
|
||||||
**Recording Setup tab:**
|
**Recording Setup tab:**
|
||||||
- Recording Mode: Continuous / Single Shot / Histogram (enum)
|
- Recording Mode: Continuous / Single Shot / Histogram / Histogram+Continuous ✅ (uint8 at anchor−3 in write, anchor−4 in read; 0x00=Single Shot, 0x01=Continuous, 0x03=Histogram, 0x04=Histogram+Continuous) — confirmed 2026-04-20
|
||||||
- Record Stop Mode: Fixed Record Time / Auto / Manual Stop (enum)
|
- Record Stop Mode: Fixed Record Time / Auto / Manual Stop (enum) ❓ (byte near recording_mode; data[40] in E5 sf1 changed 0x01→0x00 alongside Continuous→Single Shot — may be this field)
|
||||||
- Sample Rate: Standard 1024 / Fast 2048 / Faster 4096 sps ✅ (anchor−2)
|
- Sample Rate: Standard 1024 / Fast 2048 / Faster 4096 sps ✅ (anchor−2)
|
||||||
- Record Time: float, seconds ✅ (anchor+10)
|
- Record Time: float, seconds ✅ (anchor+10)
|
||||||
- Histogram Interval: 5 / 15 / 30 / 60 minutes (enum, mode-gated)
|
- Histogram Interval: 2s / 5s / 15s / 1m / 5m / 15m ✅ (uint16 BE seconds at anchor−4, same in read & write; mode-gated to Histogram/Histogram+Continuous) — confirmed 2026-04-20
|
||||||
- Storage Mode: Save All Data / Save Triggered (enum)
|
- Storage Mode: Save All Data / Save Triggered (enum)
|
||||||
- Geophone Type: Standard Triaxial / 4.5 Hz (bool/enum)
|
- Geophone Type: Standard Triaxial / 4.5 Hz (bool/enum)
|
||||||
- Geophone Channels: Enable all geophones (bool), Trigger Source (bool)
|
- Geophone Channels: Enable all geophones (bool), Trigger Source (bool)
|
||||||
- Chan 1-3 Trigger Level (float, in/s) ✅ (`trigger_level_geo`)
|
- Chan 1-3 Trigger Level (float, in/s) ✅ (`trigger_level_geo`)
|
||||||
- Chan 1-3 Maximum Range: Normal 10.000 / 1.25 in/s (enum) ✅ (`max_range_geo`)
|
- Chan 1-3 Maximum Range: Normal 10.000 / 1.25 in/s (enum) ✅ (`geo_range` uint8; **CONFIRMED 2026-04-20** from 4-20-26 geo sensitivity captures: offset = `channel_label+33` in both E5 read and SUB 71 write payloads (same bytes, round-tripped verbatim); `0x00` = Normal 10.000 in/s, `0x01` = Sensitive 1.250 in/s; applied to Tran/Vert/Long channel blocks). **IMPORTANT: `channel_label+20` reads `0x01` on ALL captures and is NOT this field** — it is a constant flag. The float32 at `channel_label+28` = 6.206053 is the ADC-to-velocity scale factor (hardware constant, do NOT write).
|
||||||
- Microphone Channels: Enable all microphones (bool), Trigger Source (bool)
|
- Microphone Channels: Enable all microphones (bool), Trigger Source (bool)
|
||||||
- Chan 4 Trigger Level (dB or psi depending on units)
|
- Chan 4 Trigger Level (dB or psi depending on units)
|
||||||
|
|
||||||
@@ -720,9 +843,344 @@ Full compliance config encoder is a future task.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Erase-all protocol (SUBs 0xA3/0xA2/0x06) — confirmed 2026-04-11
|
||||||
|
|
||||||
|
Full sequence confirmed from 4-11-26 MITM capture of a live Blastware ACH session
|
||||||
|
(`bridges/captures/mitm/ach_mitm_20260411_001912/`).
|
||||||
|
|
||||||
|
### Wire sequence
|
||||||
|
|
||||||
|
```
|
||||||
|
BW → device: SUB 0xA3 params=00 00 00 00 00 00 00 FE 00 00 (begin erase)
|
||||||
|
device → BW: SUB 0x5C (ack)
|
||||||
|
BW → device: SUB 0x1C probe (offset=0x00)
|
||||||
|
device → BW: SUB 0xE3 (probe ack)
|
||||||
|
BW → device: SUB 0x1C data (offset=0x2C)
|
||||||
|
device → BW: SUB 0xE3 (monitor status response)
|
||||||
|
BW → device: SUB 0x06 probe (offset=0x00, params same)
|
||||||
|
device → BW: SUB 0xF9 (probe ack)
|
||||||
|
BW → device: SUB 0x06 data (offset=0x24)
|
||||||
|
device → BW: SUB 0xF9 (36-byte storage range response)
|
||||||
|
BW → device: SUB 0xA2 params=00 00 00 00 00 00 00 FE 00 00 (confirm erase)
|
||||||
|
device → BW: SUB 0x5D (ack — device memory is now cleared)
|
||||||
|
```
|
||||||
|
|
||||||
|
All frames use standard `build_bw_frame` (not write-format). Response SUBs follow the
|
||||||
|
standard `0xFF - SUB` formula; no exceptions.
|
||||||
|
|
||||||
|
### SUB 0x06 — event storage range response (36 bytes)
|
||||||
|
|
||||||
|
The 36-byte response body ends with two 4-byte event keys:
|
||||||
|
|
||||||
|
| Offset (from end) | Field | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `[-8:-4]` | first stored event key | `01110000` when empty |
|
||||||
|
| `[-4:]` | last stored event key | `01110000` when empty |
|
||||||
|
|
||||||
|
Before erase: ends with `<first_key> <last_key>` (e.g. `0111ea60 0111eaa6`).
|
||||||
|
After erase: both bytes read `01110000` — device's empty/reset sentinel.
|
||||||
|
|
||||||
|
### Post-erase key counter reset
|
||||||
|
|
||||||
|
After a successful erase, the device resets its event counter. New events start from
|
||||||
|
key `0x01110000` again — the same key as the very first event ever recorded. This means
|
||||||
|
key-based deduplication in the ACH server must account for key reuse:
|
||||||
|
|
||||||
|
- After our own erase: `ach_state.json` `downloaded_keys` and `max_downloaded_key` are
|
||||||
|
cleared so the next session starts fresh.
|
||||||
|
- After an external erase: the ACH server detects it by comparing `max(device_keys)` to
|
||||||
|
`max_downloaded_key` from state. If the device max has rolled back below the historical
|
||||||
|
max, all current device keys are treated as new regardless of `seen_keys`.
|
||||||
|
|
||||||
|
### ACH server state format (v0.9.0)
|
||||||
|
|
||||||
|
`bridges/captures/ach_state.json`:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"BE11529": {
|
||||||
|
"downloaded_keys": ["01110000", "0111245a"],
|
||||||
|
"max_downloaded_key": "0111245a",
|
||||||
|
"last_seen": "2026-04-11T01:04:36",
|
||||||
|
"serial": "BE11529",
|
||||||
|
"peer": "63.43.212.232:51920"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`max_downloaded_key` is the high-water mark — the largest key ever downloaded from the
|
||||||
|
unit. It is NOT reset when events are erased from the device (only when our server does
|
||||||
|
the erase). Used for post-erase detection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Monitor log entries — SUB 0x0A partial records (confirmed 2026-04-11)
|
||||||
|
|
||||||
|
Confirmed from 4-11-26 MITM capture: 12 partial records (record type `0x2C`) and 7 full
|
||||||
|
event records (record type `0x46`) across 19 total 0x0A responses.
|
||||||
|
|
||||||
|
### Record type detection
|
||||||
|
|
||||||
|
`read_waveform_header()` returns `(raw_data, length)` where `raw_data = data_rsp.data`
|
||||||
|
(the full payload including prefix bytes). The record type is at `raw_data[0]`:
|
||||||
|
|
||||||
|
| Value | Type | How to process |
|
||||||
|
|---|---|---|
|
||||||
|
| `0x46` | Full triggered event | Normal download: 0C → 5A → 1F |
|
||||||
|
| `0x2C` | Monitor log entry (partial) | No 0C/5A; decode inline from 0A payload |
|
||||||
|
|
||||||
|
Length heuristic: `length < 0x40` (64) reliably identifies partial records across all
|
||||||
|
observed captures. Both checks (`raw_data[0] == 0x2C` and `length < 0x40`) are used.
|
||||||
|
|
||||||
|
### SUB 0x0A partial record (0x2C) payload layout
|
||||||
|
|
||||||
|
All offsets are from `raw_data` (the full `data_rsp.data` array including the 11-byte
|
||||||
|
prefix before the actual header bytes start).
|
||||||
|
|
||||||
|
```
|
||||||
|
raw_data[0] = 0x2C ← record type (partial / monitor log)
|
||||||
|
raw_data[1:11] = prefix bytes (vary; contain key4 copy, flags, length)
|
||||||
|
raw_data[11:] = timestamp and ASCII metadata payload
|
||||||
|
```
|
||||||
|
|
||||||
|
**Timestamp auto-detection** (confirmed from 4-11-26 capture):
|
||||||
|
|
||||||
|
```
|
||||||
|
raw_data[11] == 0x10 → 10-byte sub_code=0x03 format (continuous mode)
|
||||||
|
raw_data[11] != 0x10 → 9-byte sub_code=0x10 format (single-shot mode)
|
||||||
|
```
|
||||||
|
|
||||||
|
**9-byte timestamp format (sub_code=0x10):**
|
||||||
|
|
||||||
|
| Byte | Field |
|
||||||
|
|---|---|
|
||||||
|
| 0 | day |
|
||||||
|
| 1 | `0x10` (sub_code marker) |
|
||||||
|
| 2 | month |
|
||||||
|
| 3–4 | year (uint16 BE) |
|
||||||
|
| 5 | unknown (0x00) |
|
||||||
|
| 6 | hour |
|
||||||
|
| 7 | minute |
|
||||||
|
| 8 | second |
|
||||||
|
|
||||||
|
**10-byte timestamp format (sub_code=0x03):**
|
||||||
|
|
||||||
|
| Byte | Field |
|
||||||
|
|---|---|
|
||||||
|
| 0 | `0x10` (marker) |
|
||||||
|
| 1 | day |
|
||||||
|
| 2 | `0x10` (marker) |
|
||||||
|
| 3 | month |
|
||||||
|
| 4–5 | year (uint16 BE) |
|
||||||
|
| 6 | unknown (0x00) |
|
||||||
|
| 7 | hour |
|
||||||
|
| 8 | minute |
|
||||||
|
| 9 | second |
|
||||||
|
|
||||||
|
**Two timestamps:** Each partial record contains two timestamps — `start_time` and
|
||||||
|
`stop_time` — stored consecutively:
|
||||||
|
- `ts1` (start) at `raw_data[ts_offset : ts_offset + ts_size]` where `ts_offset = 11`
|
||||||
|
- `ts2` (stop) at `raw_data[ts1_end : ts1_end + ts_size]`
|
||||||
|
|
||||||
|
**Edge case — 1-byte gap between timestamps:** Occurs when ts1 and ts2 share the same
|
||||||
|
minute:second. If `try_ts(raw_data[ts1_end:])` fails, try `try_ts(raw_data[ts1_end+1:])`.
|
||||||
|
Confirmed in frames 121, 161, 165 of the 4-11-26 MITM capture. Frame 121 still shows 0s
|
||||||
|
duration (both decode to 16:02:00) — the extra byte appears in all same-second cases.
|
||||||
|
|
||||||
|
**ASCII metadata after timestamps:**
|
||||||
|
```
|
||||||
|
<separator bytes> BE<serial>\x00Geo: <float> in/s ...
|
||||||
|
```
|
||||||
|
|
||||||
|
- Serial: scan for `b"BE"`, read until `b"\x00"` (e.g. `"BE11529"`)
|
||||||
|
- Geo threshold: scan for `b"Geo: "`, read float until next space (e.g. `0.254` in/s)
|
||||||
|
|
||||||
|
A separator of variable length (4–5 bytes of `\x00` + flags) sits between the two
|
||||||
|
timestamps and the ASCII region. The `b"BE"` anchor scan is robust to separator length
|
||||||
|
variation.
|
||||||
|
|
||||||
|
### `_decode_0a_partial_header(raw_data, index, key4)` — client.py
|
||||||
|
|
||||||
|
Returns a `MonitorLogEntry` or `None`. Called by `get_monitor_log_entries()` for each
|
||||||
|
event key whose 0x0A response has `raw_data[0] == 0x2C` or `length < 0x40`.
|
||||||
|
|
||||||
|
### `MiniMateClient.get_monitor_log_entries(skip_keys=None)` — client.py
|
||||||
|
|
||||||
|
Browse-mode walk: `1E → 0A → check type → decode if partial → 1F`. No 0x0C or 5A reads
|
||||||
|
performed. Full (0x46) records are skipped without decoding. Returns `list[MonitorLogEntry]`.
|
||||||
|
|
||||||
|
`skip_keys` (optional `set[str]`): keys in this set are still advanced through the walk
|
||||||
|
(to avoid disrupting the iteration sequence), but no `MonitorLogEntry` is created for them.
|
||||||
|
|
||||||
|
### `MonitorLogEntry` model — models.py
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class MonitorLogEntry:
|
||||||
|
index: int # 0-based position
|
||||||
|
key: str # 8-hex event key
|
||||||
|
start_time: Optional[datetime.datetime] = None
|
||||||
|
stop_time: Optional[datetime.datetime] = None
|
||||||
|
serial: Optional[str] = None
|
||||||
|
geo_threshold_ips: Optional[float] = None
|
||||||
|
raw_header: Optional[bytes] = field(default=None, repr=False)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def duration_seconds(self) -> Optional[float]: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### ACH server integration (v0.10.0)
|
||||||
|
|
||||||
|
After `get_events()`, the ACH server calls `get_monitor_log_entries(skip_keys=seen_keys)`.
|
||||||
|
New entries are saved to `monitor_log.json` in the session directory. Monitor log keys are
|
||||||
|
included in `current_keys` for state persistence so they are not re-processed on the next
|
||||||
|
call-home.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Auto Call Home config (SUBs 0x2C / 0x7E / 0x7F) — confirmed 2026-04-20
|
||||||
|
|
||||||
|
Full read/write pipeline confirmed from `bridges/captures/4-20-26/call home settings/`
|
||||||
|
(10 BW TX write frames diffed against the S3 read response).
|
||||||
|
|
||||||
|
Accessible in Blastware: **Remote Access → Setup Unit**.
|
||||||
|
|
||||||
|
### Protocol
|
||||||
|
|
||||||
|
**SUB 0x2C — Call Home Config READ (response 0xD3)**
|
||||||
|
|
||||||
|
Standard two-step read: probe offset `0x0000`, data offset `0x007C` (124).
|
||||||
|
Returns 125 raw bytes (one more than DATA_LENGTH) because the device encodes
|
||||||
|
num_retries value `3` as `\x10\x03` on the wire — S3FrameParser preserves both
|
||||||
|
bytes literally, shifting all subsequent field positions by +1.
|
||||||
|
|
||||||
|
**SUB 0x7E — Call Home Config WRITE (response 0x81)**
|
||||||
|
|
||||||
|
Write format (only BW_CMD `0x10` doubled on wire; DLE-aware checksum).
|
||||||
|
Payload = 125-byte read payload + `\x00\x00` = 127 bytes.
|
||||||
|
Offset = `data[1] + 2 = 0x7C + 2 = 0x7E`.
|
||||||
|
|
||||||
|
**SUB 0x7F — Call Home WRITE CONFIRM (response 0x80)**
|
||||||
|
|
||||||
|
Confirm frame, no data payload. Required after SUB 0x7E.
|
||||||
|
|
||||||
|
### Field map (raw 125-byte array from `data_rsp.data[11:]`)
|
||||||
|
|
||||||
|
| Raw Offset | Field | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `[5]` | `auto_call_home_enabled` | `0x00`=off, `0x01`=on |
|
||||||
|
| `[6:46]` | `dial_string` | 40-byte null-padded ASCII |
|
||||||
|
| `[87]` | `after_event_recorded` | bool |
|
||||||
|
| `[91]` | `at_specified_times` | bool |
|
||||||
|
| `[93]` | `time1_enabled` | bool |
|
||||||
|
| `[101]` | `time1_hour` | 0–23 |
|
||||||
|
| `[102]` | `time1_min` | 0–59 |
|
||||||
|
| `[95]` | `time2_enabled` | bool |
|
||||||
|
| `[105]` | `time2_hour` | 0–23 |
|
||||||
|
| `[106]` | `time2_min` | 0–59 |
|
||||||
|
| `[117]` | DLE prefix `0x10` | Part of `\x10\x03` (DLE-escaped ETX encoding value 3) |
|
||||||
|
| `[118]` | `num_retries` | Value = 3; detect via `raw[117] == 0x10` |
|
||||||
|
| `[120]` | `time_between_retries_sec` | Shifted +1 from logical 119 |
|
||||||
|
| `[122]` | `wait_for_connection_sec` | Shifted +1 from logical 121 |
|
||||||
|
| `[124]` | `warm_up_time_sec` | Shifted +1 from logical 123 |
|
||||||
|
|
||||||
|
**DLE-escaped 0x03 at raw[117:119]:** The byte value `0x03` is indistinguishable from the
|
||||||
|
frame ETX terminator, so the device encodes it as `\x10\x03` (DLE + ETX inner-terminator).
|
||||||
|
S3FrameParser in `STATE_AFTER_DLE` on ETX appends both bytes as literal payload. The write
|
||||||
|
frame sends them verbatim — device accepts `\x10\x03` and interprets it as value 3.
|
||||||
|
|
||||||
|
**Unconfirmed fields:** time slots 3 and 4 (offsets unknown), `modem_power_relay_enabled`.
|
||||||
|
|
||||||
|
### `CallHomeConfig` model — models.py
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class CallHomeConfig:
|
||||||
|
raw: Optional[bytes] = None # 125-byte raw read payload
|
||||||
|
auto_call_home_enabled: Optional[bool] = None # raw[5]
|
||||||
|
dial_string: Optional[str] = None # raw[6:46]
|
||||||
|
after_event_recorded: Optional[bool] = None # raw[87]
|
||||||
|
at_specified_times: Optional[bool] = None # raw[91]
|
||||||
|
time1_enabled: Optional[bool] = None # raw[93]
|
||||||
|
time1_hour: Optional[int] = None # raw[101]
|
||||||
|
time1_min: Optional[int] = None # raw[102]
|
||||||
|
time2_enabled: Optional[bool] = None # raw[95]
|
||||||
|
time2_hour: Optional[int] = None # raw[105]
|
||||||
|
time2_min: Optional[int] = None # raw[106]
|
||||||
|
num_retries: Optional[int] = None # raw[118] (DLE-prefixed)
|
||||||
|
time_between_retries_sec: Optional[int] = None # raw[120] (shifted +1)
|
||||||
|
wait_for_connection_sec: Optional[int] = None # raw[122] (shifted +1)
|
||||||
|
warm_up_time_sec: Optional[int] = None # raw[124] (shifted +1)
|
||||||
|
```
|
||||||
|
|
||||||
|
### SFM REST API — sfm/server.py
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /device/call_home?host=1.2.3.4&tcp_port=9034 ← read call home config
|
||||||
|
POST /device/call_home?host=1.2.3.4&tcp_port=9034 ← write call home config
|
||||||
|
```
|
||||||
|
|
||||||
|
POST body fields (all optional): `auto_call_home_enabled`, `after_event_recorded`,
|
||||||
|
`at_specified_times`, `time1_enabled`, `time1_hour`, `time1_min`, `time2_enabled`,
|
||||||
|
`time2_hour`, `time2_min`.
|
||||||
|
|
||||||
|
**Note:** `dial_string` is read-only in the current implementation (omitted from POST
|
||||||
|
body) because writing a dial string may require DLE escaping for embedded control characters.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## What's next
|
## What's next
|
||||||
|
|
||||||
|
- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
|
||||||
|
- **Histograms** — decode histogram-mode A5 data (noise floor tracking)
|
||||||
|
- **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed working for Continuous mode events (2026-04-23):** SFM-generated file opens in Blastware, shows correct PPV/waveform/timestamp. File is ~200 bytes shorter than BW (missing last ADC tail slice) — all measurements correct. Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that create spurious STRT markers in the body). Extension mapping: **CONFIRMED FALSE 2026-04-21** — extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`
|
||||||
|
|
||||||
|
**Serial encoding (CONFIRMED 2026-04-22):** `prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units).
|
||||||
|
|
||||||
|
**Stem encoding (FULLY CONFIRMED 2026-04-22):** stem = 4-char base-36 of `floor(total_seconds / 1296)` where `total_seconds = (event_local_time − 1985-01-01T00:00:00_local)` in seconds. Epoch = `1985-01-01 00:00:00` device local time — confirmed against 3,248 files from 10-year production archive with zero errors. Decode: `event_time = datetime(1985,1,1) + timedelta(seconds=stem_int*1296 + ab_int)`. Example: P036L318.C80H → BE14036, 2025-05-26 15:00:08, Full Histogram.
|
||||||
|
- **Blastware filename extension — NEW FIRMWARE FULLY DECODED (confirmed 2026-04-21, further confirmed 2026-04-22 from 10-year production archive frequency analysis):**
|
||||||
|
|
||||||
|
Extension format = `AB0T` (4 chars):
|
||||||
|
- `AB` = 2-char base-36 encoding of `total_seconds % 1296` (seconds within the 21.6-min window, 0–1295); `A = value // 36`, `B = value % 36`
|
||||||
|
- `0` = always literal digit zero (third character, invariant)
|
||||||
|
- `T` = event type: `W` = Full Waveform, `H` = Full Histogram
|
||||||
|
|
||||||
|
Combined with the 4-char stem, the full filename encodes a complete second-resolution timestamp. Verified against three S353L4H0.{3M0W,8S0H,9X0W} events (all match to the second) plus large-scale frequency analysis of a 10-year archive.
|
||||||
|
|
||||||
|
**3-day cycle property (confirmed 2026-04-22):** A unit recording at a fixed daily time cycles through exactly **3 extensions** with a 3-day period. Each calendar day shifts `total_seconds % 1296` by 864 (since `86400 % 1296 = 864`). The cycle repeats every 3 days because `gcd(1296, 864) = 432` and `1296 / 432 = 3`. The three extension values are spaced 432 seconds apart. Confirmed from 10-year archive: the top 3 extensions overall were `CE0H` (95 files), `0E0H` (93), `OE0H` (91) — all three are the 3-day cycle of a 06:00:14 daily call-in time (seconds-in-window = 14, 446, 878; all three have `E` as second character because `14 = E` in base-36 and adding 864 never changes `value % 36` since `864 = 24 × 36`).
|
||||||
|
|
||||||
|
**B character invariance:** For a unit recording at a fixed time of day, the second character `B` of the extension (`value % 36`) **never changes** — only the first character `A` cycles through 3 values. This means same-time-of-day files from different dates all share the same `B` character.
|
||||||
|
|
||||||
|
**Old firmware (S338, 3-char extensions ending in `0`):** encoding unknown. Extension is NOT recording mode. `blastware_filename()` returns `.N00` as a placeholder for old-firmware units.
|
||||||
|
|
||||||
|
**Micromate Series 4** uses a different extension format entirely (observed: `IDFH`, `IDFW`). The `AB0T` formula applies only to MiniMate Plus / V10.72 firmware.
|
||||||
- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
|
- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
|
||||||
|
- **Test Histogram recording mode (0x03) write via SFM** — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state (bare 0x03 in write vs BW's DLE-escaped `10 03`)
|
||||||
|
- **Compliance write anchor-9 cleanup** — when changing recording_mode via SFM, the byte at anchor-9 is not explicitly managed. A spurious `0x10` may persist after Histogram→other mode transitions. Does not affect device operation but differs from BW's byte-perfect output.
|
||||||
- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring)
|
- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring)
|
||||||
- ACH inbound server — accept call-home connections from field units
|
- Call Home — map time slots 3/4 offsets; add dial_string write support; confirm `modem_power_relay_enabled`
|
||||||
- Modem manager — push RV50/RV55 configs via Sierra Wireless API
|
- Modem manager — push RV50/RV55 configs via Sierra Wireless API
|
||||||
|
- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't
|
||||||
|
resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred)
|
||||||
|
|
||||||
|
## BW capture reference
|
||||||
|
|
||||||
|
`bridges/captures/` contains the following BW TX + S3 response captures for protocol analysis:
|
||||||
|
|
||||||
|
| Folder / File | Contents |
|
||||||
|
|---|---|
|
||||||
|
| `3-11-26/raw_bw_20260311_170151.bin` | Full compliance write + event download (SUBs 68→83 confirmed, frames 102–112) |
|
||||||
|
| `4-20-26/raw_bw_*_recording_mode_*.bin` | Recording mode changes: Continuous→Single Shot, →Histogram, →Histogram+Continuous |
|
||||||
|
| `4-20-26/histogram interval/` | Histogram interval changes: 1min, 5min, 15min, 15sec |
|
||||||
|
| `4-20-26/geo sensitivity/` | Geo sensitivity changes: 1.25 in/s (Sensitive), 10 in/s (Normal) |
|
||||||
|
| `4-20-26/call home settings/` | Call home config read/write captures |
|
||||||
|
| `4-8-26/` | Monitor status read, start/stop monitoring, SESSION_RESET signal, sensor check |
|
||||||
|
| `4-3-26-multi_event/` | Browse-mode S3 capture with 2+ events (1E/0A/1F iteration confirmed) |
|
||||||
|
| `4-2-26/` | Download-mode BW TX capture (5A bulk stream, POLL×3 requirement confirmed) |
|
||||||
|
| `3-31-26/` | Single-event download (148 BW / 147 S3 frames) |
|
||||||
|
| `mitm/ach_mitm_20260411_001912/` | Full ACH call-home MITM (erase protocol, 0xA3/0x06/0xA2 confirmed) |
|
||||||
|
|
||||||
|
To parse BW TX captures: use `bridges/captures/` scripts or adapt the `find_write_frames()` pattern
|
||||||
|
in `/tmp/analyze_write_payload.py` — it correctly handles `0x10 0x03` DLE-escaped ETX bytes
|
||||||
|
inside write frame data (the naive parser terminates early at the escaped `0x03`). | ||||||