Compare commits
30 Commits
2db565ff9c
...
ea9c69b7c9
| Author | SHA1 | Date | |
|---|---|---|---|
| ea9c69b7c9 | |||
| 71bcf71cf7 | |||
| 3e7de848bc | |||
| 72a4209cfd | |||
| 2b5574511e | |||
| ce2c859f11 | |||
| 7f322f9ff9 | |||
| 42b7a88c3d | |||
| c474db4f69 | |||
| 2765ee6ea7 | |||
| ef88240796 | |||
| 5591d345d9 | |||
| 7883a31aa7 | |||
| b241da970d | |||
| 6acb419ebd | |||
| f6a0846bab | |||
| 3d9db8b662 | |||
| c7e7d177e6 | |||
| a3b8d10fa8 | |||
| 4921b0489a | |||
| 8688d815a0 | |||
| 9b50ec9133 | |||
| cba8b1b401 | |||
| 41a14ca468 | |||
| 1bfc6e4258 | |||
| 574d40027f | |||
| 0358acb51d | |||
| cf7d838bf4 | |||
| 5e44cdc668 | |||
| 37d32077a4 |
@@ -3,6 +3,11 @@
|
||||
|
||||
/manuals/
|
||||
|
||||
# Python build artifacts
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
|
||||
# Python bytecode
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
|
||||
+225
@@ -4,6 +4,231 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
---
|
||||
|
||||
## v0.12.0 — 2026-04-13
|
||||
|
||||
### Added
|
||||
|
||||
- **`sfm/server.py` — `_LiveCache`** — in-memory live device cache, eliminating
|
||||
redundant TCP round-trips between requests. No extra dependencies (plain Python
|
||||
dict + threading.Lock). Replaces the SQLAlchemy-based `sfm/cache.py` experiment
|
||||
from the `feature/intelligent-caching` branch.
|
||||
|
||||
Cache behaviour by endpoint:
|
||||
|
||||
| Endpoint | Cache strategy |
|
||||
|---|---|
|
||||
| `GET /device/info` | Indefinite; invalidated by `POST /device/config` |
|
||||
| `GET /device/events` | Count-probe fast path: quick `poll()+count_events()` (~2s); return cache if count matches; full download only when new events detected |
|
||||
| `GET /device/monitor/status` | 30-second TTL; invalidated by monitor start/stop |
|
||||
| `GET /device/event/{idx}/waveform` | Permanent per-index (waveforms are immutable) |
|
||||
|
||||
- **`?force=true` param** on all four cached endpoints — bypasses cache and re-reads
|
||||
from device.
|
||||
|
||||
- **`POST /device/config` cache invalidation** — marks device info + events dirty so
|
||||
the next read reflects the new compliance config.
|
||||
|
||||
- **`POST /device/monitor/start` / `stop` cache invalidation** — evicts the monitor
|
||||
status cache entry immediately so the next poll returns the updated state.
|
||||
|
||||
### Removed
|
||||
|
||||
- `sfm/cache.py` — SQLAlchemy-based cache from the experimental caching branch.
|
||||
Its logic has been ported to the sqlite3-native `_LiveCache` class above.
|
||||
`sqlalchemy` is no longer a dependency.
|
||||
|
||||
---
|
||||
|
||||
## v0.11.0 — 2026-04-13
|
||||
|
||||
### Added
|
||||
|
||||
- **`sfm/database.py` — SeismoDb** — SQLite persistence layer for all ACH data.
|
||||
Three tables, all unit-keyed by serial number:
|
||||
- `ach_sessions` — one row per inbound call-home: serial, timestamp, peer IP,
|
||||
events_downloaded, monitor_entries, duration_seconds
|
||||
- `events` — one row per triggered waveform event: serial, waveform_key (dedup),
|
||||
timestamp, Tran/Vert/Long/VectorSum/Mic PPV, project/client/operator/sensor_location
|
||||
strings, sample_rate, record_type, false_trigger flag
|
||||
- `monitor_log` — one row per monitoring interval: serial, waveform_key (dedup),
|
||||
start_time, stop_time, duration_seconds, geo_threshold_ips
|
||||
- WAL mode, per-request connections — safe for the single-writer / occasional-reader
|
||||
ACH server pattern
|
||||
- Deduplication by `(serial, waveform_key)` UNIQUE constraint — re-runs and repeat
|
||||
call-homes never produce duplicate rows
|
||||
|
||||
- **`ach_server.py` — DB integration** — after each successful call-home, writes new
|
||||
events and monitor log entries to `seismo_relay.db` then records the session in
|
||||
`ach_sessions`. DB write failures are logged as warnings and do not abort the session.
|
||||
|
||||
- **`sfm/server.py` — DB read endpoints**:
|
||||
- `GET /db/units` — distinct serials with last_seen, total_events, total_monitor_entries
|
||||
- `GET /db/events` — query events with serial / date range / false_trigger filters
|
||||
- `GET /db/monitor_log` — query monitoring intervals
|
||||
- `GET /db/sessions` — query ACH call-home sessions
|
||||
- `PATCH /db/events/{id}/false_trigger` — flag/unflag false triggers (for review UI)
|
||||
|
||||
### Architecture
|
||||
|
||||
- seismo-relay DB is unit-keyed only — no project concepts. Project aggregation is
|
||||
terra-view's responsibility via `UnitAssignment` / `DeploymentRecord` + date range
|
||||
queries against the SFM DB endpoints.
|
||||
- DB file lives at `bridges/captures/seismo_relay.db` by default.
|
||||
|
||||
---
|
||||
|
||||
## v0.10.0 — 2026-04-11
|
||||
|
||||
### Added
|
||||
|
||||
- **`MiniMateClient.get_monitor_log_entries(skip_keys=None)`** — browse-mode walk
|
||||
(`1E → 0A → 1F`) that collects partial records (`0x2C` record type) from the device's
|
||||
event list without triggering a full waveform download (no 0C or 5A). Returns
|
||||
`list[MonitorLogEntry]`. Each entry represents one continuous monitoring interval where
|
||||
no threshold was exceeded.
|
||||
|
||||
- **`_decode_0a_partial_header(raw_data, index, key4)`** in `client.py` — decodes a SUB
|
||||
0x0A response payload whose record type is `0x2C`. Extracts:
|
||||
- `start_time` / `stop_time` — two consecutive timestamps; auto-detects 9-byte
|
||||
(sub_code=0x10, single-shot) vs 10-byte (sub_code=0x03, continuous) format from
|
||||
`raw_data[11]`. Handles a 1-byte gap between the two timestamps that occurs when
|
||||
ts1 and ts2 share the same minute:second.
|
||||
- `serial` — device serial string found via `b"BE"` anchor scan.
|
||||
- `geo_threshold_ips` — trigger level found via `b"Geo: "` anchor scan.
|
||||
|
||||
- **`MonitorLogEntry` dataclass** in `models.py` — new model for partial records:
|
||||
`index`, `key`, `start_time`, `stop_time`, `serial`, `geo_threshold_ips`,
|
||||
`raw_header`, and a `duration_seconds` property.
|
||||
|
||||
- **`read_waveform_header()` return value extended** — now returns `(data_rsp.data, length)`
|
||||
(full payload) instead of `(data_rsp.data[11:11+length], length)`. Callers get the
|
||||
complete payload including the record-type byte at position 0. Full records use
|
||||
`raw_data[11:11+length]` as before; partial records are detected by `raw_data[0] == 0x2C`.
|
||||
|
||||
- **ACH server: monitor log collection** — after `get_events()`, calls
|
||||
`get_monitor_log_entries(skip_keys=seen_keys)` and saves new entries to
|
||||
`monitor_log.json` in the session directory. Monitor log keys are included in
|
||||
`downloaded_keys` for state persistence (no re-processing on next call-home).
|
||||
|
||||
- **`_monitor_log_entry_to_dict()`** in `ach_server.py` — serialises a `MonitorLogEntry`
|
||||
to a JSON-compatible dict with ISO-format timestamps.
|
||||
|
||||
### Protocol / Documentation
|
||||
|
||||
- **SUB 0x0A partial record (0x2C) format confirmed** (✅ 4-11-26 MITM capture, 12 frames):
|
||||
- Record type `0x2C` at `raw_data[0]`; length < 64 bytes.
|
||||
- Two timestamps at `raw_data[11:]` — start and stop of the monitoring interval.
|
||||
- ASCII metadata region after timestamps: `BE<serial>\x00Geo: <float> in/s`.
|
||||
- Edge case: 1-byte separator between timestamps when ts1 and ts2 share minute:second.
|
||||
- 10-byte timestamp format (sub_code=0x03) signalled by `raw_data[11] == 0x10`.
|
||||
|
||||
- **Key reuse detection for monitor log entries** — monitor log keys are tracked alongside
|
||||
event keys in `ach_state.json` so the ACH server does not re-process them after a
|
||||
call-home cycle.
|
||||
|
||||
---
|
||||
|
||||
## v0.9.0 — 2026-04-11
|
||||
|
||||
### Added
|
||||
|
||||
- **`MiniMateClient.list_event_keys()`** — fast browse-mode walk (1E → 0A → 1F, no waveform
|
||||
download) that returns the list of event key hex strings currently stored on the device.
|
||||
Used by the ACH server as a cheap pre-check before deciding whether to call `get_events()`.
|
||||
|
||||
- **`get_events(skip_waveform_for_keys=set(...))`** — new optional parameter. For any key in
|
||||
the set the function performs only 0A + 1F(browse) instead of the full
|
||||
1E-arm → 0C → POLL×3 → 5A sequence. Eliminates redundant waveform downloads on repeat
|
||||
call-homes when the device still holds previously downloaded events.
|
||||
|
||||
- **`MiniMateClient.delete_all_events()`** — erases all events from device memory using the
|
||||
confirmed 4-step sequence:
|
||||
- SUB 0xA3 `begin_erase_all` — initiate erase (token=0xFE) → ack 0x5C
|
||||
- SUB 0x1C `read_monitor_status` — intermediate status read (Blastware-required)
|
||||
- SUB 0x06 `read_event_storage_range` — verify storage state (token=0xFE) → 36-byte response
|
||||
- SUB 0xA2 `confirm_erase_all` — commit erase (token=0xFE) → ack 0x5D
|
||||
|
||||
All four steps confirmed from 4-11-26 MITM capture of a live Blastware ACH session.
|
||||
After a successful call, the device's event counter resets to `0x01110000`.
|
||||
|
||||
- **`MiniMateProtocol` erase methods**: `begin_erase_all()`, `confirm_erase_all()`,
|
||||
`read_event_storage_range()` added to `protocol.py` with documented SUB constants
|
||||
`SUB_ERASE_ALL_BEGIN = 0xA3` and `SUB_ERASE_ALL_CONFIRM = 0xA2`.
|
||||
|
||||
- **`bridges/ach_mitm.py`** — transparent TCP-to-TCP MITM proxy. Listens for inbound unit
|
||||
connections, connects upstream to a real Blastware ACH server, and saves both directions
|
||||
to `raw_bw_<ts>.bin` / `raw_s3_<ts>.bin` files matching the existing capture format.
|
||||
Used to capture the 4-11-26 Blastware ACH session including event deletion.
|
||||
Usage: `python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998`
|
||||
|
||||
- **ACH server: key-based state tracking** — `ach_state.json` now stores
|
||||
`downloaded_keys: [hex_strings]` and `max_downloaded_key: hex_string` per unit instead of
|
||||
`event_count: N`. This correctly handles the standard workflow where events are deleted
|
||||
from the device after upload — a count-based approach would see `count=0` on the next
|
||||
call-home and silently skip new events.
|
||||
|
||||
- **ACH server: `--clear-after-download` flag** — after a successful download (at least one
|
||||
new event saved), erases all events from the device using `delete_all_events()`. Mirrors
|
||||
the standard Blastware ACH workflow. On success, `downloaded_keys` and
|
||||
`max_downloaded_key` are reset to empty so the next session starts fresh.
|
||||
|
||||
- **ACH server: post-erase key-reuse detection** — after an external erase (Blastware or
|
||||
manual), device keys restart from `0x01110000`, colliding with previously downloaded keys.
|
||||
On each browse walk, if `max(device_keys) < max_downloaded_key` (device counter rolled
|
||||
back), all device keys are treated as new regardless of `seen_keys`. This also catches
|
||||
erases performed by Blastware between our sessions.
|
||||
|
||||
### Protocol / Documentation
|
||||
|
||||
- **SUB 0xA3 / SUB 0xA2 — erase-all sequence confirmed** (✅ 4-11-26 MITM capture):
|
||||
Both frames use `token=0xFE` at `params[7]` and are standard `build_bw_frame` requests
|
||||
(not write-format). Response SUBs follow the standard formula: 0x5C and 0x5D.
|
||||
The intermediate 0x1C + 0x06 reads between them are required by Blastware.
|
||||
|
||||
- **SUB 0x06 — event storage range read confirmed** (✅ 4-11-26 MITM capture):
|
||||
Two-step read, data offset = 0x24 (36 bytes). The last 8 bytes of the response contain
|
||||
the first and last stored event keys (4 bytes each). After a successful erase, both keys
|
||||
read as `01110000` (device-empty state).
|
||||
|
||||
- **Event key counter resets to `0x01110000` after erase** — confirmed by observing key
|
||||
`01110000` on the device immediately after the MITM erase session.
|
||||
|
||||
---
|
||||
|
||||
## v0.8.0 — 2026-04-07
|
||||
|
||||
### Added
|
||||
|
||||
- **Write pipeline end-to-end** — `push_config_raw(event_index_data, compliance_data,
|
||||
trigger_data, waveform_data)` on `MiniMateClient` orchestrates the full
|
||||
`68→73 | 71×3→72 | 82→83 | 69→74→72` write sequence.
|
||||
|
||||
- **`build_bw_write_frame(sub, data, *, offset, params)`** in `framing.py` — dedicated frame
|
||||
builder for write commands (SUBs 0x68–0x83). Doubles only the BW_CMD byte; all other
|
||||
bytes including offset, params, data, and checksum are written raw. Uses the large-frame
|
||||
DLE-aware checksum (`sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF`).
|
||||
|
||||
- **`MiniMateProtocol` write methods** — `write_event_index()`, `write_compliance()`,
|
||||
`write_trigger_config()`, `write_waveform_data()`, `write_confirm()`,
|
||||
`start_monitoring()`, `stop_monitoring()`.
|
||||
|
||||
- **`AchSession` inbound server** (`bridges/ach_server.py`) — accepts call-home TCP
|
||||
connections, runs the full handshake + device-info + event-download sequence, saves
|
||||
`device_info.json` + `events.json` per session.
|
||||
|
||||
### Protocol / Documentation
|
||||
|
||||
- **Write frame format confirmed** (✅ 3-11-26 BW TX capture, all 11 frames): only BW_CMD
|
||||
byte `0x10` is doubled; all other bytes sent raw. Standard `build_bw_frame` DLE-stuffing
|
||||
is incorrect for write commands.
|
||||
- **Write ack responses** confirmed as 17-byte zero-data S3 frames.
|
||||
- **Monitoring SUBs 0x96/0x97** confirmed from 4-8-26 capture.
|
||||
- **SESSION_RESET signal** (`41 03`) required before POLL for monitoring units.
|
||||
- **SUB 0x1C monitoring flag** at `section[1]`: `0x00` = idle, `0x10` = monitoring.
|
||||
Confirmed by byte-diff of all 144 data frames in 4-8-26/2ndtry capture.
|
||||
|
||||
---
|
||||
|
||||
## v0.7.0 — 2026-04-03
|
||||
|
||||
### Added
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
|
||||
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.8.0**.
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.12.0**.
|
||||
|
||||
---
|
||||
|
||||
@@ -25,9 +25,9 @@ CHANGELOG.md ← version history
|
||||
|
||||
---
|
||||
|
||||
## Current implementation state (v0.8.0)
|
||||
## Current implementation state (v0.10.0)
|
||||
|
||||
Full read pipeline + write pipeline working end-to-end over TCP/cellular:
|
||||
Full read pipeline + write pipeline + erase pipeline + monitor log working end-to-end over TCP/cellular:
|
||||
|
||||
| Step | SUB | Status |
|
||||
|---|---|---|
|
||||
@@ -41,12 +41,16 @@ Full read pipeline + write pipeline working end-to-end over TCP/cellular:
|
||||
| Waveform record (peaks, timestamp, project) | 0C | ✅ |
|
||||
| **Bulk waveform stream (event-time metadata)** | **5A** | ✅ new v0.6.0 |
|
||||
| Event advance / next key | 1F | ✅ |
|
||||
| **Write commands (push config to device)** | **68–83** | ✅ **new v0.8.0** |
|
||||
| **Write commands (push config to device)** | **68–83** | ✅ new v0.8.0 |
|
||||
| **Erase all events** | **0xA3 → 0x1C → 0x06 → 0xA2** | ✅ new v0.9.0 |
|
||||
| **Monitor log entries (partial 0x2C records)** | **0A browse** | ✅ **new v0.10.0** |
|
||||
|
||||
`get_events()` sequence per event: `1E → 0A → 0C → 5A → 1F`
|
||||
|
||||
`push_config_raw()` write sequence: `68→73 | 71×3→72 | 82→83 | 69→74→72`
|
||||
|
||||
`delete_all_events()` erase sequence: `0xA3 → 0x1C → 0x06 → 0xA2`
|
||||
|
||||
---
|
||||
|
||||
## Protocol fundamentals
|
||||
@@ -412,6 +416,8 @@ for 0x10 records).
|
||||
|
||||
## SFM REST API (sfm/server.py)
|
||||
|
||||
### Live device endpoints (connect to device per-request)
|
||||
|
||||
```
|
||||
GET /device/info?port=COM5 ← serial
|
||||
GET /device/info?host=1.2.3.4&tcp_port=9034 ← cellular
|
||||
@@ -424,6 +430,19 @@ POST /device/monitor/stop?host=1.2.3.4&tcp_port=9034 ← stop recording
|
||||
|
||||
Server retries once on `ProtocolError` for TCP connections (handles cold-boot timing).
|
||||
|
||||
### DB read endpoints (query seismo_relay.db written by ach_server.py)
|
||||
|
||||
```
|
||||
GET /db/units ← all known serials + summary stats
|
||||
GET /db/events?serial=BE11529&from_dt=&to_dt=&limit= ← triggered events, newest first
|
||||
GET /db/monitor_log?serial=BE11529&from_dt=&to_dt= ← monitoring intervals, newest first
|
||||
GET /db/sessions?serial=BE11529&limit=50 ← ACH call-home sessions, newest first
|
||||
PATCH /db/events/{id}/false_trigger?value=true ← flag/unflag false triggers
|
||||
```
|
||||
|
||||
DB file: `bridges/captures/seismo_relay.db` (default; override with `--db-path` at startup).
|
||||
All DB endpoints are read-only except `PATCH /db/events/{id}/false_trigger`.
|
||||
|
||||
---
|
||||
|
||||
## Key wire captures (reference material)
|
||||
@@ -582,28 +601,32 @@ All confirmed from 4-8-26/2ndtry BW TX/S3 capture (clean start → 30s monitor
|
||||
Standard two-step read (probe at offset 0x00, data at offset 0x2C).
|
||||
Response SUB = 0xFF − 0x1C = **0xE3** (standard formula — no exception).
|
||||
|
||||
**Payload length is ~46–49 bytes in BOTH idle and monitoring states** — length alone
|
||||
is NOT a reliable mode indicator. Earlier note claiming "12 bytes when monitoring"
|
||||
was wrong (confirmed 2026-04-08 from 4-8-26/mid-monitor captures).
|
||||
**Payload length is 46–47 bytes IDLE, 48–49 bytes MONITORING** — not a reliable sole
|
||||
indicator due to 1-byte jitter overlap at the boundary.
|
||||
|
||||
**Monitoring flag (CORRECTED 2026-04-08 — full byte diff of 2ndtry capture):**
|
||||
- `section[6] == 0x00` → unit is **idle**
|
||||
- `section[6] == 0x10` → unit is **monitoring**
|
||||
**Monitoring flag (CONFIRMED 2026-04-09 — byte diff of all 144 data frames, 2ndtry capture):**
|
||||
- `section[1] == 0x00` → unit is **idle**
|
||||
- `section[1] == 0x10` → unit is **monitoring**
|
||||
|
||||
Earlier note claiming `section[1]` was the flag was WRONG — section[1] is always 0x00 in both states. The correction was found by diffing all 0xE3 data frames across the start/stop transitions: `section[6]` is the only byte that flips cleanly at frame #36 (start) and #132 (stop) within the 2ndtry 0xE3 frame sequence.
|
||||
This is `data[12]` (= `frame.data[12]`). The flag is 0x00 in all 36 IDLE_BEFORE frames,
|
||||
0x10 in all 98 MONITORING frames, and 0x00 in all 10 IDLE_AFTER frames — 100% accurate.
|
||||
|
||||
Battery and memory fields are present in **both** states, but the payload grows by **3 bytes** when monitoring is active (section goes from ~52 to ~55 bytes), shifting subsequent fields by +3.
|
||||
**HISTORY OF THIS FIELD (do not re-derive):** The original implementation used `section[1]`.
|
||||
A re-analysis in the prior session incorrectly concluded `section[1]` is always 0x00 and
|
||||
"corrected" the flag to `section[6]`, which has non-binary values (0xea idle, 0x07 monitoring)
|
||||
and is device-specific. The 2026-04-09 re-analysis confirms `section[1]` was right.
|
||||
|
||||
**Field offsets (relative to `data[11:]` = section):**
|
||||
**IMPORTANT — `frame.data` has checksum already stripped** by `S3FrameParser._finalise()`
|
||||
(`raw_payload = body[:-1]`; `data = raw_payload[5:]`). There is NO trailing checksum byte in
|
||||
`section`. All relative-from-end offsets must account for this.
|
||||
|
||||
Battery and memory are at **relative offsets from the end** — the payload can vary by ±1–3 bytes due to counter jitter and monitoring-mode expansion, but these 10 bytes are always anchored at the end:
|
||||
Battery and memory fields are present in **both** states:
|
||||
|
||||
| Offset (relative to end) | Field | Type | Notes |
|
||||
|---|---|---|---|
|
||||
| `section[-11:-9]` | battery voltage × 100 | uint16 BE | `0x02A8` = 680 → 6.80 V |
|
||||
| `section[-9:-5]` | memory total (bytes) | uint32 BE | e.g. 983026 ≈ 960 KB |
|
||||
| `section[-5:-1]` | memory free (bytes) | uint32 BE | decreases as events are stored |
|
||||
| `section[-1]` | frame checksum | — | last byte, skip |
|
||||
| `section[-10:-8]` | battery voltage × 100 | uint16 BE | `0x02A8` = 680 → 6.80 V |
|
||||
| `section[-8:-4]` | memory total (bytes) | uint32 BE | e.g. 983026 ≈ 960 KB |
|
||||
| `section[-4:]` | memory free (bytes) | uint32 BE | decreases as events are stored |
|
||||
|
||||
### SESSION_RESET signal (`41 03`) — required for monitoring units
|
||||
|
||||
@@ -657,7 +680,7 @@ Key findings:
|
||||
|
||||
**SFM behavior after `POST /device/monitor/start`:** `_pollMonitorConfirm()` polls
|
||||
`/device/monitor/status` every 5 s for up to 60 s, updating the badge on each poll.
|
||||
Status will show MONITORING once `section[6]` flips to `0x10`.
|
||||
Status will show MONITORING once `section[1]` flips to `0x10`.
|
||||
|
||||
### SUBs known from sensor-check capture (4-8-26) — NOT YET IMPLEMENTED
|
||||
|
||||
@@ -716,9 +739,208 @@ Full compliance config encoder is a future task.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Erase-all protocol (SUBs 0xA3/0xA2/0x06) — confirmed 2026-04-11
|
||||
|
||||
Full sequence confirmed from 4-11-26 MITM capture of a live Blastware ACH session
|
||||
(`bridges/captures/mitm/ach_mitm_20260411_001912/`).
|
||||
|
||||
### Wire sequence
|
||||
|
||||
```
|
||||
BW → device: SUB 0xA3 params=00 00 00 00 00 00 00 FE 00 00 (begin erase)
|
||||
device → BW: SUB 0x5C (ack)
|
||||
BW → device: SUB 0x1C probe (offset=0x00)
|
||||
device → BW: SUB 0xE3 (probe ack)
|
||||
BW → device: SUB 0x1C data (offset=0x2C)
|
||||
device → BW: SUB 0xE3 (monitor status response)
|
||||
BW → device: SUB 0x06 probe (offset=0x00, params same)
|
||||
device → BW: SUB 0xF9 (probe ack)
|
||||
BW → device: SUB 0x06 data (offset=0x24)
|
||||
device → BW: SUB 0xF9 (36-byte storage range response)
|
||||
BW → device: SUB 0xA2 params=00 00 00 00 00 00 00 FE 00 00 (confirm erase)
|
||||
device → BW: SUB 0x5D (ack — device memory is now cleared)
|
||||
```
|
||||
|
||||
All frames use standard `build_bw_frame` (not write-format). Response SUBs follow the
|
||||
standard `0xFF - SUB` formula; no exceptions.
|
||||
|
||||
### SUB 0x06 — event storage range response (36 bytes)
|
||||
|
||||
The 36-byte response body ends with two 4-byte event keys:
|
||||
|
||||
| Offset (from end) | Field | Notes |
|
||||
|---|---|---|
|
||||
| `[-8:-4]` | first stored event key | `01110000` when empty |
|
||||
| `[-4:]` | last stored event key | `01110000` when empty |
|
||||
|
||||
Before erase: ends with `<first_key> <last_key>` (e.g. `0111ea60 0111eaa6`).
|
||||
After erase: both bytes read `01110000` — device's empty/reset sentinel.
|
||||
|
||||
### Post-erase key counter reset
|
||||
|
||||
After a successful erase, the device resets its event counter. New events start from
|
||||
key `0x01110000` again — the same key as the very first event ever recorded. This means
|
||||
key-based deduplication in the ACH server must account for key reuse:
|
||||
|
||||
- After our own erase: `ach_state.json` `downloaded_keys` and `max_downloaded_key` are
|
||||
cleared so the next session starts fresh.
|
||||
- After an external erase: the ACH server detects it by comparing `max(device_keys)` to
|
||||
`max_downloaded_key` from state. If the device max has rolled back below the historical
|
||||
max, all current device keys are treated as new regardless of `seen_keys`.
|
||||
|
||||
### ACH server state format (v0.9.0)
|
||||
|
||||
`bridges/captures/ach_state.json`:
|
||||
```json
|
||||
{
|
||||
"BE11529": {
|
||||
"downloaded_keys": ["01110000", "0111245a"],
|
||||
"max_downloaded_key": "0111245a",
|
||||
"last_seen": "2026-04-11T01:04:36",
|
||||
"serial": "BE11529",
|
||||
"peer": "63.43.212.232:51920"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`max_downloaded_key` is the high-water mark — the largest key ever downloaded from the
|
||||
unit. It is NOT reset when events are erased from the device (only when our server does
|
||||
the erase). Used for post-erase detection.
|
||||
|
||||
---
|
||||
|
||||
## Monitor log entries — SUB 0x0A partial records (confirmed 2026-04-11)
|
||||
|
||||
Confirmed from 4-11-26 MITM capture: 12 partial records (record type `0x2C`) and 7 full
|
||||
event records (record type `0x46`) across 19 total 0x0A responses.
|
||||
|
||||
### Record type detection
|
||||
|
||||
`read_waveform_header()` returns `(raw_data, length)` where `raw_data = data_rsp.data`
|
||||
(the full payload including prefix bytes). The record type is at `raw_data[0]`:
|
||||
|
||||
| Value | Type | How to process |
|
||||
|---|---|---|
|
||||
| `0x46` | Full triggered event | Normal download: 0C → 5A → 1F |
|
||||
| `0x2C` | Monitor log entry (partial) | No 0C/5A; decode inline from 0A payload |
|
||||
|
||||
Length heuristic: `length < 0x40` (64) reliably identifies partial records across all
|
||||
observed captures. Both checks (`raw_data[0] == 0x2C` and `length < 0x40`) are used.
|
||||
|
||||
### SUB 0x0A partial record (0x2C) payload layout
|
||||
|
||||
All offsets are from `raw_data` (the full `data_rsp.data` array including the 11-byte
|
||||
prefix before the actual header bytes start).
|
||||
|
||||
```
|
||||
raw_data[0] = 0x2C ← record type (partial / monitor log)
|
||||
raw_data[1:11] = prefix bytes (vary; contain key4 copy, flags, length)
|
||||
raw_data[11:] = timestamp and ASCII metadata payload
|
||||
```
|
||||
|
||||
**Timestamp auto-detection** (confirmed from 4-11-26 capture):
|
||||
|
||||
```
|
||||
raw_data[11] == 0x10 → 10-byte sub_code=0x03 format (continuous mode)
|
||||
raw_data[11] != 0x10 → 9-byte sub_code=0x10 format (single-shot mode)
|
||||
```
|
||||
|
||||
**9-byte timestamp format (sub_code=0x10):**
|
||||
|
||||
| Byte | Field |
|
||||
|---|---|
|
||||
| 0 | day |
|
||||
| 1 | `0x10` (sub_code marker) |
|
||||
| 2 | month |
|
||||
| 3–4 | year (uint16 BE) |
|
||||
| 5 | unknown (0x00) |
|
||||
| 6 | hour |
|
||||
| 7 | minute |
|
||||
| 8 | second |
|
||||
|
||||
**10-byte timestamp format (sub_code=0x03):**
|
||||
|
||||
| Byte | Field |
|
||||
|---|---|
|
||||
| 0 | `0x10` (marker) |
|
||||
| 1 | day |
|
||||
| 2 | `0x10` (marker) |
|
||||
| 3 | month |
|
||||
| 4–5 | year (uint16 BE) |
|
||||
| 6 | unknown (0x00) |
|
||||
| 7 | hour |
|
||||
| 8 | minute |
|
||||
| 9 | second |
|
||||
|
||||
**Two timestamps:** Each partial record contains two timestamps — `start_time` and
|
||||
`stop_time` — stored consecutively:
|
||||
- `ts1` (start) at `raw_data[ts_offset : ts_offset + ts_size]` where `ts_offset = 11`
|
||||
- `ts2` (stop) at `raw_data[ts1_end : ts1_end + ts_size]`
|
||||
|
||||
**Edge case — 1-byte gap between timestamps:** Occurs when ts1 and ts2 share the same
|
||||
minute:second. If `try_ts(raw_data[ts1_end:])` fails, try `try_ts(raw_data[ts1_end+1:])`.
|
||||
Confirmed in frames 121, 161, 165 of the 4-11-26 MITM capture. Frame 121 still shows 0s
|
||||
duration (both decode to 16:02:00) — the extra byte appears in all same-second cases.
|
||||
|
||||
**ASCII metadata after timestamps:**
|
||||
```
|
||||
<separator bytes> BE<serial>\x00Geo: <float> in/s ...
|
||||
```
|
||||
|
||||
- Serial: scan for `b"BE"`, read until `b"\x00"` (e.g. `"BE11529"`)
|
||||
- Geo threshold: scan for `b"Geo: "`, read float until next space (e.g. `0.254` in/s)
|
||||
|
||||
A separator of variable length (4–5 bytes of `\x00` + flags) sits between the two
|
||||
timestamps and the ASCII region. The `b"BE"` anchor scan is robust to separator length
|
||||
variation.
|
||||
|
||||
### `_decode_0a_partial_header(raw_data, index, key4)` — client.py
|
||||
|
||||
Returns a `MonitorLogEntry` or `None`. Called by `get_monitor_log_entries()` for each
|
||||
event key whose 0x0A response has `raw_data[0] == 0x2C` or `length < 0x40`.
|
||||
|
||||
### `MiniMateClient.get_monitor_log_entries(skip_keys=None)` — client.py
|
||||
|
||||
Browse-mode walk: `1E → 0A → check type → decode if partial → 1F`. No 0x0C or 5A reads
|
||||
performed. Full (0x46) records are skipped without decoding. Returns `list[MonitorLogEntry]`.
|
||||
|
||||
`skip_keys` (optional `set[str]`): keys in this set are still advanced through the walk
|
||||
(to avoid disrupting the iteration sequence), but no `MonitorLogEntry` is created for them.
|
||||
|
||||
### `MonitorLogEntry` model — models.py
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class MonitorLogEntry:
|
||||
index: int # 0-based position
|
||||
key: str # 8-hex event key
|
||||
start_time: Optional[datetime.datetime] = None
|
||||
stop_time: Optional[datetime.datetime] = None
|
||||
serial: Optional[str] = None
|
||||
geo_threshold_ips: Optional[float] = None
|
||||
raw_header: Optional[bytes] = field(default=None, repr=False)
|
||||
|
||||
@property
|
||||
def duration_seconds(self) -> Optional[float]: ...
|
||||
```
|
||||
|
||||
### ACH server integration (v0.10.0)
|
||||
|
||||
After `get_events()`, the ACH server calls `get_monitor_log_entries(skip_keys=seen_keys)`.
|
||||
New entries are saved to `monitor_log.json` in the session directory. Monitor log keys are
|
||||
included in `current_keys` for state persistence so they are not re-processed on the next
|
||||
call-home.
|
||||
|
||||
---
|
||||
|
||||
## What's next
|
||||
|
||||
- **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
|
||||
- **Histograms** — decode histogram-mode A5 data (noise floor tracking)
|
||||
- Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
|
||||
- Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring)
|
||||
- ACH inbound server — accept call-home connections from field units
|
||||
- Modem manager — push RV50/RV55 configs via Sierra Wireless API
|
||||
- RV55 DCD/DTR issue — newer RV55 firmware doesn't assert DCD by default; units don't
|
||||
resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred)
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
# seismo-relay `v0.6.0`
|
||||
# seismo-relay `v0.12.0`
|
||||
|
||||
A ground-up replacement for **Blastware** — Instantel's aging Windows-only
|
||||
software for managing MiniMate Plus seismographs.
|
||||
|
||||
Built in Python. Runs on Windows. Connects to instruments over direct RS-232
|
||||
or cellular modem (Sierra Wireless RV50 / RV55).
|
||||
Built in Python. Runs on Windows, Linux, or macOS. Connects to instruments
|
||||
over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).
|
||||
|
||||
> **Status:** Active development. Full read pipeline working end-to-end:
|
||||
> device info, compliance config (with geo thresholds), event download with
|
||||
> true event-time metadata (project / client / operator / sensor location
|
||||
> sourced from the device at record-time via SUB 5A). Write commands in progress.
|
||||
> See [CHANGELOG.md](CHANGELOG.md) for version history.
|
||||
> **Status:** Active development. Full read + write + erase + monitoring
|
||||
> pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server
|
||||
> handles inbound unit connections, downloads events, and persists everything
|
||||
> to a SQLite database. SFM REST API exposes device control and DB queries.
|
||||
> See [CHANGELOG.md](CHANGELOG.md) for full version history.
|
||||
|
||||
---
|
||||
|
||||
@@ -21,26 +21,28 @@ seismo-relay/
|
||||
├── seismo_lab.py ← Main GUI (Bridge + Analyzer + Console tabs)
|
||||
│
|
||||
├── minimateplus/ ← MiniMate Plus client library
|
||||
│ ├── transport.py ← SerialTransport and TcpTransport
|
||||
│ ├── protocol.py ← DLE frame layer (read/write/parse)
|
||||
│ ├── client.py ← High-level client (connect, get_config, etc.)
|
||||
│ ├── framing.py ← Frame builder/parser primitives
|
||||
│ └── models.py ← DeviceInfo, EventRecord, etc.
|
||||
│ ├── transport.py ← SerialTransport, TcpTransport, SocketTransport
|
||||
│ ├── protocol.py ← DLE frame layer, SUB command dispatch
|
||||
│ ├── client.py ← High-level client (connect, get_events, push_config, …)
|
||||
│ ├── framing.py ← Frame builders, DLE codec, S3FrameParser
|
||||
│ └── models.py ← DeviceInfo, Event, ComplianceConfig, MonitorLogEntry, …
|
||||
│
|
||||
├── sfm/ ← SFM REST API server (FastAPI)
|
||||
│ └── server.py ← /device/info, /device/events, /device/event
|
||||
├── sfm/ ← SFM REST API server (FastAPI, port 8200)
|
||||
│ ├── server.py ← All device + DB endpoints
|
||||
│ ├── database.py ← SeismoDb — SQLite persistence layer
|
||||
│ └── sfm_webapp.html ← Embedded web UI (served at /)
|
||||
│
|
||||
├── bridges/
|
||||
│ ├── s3-bridge/
|
||||
│ │ └── s3_bridge.py ← RS-232 serial bridge (capture tool)
|
||||
│ ├── ach_server.py ← Inbound ACH call-home server (main production server)
|
||||
│ ├── ach_mitm.py ← Transparent MITM proxy for capturing BW sessions
|
||||
│ ├── s3-bridge/ ← RS-232 serial bridge (capture tool)
|
||||
│ ├── tcp_serial_bridge.py ← Local TCP↔serial bridge (bench testing)
|
||||
│ ├── gui_bridge.py ← Standalone bridge GUI (legacy)
|
||||
│ ├── gui_bridge.py ← Standalone bridge GUI
|
||||
│ └── raw_capture.py ← Simple raw capture tool
|
||||
│
|
||||
├── parsers/
|
||||
│ ├── s3_parser.py ← DLE frame extractor
|
||||
│ ├── s3_analyzer.py ← Session parser, differ, Claude export
|
||||
│ ├── gui_analyzer.py ← Standalone analyzer GUI (legacy)
|
||||
│ ├── gui_analyzer.py ← Standalone analyzer GUI
|
||||
│ └── frame_db.py ← SQLite frame database
|
||||
│
|
||||
└── docs/
|
||||
@@ -51,123 +53,88 @@ seismo-relay/
|
||||
|
||||
## Quick start
|
||||
|
||||
### Seismo Lab (main GUI)
|
||||
### ACH inbound server (production)
|
||||
|
||||
The all-in-one tool. Three tabs: **Bridge**, **Analyzer**, **Console**.
|
||||
Listens for inbound unit call-homes, downloads all new events and monitor log
|
||||
entries, and writes everything to `bridges/captures/seismo_relay.db`.
|
||||
|
||||
```bash
|
||||
python bridges/ach_server.py --port 12345 --output bridges/captures/
|
||||
```
|
||||
python seismo_lab.py
|
||||
|
||||
Point the unit's ACEmanager **Remote Host** to this machine's IP and **Remote Port** to `12345`.
|
||||
|
||||
Options:
|
||||
```
|
||||
--port N Listen port (default 12345)
|
||||
--output DIR Capture directory (default bridges/captures/)
|
||||
--allow-ip IP Allowlist an IP (repeat for multiple; default: accept all)
|
||||
--max-events N Safety cap for first run (default: unlimited)
|
||||
--clear-after-download Erase device memory after successful download
|
||||
--verbose Debug logging
|
||||
```
|
||||
|
||||
### SFM REST server
|
||||
|
||||
Exposes MiniMate Plus commands as a REST API for integration with other systems.
|
||||
Exposes device control and DB queries as a REST API. Proxied by terra-view.
|
||||
|
||||
```
|
||||
cd sfm
|
||||
uvicorn server:app --reload
|
||||
```bash
|
||||
python sfm/server.py # default: 0.0.0.0:8200
|
||||
python -m uvicorn sfm.server:app --host 0.0.0.0 --port 8200 --reload
|
||||
```
|
||||
|
||||
**Endpoints:**
|
||||
Open `http://localhost:8200` for the embedded web UI, or `http://localhost:8200/docs`
|
||||
for the interactive API docs.
|
||||
|
||||
### Seismo Lab GUI
|
||||
|
||||
```bash
|
||||
python seismo_lab.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SFM REST API
|
||||
|
||||
### Live device endpoints
|
||||
|
||||
Each call dials the device, does its work, and closes the connection. TCP
|
||||
connections are retried once on `ProtocolError` to handle cold-boot timing.
|
||||
|
||||
**Caching** — frequently-polled endpoints are cached in-process to avoid
|
||||
redundant TCP round-trips:
|
||||
|
||||
| Method | URL | Cache |
|
||||
|--------|-----|-------|
|
||||
| `GET` | `/device/info` | Indefinite; invalidated by `POST /device/config` |
|
||||
| `GET` | `/device/events` | Count-probe fast path (~2s); full download only when new events detected |
|
||||
| `GET` | `/device/event/{idx}/waveform` | Permanent per event index |
|
||||
| `GET` | `/device/monitor/status` | 30-second TTL |
|
||||
| `POST` | `/device/connect` | — |
|
||||
| `POST` | `/device/config` | Writes compliance config; invalidates cache |
|
||||
| `POST` | `/device/monitor/start` | Sends SUB 0x96 |
|
||||
| `POST` | `/device/monitor/stop` | Sends SUB 0x97 |
|
||||
|
||||
All cached endpoints accept `?force=true` to bypass the cache.
|
||||
|
||||
Transport query params (supply one set):
|
||||
```
|
||||
Serial: ?port=COM5&baud=38400
|
||||
TCP: ?host=1.2.3.4&tcp_port=12345
|
||||
```
|
||||
|
||||
### DB read endpoints
|
||||
|
||||
Query the SQLite database written by `ach_server.py`. All read-only except
|
||||
`PATCH /db/events/{id}/false_trigger`.
|
||||
|
||||
| Method | URL | Description |
|
||||
|--------|-----|-------------|
|
||||
| `GET` | `/device/info?port=COM5` | Device info via serial |
|
||||
| `GET` | `/device/info?host=1.2.3.4&tcp_port=9034` | Device info via cellular modem |
|
||||
| `GET` | `/device/events?port=COM5` | Event index |
|
||||
| `GET` | `/device/event?port=COM5&index=0` | Single event record |
|
||||
|
||||
---
|
||||
|
||||
## Seismo Lab tabs
|
||||
|
||||
### Bridge tab
|
||||
|
||||
Captures live RS-232 traffic between Blastware and the seismograph. Sits in
|
||||
the middle as a transparent pass-through while logging everything to disk.
|
||||
|
||||
```
|
||||
Blastware → COM4 (virtual) ↔ s3_bridge ↔ COM5 (physical) → MiniMate Plus
|
||||
```
|
||||
|
||||
Set your COM ports and log directory, then hit **Start Bridge**. Use
|
||||
**Add Mark** to annotate the capture at specific moments (e.g. "changed
|
||||
trigger level"). When the bridge starts, the Analyzer tab automatically wires
|
||||
up to the live files and starts updating in real time.
|
||||
|
||||
### Analyzer tab
|
||||
|
||||
Parses raw captures into DLE-framed protocol sessions, diffs consecutive
|
||||
sessions to show exactly which bytes changed, and lets you query across all
|
||||
historical captures via the built-in SQLite database.
|
||||
|
||||
- **Inventory** — all frames in a session, click to drill in
|
||||
- **Hex Dump** — full payload hex dump with changed-byte annotations
|
||||
- **Diff** — byte-level before/after diff between sessions
|
||||
- **Full Report** — plain text session report
|
||||
- **Query DB** — search across all captures by SUB, direction, or byte value
|
||||
|
||||
Use **Export for Claude** to generate a self-contained `.md` report for
|
||||
AI-assisted field mapping.
|
||||
|
||||
### Console tab
|
||||
|
||||
Direct connection to a MiniMate Plus — no bridge, no Blastware. Useful for
|
||||
diagnosing field units over cellular without a full capture session.
|
||||
|
||||
**Connection:** choose Serial (COM port + baud) or TCP (IP + port for
|
||||
cellular modem).
|
||||
|
||||
**Commands:**
|
||||
| Button | What it does |
|
||||
|--------|-------------|
|
||||
| POLL | Startup handshake — confirms unit is alive and identifies model |
|
||||
| Serial # | Reads unit serial number |
|
||||
| Full Config | Reads full 166-byte config block (firmware version, channel scales, etc.) |
|
||||
| Event Index | Reads stored event list |
|
||||
|
||||
Output is colour-coded: TX in blue, raw RX bytes in teal, decoded fields in
|
||||
green, errors in red. **Save Log** writes a timestamped `.log` file to
|
||||
`bridges/captures/`. **Send to Analyzer** injects the captured bytes into the
|
||||
Analyzer tab for deeper inspection.
|
||||
|
||||
---
|
||||
|
||||
## Connecting over cellular (RV50 / RV55 modems)
|
||||
|
||||
Field units connect via Sierra Wireless RV50 or RV55 cellular modems. Use
|
||||
TCP mode in the Console or SFM:
|
||||
|
||||
```
|
||||
# Console tab
|
||||
Transport: TCP
|
||||
Host: <modem public IP>
|
||||
Port: 9034 ← Device Port in ACEmanager (call-up mode)
|
||||
```
|
||||
|
||||
```python
|
||||
# In code
|
||||
from minimateplus.transport import TcpTransport
|
||||
from minimateplus.client import MiniMateClient
|
||||
|
||||
client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
|
||||
info = client.connect()
|
||||
```
|
||||
|
||||
### Required ACEmanager settings (Serial tab)
|
||||
|
||||
These must match exactly — a single wrong setting causes the unit to beep
|
||||
on connect but never respond:
|
||||
|
||||
| Setting | Value | Why |
|
||||
|---------|-------|-----|
|
||||
| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate |
|
||||
| Flow Control | `None` | Hardware flow control blocks unit TX if pins unconnected |
|
||||
| **Quiet Mode** | **Enable** | **Critical.** Disabled → modem injects `RING`/`CONNECT` onto serial line, corrupting the S3 handshake |
|
||||
| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency; `5` works but is sluggish |
|
||||
| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame |
|
||||
| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect |
|
||||
| DB9 Serial Echo | `Disable` | Echo corrupts the data stream |
|
||||
| `GET` | `/db/units` | All known serials with summary stats |
|
||||
| `GET` | `/db/events` | Triggered events (filter by serial, date range, false_trigger) |
|
||||
| `GET` | `/db/monitor_log` | Monitoring intervals |
|
||||
| `GET` | `/db/sessions` | ACH call-home session history |
|
||||
| `PATCH` | `/db/events/{id}/false_trigger?value=true` | Flag / unflag false triggers |
|
||||
|
||||
---
|
||||
|
||||
@@ -175,25 +142,76 @@ on connect but never respond:
|
||||
|
||||
```python
|
||||
from minimateplus import MiniMateClient
|
||||
from minimateplus.transport import SerialTransport, TcpTransport
|
||||
from minimateplus.transport import TcpTransport
|
||||
|
||||
# Serial
|
||||
client = MiniMateClient(port="COM5")
|
||||
|
||||
# TCP (cellular modem)
|
||||
client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
|
||||
client = MiniMateClient(transport=TcpTransport("1.2.3.4", 12345), timeout=30.0)
|
||||
|
||||
with client:
|
||||
info = client.connect() # DeviceInfo — model, serial, firmware, compliance config
|
||||
serial = client.get_serial() # Serial number string
|
||||
config = client.get_config() # Full config block (bytes)
|
||||
events = client.get_events() # List[EventRecord] with true event-time metadata
|
||||
# Read
|
||||
info = client.connect() # DeviceInfo — serial, firmware, compliance config
|
||||
count = client.count_events() # Number of stored events
|
||||
keys = client.list_event_keys() # Fast browse walk — event keys only, no download
|
||||
events = client.get_events() # Full download: headers + peaks + metadata
|
||||
monitor = client.get_monitor_status() # Battery, memory, is_monitoring flag
|
||||
log = client.get_monitor_log_entries() # Monitoring intervals (partial 0x2C records)
|
||||
|
||||
# Write
|
||||
client.apply_config(
|
||||
sample_rate=1024,
|
||||
trigger_level_geo=0.5,
|
||||
project="Bridge Inspection 2026",
|
||||
client_name="City of Portland",
|
||||
operator="B. Harrison",
|
||||
)
|
||||
|
||||
# Control
|
||||
client.start_monitoring() # SUB 0x96
|
||||
client.stop_monitoring() # SUB 0x97
|
||||
client.delete_all_events() # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2)
|
||||
```
|
||||
|
||||
`get_events()` runs the full download sequence per event: `1E → 0A → 0C → 5A → 1F`.
|
||||
The SUB 5A bulk waveform stream is used to retrieve `client`, `operator`, and
|
||||
`sensor_location` as they existed at record time — not backfilled from the current
|
||||
compliance config.
|
||||
`get_events()` runs the full per-event sequence: `1E → 0A → 0C → 5A → 1F`.
|
||||
SUB 5A bulk stream provides `client`, `operator`, and `sensor_location` as they
|
||||
existed at record time — not backfilled from the current compliance config.
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
`ach_server.py` writes to `bridges/captures/seismo_relay.db` (SQLite, WAL mode).
|
||||
Three tables, all unit-keyed by serial number:
|
||||
|
||||
| Table | Key | Contents |
|
||||
|-------|-----|----------|
|
||||
| `ach_sessions` | UUID | Per-call-home audit record: serial, peer IP, events_downloaded, duration |
|
||||
| `events` | UUID, UNIQUE(serial, waveform_key) | Triggered events: timestamp, PPV per channel, project/client/operator strings, false_trigger flag |
|
||||
| `monitor_log` | UUID, UNIQUE(serial, waveform_key) | Monitoring intervals: start/stop time, duration, geo threshold |
|
||||
|
||||
Deduplication is by `(serial, waveform_key)` — repeat call-homes or re-runs
|
||||
never produce duplicate rows. Post-erase key reuse is handled automatically
|
||||
via the high-water mark in `ach_state.json`.
|
||||
|
||||
---
|
||||
|
||||
## Connecting over cellular (RV50 / RV55)
|
||||
|
||||
Field units connect via Sierra Wireless RV50 or RV55 cellular modems.
|
||||
|
||||
### Required ACEmanager settings
|
||||
|
||||
| Setting | Value | Why |
|
||||
|---------|-------|-----|
|
||||
| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate |
|
||||
| Flow Control | `None` | Hardware FC blocks TX if pins unconnected |
|
||||
| **Quiet Mode** | **Enable** | **Critical** — disabled injects `RING`/`CONNECT` onto serial, corrupting the S3 handshake |
|
||||
| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency |
|
||||
| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame |
|
||||
| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect |
|
||||
| DB9 Serial Echo | `Disable` | Echo corrupts the data stream |
|
||||
|
||||
---
|
||||
|
||||
@@ -204,23 +222,10 @@ compliance config.
|
||||
| DLE | `0x10` | Data Link Escape |
|
||||
| STX | `0x02` | Start of frame |
|
||||
| ETX | `0x03` | End of frame |
|
||||
| ACK | `0x41` (`'A'`) | Frame-start marker sent before every frame |
|
||||
| ACK | `0x41` | Frame-start marker sent before every BW frame |
|
||||
| DLE stuffing | `10 10` on wire | Literal `0x10` in payload |
|
||||
|
||||
**S3-side frame** (seismograph → Blastware): `ACK DLE+STX [payload] CHK DLE+ETX`
|
||||
|
||||
**De-stuffed payload header:**
|
||||
```
|
||||
[0] CMD 0x10 = BW request, 0x00 = S3 response
|
||||
[1] ? unknown (0x00 BW / 0x10 S3)
|
||||
[2] SUB Command/response identifier ← the key field
|
||||
[3] PAGE_HI Page address high byte
|
||||
[4] PAGE_LO Page address low byte
|
||||
[5+] DATA Payload content
|
||||
```
|
||||
|
||||
**Response SUB rule:** `response_SUB = 0xFF - request_SUB`
|
||||
Example: request SUB `0x08` (Event Index) → response SUB `0xF7`
|
||||
**Response SUB rule:** `response_SUB = 0xFF - request_SUB` (no exceptions)
|
||||
|
||||
Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/instantel_protocol_reference.md)
|
||||
|
||||
@@ -228,32 +233,36 @@ Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/insta
|
||||
|
||||
## Requirements
|
||||
|
||||
```
|
||||
```bash
|
||||
pip install pyserial fastapi uvicorn
|
||||
```
|
||||
|
||||
Python 3.10+. Tkinter is included with the standard Python installer on
|
||||
Windows (make sure "tcl/tk and IDLE" is checked during install).
|
||||
Windows (check "tcl/tk and IDLE" during install).
|
||||
|
||||
---
|
||||
|
||||
## Virtual COM ports (bridge capture)
|
||||
|
||||
The bridge needs two COM ports on the same PC — one that Blastware connects
|
||||
to, and one wired to the seismograph. Use a virtual COM port pair
|
||||
(**com0com** or **VSPD**) to give Blastware a port to talk to.
|
||||
|
||||
```
|
||||
Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus
|
||||
```
|
||||
|
||||
Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [x] Event download — pull waveform records from the unit (`1E → 0A → 0C → 5A → 1F`)
|
||||
- [x] True event-time metadata — project / client / operator / sensor location from SUB 5A
|
||||
- [ ] Write commands — push config changes to the unit (compliance setup, channel config, trigger settings)
|
||||
- [ ] ACH inbound server — accept call-home connections from field units
|
||||
- [ ] Modem manager — push standard configs to RV50/RV55 fleet via Sierra Wireless API
|
||||
- [ ] Full Blastware parity — complete read/write/download cycle without Blastware
|
||||
- [x] Full read pipeline — device info, compliance config, event download with true event-time metadata
|
||||
- [x] Write commands — push compliance config, trigger thresholds, project strings to device
|
||||
- [x] Erase all events — confirmed erase sequence from live MITM capture
|
||||
- [x] Monitor control — start/stop monitoring, read battery/memory/status
|
||||
- [x] Monitor log entries — decode partial 0x2C records (continuous monitoring intervals)
|
||||
- [x] ACH inbound server — accept call-home connections, download events, dedup by key
|
||||
- [x] SQLite persistence — events, monitor log, and session history in `seismo_relay.db`
|
||||
- [x] SFM REST API — device control + DB query endpoints, live device cache
|
||||
- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
|
||||
- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
|
||||
- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
|
||||
- [ ] Modem manager — push RV50/RV55 configs via Sierra Wireless API
|
||||
|
||||
@@ -0,0 +1,627 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
ach_bridge.py — Transparent TCP bridge / splitter for Instantel MiniMate Plus
|
||||
call-home (ACH) traffic.
|
||||
|
||||
Modes
|
||||
-----
|
||||
standalone Accept connection, capture frames, do NOT forward anywhere.
|
||||
Good for initial discovery with a test unit.
|
||||
|
||||
bridge Forward to one upstream server while capturing.
|
||||
Use this for the initial discovery phase with your test server.
|
||||
|
||||
splitter Forward to the PRIMARY upstream (production ACH server) AND
|
||||
mirror a copy to a SECONDARY server simultaneously.
|
||||
The device never knows — it talks to the primary the whole time.
|
||||
If the mirror fails, the primary connection is unaffected.
|
||||
|
||||
Think of it like a headphone splitter: one input, two outputs.
|
||||
Primary → authoritative responses back to device.
|
||||
Mirror → gets all device bytes, its responses are discarded.
|
||||
|
||||
Usage
|
||||
-----
|
||||
# Standalone capture (test/discovery — no forwarding)
|
||||
python bridges/ach_bridge.py --standalone [--port 12345]
|
||||
|
||||
# Bridge mode (forward to one server, e.g. your test server)
|
||||
python bridges/ach_bridge.py --upstream HOST:PORT [--port 12345]
|
||||
|
||||
# Splitter mode (production: forward to prod + mirror to your server)
|
||||
python bridges/ach_bridge.py --upstream PROD_HOST:PORT --mirror MY_HOST:PORT [--port 12345]
|
||||
|
||||
Setup for discovery (test server, don't touch prod)
|
||||
----------------------------------------------------
|
||||
1. Stand up your test ACH server, note its IP and port (e.g. 192.168.1.50:12345).
|
||||
2. Take ONE test unit. In ACEmanager → Call Home, point it at:
|
||||
<this machine's LAN IP> : <--port>
|
||||
3. Run: python bridges/ach_bridge.py --upstream TEST_SERVER:12345 --port 12345
|
||||
4. Trigger the unit. Raw frames are saved to bridges/captures/ach_<ts>/.
|
||||
5. Revert the unit's ACEmanager setting when done.
|
||||
|
||||
Setup for production splitter (when you're ready)
|
||||
-------------------------------------------------
|
||||
This does NOT touch the units. Instead you re-route traffic at the network
|
||||
layer so that call-home packets arrive at a machine running this script first.
|
||||
Typical approach: update the DNS entry / host record your prod ACH server is
|
||||
registered under to point at this machine. The units keep their existing
|
||||
ACEmanager settings.
|
||||
|
||||
python bridges/ach_bridge.py \\
|
||||
--upstream PROD_ACH_HOST:12345 \\
|
||||
--mirror MY_NEW_SERVER:12345 \\
|
||||
--port 12345
|
||||
|
||||
Output (each connection gets its own timestamped sub-directory)
|
||||
------
|
||||
bridges/captures/ach_<ts>/
|
||||
raw_client_<ts>.bin — raw bytes from the device (S3 side)
|
||||
raw_server_<ts>.bin — raw bytes from the primary upstream (BW side)
|
||||
raw_mirror_<ts>.bin — raw bytes from the mirror upstream (splitter mode only)
|
||||
session_<ts>.log — human-readable frame parse log
|
||||
session_<ts>.jsonl — JSON-lines frame log
|
||||
|
||||
raw_client / raw_server are byte-for-byte compatible with parse_capture.py.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import datetime
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from minimateplus.framing import S3FrameParser, S3Frame
|
||||
|
||||
log = logging.getLogger("ach_bridge")
|
||||
|
||||
|
||||
# ── Frame label helpers ──────────────────────────────────────────────────────
|
||||
|
||||
_KNOWN_RSP_SUBS = {
|
||||
0xA4: "POLL_RSP",
|
||||
0xA5: "BULK_WAVEFORM_RSP",
|
||||
0xE0: "ADVANCE_EVENT_RSP",
|
||||
0xE1: "EVENT_INDEX_FIRST_RSP",
|
||||
0xE3: "MONITOR_STATUS_RSP",
|
||||
0xEA: "SERIAL_NUM_RSP",
|
||||
0xF3: "WAVEFORM_RECORD_RSP",
|
||||
0xF5: "WAVEFORM_HEADER_RSP",
|
||||
0xF7: "EVENT_INDEX_RSP",
|
||||
0xF9: "UNK_06_RSP",
|
||||
0xFE: "DEVICE_INFO_RSP",
|
||||
# Write acks
|
||||
0x97: "EVT_IDX_WRITE_ACK",
|
||||
0x8C: "CONFIRM_B_ACK",
|
||||
0x8E: "COMPLIANCE_WRITE_ACK",
|
||||
0x8D: "CONFIRM_A_ACK",
|
||||
0x7D: "TRIGGER_WRITE_ACK",
|
||||
0x7C: "TRIGGER_CONFIRM_ACK",
|
||||
0x96: "WAVEFORM_WRITE_ACK",
|
||||
0x8B: "CONFIRM_C_ACK",
|
||||
0x69: "START_MONITOR_ACK",
|
||||
0x68: "STOP_MONITOR_ACK",
|
||||
}
|
||||
|
||||
_KNOWN_REQ_SUBS = {
|
||||
0x5B: "POLL",
|
||||
0x5A: "BULK_WAVEFORM",
|
||||
0x1F: "ADVANCE_EVENT",
|
||||
0x1E: "EVENT_INDEX_FIRST",
|
||||
0x1C: "MONITOR_STATUS",
|
||||
0x15: "SERIAL_NUM",
|
||||
0x0C: "WAVEFORM_RECORD",
|
||||
0x0A: "WAVEFORM_HEADER",
|
||||
0x08: "EVENT_INDEX",
|
||||
0x06: "UNK_06",
|
||||
0x01: "DEVICE_INFO",
|
||||
# Write commands
|
||||
0x68: "EVT_IDX_WRITE",
|
||||
0x73: "CONFIRM_B",
|
||||
0x71: "COMPLIANCE_WRITE",
|
||||
0x72: "CONFIRM_A",
|
||||
0x82: "TRIGGER_WRITE",
|
||||
0x83: "TRIGGER_CONFIRM",
|
||||
0x69: "WAVEFORM_WRITE",
|
||||
0x74: "CONFIRM_C",
|
||||
0x96: "START_MONITOR",
|
||||
0x97: "STOP_MONITOR",
|
||||
}
|
||||
|
||||
|
||||
def _label_s3_frame(frame: S3Frame) -> str:
|
||||
name = _KNOWN_RSP_SUBS.get(frame.sub, f"UNK_0x{frame.sub:02X}")
|
||||
chk = "✓" if frame.checksum_valid else "✗CHK"
|
||||
return (
|
||||
f"S3→ SUB=0x{frame.sub:02X} ({name}) "
|
||||
f"page=0x{frame.page_key:04X} data={len(frame.data)}B {chk}"
|
||||
)
|
||||
|
||||
|
||||
def _label_bw_frame(data: bytes, prefix: str = " →BW") -> str:
|
||||
"""Best-effort label for a raw BW request frame (wire bytes)."""
|
||||
# Wire layout: 41 02 10 10 00 sub ...
|
||||
if len(data) < 6:
|
||||
return f"{prefix} (short {len(data)}B)"
|
||||
sub = data[5]
|
||||
name = _KNOWN_REQ_SUBS.get(sub, f"UNK_0x{sub:02X}")
|
||||
return f"{prefix} SUB=0x{sub:02X} ({name}) {len(data)}B"
|
||||
|
||||
|
||||
# ── Per-session capture writer ─────────────────────────────────────────────────
|
||||
|
||||
class CaptureSession:
|
||||
"""Writes raw bytes + parsed log for one TCP connection."""
|
||||
|
||||
def __init__(self, capture_dir: Path, peer: str, *, has_mirror: bool = False):
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
self.dir = capture_dir / f"ach_{ts}"
|
||||
self.dir.mkdir(parents=True, exist_ok=True)
|
||||
self.peer = peer
|
||||
|
||||
self._raw_client = open(self.dir / f"raw_client_{ts}.bin", "wb")
|
||||
self._raw_server = open(self.dir / f"raw_server_{ts}.bin", "wb")
|
||||
self._raw_mirror = (
|
||||
open(self.dir / f"raw_mirror_{ts}.bin", "wb") if has_mirror else None
|
||||
)
|
||||
self._log_fh = open(self.dir / f"session_{ts}.log", "w")
|
||||
self._jsonl_fh = open(self.dir / f"session_{ts}.jsonl", "w")
|
||||
|
||||
self._s3_parser = S3FrameParser()
|
||||
self._frame_count = 0
|
||||
self._byte_count_client = 0
|
||||
self._byte_count_server = 0
|
||||
self._byte_count_mirror = 0
|
||||
|
||||
self._log(
|
||||
f"# ACH capture — peer={peer} "
|
||||
f"mirror={'yes' if has_mirror else 'no'} "
|
||||
f"started={datetime.datetime.now().isoformat()}"
|
||||
)
|
||||
self._log(f"# Output dir: {self.dir}")
|
||||
log.info("Capture session opened: %s (peer=%s)", self.dir, peer)
|
||||
|
||||
# ── public API ────────────────────────────────────────────────────────────
|
||||
|
||||
def feed_client(self, data: bytes) -> None:
|
||||
"""Bytes FROM the device (S3 response frames)."""
|
||||
self._raw_client.write(data)
|
||||
self._raw_client.flush()
|
||||
self._byte_count_client += len(data)
|
||||
|
||||
for byte in data:
|
||||
frame = self._s3_parser.feed(bytes([byte]))
|
||||
if frame:
|
||||
frames = frame if isinstance(frame, list) else [frame]
|
||||
for f in frames:
|
||||
self._frame_count += 1
|
||||
label = _label_s3_frame(f)
|
||||
self._log(f"[{self._frame_count:04d}] {label}")
|
||||
self._log(
|
||||
f" hex: {f.data[:64].hex()}"
|
||||
+ (" ..." if len(f.data) > 64 else "")
|
||||
)
|
||||
self._emit_json("s3", f)
|
||||
|
||||
def feed_server(self, data: bytes) -> None:
|
||||
"""Bytes FROM the primary upstream server (BW request frames)."""
|
||||
self._raw_server.write(data)
|
||||
self._raw_server.flush()
|
||||
self._byte_count_server += len(data)
|
||||
label = _label_bw_frame(data, prefix=" →BW[primary]")
|
||||
self._log(f" {label}")
|
||||
|
||||
def feed_mirror(self, data: bytes) -> None:
|
||||
"""Bytes FROM the mirror server (logged, not forwarded to device)."""
|
||||
if self._raw_mirror:
|
||||
self._raw_mirror.write(data)
|
||||
self._raw_mirror.flush()
|
||||
self._byte_count_mirror += len(data)
|
||||
label = _label_bw_frame(data, prefix=" →BW[mirror] ")
|
||||
self._log(f" {label} [MIRROR — not sent to device]")
|
||||
|
||||
def close(self, reason: str = "connection closed") -> None:
|
||||
self._log(f"# Session ended: {reason}")
|
||||
self._log(
|
||||
f"# Totals — client={self._byte_count_client}B "
|
||||
f"server={self._byte_count_server}B "
|
||||
f"mirror={self._byte_count_mirror}B "
|
||||
f"s3_frames={self._frame_count}"
|
||||
)
|
||||
handles = [self._raw_client, self._raw_server, self._log_fh, self._jsonl_fh]
|
||||
if self._raw_mirror:
|
||||
handles.append(self._raw_mirror)
|
||||
for fh in handles:
|
||||
try:
|
||||
fh.close()
|
||||
except Exception:
|
||||
pass
|
||||
log.info(
|
||||
"Session closed (%s): %dB client, %dB server, %dB mirror, %d S3 frames → %s",
|
||||
reason,
|
||||
self._byte_count_client, self._byte_count_server,
|
||||
self._byte_count_mirror, self._frame_count,
|
||||
self.dir,
|
||||
)
|
||||
|
||||
# ── internals ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _log(self, msg: str) -> None:
|
||||
print(msg, file=self._log_fh, flush=True)
|
||||
print(msg)
|
||||
|
||||
def _emit_json(self, direction: str, frame: S3Frame) -> None:
|
||||
record = {
|
||||
"dir": direction,
|
||||
"sub": frame.sub,
|
||||
"page_key": frame.page_key,
|
||||
"data_len": len(frame.data),
|
||||
"data_hex": frame.data.hex(),
|
||||
"checksum_valid": frame.checksum_valid,
|
||||
}
|
||||
print(json.dumps(record), file=self._jsonl_fh, flush=True)
|
||||
|
||||
|
||||
# ── Bridge / splitter connection handler ──────────────────────────────────────
|
||||
|
||||
class BridgeHandler:
|
||||
"""
|
||||
Handles inbound device connections.
|
||||
|
||||
Modes (determined by which upstreams are configured):
|
||||
standalone — no upstream_host / no mirror_host
|
||||
bridge — upstream_host set, no mirror_host
|
||||
splitter — upstream_host AND mirror_host both set
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
capture_dir: Path,
|
||||
upstream_host: Optional[str],
|
||||
upstream_port: Optional[int],
|
||||
mirror_host: Optional[str] = None,
|
||||
mirror_port: Optional[int] = None,
|
||||
):
|
||||
self.capture_dir = capture_dir
|
||||
self.upstream_host = upstream_host
|
||||
self.upstream_port = upstream_port
|
||||
self.mirror_host = mirror_host
|
||||
self.mirror_port = mirror_port
|
||||
|
||||
async def handle(
|
||||
self,
|
||||
client_reader: asyncio.StreamReader,
|
||||
client_writer: asyncio.StreamWriter,
|
||||
) -> None:
|
||||
peer = client_writer.get_extra_info("peername", ("?", 0))
|
||||
peer_str = f"{peer[0]}:{peer[1]}"
|
||||
log.info("Inbound connection from %s", peer_str)
|
||||
|
||||
has_mirror = bool(self.mirror_host)
|
||||
session = CaptureSession(self.capture_dir, peer_str, has_mirror=has_mirror)
|
||||
|
||||
if not self.upstream_host:
|
||||
# ── Standalone mode ──────────────────────────────────────────────
|
||||
log.info("Standalone mode — recording inbound traffic only")
|
||||
try:
|
||||
while True:
|
||||
data = await client_reader.read(4096)
|
||||
if not data:
|
||||
break
|
||||
session.feed_client(data)
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
except Exception as exc:
|
||||
log.warning("Standalone read error: %s", exc)
|
||||
finally:
|
||||
session.close("standalone capture ended")
|
||||
try:
|
||||
client_writer.close()
|
||||
await client_writer.wait_closed()
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
|
||||
# ── Bridge / splitter mode ───────────────────────────────────────────
|
||||
# Connect to primary upstream (required)
|
||||
try:
|
||||
up_reader, up_writer = await asyncio.open_connection(
|
||||
self.upstream_host, self.upstream_port
|
||||
)
|
||||
log.info("Connected to primary %s:%s", self.upstream_host, self.upstream_port)
|
||||
except Exception as exc:
|
||||
log.error("Failed to connect to primary upstream: %s", exc)
|
||||
session.close(f"primary connect failed: {exc}")
|
||||
client_writer.close()
|
||||
return
|
||||
|
||||
# Connect to mirror upstream (optional — failure is non-fatal)
|
||||
mir_reader: Optional[asyncio.StreamReader] = None
|
||||
mir_writer: Optional[asyncio.StreamWriter] = None
|
||||
if self.mirror_host:
|
||||
try:
|
||||
mir_reader, mir_writer = await asyncio.open_connection(
|
||||
self.mirror_host, self.mirror_port
|
||||
)
|
||||
log.info("Connected to mirror %s:%s", self.mirror_host, self.mirror_port)
|
||||
except Exception as exc:
|
||||
log.warning(
|
||||
"Mirror connect failed — continuing without mirror: %s", exc
|
||||
)
|
||||
session._log(f"# WARNING: mirror connect failed: {exc}")
|
||||
|
||||
# Build relay tasks
|
||||
#
|
||||
# ┌──────────┐ device bytes ┌─────────────┐
|
||||
# │ Device │ ─────────────► │ PRIMARY │ responses ──► device
|
||||
# └──────────┘ └─────────────┘
|
||||
# │
|
||||
# │ device bytes (copy)
|
||||
# ▼
|
||||
# ┌─────────────┐
|
||||
# │ MIRROR │ responses discarded (logged only)
|
||||
# └─────────────┘
|
||||
#
|
||||
tasks = [
|
||||
asyncio.create_task(
|
||||
self._relay_device(client_reader, up_writer, mir_writer, session),
|
||||
name="device→upstreams",
|
||||
),
|
||||
asyncio.create_task(
|
||||
self._relay_simple(up_reader, client_writer, session, "server"),
|
||||
name="primary→device",
|
||||
),
|
||||
]
|
||||
if mir_reader is not None:
|
||||
tasks.append(asyncio.create_task(
|
||||
self._relay_drain(mir_reader, session),
|
||||
name="mirror→drain",
|
||||
))
|
||||
|
||||
try:
|
||||
# Wait for the device-to-upstreams relay to exit first (device
|
||||
# disconnected or primary dropped). Then cancel the rest.
|
||||
done, pending = await asyncio.wait(
|
||||
tasks,
|
||||
return_when=asyncio.FIRST_COMPLETED,
|
||||
)
|
||||
for t in pending:
|
||||
t.cancel()
|
||||
try:
|
||||
await t
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
except Exception as exc:
|
||||
log.warning("Bridge relay error: %s", exc)
|
||||
finally:
|
||||
session.close("relay ended")
|
||||
for writer in filter(None, [client_writer, up_writer, mir_writer]):
|
||||
try:
|
||||
writer.close()
|
||||
await writer.wait_closed()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ── Relay helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
async def _relay_device(
|
||||
self,
|
||||
reader: asyncio.StreamReader,
|
||||
primary_writer: asyncio.StreamWriter,
|
||||
mirror_writer: Optional[asyncio.StreamWriter],
|
||||
session: CaptureSession,
|
||||
) -> None:
|
||||
"""
|
||||
Read bytes from the device, write to the primary server, and also
|
||||
write a copy to the mirror server (if connected). Mirror write
|
||||
failures are non-fatal — we log and continue.
|
||||
"""
|
||||
try:
|
||||
while True:
|
||||
data = await reader.read(4096)
|
||||
if not data:
|
||||
break
|
||||
session.feed_client(data)
|
||||
|
||||
# Primary write — failure IS fatal (lose primary = lose prod)
|
||||
primary_writer.write(data)
|
||||
await primary_writer.drain()
|
||||
|
||||
# Mirror write — failure is non-fatal
|
||||
if mirror_writer is not None:
|
||||
try:
|
||||
mirror_writer.write(data)
|
||||
await mirror_writer.drain()
|
||||
except Exception as exc:
|
||||
log.warning("Mirror write failed (non-fatal): %s", exc)
|
||||
session._log(f"# WARNING: mirror write failed: {exc}")
|
||||
mirror_writer = None # stop trying
|
||||
|
||||
except (asyncio.IncompleteReadError, ConnectionResetError, BrokenPipeError):
|
||||
pass
|
||||
|
||||
async def _relay_simple(
|
||||
self,
|
||||
reader: asyncio.StreamReader,
|
||||
writer: asyncio.StreamWriter,
|
||||
session: CaptureSession,
|
||||
direction: str,
|
||||
) -> None:
|
||||
"""Standard single-pipe relay (primary→device or vice-versa)."""
|
||||
try:
|
||||
while True:
|
||||
data = await reader.read(4096)
|
||||
if not data:
|
||||
break
|
||||
if direction == "server":
|
||||
session.feed_server(data)
|
||||
else:
|
||||
session.feed_client(data)
|
||||
writer.write(data)
|
||||
await writer.drain()
|
||||
except (asyncio.IncompleteReadError, ConnectionResetError, BrokenPipeError):
|
||||
pass
|
||||
|
||||
async def _relay_drain(
|
||||
self,
|
||||
reader: asyncio.StreamReader,
|
||||
session: CaptureSession,
|
||||
) -> None:
|
||||
"""
|
||||
Read mirror server responses, log them to session, do NOT forward to
|
||||
device. The device only ever sees primary server responses.
|
||||
"""
|
||||
try:
|
||||
while True:
|
||||
data = await reader.read(4096)
|
||||
if not data:
|
||||
break
|
||||
session.feed_mirror(data)
|
||||
except (asyncio.IncompleteReadError, ConnectionResetError, BrokenPipeError):
|
||||
pass
|
||||
|
||||
|
||||
# ── Main ───────────────────────────────────────────────────────────────────────
|
||||
|
||||
async def main(args: argparse.Namespace) -> None:
|
||||
capture_dir = Path(__file__).parent / "captures"
|
||||
capture_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
upstream_host: Optional[str] = None
|
||||
upstream_port: Optional[int] = None
|
||||
mirror_host: Optional[str] = None
|
||||
mirror_port: Optional[int] = None
|
||||
|
||||
if not args.standalone:
|
||||
if not args.upstream:
|
||||
print("ERROR: --upstream HOST:PORT is required unless --standalone is set.")
|
||||
sys.exit(1)
|
||||
parts = args.upstream.rsplit(":", 1)
|
||||
if len(parts) != 2:
|
||||
print("ERROR: --upstream must be HOST:PORT (e.g. 203.0.113.5:12345)")
|
||||
sys.exit(1)
|
||||
upstream_host = parts[0]
|
||||
upstream_port = int(parts[1])
|
||||
|
||||
if args.mirror:
|
||||
parts = args.mirror.rsplit(":", 1)
|
||||
if len(parts) != 2:
|
||||
print("ERROR: --mirror must be HOST:PORT (e.g. 192.168.1.50:12345)")
|
||||
sys.exit(1)
|
||||
mirror_host = parts[0]
|
||||
mirror_port = int(parts[1])
|
||||
|
||||
handler = BridgeHandler(
|
||||
capture_dir,
|
||||
upstream_host, upstream_port,
|
||||
mirror_host, mirror_port,
|
||||
)
|
||||
|
||||
server = await asyncio.start_server(
|
||||
handler.handle,
|
||||
host="0.0.0.0",
|
||||
port=args.port,
|
||||
)
|
||||
|
||||
# ── Startup banner ────────────────────────────────────────────────────────
|
||||
if args.standalone:
|
||||
mode = "STANDALONE capture (no forwarding)"
|
||||
elif mirror_host:
|
||||
mode = f"SPLITTER primary={upstream_host}:{upstream_port} mirror={mirror_host}:{mirror_port}"
|
||||
else:
|
||||
mode = f"BRIDGE → {upstream_host}:{upstream_port}"
|
||||
|
||||
addrs = ", ".join(str(s.getsockname()) for s in server.sockets)
|
||||
print(f"\n{'='*70}")
|
||||
print(f" ACH bridge/splitter listening on {addrs}")
|
||||
print(f" Mode: {mode}")
|
||||
print(f" Captures: {capture_dir}/ach_<timestamp>/")
|
||||
print(f"{'='*70}")
|
||||
|
||||
if upstream_host and not mirror_host:
|
||||
print(f"\n DISCOVERY PHASE")
|
||||
print(f" Point your TEST unit's ACEmanager call-home destination to:")
|
||||
print(f" <this machine's LAN IP> : {args.port}")
|
||||
print(f" All traffic will be forwarded to {upstream_host}:{upstream_port}")
|
||||
elif mirror_host:
|
||||
print(f"\n SPLITTER MODE — PRODUCTION SAFE")
|
||||
print(f" Units connect as normal. Every byte is forwarded to:")
|
||||
print(f" PRIMARY (authoritative): {upstream_host}:{upstream_port}")
|
||||
print(f" MIRROR (your server): {mirror_host}:{mirror_port}")
|
||||
print(f" Only PRIMARY responses reach the device.")
|
||||
print(f" Mirror failures are logged and do not affect the device.")
|
||||
else:
|
||||
print(f"\n STANDALONE MODE — capture only, nothing forwarded")
|
||||
print(f" Point a unit at <this machine's LAN IP> : {args.port}")
|
||||
|
||||
print(f"\n Waiting for inbound connections... (Ctrl-C to stop)\n")
|
||||
|
||||
async with server:
|
||||
await server.serve_forever()
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
p = argparse.ArgumentParser(
|
||||
description=(
|
||||
"Transparent TCP bridge / splitter for Instantel MiniMate Plus "
|
||||
"call-home (ACH) traffic."
|
||||
),
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__,
|
||||
)
|
||||
p.add_argument(
|
||||
"--upstream", "-u",
|
||||
metavar="HOST:PORT",
|
||||
help=(
|
||||
"Primary upstream ACH server to forward to "
|
||||
"(e.g. 203.0.113.5:12345). "
|
||||
"Omit with --standalone for capture-only mode."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--mirror", "-m",
|
||||
metavar="HOST:PORT",
|
||||
help=(
|
||||
"Mirror / secondary server to receive a copy of all device bytes "
|
||||
"(splitter mode). Mirror responses are logged but NOT forwarded "
|
||||
"to the device. Mirror failures are non-fatal."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--port", "-p",
|
||||
type=int,
|
||||
default=12345,
|
||||
help="Local port to listen on (default: 12345).",
|
||||
)
|
||||
p.add_argument(
|
||||
"--standalone", "-s",
|
||||
action="store_true",
|
||||
help="Capture-only mode: accept connection, do not forward anywhere.",
|
||||
)
|
||||
p.add_argument(
|
||||
"--verbose", "-v",
|
||||
action="store_true",
|
||||
help="Enable debug logging.",
|
||||
)
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parse_args()
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG if args.verbose else logging.INFO,
|
||||
format="%(asctime)s %(levelname)-7s %(name)s %(message)s",
|
||||
)
|
||||
try:
|
||||
asyncio.run(main(args))
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopped.")
|
||||
@@ -0,0 +1,177 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
ach_mitm.py — TCP man-in-the-middle proxy for capturing Blastware ACH sessions.
|
||||
|
||||
The unit calls home to THIS proxy instead of directly to Blastware. The proxy
|
||||
forwards every byte in both directions to the real Blastware ACH server and saves
|
||||
the traffic to separate raw capture files that the Analyzer can load directly.
|
||||
|
||||
Setup
|
||||
-----
|
||||
1. Start Blastware's ACH server on the BW PC as normal (it listens on its port).
|
||||
2. Run this proxy on any machine the unit can reach:
|
||||
|
||||
python bridges/ach_mitm.py --bw-host 192.168.1.50 --bw-port 9999
|
||||
|
||||
3. Point the unit's ACEmanager call-home destination to THIS machine's IP and
|
||||
the --listen-port (default 9999).
|
||||
4. Trigger a call-home (or wait for the unit to call in).
|
||||
5. The proxy transparently forwards everything and saves two files per session:
|
||||
|
||||
ach_mitm_<ts>/raw_bw_<ts>.bin -- bytes Blastware sent to unit (BW TX)
|
||||
ach_mitm_<ts>/raw_s3_<ts>.bin -- bytes unit sent to Blastware (S3 TX)
|
||||
|
||||
Both files load directly in the Analyzer (File > Open Capture).
|
||||
|
||||
The proxy exits cleanly when either side drops the connection.
|
||||
|
||||
Use case: capturing Blastware operations we haven't reverse-engineered yet,
|
||||
e.g. event deletion, factory reset, firmware update.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import logging
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger("ach_mitm")
|
||||
|
||||
|
||||
def _pipe(src: socket.socket, dst: socket.socket, label: str, outfile) -> None:
|
||||
"""Forward bytes from src to dst, writing everything to outfile."""
|
||||
try:
|
||||
while True:
|
||||
data = src.recv(4096)
|
||||
if not data:
|
||||
break
|
||||
dst.sendall(data)
|
||||
outfile.write(data)
|
||||
outfile.flush()
|
||||
log.debug("%s %d bytes", label, len(data))
|
||||
except OSError:
|
||||
pass
|
||||
finally:
|
||||
log.info("%s pipe closed", label)
|
||||
# Signal the other direction to stop by shutting down our end.
|
||||
try:
|
||||
dst.shutdown(socket.SHUT_WR)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def handle(unit_sock: socket.socket, peer: str, bw_host: str, bw_port: int,
|
||||
output_dir: Path) -> None:
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
session_dir = output_dir / f"ach_mitm_{ts}"
|
||||
session_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
log.info("Session %s unit=%s forwarding to %s:%d", ts, peer, bw_host, bw_port)
|
||||
|
||||
# Connect upstream to Blastware.
|
||||
bw_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
try:
|
||||
bw_sock.connect((bw_host, bw_port))
|
||||
except OSError as exc:
|
||||
log.error("Cannot reach Blastware at %s:%d: %s", bw_host, bw_port, exc)
|
||||
unit_sock.close()
|
||||
return
|
||||
|
||||
log.info("Connected to Blastware at %s:%d", bw_host, bw_port)
|
||||
|
||||
bw_path = session_dir / f"raw_bw_{ts}.bin" # Blastware → unit (BW TX)
|
||||
s3_path = session_dir / f"raw_s3_{ts}.bin" # unit → Blastware (S3 TX)
|
||||
|
||||
with open(bw_path, "wb") as bw_fh, open(s3_path, "wb") as s3_fh:
|
||||
# Two threads: one per direction.
|
||||
t_bw = threading.Thread(
|
||||
target=_pipe, args=(bw_sock, unit_sock, "BW->unit", bw_fh), daemon=True
|
||||
)
|
||||
t_s3 = threading.Thread(
|
||||
target=_pipe, args=(unit_sock, bw_sock, "unit->BW", s3_fh), daemon=True
|
||||
)
|
||||
t_bw.start()
|
||||
t_s3.start()
|
||||
t_bw.join()
|
||||
t_s3.join()
|
||||
|
||||
bw_bytes = bw_path.stat().st_size
|
||||
s3_bytes = s3_path.stat().st_size
|
||||
log.info(
|
||||
"Session %s done BW->unit: %d bytes unit->BW: %d bytes -> %s",
|
||||
ts, bw_bytes, s3_bytes, session_dir,
|
||||
)
|
||||
|
||||
unit_sock.close()
|
||||
bw_sock.close()
|
||||
|
||||
|
||||
def serve(args: argparse.Namespace) -> None:
|
||||
output_dir = Path(args.output)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
server.bind(("0.0.0.0", args.listen_port))
|
||||
server.listen(5)
|
||||
server.settimeout(1.0)
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f" ACH MITM proxy")
|
||||
print(f" Listening on 0.0.0.0:{args.listen_port}")
|
||||
print(f" Forwarding to {args.bw_host}:{args.bw_port}")
|
||||
print(f" Captures in {output_dir.resolve()}/ach_mitm_<ts>/")
|
||||
print(f"{'='*60}")
|
||||
print(f"\n Point the unit's ACEmanager call-home to this machine on port {args.listen_port}")
|
||||
print(f" Ctrl-C to stop\n")
|
||||
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
client_sock, addr = server.accept()
|
||||
except socket.timeout:
|
||||
continue
|
||||
peer = f"{addr[0]}:{addr[1]}"
|
||||
log.info("Accepted connection from %s", peer)
|
||||
t = threading.Thread(
|
||||
target=handle,
|
||||
args=(client_sock, peer, args.bw_host, args.bw_port, output_dir),
|
||||
daemon=True,
|
||||
)
|
||||
t.start()
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopping.")
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--bw-host", required=True,
|
||||
help="IP or hostname of the Blastware ACH server")
|
||||
ap.add_argument("--bw-port", type=int, default=9999,
|
||||
help="Port Blastware is listening on (default: 9999)")
|
||||
ap.add_argument("--listen-port", type=int, default=9999,
|
||||
help="Port this proxy listens on (default: 9999)")
|
||||
ap.add_argument("--output", default="bridges/captures/mitm",
|
||||
help="Directory for capture files")
|
||||
ap.add_argument("--log-level", default="INFO",
|
||||
choices=["DEBUG", "INFO", "WARNING", "ERROR"])
|
||||
args = ap.parse_args()
|
||||
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, args.log_level),
|
||||
format="%(asctime)s %(levelname)-7s %(name)s %(message)s",
|
||||
stream=sys.stdout,
|
||||
)
|
||||
|
||||
serve(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,777 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
ach_server.py — Minimal inbound ACH (Auto Call Home) server for MiniMate Plus.
|
||||
|
||||
This IS your test server. Run it on any machine on the same network, point a
|
||||
unit's ACEmanager call-home destination at it, and it will speak the full BW
|
||||
protocol to the device: handshake, pull device info, download all events, save
|
||||
everything as JSON.
|
||||
|
||||
The key thing this script tells you that no amount of packet sniffing can:
|
||||
- Does the device speak first (push) or wait for us to send POLL (pull)?
|
||||
|
||||
If startup() completes normally → it's pull protocol, same as Blastware.
|
||||
If startup() times out → the device sent something first; check raw_rx.bin.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python bridges/ach_server.py [--port 12345] [--output bridges/captures/]
|
||||
|
||||
Setup
|
||||
-----
|
||||
1. Run this script on a machine on your local network.
|
||||
2. In ACEmanager → Application → ALEOS Application Framework (or equivalent)
|
||||
find the Call Home / ACH settings. Set:
|
||||
Remote Host: <this machine's LAN IP>
|
||||
Remote Port: 12345
|
||||
3. Trigger the unit (wait for a vibration event, or use the manual call-home
|
||||
button if your firmware version has one).
|
||||
4. The unit connects. This script handshakes, downloads all events,
|
||||
and saves a timestamped session directory.
|
||||
|
||||
Output per session
|
||||
------------------
|
||||
bridges/captures/ach_inbound_<ts>/
|
||||
device_info.json — serial number, firmware version, calibration date, etc.
|
||||
events.json — all events: timestamp, PPV per channel, peaks, metadata
|
||||
raw_rx_<ts>.bin — raw bytes from the device (S3 side) for Analyzer
|
||||
session_<ts>.log — detailed protocol log
|
||||
|
||||
What to look for
|
||||
----------------
|
||||
Push vs pull: Check session_<ts>.log. If the first line after "Connected"
|
||||
shows bytes arriving BEFORE the POLL probe was sent, it's push. If POLL
|
||||
gets a clean response, it's pull.
|
||||
|
||||
Frequency: Look at raw_rx.bin in the Analyzer. SUB 5A (0xA5 responses) carry
|
||||
bulk waveform data — if frequency is sent pre-computed there will be float32
|
||||
values before the ADC sample blocks.
|
||||
|
||||
ACH-specific framing: Does the unit send anything extra before the DLE+STX
|
||||
framing starts? raw_rx.bin will show raw bytes including any preamble.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import logging
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from minimateplus.transport import SocketTransport
|
||||
from minimateplus.client import MiniMateClient
|
||||
from minimateplus.models import DeviceInfo, Event, MonitorLogEntry
|
||||
from sfm.database import SeismoDb
|
||||
|
||||
log = logging.getLogger("ach_server")
|
||||
|
||||
# ── Per-unit state (downloaded-key set) ───────────────────────────────────────
|
||||
# Persisted as <output_dir>/ach_state.json
|
||||
# Format:
|
||||
# {
|
||||
# "BE11529": {
|
||||
# "downloaded_keys": ["01110000", "0111245a"], # hex keys already on disk
|
||||
# "max_downloaded_key": "0111245a", # highest key ever seen
|
||||
# "last_seen": "2026-04-11T01:04:36"
|
||||
# }
|
||||
# }
|
||||
#
|
||||
# Key-based deduplication works well within a single "key generation" (between
|
||||
# erases). After the device memory is erased the event counter resets to
|
||||
# 0x01110000, so the first new event has the SAME key as the very first event
|
||||
# we ever downloaded. We detect this situation with max_downloaded_key:
|
||||
#
|
||||
# if max(current_device_keys) < max_downloaded_key
|
||||
# → device was wiped and keys have restarted → treat all device keys as new
|
||||
#
|
||||
# After our own erase (--clear-after-download) we also explicitly clear
|
||||
# downloaded_keys and max_downloaded_key so the next session starts fresh.
|
||||
|
||||
_state_lock = threading.Lock()
|
||||
|
||||
|
||||
def _load_state(state_path: Path) -> dict:
|
||||
if state_path.exists():
|
||||
try:
|
||||
with open(state_path) as f:
|
||||
return json.load(f)
|
||||
except Exception:
|
||||
pass
|
||||
return {}
|
||||
|
||||
|
||||
def _save_state(state_path: Path, state: dict) -> None:
|
||||
with _state_lock:
|
||||
with open(state_path, "w") as f:
|
||||
json.dump(state, f, indent=2)
|
||||
|
||||
|
||||
# ── Per-session handler ────────────────────────────────────────────────────────
|
||||
|
||||
class AchSession:
|
||||
"""
|
||||
Handles one inbound unit connection in its own thread.
|
||||
Wraps the socket in a SocketTransport → MiniMateClient, then runs the
|
||||
standard connect → get_device_info → get_events sequence.
|
||||
|
||||
State tracking (ach_state.json in output_dir):
|
||||
On each successful download we record the SET of event keys downloaded.
|
||||
On the next call-home we compare: if all device keys are already in the
|
||||
set, there's nothing new. If any key is new (including after the device
|
||||
was wiped and re-recorded), we download and save only those events.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
sock: socket.socket,
|
||||
peer: str,
|
||||
output_dir: Path,
|
||||
timeout: float,
|
||||
events_only: bool,
|
||||
max_events: Optional[int],
|
||||
state_path: Path,
|
||||
db: "SeismoDb",
|
||||
clear_after_download: bool = False,
|
||||
restart_monitoring: bool = False,
|
||||
) -> None:
|
||||
self.sock = sock
|
||||
self.peer = peer
|
||||
self.output_dir = output_dir
|
||||
self.timeout = timeout
|
||||
self.events_only = events_only
|
||||
self.max_events = max_events
|
||||
self.state_path = state_path
|
||||
self.db = db
|
||||
self.clear_after_download = clear_after_download
|
||||
self.restart_monitoring = restart_monitoring
|
||||
|
||||
def run(self) -> None:
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
|
||||
# Session dir and file handler are created lazily — only after startup
|
||||
# succeeds. This prevents internet scanners and dropped connections from
|
||||
# littering the output directory with empty session folders.
|
||||
try:
|
||||
self._run_inner(ts)
|
||||
except Exception as exc:
|
||||
log.error("Session failed (%s): %s", self.peer, exc, exc_info=True)
|
||||
finally:
|
||||
try:
|
||||
self.sock.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _run_inner(self, ts: str) -> None:
|
||||
transport = SocketTransport(self.sock, peer=self.peer)
|
||||
|
||||
# Collect raw bytes in memory until startup succeeds, then flush to disk.
|
||||
raw_buf: list[bytes] = []
|
||||
_orig_read = transport.read
|
||||
|
||||
def tapped_read(n: int) -> bytes:
|
||||
data = _orig_read(n)
|
||||
if data:
|
||||
raw_buf.append(data)
|
||||
return data
|
||||
|
||||
transport.read = tapped_read # type: ignore[method-assign]
|
||||
|
||||
serial: Optional[str] = None
|
||||
|
||||
# ── Step 1: startup handshake ─────────────────────────────────────────
|
||||
# Do this BEFORE creating the session directory so that scanner probes
|
||||
# and dropped connections leave no trace on disk.
|
||||
try:
|
||||
from minimateplus.protocol import MiniMateProtocol
|
||||
client = MiniMateClient(transport=transport, timeout=self.timeout)
|
||||
client.open()
|
||||
proto = MiniMateProtocol(transport, recv_timeout=self.timeout)
|
||||
proto.startup()
|
||||
except Exception as exc:
|
||||
log.warning("Startup failed from %s: %s -- ignoring", self.peer, exc)
|
||||
return # no session dir created
|
||||
|
||||
# Startup succeeded — this is a real unit. Create session dir now.
|
||||
session_dir = self.output_dir / f"ach_inbound_{ts}"
|
||||
session_dir.mkdir(parents=True, exist_ok=True)
|
||||
log_path = session_dir / f"session_{ts}.log"
|
||||
raw_path = session_dir / f"raw_rx_{ts}.bin"
|
||||
|
||||
# Flush buffered raw bytes to file and switch to direct file writes.
|
||||
raw_fh = open(raw_path, "wb")
|
||||
for chunk in raw_buf:
|
||||
raw_fh.write(chunk)
|
||||
raw_buf.clear()
|
||||
|
||||
def tapped_read_file(n: int) -> bytes:
|
||||
data = _orig_read(n)
|
||||
if data:
|
||||
raw_fh.write(data)
|
||||
raw_fh.flush()
|
||||
return data
|
||||
|
||||
transport.read = tapped_read_file # type: ignore[method-assign]
|
||||
|
||||
# Wire up file handler now that the session dir exists.
|
||||
fh = logging.FileHandler(log_path, encoding="utf-8")
|
||||
fh.setFormatter(logging.Formatter("%(asctime)s %(levelname)-7s %(name)s %(message)s"))
|
||||
root_logger = logging.getLogger()
|
||||
root_logger.addHandler(fh)
|
||||
|
||||
try:
|
||||
# ── Step 2: device info ───────────────────────────────────────────
|
||||
device_info = None
|
||||
if not self.events_only:
|
||||
log.info("Step 2/3: reading device info")
|
||||
try:
|
||||
device_info = client.connect()
|
||||
serial = device_info.serial
|
||||
_save_json(session_dir / "device_info.json", _device_info_to_dict(device_info))
|
||||
log.info(
|
||||
" [OK] Device: serial=%s firmware=%s model=%s events=%d",
|
||||
serial,
|
||||
device_info.firmware_version,
|
||||
device_info.model,
|
||||
device_info.event_count or 0,
|
||||
)
|
||||
except Exception as exc:
|
||||
log.error(" [FAIL] Device info failed: %s", exc)
|
||||
else:
|
||||
log.info("Step 2/3: skipping device info (--events-only)")
|
||||
|
||||
# ── Step 3: check for new events by comparing key sets ────────────
|
||||
log.info("Step 3/3: checking for new events")
|
||||
|
||||
state = _load_state(self.state_path)
|
||||
unit_key = serial or self.peer # fall back to IP if no serial
|
||||
unit_state = state.get(unit_key, {})
|
||||
seen_keys: set[str] = set(unit_state.get("downloaded_keys", []))
|
||||
# Highest event key ever downloaded from this unit (hex string, 8 chars).
|
||||
# Used to detect post-erase key reuse — see comment block above.
|
||||
max_seen_key: str = unit_state.get("max_downloaded_key", "00000000")
|
||||
|
||||
# Walk the event index (browse-mode, no 5A) to get the actual current
|
||||
# key list. The SUB 08 event_count field is a lifetime "total events
|
||||
# ever recorded" counter that does NOT decrement on erase — confirmed
|
||||
# 2026-04-13. list_event_keys() via the 1E/1F chain is the only
|
||||
# reliable way to know what is actually stored on the device right now.
|
||||
log.info(" Checking device key list (browse walk, no waveform download)...")
|
||||
try:
|
||||
device_keys = client.list_event_keys()
|
||||
except Exception as exc:
|
||||
log.warning(" list_event_keys failed: %s -- falling back to full download", exc)
|
||||
device_keys = None
|
||||
|
||||
# Use the walk result as our authoritative current count.
|
||||
current_count = len(device_keys) if device_keys is not None else 0
|
||||
|
||||
log.info(" Unit has %d stored event(s); %d key(s) previously downloaded",
|
||||
current_count, len(seen_keys))
|
||||
|
||||
if device_keys is not None and current_count == 0:
|
||||
log.info(" [OK] No events on device -- nothing to download")
|
||||
log.info("Session complete (no events) -> %s", session_dir)
|
||||
return
|
||||
|
||||
if device_keys is not None:
|
||||
# ── Post-erase detection ──────────────────────────────────────
|
||||
# After the device memory is erased, new events start from key
|
||||
# 01110000 again — the same keys we already downloaded. Detect
|
||||
# this by comparing the device's current highest key against the
|
||||
# historical maximum. If the device has rolled back below our
|
||||
# high-water mark, its counter was reset and we must treat all
|
||||
# its keys as new, regardless of what seen_keys contains.
|
||||
if device_keys and max_seen_key != "00000000":
|
||||
max_device_key = max(device_keys) # lexicographic; safe because
|
||||
# keys share the same 4-char prefix
|
||||
if max_device_key < max_seen_key:
|
||||
log.info(
|
||||
" Post-erase reset detected: "
|
||||
"device max key %s < historical max %s "
|
||||
"-- treating all device keys as new",
|
||||
max_device_key, max_seen_key,
|
||||
)
|
||||
seen_keys = set() # discard stale dedup info for this session
|
||||
|
||||
new_key_set = set(device_keys) - seen_keys
|
||||
log.info(" Device has %d key(s): %d new, %d already seen",
|
||||
len(device_keys), len(new_key_set), len(device_keys) - len(new_key_set))
|
||||
if not new_key_set:
|
||||
log.info(" [OK] All events already downloaded -- nothing to do")
|
||||
# Refresh state timestamp; preserve max_seen_key unchanged.
|
||||
state[unit_key] = {
|
||||
"downloaded_keys": sorted(seen_keys | set(device_keys)),
|
||||
"max_downloaded_key": max_seen_key,
|
||||
"last_seen": datetime.datetime.now().isoformat(),
|
||||
"serial": serial,
|
||||
"peer": self.peer,
|
||||
}
|
||||
_save_state(self.state_path, state)
|
||||
|
||||
# ── Erase even when no new events (if requested) ──────────
|
||||
# Blastware ACH always erases after every session — even when
|
||||
# nothing new was downloaded. Without the erase the device
|
||||
# still sees stored events in its memory and immediately
|
||||
# retries the call-home, causing the looping we observed.
|
||||
# Only erase when device actually has events stored; skip
|
||||
# the erase if device_keys is empty (nothing to erase).
|
||||
if self.clear_after_download and device_keys:
|
||||
log.info(
|
||||
" Clearing device memory (--clear-after-download, "
|
||||
"no new events but device has %d stored)...",
|
||||
len(device_keys),
|
||||
)
|
||||
try:
|
||||
client.delete_all_events()
|
||||
log.info(" [OK] Device memory cleared")
|
||||
# Reset state so the next session starts fresh.
|
||||
state[unit_key] = {
|
||||
"downloaded_keys": [],
|
||||
"max_downloaded_key": "00000000",
|
||||
"last_seen": datetime.datetime.now().isoformat(),
|
||||
"serial": serial,
|
||||
"peer": self.peer,
|
||||
}
|
||||
_save_state(self.state_path, state)
|
||||
except Exception as exc:
|
||||
log.error(
|
||||
" [WARN] Event deletion failed: %s -- events NOT cleared",
|
||||
exc,
|
||||
)
|
||||
|
||||
log.info("Session complete (no new events) -> %s", session_dir)
|
||||
return
|
||||
else:
|
||||
new_key_set = None # unknown; proceed with full download
|
||||
|
||||
# Apply max_events cap
|
||||
# stop_idx: when we know the count from list_event_keys, use it as
|
||||
# an upper bound. When list_event_keys failed (device_keys is None),
|
||||
# pass None — get_events will run until the null sentinel naturally.
|
||||
stop_idx: Optional[int] = (current_count - 1) if device_keys is not None else None
|
||||
if self.max_events is not None:
|
||||
cap = self.max_events - 1
|
||||
stop_idx = cap if stop_idx is None else min(stop_idx, cap)
|
||||
if device_keys is not None and self.max_events < current_count:
|
||||
log.warning(
|
||||
" max_events=%d cap: will download events 0-%d only "
|
||||
"(unit has %d total)",
|
||||
self.max_events, stop_idx, current_count,
|
||||
)
|
||||
|
||||
try:
|
||||
all_events = client.get_events(
|
||||
full_waveform=True,
|
||||
stop_after_index=stop_idx,
|
||||
skip_waveform_for_keys=seen_keys if seen_keys else None,
|
||||
)
|
||||
|
||||
# Filter to events whose keys we haven't saved before.
|
||||
new_events = [
|
||||
e for e in all_events
|
||||
if e._waveform_key is None
|
||||
or e._waveform_key.hex() not in seen_keys
|
||||
]
|
||||
skipped = len(all_events) - len(new_events)
|
||||
|
||||
log.info(" [OK] Downloaded %d event(s): %d new, %d skipped (already seen)",
|
||||
len(all_events), len(new_events), skipped)
|
||||
if skipped:
|
||||
log.info(" (skipped %d already-downloaded event(s))", skipped)
|
||||
|
||||
if new_events:
|
||||
_save_json(session_dir / "events.json", [_event_to_dict(e) for e in new_events])
|
||||
|
||||
for ev in new_events:
|
||||
pv = ev.peak_values
|
||||
pi = ev.project_info
|
||||
key_hex = ev._waveform_key.hex() if ev._waveform_key else "????????"
|
||||
log.info(
|
||||
" NEW [%s] %s Tran=%.4f Vert=%.4f Long=%.4f VS=%.4f project=%r",
|
||||
key_hex,
|
||||
str(ev.timestamp) if ev.timestamp else "?",
|
||||
pv.tran if pv else 0,
|
||||
pv.vert if pv else 0,
|
||||
pv.long if pv else 0,
|
||||
pv.peak_vector_sum if pv else 0,
|
||||
pi.project if pi else "",
|
||||
)
|
||||
else:
|
||||
log.info(" [OK] No new events since last call-home -- nothing to save")
|
||||
|
||||
# ── Monitor log entries (partial records / continuous monitoring) ──
|
||||
# Browse walk (0A + 1F only) to collect monitor log entries for
|
||||
# recording intervals where no threshold was crossed. This is a
|
||||
# second 1E-based pass over the device's record list, separate from
|
||||
# the get_events() download loop above.
|
||||
log.info(" Collecting monitor log entries (browse walk)...")
|
||||
new_monitor_entries: list[MonitorLogEntry] = []
|
||||
try:
|
||||
new_monitor_entries = client.get_monitor_log_entries(
|
||||
skip_keys=seen_keys if seen_keys else None,
|
||||
)
|
||||
if new_monitor_entries:
|
||||
_save_json(
|
||||
session_dir / "monitor_log.json",
|
||||
[_monitor_log_entry_to_dict(e) for e in new_monitor_entries],
|
||||
)
|
||||
log.info(
|
||||
" [OK] %d new monitor log entry(s) saved",
|
||||
len(new_monitor_entries),
|
||||
)
|
||||
for ml in new_monitor_entries:
|
||||
log.info(
|
||||
" MONLOG [%s] %s → %s (%s)",
|
||||
ml.key,
|
||||
ml.start_time.isoformat() if ml.start_time else "?",
|
||||
ml.stop_time.isoformat() if ml.stop_time else "?",
|
||||
f"{ml.duration_seconds:.0f}s" if ml.duration_seconds is not None else "?s",
|
||||
)
|
||||
else:
|
||||
log.info(" [OK] No new monitor log entries")
|
||||
except Exception as exc:
|
||||
log.warning(
|
||||
" [WARN] Monitor log collection failed: %s -- continuing",
|
||||
exc,
|
||||
)
|
||||
|
||||
# ── Persist to SQLite DB ─────────────────────────────────────
|
||||
_session_start = datetime.datetime.now()
|
||||
try:
|
||||
_ev_ins, _ev_skip = self.db.insert_events(
|
||||
new_events, serial=serial or self.peer, session_id=None
|
||||
)
|
||||
_ml_ins, _ml_skip = self.db.insert_monitor_log(
|
||||
new_monitor_entries, session_id=None
|
||||
)
|
||||
_session_id = self.db.insert_ach_session(
|
||||
serial=serial or self.peer,
|
||||
peer=self.peer,
|
||||
events_downloaded=_ev_ins,
|
||||
monitor_entries=_ml_ins,
|
||||
duration_seconds=(datetime.datetime.now() - _session_start).total_seconds(),
|
||||
session_time=_session_start,
|
||||
)
|
||||
log.info(
|
||||
" [DB] session=%s events +%d (skip %d) monitor +%d (skip %d)",
|
||||
_session_id[:8], _ev_ins, _ev_skip, _ml_ins, _ml_skip,
|
||||
)
|
||||
except Exception as exc:
|
||||
log.warning(" [WARN] DB write failed: %s -- continuing", exc)
|
||||
|
||||
# ── Optional: erase device memory after successful download ────
|
||||
erased_successfully = False
|
||||
if self.clear_after_download and new_events:
|
||||
log.info(" Clearing device memory (--clear-after-download)...")
|
||||
try:
|
||||
client.delete_all_events()
|
||||
log.info(" [OK] Device memory cleared")
|
||||
erased_successfully = True
|
||||
except Exception as exc:
|
||||
log.error(
|
||||
" [WARN] Event deletion failed: %s -- events NOT cleared",
|
||||
exc,
|
||||
)
|
||||
|
||||
# ── Update persistent state ───────────────────────────────────
|
||||
# Include both triggered-event keys and monitor-log keys in the
|
||||
# downloaded set so they are not re-processed on the next call-home.
|
||||
current_event_keys = [
|
||||
e._waveform_key.hex()
|
||||
for e in all_events
|
||||
if e._waveform_key is not None
|
||||
]
|
||||
current_monitor_keys = [e.key for e in new_monitor_entries]
|
||||
current_keys = current_event_keys + current_monitor_keys
|
||||
|
||||
if erased_successfully:
|
||||
# Device memory is clear. Reset downloaded_keys and the
|
||||
# high-water mark so the next call-home starts fresh and
|
||||
# doesn't mis-identify the recycled key 01110000 as "seen".
|
||||
updated_keys = []
|
||||
new_max_key = "00000000"
|
||||
log.info(
|
||||
" State reset after erase -- next session will download "
|
||||
"from key 0 (device counter resets after erase)"
|
||||
)
|
||||
else:
|
||||
# Normal (no erase): union of previously-seen + all keys on
|
||||
# device now. Includes already-seen survivors so we never
|
||||
# re-download them if the device somehow keeps old records.
|
||||
updated_keys = sorted(set(seen_keys) | set(current_keys))
|
||||
new_max_key = updated_keys[-1] if updated_keys else max_seen_key
|
||||
|
||||
state[unit_key] = {
|
||||
"downloaded_keys": updated_keys,
|
||||
"max_downloaded_key": new_max_key,
|
||||
"last_seen": datetime.datetime.now().isoformat(),
|
||||
"serial": serial,
|
||||
"peer": self.peer,
|
||||
}
|
||||
_save_state(self.state_path, state)
|
||||
|
||||
except Exception as exc:
|
||||
log.error(" [FAIL] Event download failed: %s", exc, exc_info=True)
|
||||
|
||||
# ── Optional: restart monitoring after successful download ─────────
|
||||
if self.restart_monitoring:
|
||||
log.info(" Restarting monitoring on device (--restart-monitoring)...")
|
||||
try:
|
||||
client.start_monitoring()
|
||||
log.info(" [OK] Monitoring restarted")
|
||||
except Exception as exc:
|
||||
log.warning(" [WARN] Failed to restart monitoring: %s", exc)
|
||||
|
||||
finally:
|
||||
raw_fh.close()
|
||||
client.close() # closes transport / socket cleanly
|
||||
root_logger.removeHandler(fh)
|
||||
fh.close()
|
||||
|
||||
log.info("Session complete -> %s", session_dir)
|
||||
log.info("="*60)
|
||||
|
||||
|
||||
# ── JSON helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
def _save_json(path: Path, obj: object) -> None:
|
||||
with open(path, "w") as f:
|
||||
json.dump(obj, f, indent=2, default=str)
|
||||
log.debug("Saved %s", path)
|
||||
|
||||
|
||||
def _device_info_to_dict(d: DeviceInfo) -> dict:
|
||||
cc = d.compliance_config
|
||||
return {
|
||||
"serial": d.serial,
|
||||
"firmware_version": d.firmware_version,
|
||||
"dsp_version": d.dsp_version,
|
||||
"model": d.model,
|
||||
"event_count": d.event_count,
|
||||
# compliance config fields (None if 1A read failed)
|
||||
"setup_name": cc.setup_name if cc else None,
|
||||
"sample_rate": cc.sample_rate if cc else None,
|
||||
"record_time": cc.record_time if cc else None,
|
||||
"trigger_level_geo": cc.trigger_level_geo if cc else None,
|
||||
"alarm_level_geo": cc.alarm_level_geo if cc else None,
|
||||
"max_range_geo": cc.max_range_geo if cc else None,
|
||||
"project": cc.project if cc else None,
|
||||
"client": cc.client if cc else None,
|
||||
"operator": cc.operator if cc else None,
|
||||
"sensor_location": cc.sensor_location if cc else None,
|
||||
}
|
||||
|
||||
|
||||
def _event_to_dict(e: Event) -> dict:
|
||||
pv = e.peak_values
|
||||
pi = e.project_info
|
||||
peaks = {}
|
||||
if pv:
|
||||
peaks = {
|
||||
"transverse": pv.tran,
|
||||
"vertical": pv.vert,
|
||||
"longitudinal": pv.long,
|
||||
"vector_sum": pv.peak_vector_sum,
|
||||
"mic": pv.micl,
|
||||
}
|
||||
samples = {}
|
||||
if e.raw_samples:
|
||||
samples = {
|
||||
ch: vals[:20] # first 20 sample-sets to keep the file sane
|
||||
for ch, vals in e.raw_samples.items()
|
||||
}
|
||||
samples["__note__"] = "first 20 sample-sets only; see raw_rx.bin for full waveform"
|
||||
return {
|
||||
"timestamp": str(e.timestamp) if e.timestamp else None,
|
||||
"project": pi.project if pi else None,
|
||||
"client": pi.client if pi else None,
|
||||
"operator": pi.operator if pi else None,
|
||||
"sensor_location": pi.sensor_location if pi else None,
|
||||
"peaks": peaks,
|
||||
"raw_samples_preview": samples,
|
||||
}
|
||||
|
||||
|
||||
def _monitor_log_entry_to_dict(e: MonitorLogEntry) -> dict:
|
||||
return {
|
||||
"key": e.key,
|
||||
"start_time": e.start_time.isoformat() if e.start_time else None,
|
||||
"stop_time": e.stop_time.isoformat() if e.stop_time else None,
|
||||
"duration_seconds": e.duration_seconds,
|
||||
"serial": e.serial,
|
||||
"geo_threshold_ips": e.geo_threshold_ips,
|
||||
}
|
||||
|
||||
|
||||
# ── Main server loop ───────────────────────────────────────────────────────────
|
||||
|
||||
def serve(args: argparse.Namespace) -> None:
|
||||
output_dir = Path(args.output)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
state_path = output_dir / "ach_state.json"
|
||||
db = SeismoDb(output_dir / "seismo_relay.db")
|
||||
|
||||
server_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
server_sock.bind(("0.0.0.0", args.port))
|
||||
server_sock.listen(5)
|
||||
# Wake up every second so Ctrl-C is handled promptly on Windows.
|
||||
# Without this, accept() blocks indefinitely and ignores KeyboardInterrupt.
|
||||
server_sock.settimeout(1.0)
|
||||
|
||||
max_ev = args.max_events
|
||||
print(f"\n{'='*60}")
|
||||
print(f" ACH inbound server listening on 0.0.0.0:{args.port}")
|
||||
print(f" Output: {output_dir.resolve()}/ach_inbound_<timestamp>/")
|
||||
print(f" State file: {state_path}")
|
||||
print(f" Max events per session: {max_ev if max_ev else 'unlimited'}")
|
||||
print(f" Clear device after download: {'YES' if args.clear_after_download else 'no'}")
|
||||
print(f" Restart monitoring after download: {'YES' if args.restart_monitoring else 'no'}")
|
||||
print(f"{'='*60}")
|
||||
print(f"\n Point your test unit's ACEmanager call-home settings to:")
|
||||
print(f" Remote Host: <this machine's LAN IP>")
|
||||
print(f" Remote Port: {args.port}")
|
||||
print(f"\n Waiting for inbound connections... (Ctrl-C to stop)\n")
|
||||
|
||||
allow_ips = set(args.allow_ips)
|
||||
if allow_ips:
|
||||
print(f" Allowlist: {', '.join(sorted(allow_ips))}")
|
||||
else:
|
||||
print(" Allowlist: NONE -- accepting all IPs (add --allow-ip to restrict)")
|
||||
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
client_sock, addr = server_sock.accept()
|
||||
except socket.timeout:
|
||||
continue # no connection this second; loop back and check for Ctrl-C
|
||||
try:
|
||||
peer_ip = addr[0]
|
||||
peer = f"{addr[0]}:{addr[1]}"
|
||||
|
||||
if allow_ips and peer_ip not in allow_ips:
|
||||
log.info("Rejected connection from %s (not in allowlist)", peer)
|
||||
client_sock.close()
|
||||
continue
|
||||
|
||||
log.info("Accepted connection from %s", peer)
|
||||
session = AchSession(
|
||||
sock=client_sock,
|
||||
peer=peer,
|
||||
output_dir=output_dir,
|
||||
timeout=args.timeout,
|
||||
events_only=args.events_only,
|
||||
max_events=max_ev,
|
||||
state_path=state_path,
|
||||
db=db,
|
||||
clear_after_download=args.clear_after_download,
|
||||
restart_monitoring=args.restart_monitoring,
|
||||
)
|
||||
t = threading.Thread(target=session.run, daemon=True, name=f"ach-{peer}")
|
||||
t.start()
|
||||
except KeyboardInterrupt:
|
||||
raise
|
||||
except Exception as exc:
|
||||
log.error("Accept error: %s", exc)
|
||||
finally:
|
||||
server_sock.close()
|
||||
print("\nServer stopped.")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
p = argparse.ArgumentParser(
|
||||
description="Minimal inbound ACH server — speak BW protocol to calling MiniMate Plus units.",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__,
|
||||
)
|
||||
p.add_argument(
|
||||
"--port", "-p",
|
||||
type=int,
|
||||
default=12345,
|
||||
help="Port to listen on (default: 12345).",
|
||||
)
|
||||
p.add_argument(
|
||||
"--output", "-o",
|
||||
default=str(Path(__file__).parent / "captures"),
|
||||
metavar="DIR",
|
||||
help="Directory to write session captures (default: bridges/captures/).",
|
||||
)
|
||||
p.add_argument(
|
||||
"--timeout", "-t",
|
||||
type=float,
|
||||
default=30.0,
|
||||
help="Protocol receive timeout in seconds (default: 30.0).",
|
||||
)
|
||||
p.add_argument(
|
||||
"--events-only",
|
||||
action="store_true",
|
||||
help="Skip the device-info step and go straight to event download.",
|
||||
)
|
||||
p.add_argument(
|
||||
"--max-events",
|
||||
type=int,
|
||||
default=None,
|
||||
metavar="N",
|
||||
help=(
|
||||
"Safety cap: download at most N events per session (default: unlimited). "
|
||||
"Useful if a unit has many old events stored — prevents a very long first run."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--allow-ip",
|
||||
metavar="IP",
|
||||
action="append",
|
||||
dest="allow_ips",
|
||||
default=[],
|
||||
help=(
|
||||
"Only accept connections from this IP address (repeat for multiple). "
|
||||
"Example: --allow-ip 63.43.212.232 "
|
||||
"If not specified, all IPs are accepted (not recommended for public servers)."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--restart-monitoring",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help=(
|
||||
"After downloading events, send SUB 0x96 (start monitoring) before "
|
||||
"disconnecting. Required for RV55 units whose firmware does not assert "
|
||||
"DCD on disconnect — without this the unit stays idle after a call-home."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--clear-after-download",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help=(
|
||||
"After successfully downloading new events, erase all events from the "
|
||||
"device memory (SUB 0xA3 → 0x1C → 0x06 → 0xA2 sequence, confirmed from "
|
||||
"4-11-26 MITM capture). Only fires when at least one new event was saved. "
|
||||
"This mirrors the standard Blastware ACH workflow."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--verbose", "-v",
|
||||
action="store_true",
|
||||
help="Enable debug logging.",
|
||||
)
|
||||
return p.parse_args()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parse_args()
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG if args.verbose else logging.INFO,
|
||||
format="%(asctime)s %(levelname)-7s %(name)s %(message)s",
|
||||
)
|
||||
try:
|
||||
serve(args)
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopped.")
|
||||
@@ -0,0 +1,435 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
serial_watch.py — Instantel Series-3 serial monitor with S3 frame parsing.
|
||||
|
||||
Taps the RS-232 line between the MiniMate Plus and its modem (RV50/RV55).
|
||||
Saves raw binary captures compatible with the rest of the analysis toolchain,
|
||||
plus a human-readable frame log.
|
||||
|
||||
Usage
|
||||
-----
|
||||
python bridges/serial_watch.py # interactive COM picker
|
||||
python bridges/serial_watch.py --port COM3 # specify port
|
||||
python bridges/serial_watch.py --port COM3 --ack-ok # reply OK to AT commands
|
||||
# (useful if modem is absent
|
||||
# and you want the device to
|
||||
# proceed past AT negotiation)
|
||||
python bridges/serial_watch.py --list # list available ports
|
||||
|
||||
Output
|
||||
------
|
||||
bridges/captures/serial_<ISO-timestamp>/
|
||||
raw_s3_<ts>.bin — raw bytes from device (feeds directly into S3FrameParser)
|
||||
session_<ts>.log — human-readable frame + control-line log
|
||||
session_<ts>.jsonl — JSON-lines frame log
|
||||
|
||||
The raw_s3_*.bin file is byte-for-byte compatible with the existing capture
|
||||
format used by bridges/parse_capture.py and all analysis scripts.
|
||||
|
||||
What to look for in a call-home capture
|
||||
----------------------------------------
|
||||
1. Does the device talk first after CONNECT, or does it wait?
|
||||
- If raw_s3_*.bin has bytes before any AT/POLL exchange → PUSH protocol
|
||||
- If it stays silent → PULL protocol (same as Blastware manual download)
|
||||
|
||||
2. Look for "Operating System" ASCII at the start — the device sends this 16-byte
|
||||
boot string on cold start before entering DLE-framed mode.
|
||||
|
||||
3. RING/CONNECT from the modem appear as ASCII before the DLE frames — the parser
|
||||
handles these automatically (scans forward to DLE+STX).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import serial
|
||||
from serial.tools import list_ports
|
||||
except ModuleNotFoundError:
|
||||
print(
|
||||
"pyserial not found. Install with:\n python -m pip install pyserial",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Add project root so we can import the frame parser
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
from minimateplus.framing import S3FrameParser, S3Frame
|
||||
|
||||
import json
|
||||
|
||||
|
||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def _ts() -> str:
|
||||
return datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]
|
||||
|
||||
|
||||
def _hexdump(b: bytes) -> str:
|
||||
return " ".join(f"{x:02X}" for x in b)
|
||||
|
||||
|
||||
def _printable(b: bytes) -> str:
|
||||
return b.decode("latin1", errors="replace")
|
||||
|
||||
|
||||
_KNOWN_SUBS = {
|
||||
0xA4: "POLL_RSP", 0xA5: "BULK_WAVEFORM_RSP", 0xE0: "ADVANCE_EVENT_RSP",
|
||||
0xE1: "EVENT_IDX_FIRST_RSP", 0xE3: "MONITOR_STATUS_RSP", 0xEA: "SERIAL_NUM_RSP",
|
||||
0xF3: "WAVEFORM_RECORD_RSP", 0xF5: "WAVEFORM_HEADER_RSP", 0xF7: "EVENT_INDEX_RSP",
|
||||
0xF9: "UNK_06_RSP", 0xFE: "DEVICE_INFO_RSP",
|
||||
0x69: "START_MONITOR_ACK", 0x68: "STOP_MONITOR_ACK",
|
||||
0x97: "EVT_IDX_WRITE_ACK", 0x8C: "CONFIRM_B_ACK", 0x8E: "COMPLIANCE_WRITE_ACK",
|
||||
0x8D: "CONFIRM_A_ACK", 0x7D: "TRIGGER_WRITE_ACK", 0x7C: "TRIGGER_CONFIRM_ACK",
|
||||
0x96: "WAVEFORM_WRITE_ACK", 0x8B: "CONFIRM_C_ACK",
|
||||
}
|
||||
|
||||
|
||||
def _label_frame(frame: S3Frame) -> str:
|
||||
name = _KNOWN_SUBS.get(frame.sub, f"UNK_0x{frame.sub:02X}")
|
||||
chk = "✓" if frame.checksum_valid else "✗ BAD_CHK"
|
||||
peek = frame.data[:24].hex() + ("…" if len(frame.data) > 24 else "")
|
||||
return (
|
||||
f"S3 SUB=0x{frame.sub:02X} ({name:<22}) "
|
||||
f"page=0x{frame.page_key:04X} data={len(frame.data):4d}B {chk} {peek}"
|
||||
)
|
||||
|
||||
|
||||
# ── Logger ────────────────────────────────────────────────────────────────────
|
||||
|
||||
class Logger:
|
||||
def __init__(self, log_path: Path, jsonl_path: Path, raw_path: Path) -> None:
|
||||
self._log = log_path.open("a", encoding="utf-8", newline="")
|
||||
self._jl = jsonl_path.open("a", encoding="utf-8", newline="")
|
||||
self._raw = raw_path.open("ab")
|
||||
self._lock = threading.Lock()
|
||||
self._frame_count = 0
|
||||
|
||||
def info(self, msg: str) -> None:
|
||||
line = f"[{_ts()}] INFO | {msg}"
|
||||
with self._lock:
|
||||
print(line)
|
||||
print(line, file=self._log, flush=True)
|
||||
|
||||
def ctrl(self, msg: str) -> None:
|
||||
line = f"[{_ts()}] CTRL | {msg}"
|
||||
with self._lock:
|
||||
print(line)
|
||||
print(line, file=self._log, flush=True)
|
||||
|
||||
def data_hex(self, msg: str) -> None:
|
||||
line = f"[{_ts()}] HEX | {msg}"
|
||||
with self._lock:
|
||||
print(line)
|
||||
print(line, file=self._log, flush=True)
|
||||
|
||||
def data_ascii(self, msg: str) -> None:
|
||||
line = f"[{_ts()}] DATA | {msg}"
|
||||
with self._lock:
|
||||
print(line)
|
||||
print(line, file=self._log, flush=True)
|
||||
|
||||
def frame(self, f: S3Frame) -> None:
|
||||
with self._lock:
|
||||
self._frame_count += 1
|
||||
label = f"[{_ts()}] FRAME | #{self._frame_count:04d} {_label_frame(f)}"
|
||||
print(label)
|
||||
print(label, file=self._log, flush=True)
|
||||
record = {
|
||||
"frame": self._frame_count,
|
||||
"sub": f.sub,
|
||||
"page_key": f.page_key,
|
||||
"data_len": len(f.data),
|
||||
"data_hex": f.data.hex(),
|
||||
"checksum_valid": f.checksum_valid,
|
||||
}
|
||||
print(json.dumps(record), file=self._jl, flush=True)
|
||||
|
||||
def write_raw(self, data: bytes) -> None:
|
||||
with self._lock:
|
||||
self._raw.write(data)
|
||||
self._raw.flush()
|
||||
|
||||
def close(self) -> None:
|
||||
with self._lock:
|
||||
for fh in (self._log, self._jl, self._raw):
|
||||
try:
|
||||
fh.flush()
|
||||
fh.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ── Control-line monitor thread ───────────────────────────────────────────────
|
||||
|
||||
def _monitor_control_lines(
|
||||
ser: serial.Serial,
|
||||
logger: Logger,
|
||||
stop: threading.Event,
|
||||
interval: float,
|
||||
) -> None:
|
||||
prev = dict(CTS=None, DSR=None, DCD=None, RI=None)
|
||||
try:
|
||||
prev.update(CTS=ser.cts, DSR=ser.dsr, DCD=ser.cd)
|
||||
try:
|
||||
prev["RI"] = ser.ri
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as exc:
|
||||
logger.ctrl(f"Init error: {exc}")
|
||||
return
|
||||
|
||||
logger.ctrl(
|
||||
f"Initial: CTS={prev['CTS']} DSR={prev['DSR']} DCD={prev['DCD']} RI={prev['RI']}"
|
||||
)
|
||||
while not stop.is_set():
|
||||
try:
|
||||
cur = dict(CTS=ser.cts, DSR=ser.dsr, DCD=ser.cd, RI=None)
|
||||
try:
|
||||
cur["RI"] = ser.ri
|
||||
except Exception:
|
||||
pass
|
||||
for name, val in cur.items():
|
||||
if val != prev[name]:
|
||||
logger.ctrl(f"{name} → {val}")
|
||||
prev[name] = val
|
||||
except serial.SerialException as exc:
|
||||
logger.ctrl(f"Poll error: {exc}")
|
||||
break
|
||||
stop.wait(interval)
|
||||
|
||||
|
||||
# ── Serial open ───────────────────────────────────────────────────────────────
|
||||
|
||||
_PARITY = {
|
||||
"N": serial.PARITY_NONE, "E": serial.PARITY_EVEN, "O": serial.PARITY_ODD,
|
||||
"M": serial.PARITY_MARK, "S": serial.PARITY_SPACE,
|
||||
}
|
||||
_STOPBITS = {
|
||||
1: serial.STOPBITS_ONE, 1.5: serial.STOPBITS_ONE_POINT_FIVE, 2: serial.STOPBITS_TWO,
|
||||
}
|
||||
|
||||
|
||||
def _open_serial(args: argparse.Namespace, logger: Logger) -> serial.Serial | None:
|
||||
for attempt in range(1, args.open_retries + 2):
|
||||
logger.info(
|
||||
f"Opening {args.port} @ {args.baud},{args.bytesize}{args.parity}{args.stopbits} "
|
||||
f"rtscts={args.rtscts} xonxoff={args.xonxoff} dsrdtr={args.dsrdtr} "
|
||||
f"(attempt {attempt})"
|
||||
)
|
||||
try:
|
||||
ser = serial.Serial(
|
||||
port=args.port,
|
||||
baudrate=args.baud,
|
||||
bytesize=args.bytesize,
|
||||
parity=_PARITY[args.parity],
|
||||
stopbits=_STOPBITS[args.stopbits],
|
||||
timeout=args.timeout,
|
||||
xonxoff=args.xonxoff,
|
||||
rtscts=args.rtscts,
|
||||
dsrdtr=args.dsrdtr,
|
||||
write_timeout=0,
|
||||
)
|
||||
try:
|
||||
ser.setDTR(args.dtr == "on")
|
||||
ser.setRTS(args.rts == "on")
|
||||
logger.ctrl(f"Set DTR={args.dtr} RTS={args.rts}")
|
||||
except Exception as exc:
|
||||
logger.ctrl(f"DTR/RTS set failed: {exc}")
|
||||
|
||||
if args.send_break > 0:
|
||||
try:
|
||||
ser.break_condition = True
|
||||
time.sleep(args.send_break / 1000.0)
|
||||
ser.break_condition = False
|
||||
logger.ctrl(f"BREAK held {args.send_break} ms")
|
||||
except Exception as exc:
|
||||
logger.ctrl(f"BREAK failed: {exc}")
|
||||
|
||||
return ser
|
||||
|
||||
except serial.SerialException as exc:
|
||||
logger.info(f"Open failed: {exc}")
|
||||
if attempt <= args.open_retries:
|
||||
time.sleep(args.open_retry_delay)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ── Port picker ───────────────────────────────────────────────────────────────
|
||||
|
||||
def _list_ports() -> list:
|
||||
ports = list(list_ports.comports())
|
||||
if not ports:
|
||||
print("No serial ports found.")
|
||||
return []
|
||||
print("Available serial ports:")
|
||||
for i, p in enumerate(ports, 1):
|
||||
print(f" {i:2d}) {p.device:<12} {p.description or ''}")
|
||||
return ports
|
||||
|
||||
|
||||
def _pick_port() -> str:
|
||||
ports = _list_ports()
|
||||
if not ports:
|
||||
sys.exit(1)
|
||||
if len(ports) == 1:
|
||||
print(f"Auto-selecting: {ports[0].device}")
|
||||
return ports[0].device
|
||||
while True:
|
||||
sel = input("Select port (number or name, e.g. COM3): ").strip()
|
||||
if sel.isdigit() and 1 <= int(sel) <= len(ports):
|
||||
return ports[int(sel) - 1].device
|
||||
for p in ports:
|
||||
if p.device.upper() == sel.upper():
|
||||
return p.device
|
||||
print("Not recognised. Enter list number or exact port name.")
|
||||
|
||||
|
||||
# ── Main loop ─────────────────────────────────────────────────────────────────
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Monitor Instantel Series-3 serial traffic with S3 frame parsing."
|
||||
)
|
||||
ap.add_argument("--port", "-p",
|
||||
help="COM port (e.g. COM3). Omit to be prompted.")
|
||||
ap.add_argument("--baud", "-b", type=int, default=38400)
|
||||
ap.add_argument("--bytesize", type=int, choices=[5, 6, 7, 8], default=8)
|
||||
ap.add_argument("--parity", choices=["N", "E", "O", "M", "S"], default="N")
|
||||
ap.add_argument("--stopbits", type=float, choices=[1, 1.5, 2], default=1)
|
||||
ap.add_argument("--rtscts", action="store_true")
|
||||
ap.add_argument("--xonxoff", action="store_true")
|
||||
ap.add_argument("--dsrdtr", action="store_true")
|
||||
ap.add_argument("--dtr", choices=["on", "off"], default="on")
|
||||
ap.add_argument("--rts", choices=["on", "off"], default="on")
|
||||
ap.add_argument("--send-break", type=int, default=0,
|
||||
help="Hold BREAK for N ms after open.")
|
||||
ap.add_argument("--show", choices=["ascii", "hex", "both", "frames"],
|
||||
default="frames",
|
||||
help="'frames' (default) shows only parsed S3 frames. "
|
||||
"'ascii'/'hex'/'both' also show raw bytes.")
|
||||
ap.add_argument("--encoding", default="latin1")
|
||||
ap.add_argument("--read-chunk", type=int, default=4096)
|
||||
ap.add_argument("--timeout", type=float, default=0.05)
|
||||
ap.add_argument("--poll-lines-interval", type=float, default=0.2)
|
||||
ap.add_argument("--open-retries", type=int, default=0)
|
||||
ap.add_argument("--open-retry-delay", type=float, default=0.8)
|
||||
ap.add_argument("--ack-ok", action="store_true",
|
||||
help="Auto-reply OK to AT* commands (except ATDT). "
|
||||
"Useful for testing without a real modem.")
|
||||
ap.add_argument("--list", action="store_true",
|
||||
help="List available serial ports and exit.")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.list:
|
||||
_list_ports()
|
||||
return
|
||||
|
||||
args.port = args.port or _pick_port()
|
||||
|
||||
# Build output paths
|
||||
ts_str = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
out_dir = Path(__file__).parent / "captures" / f"serial_{ts_str}"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
log_path = out_dir / f"session_{ts_str}.log"
|
||||
jsonl_path = out_dir / f"session_{ts_str}.jsonl"
|
||||
raw_path = out_dir / f"raw_s3_{ts_str}.bin"
|
||||
|
||||
logger = Logger(log_path, jsonl_path, raw_path)
|
||||
logger.info(f"Output directory: {out_dir}")
|
||||
logger.info(f"raw_s3 → {raw_path.name} (compatible with parse_capture.py)")
|
||||
|
||||
ser = _open_serial(args, logger)
|
||||
if ser is None:
|
||||
logger.info("Could not open serial port. Exiting.")
|
||||
logger.close()
|
||||
sys.exit(1)
|
||||
|
||||
s3_parser = S3FrameParser()
|
||||
rx_buf = bytearray()
|
||||
stop_evt = threading.Event()
|
||||
|
||||
ctrl_thread = threading.Thread(
|
||||
target=_monitor_control_lines,
|
||||
args=(ser, logger, stop_evt, args.poll_lines_interval),
|
||||
daemon=True,
|
||||
)
|
||||
ctrl_thread.start()
|
||||
logger.info("Monitoring started. Waiting for call-home. Press Ctrl+C to stop.")
|
||||
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
data = ser.read(args.read_chunk)
|
||||
except serial.SerialException as exc:
|
||||
logger.info(f"Read error: {exc}")
|
||||
break
|
||||
|
||||
if not data:
|
||||
continue
|
||||
|
||||
# 1. Save raw bytes
|
||||
logger.write_raw(data)
|
||||
|
||||
# 2. Optional raw display
|
||||
if args.show in ("ascii", "both"):
|
||||
txt = _printable(data)
|
||||
for line in txt.splitlines():
|
||||
logger.data_ascii(line)
|
||||
if args.show in ("hex", "both"):
|
||||
logger.data_hex(_hexdump(data))
|
||||
|
||||
# 3. Parse S3 frames
|
||||
for byte in data:
|
||||
result = s3_parser.feed(bytes([byte]))
|
||||
if result:
|
||||
frames = result if isinstance(result, list) else [result]
|
||||
for f in frames:
|
||||
logger.frame(f)
|
||||
|
||||
# 4. AT command handling for --ack-ok
|
||||
if args.ack_ok:
|
||||
rx_buf.extend(data)
|
||||
while b"\r" in rx_buf or b"\n" in rx_buf:
|
||||
for sep in (b"\r", b"\n"):
|
||||
idx = rx_buf.find(sep)
|
||||
if idx != -1:
|
||||
line_bytes = bytes(rx_buf[:idx])
|
||||
del rx_buf[:idx + 1]
|
||||
break
|
||||
else:
|
||||
break
|
||||
|
||||
line_str = line_bytes.decode("latin1", errors="ignore").strip().upper()
|
||||
if line_str.startswith("AT") and not line_str.startswith("ATDT"):
|
||||
try:
|
||||
ser.write(b"\r\nOK\r\n")
|
||||
ser.flush()
|
||||
logger.info(f"AT ack: {line_str!r} → OK")
|
||||
except Exception as exc:
|
||||
logger.info(f"AT ack write failed: {exc}")
|
||||
|
||||
except KeyboardInterrupt:
|
||||
logger.info("Ctrl+C — stopping.")
|
||||
|
||||
finally:
|
||||
stop_evt.set()
|
||||
try:
|
||||
ser.close()
|
||||
except Exception:
|
||||
pass
|
||||
ctrl_thread.join(timeout=1.0)
|
||||
logger.info(f"Capture saved to: {out_dir}")
|
||||
logger.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -93,11 +93,16 @@
|
||||
| 2026-04-06 | §7.8.4 | **NEW — 5A chunk timing and count (empirical, BE11529 at 1024 sps).** Each chunk response arrives within ~1 second over TCP/cellular. A 9,306-sample event (≈9.1 s at 1024 sps) produces **35 chunks** before end-of-stream. Chunks 1–16 have varying data lengths (1036–1123 bytes); chunks 17–35 are uniformly 1036 bytes each (post-event silence, all-zero ADC samples). Safe recv timeout for chunk loop: **10 s** (10× typical response time). Default transport timeout (120 s) results in a ~2-minute stall per event at end-of-stream. |
|
||||
| 2026-04-06 | §7.8.3 | **KNOWN ISSUE — `_decode_a5_waveform` hardcoded fi==9 skip.** The decoder contains `elif fi == 9: continue` which was written for the 9-frame original blast capture where frame 9 was a device terminator. For streams with >9 frames (current device produces 35+), frame index 9 is live waveform data — this skip discards ~1,070 bytes (~133 sample-sets) per event. The terminator is now detected via `page_key == 0x0000`, not by frame index. The fi==9 skip should be removed. |
|
||||
| 2026-04-06 | §7.8 | **CONFIRMED — ADC count-to-physical-unit conversion.** Raw waveform samples are signed 16-bit integers (counts). Conversion: `value = counts × (range / 32767)`. For geo channels: range = 10.000 in/s (from the device's compliance config geo range field). For the mic channel: range is in psi (device-specific). Near-full-scale counts (≈32,700) on all four channels simultaneously indicate ADC saturation (clipping) from a high-amplitude event. |
|
||||
| 2026-04-08 | §5.1, §7.10, §12 | **NEW — Monitoring commands confirmed.** SUB 0x1C (monitor status), 0x96 (start monitoring), 0x97 (stop monitoring) all confirmed from 4-8-26/2ndtry capture. SESSION_RESET (`41 03`) required before POLL to wake a monitoring unit. `section[6] == 0x10` is the monitoring flag (CORRECTED 2026-04-08 — was wrongly `section[1]`). Battery/memory at relative-from-end offsets: `section[-11:-9]` (battery×100), `section[-9:-5]` (memory_total), `section[-5:-1]` (memory_free) — stable across all payload size variants (52–55 bytes). |
|
||||
| 2026-04-08 | §5.1, §7.10, §12 | **NEW — Monitoring commands confirmed.** SUB 0x1C (monitor status), 0x96 (start monitoring), 0x97 (stop monitoring) all confirmed from 4-8-26/2ndtry capture. SESSION_RESET (`41 03`) required before POLL to wake a monitoring unit. |
|
||||
| 2026-04-09 | §7.10 | **CORRECTED — monitoring flag and battery/memory offsets.** `section[1] == 0x10` is the monitoring flag (100% accurate across 144 data frames in 2ndtry capture). Previous note claiming `section[6]` was wrong — section[6] has device-specific non-binary values (0xea/0x07). Battery/memory offsets corrected: `section[-10:-8]` (battery×100), `section[-8:-4]` (memory_total), `section[-4:]` (memory_free). NOTE: `frame.data` has checksum stripped by parser — earlier offsets of `[-11:-9]`/`[-9:-5]`/`[-5:-1]` were wrong because they assumed a trailing checksum byte that isn't there. |
|
||||
| 2026-04-08 | §7.10 | **NEW — SUBs 0x0E (channel sensor data) and 0x98 (trigger test) observed** in 4-8-26/sensor-check capture (Blastware "Unit Channel Test" comms check). SUB 0x0E: 2-step read with channel selector in `params[6:8]`, data length 0x0A per channel, RSP SUB = 0xF1. SUB 0x98: single probe frame with `params[0] = 0xFF`, RSP SUB = 0x67; sent twice per test cycle. Not yet implemented in SFM. |
|
||||
| 2026-04-08 | §7.10 | **NEW — SUBs 0x15 and 0x01 observed in sensor-check capture.** SUB 0x15 (serial number short form, data length 0x0A, RSP 0xEA) and SUB 0x01 (device info block, data length 0x98 = 152 bytes, RSP 0xFE) seen in Blastware's "Unit Channel Test" init sequence. Note: SUB 0x01 response SUB 0xFE collides with the existing SUB 0xFE → RSP 0x01 naming convention — they are inverse commands. |
|
||||
| 2026-04-08 | §12 | **CONFIRMED — Unit partially reachable during on-device sensor check.** 4-8-26/sensor-check capture shows: POLL responds normally throughout; SUB 0x0E channel reads partially served (channels 0–4 responded), then ~40s silent gap while sensor check ran, then channels 5–7 responded. On-device sensor check duration ≈ 40 s. SFM `_pollMonitorConfirm()` polls status every 5 s for up to 60 s after start_monitoring. |
|
||||
| 2026-04-08 | §7.9 (NEW) | **NEW — Compliance config field inventory captured from Blastware UI.** See §7.9 for full field list (Recording Setup, Notes, Special Setups tabs). Most fields NOT yet mapped to raw byte offsets. Confirmed decoded: sample_rate, record_time, trigger_level_geo, alarm_level_geo, max_range_geo, backlight_on_time, power_saving_timeout, monitoring_lcd_cycle, project/client/operator/sensor_location/notes. Sensor Check dropdown (Before monitoring / After each event / Disabled) NOT YET LOCATED in raw config bytes. |
|
||||
| 2026-04-11 | §5.1, §5.2 | **NEW — Erase-all command sequence confirmed from MITM capture.** SUB 0xA3 (begin erase, token=0xFE → ack 0x5C) + SUB 0xA2 (confirm erase, token=0xFE → ack 0x5D). Standard `build_bw_frame` format (not write-format). Required intermediate steps: 0x1C probe+data (monitor status read) + 0x06 probe+data (event storage range). All response SUBs follow the standard 0xFF−SUB formula with no exceptions. |
|
||||
| 2026-04-11 | §5.1 | **CONFIRMED — SUB 0x06 (CHANNEL CONFIG READ) now confirmed as event storage range.** Two-step read, data offset = 0x24 (36 bytes). Token=0xFE at params[7]. Last 8 bytes of response: first stored event key (bytes −8:−4) and last stored event key (bytes −4:). Both equal `01110000` when device memory is empty. Used by Blastware to verify erase completion. |
|
||||
| 2026-04-11 | §7.11 (NEW) | **NEW — §7.11 Erase-All Protocol added.** Full wire sequence, SUB 0x06 storage range payload layout, post-erase key counter reset (resets to `0x01110000`). Confirmed from 4-11-26 MITM capture of live Blastware ACH session. |
|
||||
| 2026-04-11 | §14.6 | **RESOLVED — ACH Session Lifecycle is no longer "Future".** `bridges/ach_server.py` fully implements inbound ACH: POLL handshake, device info, event download. State tracked via `ach_state.json` (key-based, with `max_downloaded_key` for post-erase detection). `--clear-after-download` flag added for the standard delete-after-upload workflow. |
|
||||
|
||||
---
|
||||
|
||||
@@ -242,7 +247,7 @@ Step 4 — Device sends actual data payload:
|
||||
| `15` | **SERIAL NUMBER REQUEST** | Requests device serial number. | ✅ CONFIRMED |
|
||||
| `01` | **FULL CONFIG READ** | Requests complete device configuration block (~0x98 bytes). Firmware, model, serial, channel config, scaling factors. | ✅ CONFIRMED |
|
||||
| `08` | **EVENT INDEX READ** | Requests the event record index (0x58 bytes). Event count and record pointers. | ✅ CONFIRMED |
|
||||
| `06` | **CHANNEL CONFIG READ** | Requests channel configuration block (0x24 bytes). | ✅ CONFIRMED |
|
||||
| `06` | **EVENT STORAGE RANGE READ** | Requests event storage range block (0x24 = 36 bytes). Token=0xFE at params[7]. Last 8 bytes of response: first stored event key (`[-8:-4]`) and last stored event key (`[-4:]`). Both equal `01110000` when device is empty. Used by Blastware as part of the erase-all verification step. Previously labelled "CHANNEL CONFIG READ" — function now confirmed from 4-11-26 MITM capture. | ✅ CONFIRMED 2026-04-11 |
|
||||
| `1C` | **TRIGGER CONFIG READ** | Requests trigger settings block (0x2C bytes). | ✅ CONFIRMED |
|
||||
| `1E` | **EVENT HEADER READ** | Gets first waveform key. Token byte at params[7] (0x00=browse, 0xFE=download-arm). Key at data[11:15]; trailing offset at data[15:19] (0 = only one event). Two uses: (1) all-zero to get key0; (2) token=0xFE after 0A, before 0C — REQUIRED to arm device for SUB 5A. | ✅ CONFIRMED 2026-04-06 |
|
||||
| `0A` | **WAVEFORM HEADER READ** | Checks record type for a given waveform key. Variable DATA_LENGTH: 0x30=full bin, 0x26=partial bin. Key at params[4..7]. Required before every 1F call to establish device waveform context. | ✅ CONFIRMED 2026-03-31 |
|
||||
@@ -256,9 +261,11 @@ Step 4 — Device sends actual data payload:
|
||||
| `2E` | **UNKNOWN READ B** | Read command, response (`D1`) returns 0x1A (26) bytes. Purpose unknown. | 🔶 INFERRED |
|
||||
| `0E` | **CHANNEL SENSOR DATA** | Real-time sensor reading for one channel. Two-step read, data length 0x0A (10 bytes). Channel selector in params[6:8] (0x0000–0x0007 for 8 channels). Response (F1) carries amplitude, frequency, overswing data for that channel. Used by Blastware "Unit Channel Test" comms check. | ✅ CONFIRMED 2026-04-08 |
|
||||
| `98` | **TRIGGER TEST** | Trigger-test command. Single probe frame; `params[0] = 0xFF`. Response (0x67) is all-zero data. Sent twice per Blastware comms-check cycle. Not a full POLL, no monitor state change. | ✅ CONFIRMED 2026-04-08 |
|
||||
| `1C` | **MONITOR STATUS READ** | Two-step read, data offset 0x2C (44 bytes). `section[6] == 0x10` → monitoring; `0x00` → idle (CORRECTED 2026-04-08 — was wrongly documented as section[1]). Payload length varies (52–55 bytes) but battery/memory block is always the last 10 bytes before checksum: `section[-11:-9]` = battery×100 (uint16 BE), `section[-9:-5]` = memory_total (uint32 BE), `section[-5:-1]` = memory_free (uint32 BE). Confirmed from 2ndtry 4-8-26 full byte diff across 3 payload size variants. | ✅ CONFIRMED 2026-04-08 |
|
||||
| `1C` | **MONITOR STATUS READ** | Two-step read, data offset 0x2C (44 bytes). `section[1] == 0x10` → monitoring; `0x00` → idle (CONFIRMED 2026-04-09, 100% accuracy on 144 frames). Payload length: 46–47 bytes IDLE, 48–49 bytes MONITORING. `frame.data` has checksum stripped — no trailing byte to skip. Battery/memory at end: `section[-10:-8]` = battery×100 (uint16 BE), `section[-8:-4]` = memory_total (uint32 BE), `section[-4:]` = memory_free (uint32 BE). | ✅ CONFIRMED 2026-04-09 |
|
||||
| `96` | **START MONITORING** | Single write frame, no data payload. Transitions unit from idle to monitoring mode (after optional on-device sensor check ~40 s). | ✅ CONFIRMED 2026-04-08 |
|
||||
| `97` | **STOP MONITORING** | Single write frame, no data payload. Stops monitoring, unit returns to idle. | ✅ CONFIRMED 2026-04-08 |
|
||||
| `A3` | **ERASE ALL BEGIN** | Single frame, token=0xFE at params[7]. Initiates device memory erase. Must be followed by 0x1C probe+data + 0x06 probe+data + 0xA2 to complete. Standard `build_bw_frame` (not write-format). Response ack SUB = 0x5C. | ✅ CONFIRMED 2026-04-11 |
|
||||
| `A2` | **ERASE ALL CONFIRM** | Single frame, token=0xFE at params[7]. Commits the erase initiated by 0xA3. After this ack (SUB 0x5D), device memory is cleared and the event counter resets to `0x01110000`. | ✅ CONFIRMED 2026-04-11 |
|
||||
|
||||
All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which, after de-stuffing, is just the DLE+CMD combination — see §3).
|
||||
|
||||
@@ -272,7 +279,7 @@ All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which,
|
||||
| `15` | `EA` | ✅ CONFIRMED |
|
||||
| `01` | `FE` | ✅ CONFIRMED |
|
||||
| `08` | `F7` | ✅ CONFIRMED |
|
||||
| `06` | `F9` | ✅ CONFIRMED |
|
||||
| `06` | `F9` | ✅ CONFIRMED 2026-04-11 |
|
||||
| `1C` | `E3` | ✅ CONFIRMED 2026-04-08 |
|
||||
| `1E` | `E1` | ✅ CONFIRMED |
|
||||
| `0A` | `F5` | ✅ CONFIRMED |
|
||||
@@ -286,6 +293,8 @@ All requests use CMD byte `0x02`. All responses use CMD byte `0x10 0x02` (which,
|
||||
| `98` | `67` | ✅ CONFIRMED 2026-04-08 |
|
||||
| `96` | `69` | ✅ CONFIRMED 2026-04-08 |
|
||||
| `97` | `68` | ✅ CONFIRMED 2026-04-08 |
|
||||
| `A3` | `5C` | ✅ CONFIRMED 2026-04-11 |
|
||||
| `A2` | `5D` | ✅ CONFIRMED 2026-04-11 |
|
||||
|
||||
---
|
||||
|
||||
@@ -1385,6 +1394,77 @@ Contains serial number, firmware bytes, and floating-point calibration fields. F
|
||||
|
||||
---
|
||||
|
||||
## 7.11 Erase-All Protocol (SUBs 0xA3 / 0xA2 / 0x06) ✅ 2026-04-11
|
||||
|
||||
> ✅ **Confirmed 2026-04-11** from MITM capture of a live Blastware ACH session
|
||||
> (`bridges/captures/mitm/ach_mitm_20260411_001912/`).
|
||||
|
||||
Blastware uses a 4-step sequence to erase all stored events from device memory.
|
||||
All frames use standard `build_bw_frame` format (NOT write-format).
|
||||
|
||||
### 7.11.1 Wire Sequence
|
||||
|
||||
```
|
||||
BW → device: SUB 0xA3 offset=0x0000 params=00 00 00 00 00 00 00 FE 00 00
|
||||
device → BW: SUB 0x5C (begin-erase ack)
|
||||
|
||||
BW → device: SUB 0x1C offset=0x0000 params=00 00 00 00 00 00 00 FE 00 00 (probe)
|
||||
device → BW: SUB 0xE3 (probe ack)
|
||||
BW → device: SUB 0x1C offset=0x002C params=(same) (data)
|
||||
device → BW: SUB 0xE3 (44-byte monitor status response)
|
||||
|
||||
BW → device: SUB 0x06 offset=0x0000 params=00 00 00 00 00 00 00 FE 00 00 (probe)
|
||||
device → BW: SUB 0xF9 (probe ack)
|
||||
BW → device: SUB 0x06 offset=0x0024 params=(same) (data)
|
||||
device → BW: SUB 0xF9 (36-byte storage range response)
|
||||
|
||||
BW → device: SUB 0xA2 offset=0x0000 params=00 00 00 00 00 00 00 FE 00 00
|
||||
device → BW: SUB 0x5D (confirm-erase ack — device memory is now cleared)
|
||||
```
|
||||
|
||||
All response SUBs follow the standard formula `0xFF − request_SUB`. No exceptions.
|
||||
The `token=0xFE` at `params[7]` is required for 0xA3, 0x06, and 0xA2.
|
||||
|
||||
### 7.11.2 SUB 0x06 Storage Range Response (36 bytes)
|
||||
|
||||
The 36-byte response from the data step ends with two 4-byte event keys:
|
||||
|
||||
| Offset (from response end) | Field | Notes |
|
||||
|---|---|---|
|
||||
| `[-8:-4]` | First stored event key | e.g. `0111ea60` before erase |
|
||||
| `[-4:]` | Last stored event key | e.g. `0111eaa6` before erase |
|
||||
|
||||
After a successful erase:
|
||||
- Both keys read `01110000` (device-empty sentinel)
|
||||
- The device's internal event counter has reset
|
||||
|
||||
Example pre-erase: `... 0111ea60 0111eaa6`
|
||||
Example post-erase: `... 01110000 01110000`
|
||||
|
||||
### 7.11.3 Post-Erase Key Counter Reset
|
||||
|
||||
After a successful erase the device resets its event counter. New events start
|
||||
from key `0x01110000` — the same key as the very first event ever recorded on
|
||||
the device. This means:
|
||||
|
||||
- Any system using event keys for deduplication must clear its "seen keys" state
|
||||
after an erase, or risk treating fresh events as already downloaded.
|
||||
- Detection heuristic: if `max(device_keys) < historical_max_key`, the counter
|
||||
was reset. All device keys should be treated as new regardless of prior state.
|
||||
|
||||
The `ach_server.py` implementation stores `max_downloaded_key` in `ach_state.json`
|
||||
and applies this heuristic on every call-home.
|
||||
|
||||
### 7.11.4 Implementation Notes
|
||||
|
||||
- `MiniMateClient.delete_all_events()` in `client.py` orchestrates the full sequence.
|
||||
- `MiniMateProtocol` exposes `begin_erase_all()`, `confirm_erase_all()`, and
|
||||
`read_event_storage_range()` as separate methods.
|
||||
- The ACH server `--clear-after-download` flag calls `delete_all_events()` after a
|
||||
successful event download and resets `ach_state.json` state for the unit.
|
||||
|
||||
---
|
||||
|
||||
## 8. Timestamp Format
|
||||
|
||||
Two timestamp wire formats are used:
|
||||
@@ -1775,7 +1855,7 @@ The TCP port is **user-configurable** in both Blastware and the modem. There is
|
||||
|
||||
---
|
||||
|
||||
### 14.6 ACH Session Lifecycle (Call Home Mode — Future)
|
||||
### 14.6 ACH Session Lifecycle (Call Home Mode) ✅ IMPLEMENTED 2026-04-11
|
||||
|
||||
When the unit calls home under ACH, the session lifecycle from the unit's perspective is:
|
||||
|
||||
@@ -1784,10 +1864,28 @@ When the unit calls home under ACH, the session lifecycle from the unit's perspe
|
||||
3. Unit waits for "Wait for Connection" window for first BW frame from server
|
||||
4. Server sends POLL_PROBE → unit responds with POLL_RESPONSE (same as serial)
|
||||
5. Server reads serial number, full config, events as needed
|
||||
6. Server disconnects (or unit disconnects on Serial Idle Time expiry)
|
||||
7. Unit powers modem down, returns to monitor mode
|
||||
6. (Optional) Server erases device memory: SUB 0xA3 → 0x1C → 0x06 → 0xA2
|
||||
7. Server disconnects (or unit disconnects on Serial Idle Time expiry)
|
||||
8. Unit detects DCD/DTR going low (modem signals line drop), returns to monitor mode automatically
|
||||
|
||||
Step 4 onward is **identical to the serial/call-up protocol**. The only difference from our perspective is that we are the **listener** rather than the **connector**. A future `AchServer` class will accept the incoming TCP connection and hand the socket to `TcpTransport` for processing.
|
||||
Step 4 onward is **identical to the serial/call-up protocol**. The only difference
|
||||
from our perspective is that we are the **listener** rather than the **connector**.
|
||||
|
||||
**Implementation: `bridges/ach_server.py`** — run with `python bridges/ach_server.py`.
|
||||
Key flags:
|
||||
- `--clear-after-download` — erase device memory after a successful event download
|
||||
- `--allow-ip IP` — restrict to specific unit IPs
|
||||
- `--max-events N` — cap events per session for safety
|
||||
|
||||
**State persistence: `ach_state.json`** — tracks `downloaded_keys` (set of event key
|
||||
hex strings) and `max_downloaded_key` (high-water mark) per unit serial number.
|
||||
Post-erase key reuse (`0x01110000` recycled) is detected via the high-water mark.
|
||||
|
||||
**Note on DCD/DTR:** The MiniMate Plus monitors the RS-232 DCD line. When the TCP
|
||||
connection closes, the Sierra Wireless modem drops DCD, which the unit interprets as
|
||||
"serial connection ended" and automatically resumes monitoring. No `start_monitoring()`
|
||||
(SUB 0x96) command is needed from the server. ⚠️ Newer RV55 firmware may not assert DCD
|
||||
by default — known issue, not yet resolved.
|
||||
|
||||
---
|
||||
|
||||
@@ -1840,6 +1938,11 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger
|
||||
| Backlight offset — **RESOLVED: +4B in event index data**, uint8, seconds | RESOLVED | 2026-03-02 | |
|
||||
| Power save offset — **RESOLVED: +53 in event index data**, uint8, minutes | RESOLVED | 2026-03-02 | |
|
||||
| Monitoring LCD Cycle — **RESOLVED: +54/+55 in event index data**, uint16 BE, seconds (65500 = disabled) | RESOLVED | 2026-03-02 | |
|
||||
| **SUB 0x06 purpose — RESOLVED: event storage range.** Previously labeled "CHANNEL CONFIG READ". 4-11-26 MITM capture confirms it returns first/last stored event keys in the final 8 bytes of the 36-byte response. Used by Blastware as part of the erase-all verification step. | RESOLVED | 2026-04-11 | |
|
||||
| **Erase-all command sequence — RESOLVED.** SUB 0xA3 (begin) + 0x1C (monitor status) + 0x06 (storage range) + 0xA2 (confirm). Confirmed from 4-11-26 MITM capture. All frames standard `build_bw_frame`, token=0xFE. | RESOLVED | 2026-04-11 | |
|
||||
| **ACH inbound server — RESOLVED.** `bridges/ach_server.py` implements full inbound ACH pipeline. `--clear-after-download` flag for delete-after-upload workflow. Post-erase key-reuse detection via `max_downloaded_key` high-water mark. | RESOLVED | 2026-04-11 | |
|
||||
| **Sensor Check dropdown byte location** — byte offset in 1A compliance config payload for the "Sensor Check: Before monitoring / After each event / Disabled" setting is NOT YET LOCATED. Confirmed: unit always runs with "Before monitoring" set. Need a capture with "Disabled" to diff. | MEDIUM | 2026-04-08 | Still open |
|
||||
| **RV55 DCD/DTR default** — newer Sierra Wireless RV55 firmware does not assert DCD/DTR by default, so the MiniMate Plus never detects TCP disconnect and stays idle instead of resuming monitoring. Root cause: RV55 ACEmanager `DCD Control` setting. Workaround not yet found. | MEDIUM | 2026-04-11 | Still open |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+634
@@ -0,0 +1,634 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
experiments.py — Protocol minimization experiments for MiniMate Plus.
|
||||
|
||||
Goal: figure out which steps in Blastware's sequences are truly required vs.
|
||||
cargo-culted, so we can build a faster, smarter client.
|
||||
|
||||
Each experiment is self-contained (opens its own TCP connection) and reports
|
||||
PASS / FAIL / INCONCLUSIVE with timing and notes.
|
||||
|
||||
Usage:
|
||||
python experiments.py [--host IP] [--port PORT] [exp1 exp2 ...]
|
||||
|
||||
Run all: python experiments.py
|
||||
Run specific: python experiments.py cold_status fast_event_count no_5a
|
||||
|
||||
Available experiments
|
||||
---------------------
|
||||
cold_status EXP1 Monitor status (1C) with NO prior POLL
|
||||
fast_event_count EXP2 Event count via POLL+08 only — skip identity reads
|
||||
no_5a EXP3 Event record (0C) without bulk waveform stream (5A)
|
||||
skip_1e EXP4 0A/0C directly with cached key — skip initial 1E
|
||||
fewer_polls EXP5 Only 1 POLL before 5A instead of Blastware's 3
|
||||
compliance_only EXP6 Write compliance ONLY (71x3→72), skip event index+trigger+waveform
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.WARNING, # experiment output is via print(); set DEBUG for wire trace
|
||||
format="%(asctime)s %(levelname)-7s %(name)-20s %(message)s",
|
||||
datefmt="%H:%M:%S",
|
||||
)
|
||||
log = logging.getLogger("experiments")
|
||||
|
||||
# ── Imports ───────────────────────────────────────────────────────────────────
|
||||
|
||||
from minimateplus.transport import TcpTransport
|
||||
from minimateplus.protocol import (
|
||||
MiniMateProtocol,
|
||||
ProtocolError,
|
||||
TimeoutError as ProtoTimeout,
|
||||
SUB_MONITOR_STATUS,
|
||||
SUB_SERIAL_NUMBER,
|
||||
SUB_FULL_CONFIG,
|
||||
SUB_EVENT_INDEX,
|
||||
SUB_COMPLIANCE,
|
||||
SUB_WRITE_CONFIRM_A,
|
||||
SUB_WRITE_CONFIRM_B,
|
||||
)
|
||||
from minimateplus.framing import build_bw_frame, SESSION_RESET
|
||||
from minimateplus.client import (
|
||||
MiniMateClient,
|
||||
_decode_compliance_config_into,
|
||||
_encode_compliance_config,
|
||||
)
|
||||
from minimateplus.models import DeviceInfo
|
||||
|
||||
|
||||
DEFAULT_HOST = "63.43.212.232"
|
||||
DEFAULT_PORT = 9034
|
||||
|
||||
|
||||
# ── Result container ──────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class Result:
|
||||
name: str
|
||||
outcome: str # "PASS" | "FAIL" | "INCONCLUSIVE"
|
||||
elapsed: float = 0.0
|
||||
notes: str = ""
|
||||
details: dict = field(default_factory=dict)
|
||||
|
||||
def __str__(self) -> str:
|
||||
sym = {"PASS": "✅", "FAIL": "❌", "INCONCLUSIVE": "⚠️ "}.get(self.outcome, "?")
|
||||
lines = [f" {sym} {self.outcome:13s} {self.name} ({self.elapsed:.1f}s)"]
|
||||
if self.notes:
|
||||
lines.append(f" {self.notes}")
|
||||
for k, v in self.details.items():
|
||||
lines.append(f" {k}: {v}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ── Connection helpers ────────────────────────────────────────────────────────
|
||||
|
||||
def connect_proto(host: str, port: int, timeout: float = 15.0) -> tuple[TcpTransport, MiniMateProtocol]:
|
||||
"""Open a raw TCP connection and return (transport, proto) without any handshake."""
|
||||
t = TcpTransport(host, port)
|
||||
t.connect()
|
||||
proto = MiniMateProtocol(t, recv_timeout=timeout)
|
||||
return t, proto
|
||||
|
||||
|
||||
def connect_client(host: str, port: int, timeout: float = 30.0) -> tuple[MiniMateClient, DeviceInfo]:
|
||||
"""Open a MiniMateClient and run the full connect() handshake."""
|
||||
transport = TcpTransport(host, port)
|
||||
client = MiniMateClient(transport=transport, timeout=timeout)
|
||||
client.open()
|
||||
info = client.connect()
|
||||
return client, info
|
||||
|
||||
|
||||
# ── Experiment runner ─────────────────────────────────────────────────────────
|
||||
|
||||
def run(name: str, fn, *args, **kwargs) -> Result:
|
||||
print(f"\n{'─'*60}")
|
||||
print(f" Running: {name}")
|
||||
print(f"{'─'*60}")
|
||||
t0 = time.time()
|
||||
try:
|
||||
outcome, notes, details = fn(*args, **kwargs)
|
||||
except Exception as exc:
|
||||
outcome = "FAIL"
|
||||
notes = f"Uncaught exception: {exc}"
|
||||
details = {}
|
||||
log.exception("Experiment %s raised:", name)
|
||||
elapsed = time.time() - t0
|
||||
r = Result(name=name, outcome=outcome, elapsed=elapsed, notes=notes, details=details)
|
||||
print(str(r))
|
||||
return r
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP1 — Monitor status (1C) with NO prior POLL
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# Blastware always does a full POLL handshake before any other command.
|
||||
# We want to know: can we query SUB 1C (battery, memory, monitoring state)
|
||||
# cold, with only a SESSION_RESET signal and no POLL at all?
|
||||
#
|
||||
# If PASS: status checks become near-instant (no ~1s POLL round-trip).
|
||||
# If FAIL: we need POLL first, but maybe we can cache it.
|
||||
|
||||
def exp_cold_status(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""SUB 1C without any POLL — just SESSION_RESET + 1C probe + 1C data."""
|
||||
t, proto = connect_proto(host, port)
|
||||
try:
|
||||
print(" Sending SESSION_RESET only (no POLL)")
|
||||
t.write(SESSION_RESET)
|
||||
time.sleep(0.1)
|
||||
|
||||
print(" Sending SUB 1C probe (no POLL first)…")
|
||||
rsp_sub = (0xFF - SUB_MONITOR_STATUS) & 0xFF # 0xE3
|
||||
t.write(build_bw_frame(SUB_MONITOR_STATUS, 0x00))
|
||||
probe = proto._recv_one(expected_sub=rsp_sub, timeout=8.0)
|
||||
print(f" 1C probe OK page_key=0x{probe.page_key:04X} data={probe.data.hex()}")
|
||||
|
||||
t.write(build_bw_frame(SUB_MONITOR_STATUS, 0x2C))
|
||||
data_rsp = proto._recv_one(expected_sub=rsp_sub, timeout=8.0)
|
||||
|
||||
section = data_rsp.data
|
||||
print(f" 1C data OK {len(section)} bytes hex: {section.hex()}")
|
||||
|
||||
# Decode battery + memory from the end of the section
|
||||
details = {"raw_bytes": len(section)}
|
||||
if len(section) >= 10:
|
||||
batt_raw = struct.unpack_from(">H", section, len(section) - 10)[0]
|
||||
mem_total = struct.unpack_from(">I", section, len(section) - 8)[0]
|
||||
mem_free = struct.unpack_from(">I", section, len(section) - 4)[0]
|
||||
is_monitoring = (section[1] == 0x10)
|
||||
details["battery_v"] = f"{batt_raw / 100:.2f} V"
|
||||
details["memory_total"] = f"{mem_total:,} bytes"
|
||||
details["memory_free"] = f"{mem_free:,} bytes"
|
||||
details["monitoring"] = is_monitoring
|
||||
print(f" battery={batt_raw/100:.2f}V mem_free={mem_free:,} monitoring={is_monitoring}")
|
||||
|
||||
return "PASS", "SUB 1C responded without any POLL — cold status read works!", details
|
||||
|
||||
except ProtoTimeout:
|
||||
return "FAIL", "Device did not respond to 1C without POLL (timeout)", {}
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"Protocol error: {exc}", {}
|
||||
finally:
|
||||
t.disconnect()
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP2 — Fast event count: POLL + SUB 08 only (skip identity reads)
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# Blastware's connect() does: POLL → 15 → 01 → 1A → 08
|
||||
# We want to know: can we skip 15/01/1A and go straight from POLL to 08?
|
||||
#
|
||||
# Reading identity (15, 01) and full compliance (1A, ~2126 bytes over TCP)
|
||||
# takes several seconds each connect. If we only need event count, skipping
|
||||
# them would be a huge win.
|
||||
#
|
||||
# If PASS: fast status poll = POLL + 08 only (~2 round trips vs ~8+).
|
||||
|
||||
def exp_fast_event_count(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""POLL startup → SUB 08 only, skip serial/config/compliance reads."""
|
||||
t, proto = connect_proto(host, port)
|
||||
try:
|
||||
print(" Running startup (POLL only)…")
|
||||
proto.startup()
|
||||
print(" POLL OK — now reading SUB 08 (event index) directly…")
|
||||
|
||||
idx_raw = proto.read_event_index()
|
||||
print(f" SUB 08 OK {len(idx_raw)} bytes")
|
||||
|
||||
# Try to decode event count from SUB 08 payload
|
||||
# The raw block is 88 bytes; bytes [3:7] may be a count (uint32 BE)
|
||||
details = {"idx_raw_len": len(idx_raw)}
|
||||
if len(idx_raw) >= 7:
|
||||
count_candidate = struct.unpack_from(">I", idx_raw, 3)[0]
|
||||
details["count_candidate"] = count_candidate
|
||||
print(f" idx[3:7] as uint32 BE = {count_candidate} (may or may not be event count)")
|
||||
|
||||
# Also verify we can read 1E without the identity reads having been done
|
||||
print(" Reading 1E (event header) to confirm event access works…")
|
||||
key4, data8 = proto.read_event_first()
|
||||
is_empty = data8[4:8] == b"\x00\x00\x00\x00"
|
||||
details["first_key"] = key4.hex()
|
||||
details["is_empty"] = is_empty
|
||||
print(f" 1E OK key={key4.hex()} empty={is_empty}")
|
||||
|
||||
return "PASS", "POLL+08+1E all work without identity reads (15/01/1A skipped)", details
|
||||
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"Protocol error: {exc}", {}
|
||||
finally:
|
||||
t.disconnect()
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP3 — Get event record (0C) without bulk waveform stream (5A)
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# Blastware's event download = 1E → 0A → 1E-arm → 0C → 1F(dl) → POLL×3 → 5A → 1F(browse)
|
||||
#
|
||||
# The 5A bulk stream is the slow part (several large frames, ~1s+ per event).
|
||||
# We only need 5A for: client, operator, seis_loc, notes (not in 0C).
|
||||
# If you don't need those fields, can we do: 1E → 0A → 0C → 1F(browse) ?
|
||||
#
|
||||
# Two variants tested:
|
||||
# 3a: Skip 1E-arm AND 5A — just 0A → 0C → 1F(browse)
|
||||
# 3b: Include 1E-arm but skip 5A+POLL — 0A → 1E-arm → 0C → 1F(browse)
|
||||
#
|
||||
# If PASS: event peak values available without the slow bulk stream.
|
||||
# If FAIL on 3a but PASS on 3b: 1E-arm required even without 5A.
|
||||
|
||||
def exp_no_5a(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""Event record via 0A→0C without 5A or POLL×3. Tests both with and without 1E-arm."""
|
||||
t, proto = connect_proto(host, port)
|
||||
try:
|
||||
print(" Startup (POLL)…")
|
||||
proto.startup()
|
||||
|
||||
# Get the first event key via 1E
|
||||
key4, data8 = proto.read_event_first()
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
return "INCONCLUSIVE", "Device has no stored events — cannot test", {}
|
||||
print(f" First event key: {key4.hex()}")
|
||||
|
||||
details: dict = {"key": key4.hex()}
|
||||
|
||||
# ── Variant 3a: 0A → 0C → 1F(browse), no 1E-arm ─────────────────────
|
||||
print("\n [3a] 0A → 0C → 1F(browse) (NO 1E-arm, NO 5A)")
|
||||
try:
|
||||
_hdr, rec_len = proto.read_waveform_header(key4)
|
||||
print(f" 0A OK rec_len=0x{rec_len:02X}")
|
||||
record_3a = proto.read_waveform_record(key4)
|
||||
print(f" 0C OK {len(record_3a)} bytes")
|
||||
# Check for recognizable content
|
||||
has_tran = b"Tran" in record_3a
|
||||
has_vert = b"Vert" in record_3a
|
||||
has_long = b"Long" in record_3a
|
||||
print(f" 0C content check: Tran={has_tran} Vert={has_vert} Long={has_long}")
|
||||
details["3a_0c_bytes"] = len(record_3a)
|
||||
details["3a_has_peaks"] = has_tran and has_vert and has_long
|
||||
|
||||
# Now try browse 1F without any 5A
|
||||
key4_next, data8_next = proto.advance_event(browse=True)
|
||||
null_sentinel = data8_next[4:8] == b"\x00\x00\x00\x00"
|
||||
print(f" 1F(browse) → key={key4_next.hex()} null={null_sentinel}")
|
||||
details["3a_1f_ok"] = True
|
||||
details["3a_outcome"] = "PASS"
|
||||
except ProtocolError as exc:
|
||||
print(f" 3a FAILED: {exc}")
|
||||
details["3a_outcome"] = f"FAIL: {exc}"
|
||||
# Try to recover by reconnecting for 3b
|
||||
t.disconnect()
|
||||
t2, proto2 = connect_proto(host, port)
|
||||
proto2.startup()
|
||||
key4, data8 = proto2.read_event_first()
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
return "FAIL", f"3a failed and device empty on retry: {exc}", details
|
||||
t, proto = t2, proto2
|
||||
|
||||
# ── Variant 3b: 0A → 1E-arm → 0C → 1F(browse), no 5A ───────────────
|
||||
print("\n [3b] 0A → 1E-arm(0xFE) → 0C → 1F(browse) (NO POLL×3, NO 5A)")
|
||||
try:
|
||||
_hdr, rec_len = proto.read_waveform_header(key4)
|
||||
print(f" 0A OK rec_len=0x{rec_len:02X}")
|
||||
|
||||
# 1E download-arm (token=0xFE) between 0A and 0C
|
||||
proto.read_event_first(token=0xFE)
|
||||
print(" 1E-arm OK")
|
||||
|
||||
record_3b = proto.read_waveform_record(key4)
|
||||
print(f" 0C OK {len(record_3b)} bytes")
|
||||
has_tran = b"Tran" in record_3b
|
||||
print(f" 0C content check: Tran={has_tran} Vert={b'Vert' in record_3b}")
|
||||
details["3b_0c_bytes"] = len(record_3b)
|
||||
details["3b_has_peaks"] = has_tran
|
||||
|
||||
# Browse 1F without 5A / POLL×3
|
||||
key4_next2, data8_next2 = proto.advance_event(browse=True)
|
||||
null_sentinel2 = data8_next2[4:8] == b"\x00\x00\x00\x00"
|
||||
print(f" 1F(browse) → key={key4_next2.hex()} null={null_sentinel2}")
|
||||
details["3b_1f_ok"] = True
|
||||
details["3b_outcome"] = "PASS"
|
||||
except ProtocolError as exc:
|
||||
print(f" 3b FAILED: {exc}")
|
||||
details["3b_outcome"] = f"FAIL: {exc}"
|
||||
|
||||
# Summarize
|
||||
a_ok = details.get("3a_outcome") == "PASS"
|
||||
b_ok = details.get("3b_outcome") == "PASS"
|
||||
if a_ok:
|
||||
return "PASS", "3a: 0A→0C works with NO 1E-arm and NO 5A. Huge speedup possible!", details
|
||||
elif b_ok:
|
||||
return "PASS", "3b: 0A→1E-arm→0C works without 5A (1E-arm still needed before 0C)", details
|
||||
else:
|
||||
return "FAIL", "Both 3a and 3b failed — 5A may be required for device state", details
|
||||
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"Protocol error during setup: {exc}", {}
|
||||
finally:
|
||||
try:
|
||||
t.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP4 — Skip initial 1E if we already know the event key
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# In Blastware, every session starts with 1E to discover the first key.
|
||||
# But if we already fetched and cached the event keys from a previous session,
|
||||
# can we skip 1E entirely and go straight to 0A(cached_key)?
|
||||
#
|
||||
# Practical use case: we poll the device every N minutes. We already know
|
||||
# all the event keys from last time. On re-connect, can we go direct to 0A?
|
||||
#
|
||||
# If PASS: subsequent polls that don't add new events can skip 1E discovery.
|
||||
|
||||
def exp_skip_1e(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""Get the first event key, disconnect, reconnect, go straight to 0A (skip 1E)."""
|
||||
# Phase 1: get the key
|
||||
t, proto = connect_proto(host, port)
|
||||
try:
|
||||
proto.startup()
|
||||
key4, data8 = proto.read_event_first()
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
return "INCONCLUSIVE", "No events stored — cannot test", {}
|
||||
print(f" Phase 1: got event key = {key4.hex()}")
|
||||
finally:
|
||||
t.disconnect()
|
||||
time.sleep(0.5)
|
||||
|
||||
# Phase 2: fresh connection, skip 1E, go straight to 0A with cached key
|
||||
t2, proto2 = connect_proto(host, port)
|
||||
try:
|
||||
print(" Phase 2: fresh connection — startup + 0A directly (no 1E)")
|
||||
proto2.startup()
|
||||
|
||||
_hdr, rec_len = proto2.read_waveform_header(key4)
|
||||
print(f" 0A OK rec_len=0x{rec_len:02X}")
|
||||
|
||||
record = proto2.read_waveform_record(key4)
|
||||
has_peaks = b"Tran" in record
|
||||
print(f" 0C OK {len(record)} bytes has_peaks={has_peaks}")
|
||||
|
||||
details = {
|
||||
"cached_key": key4.hex(),
|
||||
"0c_bytes": len(record),
|
||||
"has_peaks": has_peaks,
|
||||
}
|
||||
return "PASS", "0A works with cached key — 1E discovery can be skipped on known sessions", details
|
||||
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"0A failed with cached key (device needs 1E first?): {exc}", {"key": key4.hex()}
|
||||
finally:
|
||||
t2.disconnect()
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP5 — Fewer POLLs before 5A (try POLL×1 instead of Blastware's POLL×3)
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# Blastware always sends 3 full POLL probe+data cycles between 1F and 5A.
|
||||
# Each POLL is a round trip. Can we get away with just 1?
|
||||
#
|
||||
# WARNING: If POLL×1 fails, the device may be in a bad state. We try to
|
||||
# recover with an extra POLL×2 and a fresh 5A attempt. Even on failure we
|
||||
# try to leave the device in a usable state.
|
||||
#
|
||||
# Strategy: run the full event sequence up to 1F(download), then try 5A
|
||||
# with only 1 POLL. If 5A responds → PASS. If timeout → try 2 more POLLs
|
||||
# and check if the device recovers.
|
||||
|
||||
def exp_fewer_polls(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""Full sequence to 1F, then only 1 POLL before 5A (Blastware does 3)."""
|
||||
t, proto = connect_proto(host, port)
|
||||
try:
|
||||
proto.startup()
|
||||
|
||||
key4, data8 = proto.read_event_first()
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
return "INCONCLUSIVE", "No events stored — cannot test", {}
|
||||
print(f" Event key: {key4.hex()}")
|
||||
|
||||
# Full setup: 0A → 1E-arm → 0C → 1F(download)
|
||||
_hdr, rec_len = proto.read_waveform_header(key4)
|
||||
print(f" 0A OK rec_len=0x{rec_len:02X}")
|
||||
proto.read_event_first(token=0xFE) # 1E-arm
|
||||
print(" 1E-arm OK")
|
||||
proto.read_waveform_record(key4)
|
||||
print(" 0C OK")
|
||||
arm_key4, _ = proto.advance_event(browse=False) # 1F(download) — arms 5A
|
||||
print(f" 1F(download) OK arm_key={arm_key4.hex()}")
|
||||
|
||||
# Only 1 POLL (Blastware does 3)
|
||||
print(" Sending 1 POLL (instead of 3)…")
|
||||
proto.poll()
|
||||
print(" POLL ok — now probing 5A…")
|
||||
|
||||
try:
|
||||
frames = proto.read_bulk_waveform_stream(key4, stop_after_metadata=True, max_chunks=12)
|
||||
print(f" 5A OK after 1 POLL — {len(frames)} frames received")
|
||||
details = {"poll_count": 1, "frames": len(frames)}
|
||||
return "PASS", "5A works with only 1 POLL (saved 2 round-trips per event)!", details
|
||||
|
||||
except ProtoTimeout:
|
||||
print(" 5A timed out after 1 POLL — device needs more POLLs")
|
||||
# Attempt recovery: send 2 more POLLs and see if 5A then works
|
||||
print(" Attempting recovery: 2 more POLLs…")
|
||||
try:
|
||||
proto.poll()
|
||||
proto.poll()
|
||||
frames2 = proto.read_bulk_waveform_stream(key4, stop_after_metadata=True, max_chunks=12)
|
||||
print(f" 5A worked after total 3 POLLs ({len(frames2)} frames)")
|
||||
return "FAIL", "5A needs 3 POLLs — 1 is not enough (recovery confirmed 3 still works)", {
|
||||
"poll_count_tried": 1, "recovery_polls": 3, "recovery_frames": len(frames2)
|
||||
}
|
||||
except ProtocolError as exc2:
|
||||
return "FAIL", f"5A failed even after 3 total POLLs — device may need reconnect: {exc2}", {}
|
||||
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"Setup failed: {exc}", {}
|
||||
finally:
|
||||
t.disconnect()
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# EXP6 — Compliance-only write (71×3→72), skip event index + trigger + waveform
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
#
|
||||
# Blastware's full write sequence: 68→73 | 71×3→72 | 82→83 | 69→74→72
|
||||
# We want to know: can we write ONLY the compliance block (71×3→72)?
|
||||
#
|
||||
# Test procedure:
|
||||
# 1. Read current compliance config (SUB 1A)
|
||||
# 2. Patch the "notes" field to a test marker
|
||||
# 3. Write ONLY 71×3→72 (skip 68, 73, 82, 83, 69, 74, final 72)
|
||||
# 4. Read back (SUB 1A) and verify the change was written
|
||||
# 5. Restore original value
|
||||
#
|
||||
# If PASS: we can push individual config fields without touching event index,
|
||||
# trigger config, or waveform data — huge simplification.
|
||||
# If FAIL: the device needs the full write sequence (may reject partial write).
|
||||
#
|
||||
# SAFETY: We restore original data in a finally block. If the restore write
|
||||
# fails, the device will have the test marker in "notes" — harmless but visible.
|
||||
|
||||
_EXP6_MARKER = "[exp6-test]"
|
||||
|
||||
def exp_compliance_only(host: str, port: int) -> tuple[str, str, dict]:
|
||||
"""Write compliance block alone (71×3→72), verify, and restore."""
|
||||
client, info = connect_client(host, port)
|
||||
original_raw: Optional[bytes] = None
|
||||
try:
|
||||
proto = client._proto
|
||||
if proto is None:
|
||||
return "FAIL", "Could not get protocol handle from client", {}
|
||||
|
||||
# 1. Read current compliance
|
||||
print(" Reading current compliance config (SUB 1A)…")
|
||||
original_raw = proto.read_compliance_config()
|
||||
print(f" Got {len(original_raw)} bytes of compliance config")
|
||||
|
||||
# Find current notes value for display
|
||||
info_obj = DeviceInfo()
|
||||
_decode_compliance_config_into(original_raw, info_obj)
|
||||
cc = info_obj.compliance_config
|
||||
orig_notes = cc.notes if cc else "(unknown)"
|
||||
print(f" Current notes field: {orig_notes!r}")
|
||||
|
||||
# 2. Build modified payload with test marker in notes
|
||||
test_notes = _EXP6_MARKER
|
||||
modified_raw = _encode_compliance_config(
|
||||
original_raw,
|
||||
notes=test_notes,
|
||||
)
|
||||
print(f" Encoded modified compliance payload ({len(modified_raw)} bytes)")
|
||||
print(f" Patching notes: {orig_notes!r} → {test_notes!r}")
|
||||
|
||||
# 3. Write ONLY the compliance block: 71×3 → 72
|
||||
print(" Writing compliance ONLY (71×3→72) — skipping 68/73/82/83/69/74…")
|
||||
proto.write_compliance_config_raw(modified_raw)
|
||||
print(" Write complete — device acked 71×3→72")
|
||||
|
||||
# 4. Read back and verify
|
||||
print(" Reading back compliance config to verify…")
|
||||
readback_raw = proto.read_compliance_config()
|
||||
readback_info = DeviceInfo()
|
||||
_decode_compliance_config_into(readback_raw, readback_info)
|
||||
rb_cc = readback_info.compliance_config
|
||||
readback_notes = rb_cc.notes if rb_cc else "(decode failed)"
|
||||
print(f" Read-back notes: {readback_notes!r}")
|
||||
|
||||
write_worked = (readback_notes == test_notes)
|
||||
print(f" Write verified: {write_worked}")
|
||||
|
||||
details = {
|
||||
"original_notes": orig_notes,
|
||||
"written_notes": test_notes,
|
||||
"readback_notes": readback_notes,
|
||||
"write_verified": write_worked,
|
||||
}
|
||||
|
||||
if write_worked:
|
||||
return "PASS", "Compliance-only write works! No event index or trigger writes needed.", details
|
||||
else:
|
||||
return "FAIL", f"Write was not reflected in read-back (got {readback_notes!r})", details
|
||||
|
||||
except ProtocolError as exc:
|
||||
return "FAIL", f"Protocol error: {exc}", {}
|
||||
|
||||
finally:
|
||||
# Restore original compliance data regardless of outcome
|
||||
if original_raw is not None:
|
||||
print(" Restoring original compliance config…")
|
||||
try:
|
||||
proto2 = client._proto
|
||||
if proto2:
|
||||
proto2.write_compliance_config_raw(
|
||||
_encode_compliance_config(original_raw) # no-op patch = verbatim
|
||||
)
|
||||
print(" Restore complete")
|
||||
else:
|
||||
print(" WARNING: protocol handle gone — could not restore")
|
||||
except Exception as exc_r:
|
||||
print(f" WARNING: restore failed: {exc_r}")
|
||||
client.close()
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
# Registry + main
|
||||
# ══════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
EXPERIMENTS = {
|
||||
"cold_status": ("EXP1", exp_cold_status, "Monitor status (1C) with no POLL"),
|
||||
"fast_event_count": ("EXP2", exp_fast_event_count, "Event count via POLL+08, skip identity reads"),
|
||||
"no_5a": ("EXP3", exp_no_5a, "Event record (0C) without bulk waveform (5A)"),
|
||||
"skip_1e": ("EXP4", exp_skip_1e, "0A/0C with cached key — skip initial 1E"),
|
||||
"fewer_polls": ("EXP5", exp_fewer_polls, "1 POLL before 5A instead of Blastware's 3"),
|
||||
"compliance_only": ("EXP6", exp_compliance_only, "Compliance-only write (71×3→72), no other blocks"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description="MiniMate Plus protocol minimization experiments")
|
||||
ap.add_argument("--host", default=DEFAULT_HOST)
|
||||
ap.add_argument("--port", type=int, default=DEFAULT_PORT)
|
||||
ap.add_argument("--debug", action="store_true", help="Enable DEBUG wire logging")
|
||||
ap.add_argument("experiments", nargs="*",
|
||||
help=f"Which to run (default: all). Choices: {', '.join(EXPERIMENTS)}")
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.debug:
|
||||
logging.getLogger().setLevel(logging.DEBUG)
|
||||
|
||||
which = args.experiments or list(EXPERIMENTS.keys())
|
||||
unknown = [e for e in which if e not in EXPERIMENTS]
|
||||
if unknown:
|
||||
print(f"Unknown experiments: {unknown}")
|
||||
print(f"Available: {', '.join(EXPERIMENTS)}")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"\n{'═'*60}")
|
||||
print(f" MiniMate Plus Protocol Minimization Experiments")
|
||||
print(f" Target: {args.host}:{args.port}")
|
||||
print(f" Running: {', '.join(which)}")
|
||||
print(f"{'═'*60}")
|
||||
|
||||
results: list[Result] = []
|
||||
for key in which:
|
||||
tag, fn, desc = EXPERIMENTS[key]
|
||||
label = f"{tag}: {desc}"
|
||||
r = run(label, fn, args.host, args.port)
|
||||
results.append(r)
|
||||
time.sleep(1.5) # brief pause between experiments — let device settle
|
||||
|
||||
print(f"\n\n{'═'*60}")
|
||||
print(" SUMMARY")
|
||||
print(f"{'═'*60}")
|
||||
for r in results:
|
||||
sym = {"PASS": "✅", "FAIL": "❌", "INCONCLUSIVE": "⚠️ "}.get(r.outcome, "?")
|
||||
print(f" {sym} {r.outcome:13s} {r.name}")
|
||||
print(f"{'═'*60}")
|
||||
|
||||
passed = sum(1 for r in results if r.outcome == "PASS")
|
||||
failed = sum(1 for r in results if r.outcome == "FAIL")
|
||||
skipped = sum(1 for r in results if r.outcome == "INCONCLUSIVE")
|
||||
print(f" {passed} passed {failed} failed {skipped} inconclusive")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except KeyboardInterrupt:
|
||||
print("\nInterrupted.")
|
||||
sys.exit(0)
|
||||
@@ -20,8 +20,8 @@ Typical usage (TCP / modem):
|
||||
"""
|
||||
|
||||
from .client import MiniMateClient
|
||||
from .models import DeviceInfo, Event
|
||||
from .models import DeviceInfo, Event, MonitorLogEntry
|
||||
from .transport import SerialTransport, TcpTransport
|
||||
|
||||
__version__ = "0.1.0"
|
||||
__all__ = ["MiniMateClient", "DeviceInfo", "Event", "SerialTransport", "TcpTransport"]
|
||||
__all__ = ["MiniMateClient", "DeviceInfo", "Event", "MonitorLogEntry", "SerialTransport", "TcpTransport"]
|
||||
|
||||
+394
-63
@@ -28,6 +28,7 @@ Example (TCP / modem):
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import struct
|
||||
from typing import Optional
|
||||
@@ -37,6 +38,7 @@ from .models import (
|
||||
ComplianceConfig,
|
||||
DeviceInfo,
|
||||
Event,
|
||||
MonitorLogEntry,
|
||||
MonitorStatus,
|
||||
PeakValues,
|
||||
ProjectInfo,
|
||||
@@ -179,8 +181,21 @@ class MiniMateClient:
|
||||
log.info("connect: reading event index (SUB 08)")
|
||||
try:
|
||||
idx_raw = proto.read_event_index()
|
||||
device_info.event_count = _decode_event_count(idx_raw)
|
||||
log.info("connect: device has %d stored event(s)", device_info.event_count)
|
||||
# NOTE: _decode_event_count reads data[10:12] from the SUB 08 payload,
|
||||
# which was believed to be the stored event count. Empirically it turns
|
||||
# out to be a monotonically-increasing "total events ever recorded" counter
|
||||
# that does NOT decrement when events are erased — confirmed 2026-04-13:
|
||||
# device reported 6 via SUB 08 while list_event_keys() returned 0 (empty).
|
||||
# We preserve the raw read here for the index data but do NOT use this
|
||||
# count for logic; ach_server uses list_event_keys() as the authoritative
|
||||
# source instead.
|
||||
_raw_idx_count = _decode_event_count(idx_raw)
|
||||
log.info(
|
||||
"connect: SUB 08 index count=%d (lifetime counter, not current storage)",
|
||||
_raw_idx_count,
|
||||
)
|
||||
# Leave device_info.event_count as None — callers should use
|
||||
# list_event_keys() to get the actual current event count.
|
||||
except ProtocolError as exc:
|
||||
log.warning("connect: event index read failed: %s — continuing", exc)
|
||||
|
||||
@@ -216,8 +231,8 @@ class MiniMateClient:
|
||||
log.warning("count_events: 1E failed: %s — returning 0", exc)
|
||||
return 0
|
||||
|
||||
log.warning(
|
||||
"count_events: 1E → key=%s data8=%s trailing=%s",
|
||||
log.debug(
|
||||
"count_events: 1E -> key=%s data8=%s trailing=%s",
|
||||
key4.hex(), data8.hex(), data8[4:8].hex(),
|
||||
)
|
||||
|
||||
@@ -241,8 +256,8 @@ class MiniMateClient:
|
||||
break
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
log.warning(
|
||||
"count_events: 1F [iter %d] → key=%s data8=%s trailing=%s",
|
||||
log.debug(
|
||||
"count_events: 1F [iter %d] -> key=%s data8=%s trailing=%s",
|
||||
count, key4.hex(), data8.hex(), data8[4:8].hex(),
|
||||
)
|
||||
except ProtocolError as exc:
|
||||
@@ -252,7 +267,188 @@ class MiniMateClient:
|
||||
log.info("count_events: %d event(s) found via 1E/1F chain", count)
|
||||
return count
|
||||
|
||||
def get_events(self, full_waveform: bool = False, debug: bool = False, stop_after_index: Optional[int] = None) -> list[Event]:
|
||||
def list_event_keys(self) -> list[str]:
|
||||
"""
|
||||
Return the hex key strings for all stored events without downloading
|
||||
any waveform data. Uses the same browse-mode 1E -> 0A -> 1F walk as
|
||||
count_events() but collects the key at each step.
|
||||
|
||||
Returns:
|
||||
List of 8-char lowercase hex strings, e.g. ["01110000", "0111245a"].
|
||||
Empty list if device has no stored events or 1E fails.
|
||||
"""
|
||||
proto = self._require_proto()
|
||||
try:
|
||||
key4, data8 = proto.read_event_first()
|
||||
except ProtocolError as exc:
|
||||
log.warning("list_event_keys: 1E failed: %s -- returning []", exc)
|
||||
return []
|
||||
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
log.info("list_event_keys: device is empty")
|
||||
return []
|
||||
|
||||
keys: list[str] = []
|
||||
while data8[4:8] != b"\x00\x00\x00\x00":
|
||||
keys.append(key4.hex())
|
||||
try:
|
||||
proto.read_waveform_header(key4)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"list_event_keys: 0A failed for key=%s: %s -- stopping",
|
||||
key4.hex(), exc,
|
||||
)
|
||||
break
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
log.debug(
|
||||
"list_event_keys: 1F -> key=%s trailing=%s",
|
||||
key4.hex(), data8[4:8].hex(),
|
||||
)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"list_event_keys: 1F failed after %d event(s): %s -- stopping",
|
||||
len(keys), exc,
|
||||
)
|
||||
break
|
||||
|
||||
log.info("list_event_keys: %d key(s): %s", len(keys), keys)
|
||||
return keys
|
||||
|
||||
def get_monitor_log_entries(
|
||||
self,
|
||||
skip_keys: Optional[set] = None,
|
||||
) -> list[MonitorLogEntry]:
|
||||
"""
|
||||
Collect all monitor log entries (partial records, type 0x2C) from the
|
||||
device using the browse-mode 1E → 0A → 1F walk.
|
||||
|
||||
This is the fast path for monitor log data. No 0C or 5A commands are
|
||||
issued — all available monitor log information is in the 0x0A response
|
||||
header alone.
|
||||
|
||||
Full triggered events (0x0A response type 0x46) are silently skipped.
|
||||
Only partial records (type 0x2C) are returned as MonitorLogEntry objects.
|
||||
|
||||
Confirmed from 4-11-26 MITM capture: Blastware's ACH mode performs a
|
||||
full browse walk (Phase 3: 0x0A + 1F × all records) AFTER the triggered-
|
||||
event download phase. The partial records encountered in this walk are
|
||||
the monitor log entries.
|
||||
|
||||
Args:
|
||||
skip_keys: optional set of 8-hex key strings to skip (already seen).
|
||||
Keys in this set still advance the walk (0A + 1F) but are
|
||||
not decoded or returned.
|
||||
|
||||
Returns:
|
||||
List of MonitorLogEntry objects in device storage order.
|
||||
|
||||
Raises:
|
||||
ProtocolError: on unrecoverable communication failure.
|
||||
"""
|
||||
proto = self._require_proto()
|
||||
try:
|
||||
key4, data8 = proto.read_event_first()
|
||||
except ProtocolError as exc:
|
||||
log.warning("get_monitor_log_entries: 1E failed: %s -- returning []", exc)
|
||||
return []
|
||||
|
||||
if data8[4:8] == b"\x00\x00\x00\x00":
|
||||
log.info("get_monitor_log_entries: device is empty")
|
||||
return []
|
||||
|
||||
entries: list[MonitorLogEntry] = []
|
||||
idx = 0
|
||||
|
||||
while data8[4:8] != b"\x00\x00\x00\x00":
|
||||
cur_key = key4
|
||||
key_hex = cur_key.hex()
|
||||
|
||||
try:
|
||||
raw_data, rec_len = proto.read_waveform_header(cur_key)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"get_monitor_log_entries: 0A failed for key=%s: %s -- stopping",
|
||||
key_hex, exc,
|
||||
)
|
||||
break
|
||||
|
||||
# Only decode partial records (0x2C); full records (0x46) are silently skipped.
|
||||
if rec_len < 0x40 and raw_data and (not skip_keys or key_hex not in skip_keys):
|
||||
entry = _decode_0a_partial_header(raw_data, idx, cur_key)
|
||||
if entry is not None:
|
||||
entries.append(entry)
|
||||
log.debug(
|
||||
"get_monitor_log_entries: [%d] key=%s %s → %s",
|
||||
idx, key_hex, entry.start_time, entry.stop_time,
|
||||
)
|
||||
else:
|
||||
log.debug(
|
||||
"get_monitor_log_entries: [%d] key=%s type=0x%02X %s",
|
||||
idx, key_hex, rec_len,
|
||||
"skip (already seen)" if skip_keys and key_hex in skip_keys else "skip (full record)",
|
||||
)
|
||||
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"get_monitor_log_entries: 1F failed after %d record(s): %s -- stopping",
|
||||
idx, exc,
|
||||
)
|
||||
break
|
||||
idx += 1
|
||||
|
||||
log.info(
|
||||
"get_monitor_log_entries: walked %d record(s), found %d monitor log entry(s)",
|
||||
idx, len(entries),
|
||||
)
|
||||
return entries
|
||||
|
||||
def delete_all_events(self) -> None:
|
||||
"""
|
||||
Erase all stored events from the device memory.
|
||||
|
||||
This performs the complete erase sequence confirmed from the 4-11-26
|
||||
MITM capture of a Blastware ACH session:
|
||||
|
||||
1. SUB 0xA3 (begin_erase_all) — initiate erase, token=0xFE
|
||||
2. SUB 0x1C (read_monitor_status) — status read between erase commands
|
||||
3. SUB 0x06 (read_event_storage_range) — verify storage state, token=0xFE
|
||||
4. SUB 0xA2 (confirm_erase_all) — commit erase, token=0xFE
|
||||
|
||||
After this call the device's event memory is empty. The unit returns to
|
||||
its normal operating state automatically (no restart-monitoring call needed).
|
||||
|
||||
Raises:
|
||||
ProtocolError: on timeout or unexpected device response.
|
||||
"""
|
||||
proto = self._require_proto()
|
||||
|
||||
log.info("delete_all_events: step 1/4 — begin erase (SUB 0xA3)")
|
||||
proto.begin_erase_all()
|
||||
log.debug("delete_all_events: 0xA3 ack received")
|
||||
|
||||
log.info("delete_all_events: step 2/4 — monitor status read (SUB 0x1C)")
|
||||
proto.read_monitor_status()
|
||||
log.debug("delete_all_events: 0x1C read complete")
|
||||
|
||||
log.info("delete_all_events: step 3/4 — event storage range read (SUB 0x06)")
|
||||
rng = proto.read_event_storage_range()
|
||||
if len(rng.data) >= 8:
|
||||
first_key = rng.data[-8:-4].hex()
|
||||
last_key = rng.data[-4:].hex()
|
||||
log.info(
|
||||
"delete_all_events: storage range — first=%s last=%s",
|
||||
first_key, last_key,
|
||||
)
|
||||
log.debug("delete_all_events: 0x06 read complete")
|
||||
|
||||
log.info("delete_all_events: step 4/4 — confirm erase (SUB 0xA2)")
|
||||
proto.confirm_erase_all()
|
||||
log.info("delete_all_events: erase confirmed — device memory cleared")
|
||||
|
||||
def get_events(self, full_waveform: bool = False, debug: bool = False, stop_after_index: Optional[int] = None, skip_waveform_for_keys: Optional[set] = None) -> list[Event]:
|
||||
"""
|
||||
Download all stored events from the device using the confirmed
|
||||
1E → 0A → 0C → 5A → 1F event-iterator protocol.
|
||||
@@ -303,6 +499,34 @@ class MiniMateClient:
|
||||
while data8[4:8] != b"\x00\x00\x00\x00":
|
||||
cur_key = key4 # key for this event's 0A/1E-arm/0C/5A calls
|
||||
log.info("get_events: record %d key=%s", idx, cur_key.hex())
|
||||
|
||||
# Fast-advance path: if this key is already downloaded, skip
|
||||
# 1E-arm/0C/POLL/5A entirely. Only 0A + 1F(browse) are needed
|
||||
# to advance the device's internal pointer to the next event.
|
||||
# This is identical to the browse-mode walk in count_events().
|
||||
if skip_waveform_for_keys and cur_key.hex() in skip_waveform_for_keys:
|
||||
log.debug("get_events: key=%s already seen -- fast-advance only", cur_key.hex())
|
||||
try:
|
||||
proto.read_waveform_header(cur_key)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"get_events: 0A failed for key=%s (skip path): %s -- stopping",
|
||||
cur_key.hex(), exc,
|
||||
)
|
||||
break
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
except ProtocolError as exc:
|
||||
log.warning(
|
||||
"get_events: 1F failed for key=%s (skip path): %s -- stopping",
|
||||
cur_key.hex(), exc,
|
||||
)
|
||||
break
|
||||
idx += 1
|
||||
if stop_after_index is not None and idx > stop_after_index:
|
||||
break
|
||||
continue
|
||||
|
||||
ev = Event(index=idx)
|
||||
ev._waveform_key = cur_key
|
||||
|
||||
@@ -426,7 +650,7 @@ class MiniMateClient:
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
log.info(
|
||||
"get_events: 1F(browse) → key=%s trailing=%s",
|
||||
"get_events: 1F(browse) -> key=%s trailing=%s",
|
||||
key4.hex(), data8[4:8].hex(),
|
||||
)
|
||||
except ProtocolError as exc:
|
||||
@@ -481,7 +705,7 @@ class MiniMateClient:
|
||||
try:
|
||||
key4, data8 = proto.advance_event(browse=True)
|
||||
log.info(
|
||||
"get_events: 1F → key=%s trailing=%s",
|
||||
"get_events: 1F -> key=%s trailing=%s",
|
||||
key4.hex(), data8[4:8].hex(),
|
||||
)
|
||||
except ProtocolError as exc:
|
||||
@@ -910,36 +1134,25 @@ def _decode_event_count(data: bytes) -> int:
|
||||
"""
|
||||
Extract stored event count from SUB F7 (EVENT_INDEX_RESPONSE) payload.
|
||||
|
||||
Layout per §7.4 (offsets from data section start):
|
||||
+00: 00 58 09 — total index size or record count ❓
|
||||
+03: 00 00 00 01 — possibly stored event count = 1 ❓
|
||||
Confirmed 2026-04-10 from live BE11529 event index (88 bytes):
|
||||
data[10:12] uint16 BE = stored event count (confirmed: 0x0006 = 6, matches LCD)
|
||||
data[3:7] uint32 BE = 0x00000001 (NOT the count — meaning TBD)
|
||||
|
||||
We use bytes +03..+06 interpreted as uint32 BE as the event count.
|
||||
This is inferred (🔶) — the exact meaning of the first 3 bytes is unclear.
|
||||
Previous implementation read uint32 at offset 3, which returned 1 regardless
|
||||
of how many events were stored.
|
||||
"""
|
||||
if len(data) < 7:
|
||||
if len(data) < 12:
|
||||
log.warning("event index payload too short (%d bytes), assuming 0 events", len(data))
|
||||
return 0
|
||||
|
||||
# Log the full payload so we can reverse-engineer the format
|
||||
log.warning("event_index raw (%d bytes total):", len(data))
|
||||
for off in range(0, len(data), 16):
|
||||
chunk = data[off:off+16]
|
||||
hex_part = " ".join(f"{b:02x}" for b in chunk)
|
||||
asc_part = "".join(chr(b) if 0x20 <= b < 0x7f else "." for b in chunk)
|
||||
log.warning(" [%04x]: %-47s %s", off, hex_part, asc_part)
|
||||
count = struct.unpack_from(">H", data, 10)[0]
|
||||
|
||||
# Try the uint32 at +3 first
|
||||
count = struct.unpack_from(">I", data, 3)[0]
|
||||
|
||||
# Sanity check: MiniMate Plus manual says max ~1000 events
|
||||
# Sanity check: MiniMate Plus max storage is ~1000 events
|
||||
if count > 1000:
|
||||
log.warning(
|
||||
"event count %d looks unreasonably large — clamping to 0", count
|
||||
)
|
||||
log.warning("event count %d looks unreasonably large — clamping to 0", count)
|
||||
return 0
|
||||
|
||||
log.warning("event_index decoded count=%d (uint32 BE at offset +3)", count)
|
||||
log.debug("event_index decoded count=%d (uint16 BE at offset 10)", count)
|
||||
return count
|
||||
|
||||
|
||||
@@ -1499,14 +1712,14 @@ def _encode_compliance_config(
|
||||
log.warning("_encode_compliance_config: anchor not found — cannot write sample_rate")
|
||||
else:
|
||||
struct.pack_into(">H", buf, _anc - 6, sample_rate)
|
||||
log.debug("_encode_compliance_config: sample_rate=%d → offset %d", sample_rate, _anc - 6)
|
||||
log.debug("_encode_compliance_config: sample_rate=%d -> offset %d", sample_rate, _anc - 6)
|
||||
|
||||
if record_time is not None:
|
||||
if _anc < 0 or _anc + 10 > len(buf):
|
||||
log.warning("_encode_compliance_config: anchor not found — cannot write record_time")
|
||||
else:
|
||||
struct.pack_into(">f", buf, _anc + 6, record_time)
|
||||
log.debug("_encode_compliance_config: record_time=%.3f → offset %d", record_time, _anc + 6)
|
||||
log.debug("_encode_compliance_config: record_time=%.3f -> offset %d", record_time, _anc + 6)
|
||||
|
||||
# ── Numeric: channel block (Tran label + unit-string guard) ───────────────
|
||||
_needs_channel = any(
|
||||
@@ -1529,13 +1742,13 @@ def _encode_compliance_config(
|
||||
else:
|
||||
if max_range_geo is not None:
|
||||
struct.pack_into(">f", buf, _tran + 28, max_range_geo)
|
||||
log.debug("_encode_compliance_config: max_range_geo=%.4f → offset %d", max_range_geo, _tran + 28)
|
||||
log.debug("_encode_compliance_config: max_range_geo=%.4f -> offset %d", max_range_geo, _tran + 28)
|
||||
if trigger_level_geo is not None:
|
||||
struct.pack_into(">f", buf, _tran + 34, trigger_level_geo)
|
||||
log.debug("_encode_compliance_config: trigger_level_geo=%.4f → offset %d", trigger_level_geo, _tran + 34)
|
||||
log.debug("_encode_compliance_config: trigger_level_geo=%.4f -> offset %d", trigger_level_geo, _tran + 34)
|
||||
if alarm_level_geo is not None:
|
||||
struct.pack_into(">f", buf, _tran + 42, alarm_level_geo)
|
||||
log.debug("_encode_compliance_config: alarm_level_geo=%.4f → offset %d", alarm_level_geo, _tran + 42)
|
||||
log.debug("_encode_compliance_config: alarm_level_geo=%.4f -> offset %d", alarm_level_geo, _tran + 42)
|
||||
|
||||
# ── ASCII strings (64-byte slot, value at label_pos+22) ───────────────────
|
||||
def _set_string(label: bytes, value: Optional[str]) -> None:
|
||||
@@ -1548,7 +1761,7 @@ def _encode_compliance_config(
|
||||
val_bytes = value.encode("ascii", errors="replace")[:_COMPLIANCE_VALUE_MAX - 1]
|
||||
padded = val_bytes + b"\x00" * (_COMPLIANCE_VALUE_MAX - len(val_bytes))
|
||||
buf[idx + _COMPLIANCE_VALUE_OFFSET : idx + _COMPLIANCE_SLOT_SIZE] = padded
|
||||
log.debug("_encode_compliance_config: %r → %r", label, value)
|
||||
log.debug("_encode_compliance_config: %r -> %r", label, value)
|
||||
|
||||
_set_string(b"Project:", project)
|
||||
_set_string(b"Client:", client_name)
|
||||
@@ -1748,6 +1961,123 @@ def _find_first_string(data: bytes, start: int, end: int, min_len: int) -> Optio
|
||||
|
||||
|
||||
|
||||
def _decode_0a_partial_header(raw_data: bytes, index: int, key4: bytes) -> Optional[MonitorLogEntry]:
|
||||
"""
|
||||
Decode a SUB 0x0A response for a partial (monitor log) record into a
|
||||
MonitorLogEntry.
|
||||
|
||||
Called when read_waveform_header() returns rec_len < 0x40 (i.e. 0x2C = 44).
|
||||
raw_data is the complete data_rsp.data from the protocol layer.
|
||||
|
||||
Layout of raw_data:
|
||||
[0] = 0x2C (partial record type)
|
||||
[1:5] = 0x00 × 4
|
||||
[5:9] = event key (big-endian)
|
||||
[9:11] = 0x00 × 2
|
||||
[11:] = timestamp_start + timestamp_stop + sep + serial + geo_string
|
||||
|
||||
Timestamp format detection (auto):
|
||||
raw_data[11] == 0x10 → 10-byte sub_code=0x03 continuous format
|
||||
raw_data[12] == 0x10 → 9-byte sub_code=0x10 single-shot format
|
||||
|
||||
Both timestamps use the same format (detected from the first byte).
|
||||
A 1-byte gap can appear between ts1 and ts2 for certain timestamps
|
||||
(observed empirically when both timestamps share the same minute:second).
|
||||
The parser handles this by trying ts2 immediately after ts1, then with
|
||||
a 1-byte skip if that fails.
|
||||
|
||||
Returns:
|
||||
MonitorLogEntry if decoding succeeds, None on error.
|
||||
"""
|
||||
if len(raw_data) < 20 or raw_data[0] != 0x2C:
|
||||
return None
|
||||
|
||||
key_hex = key4.hex()
|
||||
|
||||
def try_ts9(b: bytes):
|
||||
"""9-byte sub_code=0x10 format. Returns datetime or None."""
|
||||
if len(b) < 9 or b[1] != 0x10:
|
||||
return None
|
||||
day = b[0]; month = b[2]; year = (b[3] << 8) | b[4]
|
||||
hr = b[6]; mn = b[7]; sec = b[8]
|
||||
if not (1 <= day <= 31 and 1 <= month <= 12 and 2000 <= year <= 2050
|
||||
and hr <= 23 and mn <= 59 and sec <= 59):
|
||||
return None
|
||||
try:
|
||||
return datetime.datetime(year, month, day, hr, mn, sec)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
def try_ts10(b: bytes):
|
||||
"""10-byte sub_code=0x03 format. Returns datetime or None."""
|
||||
if len(b) < 10 or b[0] != 0x10 or b[2] != 0x10:
|
||||
return None
|
||||
day = b[1]; month = b[3]; year = (b[4] << 8) | b[5]
|
||||
hr = b[7]; mn = b[8]; sec = b[9]
|
||||
if not (1 <= day <= 31 and 1 <= month <= 12 and 2000 <= year <= 2050
|
||||
and hr <= 23 and mn <= 59 and sec <= 59):
|
||||
return None
|
||||
try:
|
||||
return datetime.datetime(year, month, day, hr, mn, sec)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
ts_offset = 11
|
||||
if len(raw_data) <= ts_offset:
|
||||
return MonitorLogEntry(index=index, key=key_hex, raw_header=raw_data)
|
||||
|
||||
# Detect timestamp format.
|
||||
if raw_data[ts_offset] == 0x10:
|
||||
ts_size = 10
|
||||
try_ts = try_ts10
|
||||
else:
|
||||
ts_size = 9
|
||||
try_ts = try_ts9
|
||||
|
||||
# Parse ts1.
|
||||
ts1 = try_ts(raw_data[ts_offset:ts_offset + ts_size])
|
||||
ts1_end = ts_offset + ts_size
|
||||
|
||||
# Parse ts2 immediately after ts1, then with 1-byte skip if needed.
|
||||
ts2 = try_ts(raw_data[ts1_end:ts1_end + ts_size])
|
||||
if ts2 is None:
|
||||
ts2 = try_ts(raw_data[ts1_end + 1:ts1_end + 1 + ts_size])
|
||||
|
||||
# Extract serial and geo threshold from "BE11529\0" and "Geo: X.XXX in/s\0".
|
||||
serial: Optional[str] = None
|
||||
geo_ips: Optional[float] = None
|
||||
|
||||
serial_pos = raw_data.find(b"BE")
|
||||
if serial_pos >= 0:
|
||||
# Read null-terminated serial starting at serial_pos.
|
||||
null_pos = raw_data.find(b"\x00", serial_pos)
|
||||
if null_pos > serial_pos:
|
||||
serial = raw_data[serial_pos:null_pos].decode("ascii", errors="replace")
|
||||
# Geo string follows the null byte.
|
||||
geo_start = (null_pos + 1) if null_pos > serial_pos else serial_pos + 7
|
||||
geo_bytes = raw_data[geo_start:]
|
||||
# "Geo: X.XXX in/s\0" — extract float after "Geo: ".
|
||||
geo_str_pos = geo_bytes.find(b"Geo: ")
|
||||
if geo_str_pos >= 0:
|
||||
geo_val_bytes = geo_bytes[geo_str_pos + 5:] # after "Geo: "
|
||||
geo_val_end = geo_val_bytes.find(b" ") # before " in/s"
|
||||
if geo_val_end > 0:
|
||||
try:
|
||||
geo_ips = float(geo_val_bytes[:geo_val_end].decode("ascii"))
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
return MonitorLogEntry(
|
||||
index=index,
|
||||
key=key_hex,
|
||||
start_time=ts1,
|
||||
stop_time=ts2,
|
||||
serial=serial,
|
||||
geo_threshold_ips=geo_ips,
|
||||
raw_header=raw_data,
|
||||
)
|
||||
|
||||
|
||||
def _decode_monitor_status(data: bytes) -> MonitorStatus:
|
||||
"""
|
||||
Decode SUB 0x1C response payload into a MonitorStatus object.
|
||||
@@ -1755,17 +2085,20 @@ def _decode_monitor_status(data: bytes) -> MonitorStatus:
|
||||
data is the raw S3 frame .data attribute (includes the 11-byte section
|
||||
header, so field offsets below are relative to data[11]).
|
||||
|
||||
Monitoring flag (confirmed 4-8-26/2ndtry, full byte diff analysis):
|
||||
section[6] == 0x00 → idle
|
||||
section[6] == 0x10 → monitoring
|
||||
NOTE: frame.data has the checksum byte already stripped by S3FrameParser
|
||||
(_finalise returns raw_payload[5:] where raw_payload = body[:-1]).
|
||||
There is NO trailing checksum byte in section.
|
||||
|
||||
The payload size varies (52–55+ bytes) but the battery/memory block is
|
||||
always the last 10 bytes before the trailing checksum byte:
|
||||
Monitoring flag (confirmed 4-8-26/2ndtry, byte diff of all 144 data frames):
|
||||
section[1] == 0x00 → idle
|
||||
section[1] == 0x10 → monitoring
|
||||
|
||||
section[-11:-9] battery × 100 uint16 BE (0x02A8 = 6.80 V)
|
||||
section[-9 :-5] memory_total uint32 BE bytes
|
||||
section[-5 :-1] memory_free uint32 BE bytes
|
||||
section[-1] checksum (not data)
|
||||
The payload length varies (46–49 bytes) — IDLE is 46-47, MONITORING is 48-49.
|
||||
The battery/memory block is always the last 10 bytes of section (no checksum):
|
||||
|
||||
section[-10:-8] battery × 100 uint16 BE (0x02A8 = 6.80 V)
|
||||
section[-8 :-4] memory_total uint32 BE bytes
|
||||
section[-4:] memory_free uint32 BE bytes
|
||||
|
||||
Values confirmed from 4-8-26/2ndtry capture (BE11529):
|
||||
battery 0x02A8 = 680 → 6.80 V
|
||||
@@ -1780,32 +2113,30 @@ def _decode_monitor_status(data: bytes) -> MonitorStatus:
|
||||
len(data), len(section), section.hex(),
|
||||
)
|
||||
|
||||
# Monitoring flag: section[6] (CORRECTED 2026-04-08 — was wrongly section[1]).
|
||||
# Byte diff of 2ndtry BW-S3 captures confirms section[6] flips 0x00↔0x10
|
||||
# exactly at the start/stop monitoring transitions (0xE3 frame #36 / #132).
|
||||
is_monitoring = len(section) > 6 and section[6] == 0x10
|
||||
# Monitoring flag: section[1] == 0x10.
|
||||
# Confirmed from byte diff of all 144 0xE3 data frames in 4-8-26/2ndtry capture:
|
||||
# section[1] = 0x00 in all IDLE frames, 0x10 in all MONITORING frames.
|
||||
# (section[6] also changes but has non-binary values 0xea/0x07 — device-specific.)
|
||||
is_monitoring = len(section) > 1 and section[1] == 0x10
|
||||
|
||||
battery_v = None
|
||||
memory_total = None
|
||||
memory_free = None
|
||||
|
||||
# Battery and memory offsets are RELATIVE TO THE END of the section.
|
||||
# The payload length varies (52–55+ bytes) depending on monitoring state and
|
||||
# internal counters, but the battery/memory block is always the last 10 bytes
|
||||
# before the checksum (section[-1]).
|
||||
# Battery and memory at relative-from-end offsets.
|
||||
# Payload length varies (46–49 bytes) but the battery/memory block is always
|
||||
# the last 10 bytes. No checksum byte — it was stripped by S3FrameParser.
|
||||
#
|
||||
# section[-11:-9] battery × 100 uint16 BE 0x02A8 = 6.80 V
|
||||
# section[-9 :-5] memory_total uint32 BE ≈ 960 KB on BE11529
|
||||
# section[-5 :-1] memory_free uint32 BE decreases as events fill
|
||||
# section[-1] frame checksum (not data)
|
||||
# section[-10:-8] battery × 100 uint16 BE 0x02A8 = 6.80 V
|
||||
# section[-8 :-4] memory_total uint32 BE ≈ 960 KB on BE11529
|
||||
# section[-4:] memory_free uint32 BE decreases as events fill
|
||||
#
|
||||
# Confirmed stable across IDLE (52b), MONITORING (55b), and counter-jitter
|
||||
# IDLE variants (53b) from 4-8-26/2ndtry full capture analysis.
|
||||
if len(section) >= 11:
|
||||
batt_raw = struct.unpack(">H", section[-11:-9])[0]
|
||||
# Confirmed stable across IDLE (46b), MONITORING (48-49b) variants.
|
||||
if len(section) >= 10:
|
||||
batt_raw = struct.unpack(">H", section[-10:-8])[0]
|
||||
battery_v = batt_raw / 100.0
|
||||
memory_total = struct.unpack(">I", section[-9:-5])[0]
|
||||
memory_free = struct.unpack(">I", section[-5:-1])[0]
|
||||
memory_total = struct.unpack(">I", section[-8:-4])[0]
|
||||
memory_free = struct.unpack(">I", section[-4:])[0]
|
||||
|
||||
return MonitorStatus(
|
||||
is_monitoring=is_monitoring,
|
||||
|
||||
@@ -14,6 +14,7 @@ Notes on certainty:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import struct
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
@@ -419,6 +420,65 @@ class Event:
|
||||
return f"Event#{self.index} {ts}{ppv}"
|
||||
|
||||
|
||||
# ── MonitorLogEntry ───────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class MonitorLogEntry:
|
||||
"""
|
||||
A monitor log entry decoded from a SUB 0x0A (WAVEFORM_HEADER) response
|
||||
whose first byte is 0x2C (partial record, recording mode = continuous
|
||||
monitoring without a triggered event).
|
||||
|
||||
These are the "partial bins" that Blastware stores between triggered events.
|
||||
Each entry represents one monitoring interval — the span of time during
|
||||
which the unit was actively monitoring but no threshold crossing occurred.
|
||||
|
||||
Confirmed from 4-11-26 MITM capture analysis (2026-04-11):
|
||||
|
||||
Header layout (full response data[0:]):
|
||||
data[0] = 0x2C (partial record type / data length in probe response)
|
||||
data[1:5] = 0x00 × 4
|
||||
data[5:9] = event key (4 bytes, big-endian hex)
|
||||
data[9:11] = 0x00 × 2
|
||||
data[11:] = timestamp_start (9 or 10 bytes depending on recording mode)
|
||||
+ timestamp_stop (same format)
|
||||
+ separator (4–5 bytes, variable)
|
||||
+ serial null-terminated (e.g. "BE11529\\0")
|
||||
+ "Geo: X.XXX in/s\\0" (trigger threshold string)
|
||||
|
||||
Timestamp format detection:
|
||||
data[11] == 0x10 → 10-byte sub_code=0x03 (continuous) format
|
||||
data[12] == 0x10 → 9-byte sub_code=0x10 (single-shot) format
|
||||
|
||||
In contrast to Event (triggered records, type 0x46), MonitorLogEntry
|
||||
records do NOT have a waveform record (SUB 0x0C) or bulk waveform stream
|
||||
(SUB 5A). All available metadata is in the 0x0A header alone.
|
||||
"""
|
||||
index: int # 0-based position in device record list
|
||||
key: str # 8-hex event key (e.g. "01114290") ✅
|
||||
|
||||
start_time: Optional[datetime.datetime] = None # monitoring session start ✅
|
||||
stop_time: Optional[datetime.datetime] = None # monitoring session stop ✅
|
||||
serial: Optional[str] = None # device serial (e.g. "BE11529") ✅
|
||||
geo_threshold_ips: Optional[float] = None # trigger level from "Geo: X.XXX in/s" ✅
|
||||
|
||||
# Raw bytes for debugging / future decoding
|
||||
raw_header: Optional[bytes] = field(default=None, repr=False)
|
||||
|
||||
@property
|
||||
def duration_seconds(self) -> Optional[float]:
|
||||
"""Duration of monitoring interval in seconds, or None if times unavailable."""
|
||||
if self.start_time and self.stop_time:
|
||||
return (self.stop_time - self.start_time).total_seconds()
|
||||
return None
|
||||
|
||||
def __str__(self) -> str:
|
||||
start = self.start_time.isoformat() if self.start_time else "?"
|
||||
stop = self.stop_time.isoformat() if self.stop_time else "?"
|
||||
dur = f" ({self.duration_seconds:.0f}s)" if self.duration_seconds is not None else ""
|
||||
return f"MonitorLog#{self.index} key={self.key} {start}→{stop}{dur}"
|
||||
|
||||
|
||||
# ── MonitorStatus ─────────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
|
||||
+99
-12
@@ -57,7 +57,7 @@ SUB_POLL = 0x5B
|
||||
SUB_SERIAL_NUMBER = 0x15
|
||||
SUB_FULL_CONFIG = 0x01
|
||||
SUB_EVENT_INDEX = 0x08
|
||||
SUB_CHANNEL_CONFIG = 0x06
|
||||
SUB_CHANNEL_CONFIG = 0x06 # Event storage range read (first/last key) ✅
|
||||
SUB_MONITOR_STATUS = 0x1C # Monitoring status read (battery, memory, mode) ✅
|
||||
SUB_EVENT_HEADER = 0x1E
|
||||
SUB_EVENT_ADVANCE = 0x1F
|
||||
@@ -82,6 +82,12 @@ SUB_TRIGGER_CONFIRM = 0x83 # Confirm trigger write ✅
|
||||
SUB_START_MONITORING = 0x96 # Start monitoring → response 0x69 ✅
|
||||
SUB_STOP_MONITORING = 0x97 # Stop monitoring → response 0x68 ✅
|
||||
|
||||
# Erase-all SUBs (confirmed from 4-11-26 MITM capture)
|
||||
# Both use token=0xFE at params[7] and return minimal 11-byte acks.
|
||||
# Standard response formula applies: 0xFF - SUB.
|
||||
SUB_ERASE_ALL_BEGIN = 0xA3 # Begin erase all events → response 0x5C ✅
|
||||
SUB_ERASE_ALL_CONFIRM = 0xA2 # Confirm erase all events → response 0x5D ✅
|
||||
|
||||
# Hardcoded data lengths for the two-step read protocol.
|
||||
#
|
||||
# The S3 probe response page_key is always 0x0000 — it does NOT carry the
|
||||
@@ -96,6 +102,7 @@ DATA_LENGTHS: dict[int, int] = {
|
||||
SUB_SERIAL_NUMBER: 0x0A, # 10-byte serial number block ✅
|
||||
SUB_FULL_CONFIG: 0x98, # 152-byte full config block ✅
|
||||
SUB_EVENT_INDEX: 0x58, # 88-byte event index ✅
|
||||
SUB_CHANNEL_CONFIG: 0x24, # 36-byte event storage range (first/last key) ✅
|
||||
SUB_MONITOR_STATUS: 0x2C, # 44-byte monitor status block (idle) ✅
|
||||
SUB_EVENT_HEADER: 0x08, # 8-byte event header (waveform key + event data) ✅
|
||||
SUB_EVENT_ADVANCE: 0x08, # 8-byte next-key response ✅
|
||||
@@ -387,23 +394,32 @@ class MiniMateProtocol:
|
||||
Send the SUB 0A (WAVEFORM_HEADER) two-step read for *key4*.
|
||||
|
||||
The data length for 0A is VARIABLE and must be read from the probe
|
||||
response at data[4]. Two known values:
|
||||
0x30 — full histogram bin (has a waveform record to follow)
|
||||
0x26 — partial histogram bin (no waveform record)
|
||||
response at data[4]. Two confirmed values:
|
||||
0x46 (70) — full triggered event (has 0C waveform record to follow)
|
||||
0x2C (44) — partial / monitor-log entry (no 0C record; 0A header only)
|
||||
|
||||
Args:
|
||||
key4: 4-byte waveform record address from 1E or 1F.
|
||||
|
||||
Returns:
|
||||
(header_bytes, record_length) where:
|
||||
header_bytes — raw data section starting at data[11]
|
||||
record_length — DATA_LENGTH read from probe (0x30 or 0x26)
|
||||
(raw_data, record_length) where:
|
||||
raw_data — complete data_rsp.data bytes (full response payload)
|
||||
record_length — DATA_LENGTH read from probe (0x46 for full, 0x2C for partial)
|
||||
|
||||
The raw_data layout:
|
||||
raw_data[0] = record type (0x46 = full triggered event, 0x2C = partial/monitor)
|
||||
raw_data[1:5] = 0x00 × 4
|
||||
raw_data[5:9] = event key (4 bytes)
|
||||
raw_data[9:11] = 0x00 × 2
|
||||
raw_data[11:] = timestamps + separator + serial + channel strings
|
||||
(see MonitorLogEntry in models.py for full layout)
|
||||
|
||||
Raises:
|
||||
ProtocolError: on timeout, bad checksum, or wrong response SUB.
|
||||
|
||||
Confirmed from 3-31-26 capture: 0A probe response data[4] carries
|
||||
Confirmed from 4-11-26 MITM capture: 0A probe response data[4] carries
|
||||
the variable length; data-request uses that length as the offset byte.
|
||||
record_length == data[0] in virtually all cases (confirmed empirically).
|
||||
"""
|
||||
rsp_sub = _expected_rsp_sub(SUB_WAVEFORM_HEADER)
|
||||
params = waveform_key_params(key4)
|
||||
@@ -413,7 +429,7 @@ class MiniMateProtocol:
|
||||
probe_rsp = self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
# Variable length — read from probe response data[4]
|
||||
length = probe_rsp.data[4] if len(probe_rsp.data) > 4 else 0x30
|
||||
length = probe_rsp.data[4] if len(probe_rsp.data) > 4 else 0x46
|
||||
log.debug("read_waveform_header: 0A data request offset=0x%02X", length)
|
||||
|
||||
if length == 0:
|
||||
@@ -422,12 +438,11 @@ class MiniMateProtocol:
|
||||
self._send(build_bw_frame(SUB_WAVEFORM_HEADER, length, params))
|
||||
data_rsp = self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
header_bytes = data_rsp.data[11:11 + length]
|
||||
log.debug(
|
||||
"read_waveform_header: key=%s length=0x%02X is_full=%s",
|
||||
key4.hex(), length, length == 0x30,
|
||||
key4.hex(), length, length >= 0x40,
|
||||
)
|
||||
return header_bytes, length
|
||||
return data_rsp.data, length
|
||||
|
||||
def read_waveform_data_raw(self) -> bytes:
|
||||
"""
|
||||
@@ -1137,6 +1152,78 @@ class MiniMateProtocol:
|
||||
self._send(frame)
|
||||
return self.recv_write_ack(expected_sub=rsp_sub)
|
||||
|
||||
def read_event_storage_range(self) -> S3Frame:
|
||||
"""
|
||||
Read event storage range (SUB 0x06 → response 0xF9).
|
||||
|
||||
Two-step read: probe (offset=0x00) then data (offset=0x24 = 36 bytes).
|
||||
Uses token=0xFE at params[7] — same as the erase sequence.
|
||||
|
||||
The 36-byte response ends with two 4-byte event keys (first and last
|
||||
stored event key). After a successful erase, both keys are 0x01110000
|
||||
(device-empty sentinel). Confirmed from 4-11-26 MITM capture.
|
||||
|
||||
Returns:
|
||||
S3Frame with 36 bytes of storage range data.
|
||||
|
||||
Raises:
|
||||
ProtocolError: on timeout or wrong response SUB.
|
||||
"""
|
||||
rsp_sub = _expected_rsp_sub(SUB_CHANNEL_CONFIG) # 0xFF - 0x06 = 0xF9
|
||||
params = token_params(0xFE)
|
||||
log.debug("read_event_storage_range: probe step rsp_sub=0x%02X", rsp_sub)
|
||||
self._send(build_bw_frame(SUB_CHANNEL_CONFIG, offset=0x00, params=params))
|
||||
self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
log.debug(
|
||||
"read_event_storage_range: data step offset=0x%02X",
|
||||
DATA_LENGTHS[SUB_CHANNEL_CONFIG],
|
||||
)
|
||||
self._send(build_bw_frame(SUB_CHANNEL_CONFIG,
|
||||
offset=DATA_LENGTHS[SUB_CHANNEL_CONFIG],
|
||||
params=params))
|
||||
return self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
def begin_erase_all(self) -> S3Frame:
|
||||
"""
|
||||
Send Begin-Erase-All command (SUB 0xA3 → response 0x5C).
|
||||
|
||||
Single frame with token=0xFE at params[7]. The device acknowledges with
|
||||
a minimal ack and begins the erase process. Follow up with
|
||||
read_monitor_status() + read_event_storage_range() + confirm_erase_all()
|
||||
to complete the sequence. Confirmed from 4-11-26 MITM capture.
|
||||
|
||||
Returns:
|
||||
S3Frame ack from device (SUB 0x5C).
|
||||
|
||||
Raises:
|
||||
ProtocolError: on timeout or wrong response SUB.
|
||||
"""
|
||||
rsp_sub = _expected_rsp_sub(SUB_ERASE_ALL_BEGIN) # 0xFF - 0xA3 = 0x5C
|
||||
log.debug("begin_erase_all: rsp_sub=0x%02X", rsp_sub)
|
||||
self._send(build_bw_frame(SUB_ERASE_ALL_BEGIN, params=token_params(0xFE)))
|
||||
return self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
def confirm_erase_all(self) -> S3Frame:
|
||||
"""
|
||||
Send Confirm-Erase-All command (SUB 0xA2 → response 0x5D).
|
||||
|
||||
Single frame with token=0xFE at params[7]. Must be preceded by
|
||||
begin_erase_all() + read_monitor_status() + read_event_storage_range().
|
||||
After this call the device memory is cleared. Confirmed from 4-11-26
|
||||
MITM capture.
|
||||
|
||||
Returns:
|
||||
S3Frame ack from device (SUB 0x5D).
|
||||
|
||||
Raises:
|
||||
ProtocolError: on timeout or wrong response SUB.
|
||||
"""
|
||||
rsp_sub = _expected_rsp_sub(SUB_ERASE_ALL_CONFIRM) # 0xFF - 0xA2 = 0x5D
|
||||
log.debug("confirm_erase_all: rsp_sub=0x%02X", rsp_sub)
|
||||
self._send(build_bw_frame(SUB_ERASE_ALL_CONFIRM, params=token_params(0xFE)))
|
||||
return self._recv_one(expected_sub=rsp_sub)
|
||||
|
||||
# ── Internal helpers ──────────────────────────────────────────────────────
|
||||
|
||||
def _send(self, frame: bytes) -> None:
|
||||
|
||||
@@ -418,3 +418,39 @@ class TcpTransport(BaseTransport):
|
||||
def __repr__(self) -> str:
|
||||
state = "connected" if self.is_connected else "disconnected"
|
||||
return f"TcpTransport({self.host!r}, port={self.port}, {state})"
|
||||
|
||||
|
||||
# ── Inbound / accepted-socket transport ───────────────────────────────────────
|
||||
|
||||
class SocketTransport(TcpTransport):
|
||||
"""
|
||||
Like TcpTransport but wraps an already-accepted inbound socket.
|
||||
|
||||
Used by the ACH inbound server (bridges/ach_server.py) — the device dials
|
||||
IN to us, so by the time we create this transport the socket is already live.
|
||||
connect() is a no-op; everything else (read, write, read_until_idle, …) is
|
||||
inherited unchanged from TcpTransport.
|
||||
|
||||
Args:
|
||||
sock: An already-connected socket.socket returned by server_socket.accept().
|
||||
peer: Human-readable peer label for repr / logging (e.g. "203.0.113.5:54321").
|
||||
"""
|
||||
|
||||
def __init__(self, sock: socket.socket, peer: str = "inbound") -> None:
|
||||
# Bypass TcpTransport.__init__ — we already have a live socket.
|
||||
self.host = peer
|
||||
self.port = 0
|
||||
self.connect_timeout = 0.0
|
||||
self._sock = sock
|
||||
sock.settimeout(self._RECV_TIMEOUT)
|
||||
|
||||
def connect(self) -> None:
|
||||
"""No-op — socket was already accepted inbound."""
|
||||
pass # Already have a live socket; nothing to open.
|
||||
|
||||
@property
|
||||
def is_connected(self) -> bool:
|
||||
return self._sock is not None
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return f"SocketTransport(peer={self.host!r})"
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "seismo-relay"
|
||||
version = "0.12.0"
|
||||
description = "Python client and REST server for MiniMate Plus seismographs"
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"fastapi>=0.104",
|
||||
"uvicorn[standard]>=0.24",
|
||||
"pyserial>=3.5",
|
||||
"sqlalchemy>=2.0",
|
||||
]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
# Auto-discovers minimateplus/, sfm/, bridges/ as packages
|
||||
where = ["."]
|
||||
include = ["minimateplus*", "sfm*", "bridges*"]
|
||||
+404
@@ -1071,6 +1071,398 @@ class AnalyzerPanel(tk.Frame):
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# Serial Watch panel — tap the RS-232 line between device and modem
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
try:
|
||||
import serial as _serial
|
||||
from serial.tools import list_ports as _list_ports
|
||||
_SERIAL_OK = True
|
||||
except ImportError:
|
||||
_SERIAL_OK = False
|
||||
|
||||
from minimateplus.framing import S3FrameParser as _S3FrameParser # noqa: E402
|
||||
|
||||
_SW_KNOWN_SUBS = {
|
||||
0xA4: "POLL_RSP", 0xA5: "BULK_WAVEFORM_RSP", 0xE0: "ADV_EVENT_RSP",
|
||||
0xE1: "EVT_IDX_FIRST_RSP", 0xE3: "MONITOR_STATUS_RSP", 0xEA: "SERIAL_NUM_RSP",
|
||||
0xF3: "WAVEFORM_REC_RSP", 0xF5: "WAVEFORM_HDR_RSP", 0xF7: "EVENT_INDEX_RSP",
|
||||
0xF9: "UNK_06_RSP", 0xFE: "DEVICE_INFO_RSP",
|
||||
0x69: "START_MON_ACK", 0x68: "STOP_MON_ACK",
|
||||
}
|
||||
|
||||
|
||||
class SerialWatchPanel(tk.Frame):
|
||||
"""
|
||||
Tap the RS-232 line between the MiniMate Plus and its modem (RV50/RV55).
|
||||
Runs the serial reader in a background thread; surfaces parsed S3 frames
|
||||
live in the log view. Writes raw_s3_<ts>.bin compatible with Analyzer.
|
||||
|
||||
Typical use for call-home capture:
|
||||
1. Connect a USB-to-serial tap to the RS-232 line.
|
||||
2. Pick that COM port here, click Start.
|
||||
3. Wait for the unit to trigger / call home.
|
||||
4. Click Stop, then 'Open in Analyzer' to inspect the frames.
|
||||
"""
|
||||
|
||||
_COL_FRAME = "#4ec9b0" # teal — parsed S3 frame
|
||||
_COL_CTRL = "#dcdcaa" # yellow — control-line change
|
||||
_COL_AT = "#9cdcfe" # blue — AT command / ASCII noise
|
||||
_COL_ERR = "#f44747" # red — error
|
||||
|
||||
def __init__(self, parent: tk.Widget, on_capture_ready=None, **kw):
|
||||
"""
|
||||
on_capture_ready(raw_s3_path: str) — called when capture stops,
|
||||
so the parent can inject the file into the Analyzer.
|
||||
"""
|
||||
super().__init__(parent, bg=BG2, **kw)
|
||||
self._on_capture_ready = on_capture_ready
|
||||
self._serial: Optional[object] = None # serial.Serial instance
|
||||
self._reader_thread: Optional[threading.Thread] = None
|
||||
self._stop_evt = threading.Event()
|
||||
self._log_q: queue.Queue[tuple[str, str]] = queue.Queue() # (text, colour)
|
||||
self._raw_fh = None # open binary file handle
|
||||
self._raw_path: Optional[str] = None
|
||||
self._frame_count = 0
|
||||
self._build()
|
||||
self._poll_log_queue()
|
||||
|
||||
# ── build ─────────────────────────────────────────────────────────────
|
||||
|
||||
def _build(self) -> None:
|
||||
pad = {"padx": 6, "pady": 4}
|
||||
|
||||
cfg = tk.Frame(self, bg=BG2)
|
||||
cfg.pack(side=tk.TOP, fill=tk.X, padx=4, pady=4)
|
||||
|
||||
# Row 0 — port picker
|
||||
tk.Label(cfg, text="COM port:", bg=BG2, fg=FG, font=MONO
|
||||
).grid(row=0, column=0, sticky="e", **pad)
|
||||
|
||||
self._port_var = tk.StringVar()
|
||||
self._port_cb = ttk.Combobox(cfg, textvariable=self._port_var,
|
||||
width=12, font=MONO, state="normal")
|
||||
self._port_cb.grid(row=0, column=1, sticky="w", **pad)
|
||||
|
||||
tk.Button(cfg, text="↺", bg=BG3, fg=FG, relief="flat", cursor="hand2",
|
||||
font=MONO, command=self._refresh_ports
|
||||
).grid(row=0, column=2, **pad)
|
||||
|
||||
tk.Label(cfg, text=" Baud:", bg=BG2, fg=FG, font=MONO
|
||||
).grid(row=0, column=3, sticky="e", **pad)
|
||||
self._baud_var = tk.StringVar(value="38400")
|
||||
tk.Entry(cfg, textvariable=self._baud_var, width=8,
|
||||
bg=BG3, fg=FG, insertbackground=FG, relief="flat", font=MONO
|
||||
).grid(row=0, column=4, sticky="w", **pad)
|
||||
|
||||
self._ack_ok_var = tk.BooleanVar(value=False)
|
||||
tk.Checkbutton(cfg, text="Ack OK to AT commands",
|
||||
variable=self._ack_ok_var,
|
||||
bg=BG2, fg=FG, selectcolor=BG3, activebackground=BG2,
|
||||
font=MONO).grid(row=0, column=5, sticky="w", **pad)
|
||||
|
||||
# Row 1 — capture dir
|
||||
tk.Label(cfg, text="Save to:", bg=BG2, fg=FG, font=MONO
|
||||
).grid(row=1, column=0, sticky="e", **pad)
|
||||
self._dir_var = tk.StringVar(
|
||||
value=str(SCRIPT_DIR / "bridges" / "captures"))
|
||||
tk.Entry(cfg, textvariable=self._dir_var, width=40,
|
||||
bg=BG3, fg=FG, insertbackground=FG, relief="flat", font=MONO
|
||||
).grid(row=1, column=1, columnspan=4, sticky="we", **pad)
|
||||
tk.Button(cfg, text="Browse", bg=BG3, fg=FG, relief="flat",
|
||||
cursor="hand2", font=MONO, command=self._choose_dir
|
||||
).grid(row=1, column=5, **pad)
|
||||
|
||||
# Button row
|
||||
btn_row = tk.Frame(self, bg=BG2)
|
||||
btn_row.pack(side=tk.TOP, fill=tk.X, padx=4, pady=2)
|
||||
|
||||
self._start_btn = tk.Button(
|
||||
btn_row, text="Start Watch", bg=GREEN, fg="#000000",
|
||||
relief="flat", padx=12, cursor="hand2", font=MONO_B,
|
||||
command=self._start)
|
||||
self._start_btn.pack(side=tk.LEFT, padx=6)
|
||||
|
||||
self._stop_btn = tk.Button(
|
||||
btn_row, text="Stop", bg=BG3, fg=FG,
|
||||
relief="flat", padx=12, cursor="hand2", font=MONO,
|
||||
command=self._stop, state="disabled")
|
||||
self._stop_btn.pack(side=tk.LEFT, padx=4)
|
||||
|
||||
self._analyzer_btn = tk.Button(
|
||||
btn_row, text="Open in Analyzer", bg=BG3, fg=FG,
|
||||
relief="flat", padx=10, cursor="hand2", font=MONO,
|
||||
command=self._send_to_analyzer, state="disabled")
|
||||
self._analyzer_btn.pack(side=tk.LEFT, padx=4)
|
||||
|
||||
tk.Button(btn_row, text="Clear", bg=BG3, fg=FG,
|
||||
relief="flat", padx=8, cursor="hand2", font=MONO,
|
||||
command=self._clear_log).pack(side=tk.LEFT, padx=4)
|
||||
|
||||
self._status_var = tk.StringVar(value="Idle")
|
||||
tk.Label(btn_row, textvariable=self._status_var,
|
||||
bg=BG2, fg=FG_DIM, font=MONO).pack(side=tk.LEFT, padx=10)
|
||||
|
||||
# Log view
|
||||
self._log = scrolledtext.ScrolledText(
|
||||
self, height=24, font=MONO_SM,
|
||||
bg=BG, fg=FG, insertbackground=FG,
|
||||
relief="flat", state="disabled",
|
||||
)
|
||||
self._log.pack(fill=tk.BOTH, expand=True, padx=4, pady=4)
|
||||
self._log.tag_config("frame", foreground=self._COL_FRAME)
|
||||
self._log.tag_config("ctrl", foreground=self._COL_CTRL)
|
||||
self._log.tag_config("at", foreground=self._COL_AT)
|
||||
self._log.tag_config("err", foreground=self._COL_ERR)
|
||||
self._log.tag_config("dim", foreground=FG_DIM)
|
||||
|
||||
# Populate ports on first load
|
||||
self._refresh_ports()
|
||||
|
||||
# ── port helpers ──────────────────────────────────────────────────────
|
||||
|
||||
def _refresh_ports(self) -> None:
|
||||
if not _SERIAL_OK:
|
||||
self._port_cb["values"] = ["(pyserial not installed)"]
|
||||
return
|
||||
ports = [p.device for p in _list_ports.comports()]
|
||||
self._port_cb["values"] = ports
|
||||
if ports and not self._port_var.get():
|
||||
self._port_var.set(ports[0])
|
||||
|
||||
def _choose_dir(self) -> None:
|
||||
d = filedialog.askdirectory(initialdir=self._dir_var.get())
|
||||
if d:
|
||||
self._dir_var.set(d)
|
||||
|
||||
# ── start / stop ──────────────────────────────────────────────────────
|
||||
|
||||
def _start(self) -> None:
|
||||
if not _SERIAL_OK:
|
||||
messagebox.showerror(
|
||||
"pyserial missing",
|
||||
"Install pyserial first:\n pip install pyserial")
|
||||
return
|
||||
|
||||
port = self._port_var.get().strip()
|
||||
if not port or "not installed" in port:
|
||||
messagebox.showerror("Error", "Select a valid COM port first.")
|
||||
return
|
||||
|
||||
try:
|
||||
baud = int(self._baud_var.get().strip())
|
||||
except ValueError:
|
||||
messagebox.showerror("Error", "Invalid baud rate.")
|
||||
return
|
||||
|
||||
# Open output files
|
||||
ts = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
out_dir = Path(self._dir_var.get()) / f"serial_{ts}"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._raw_path = str(out_dir / f"raw_s3_{ts}.bin")
|
||||
try:
|
||||
self._raw_fh = open(self._raw_path, "wb")
|
||||
except OSError as exc:
|
||||
messagebox.showerror("Error", f"Cannot open capture file:\n{exc}")
|
||||
return
|
||||
|
||||
# Open serial port
|
||||
try:
|
||||
ser = _serial.Serial(
|
||||
port=port, baudrate=baud,
|
||||
bytesize=8, parity=_serial.PARITY_NONE,
|
||||
stopbits=_serial.STOPBITS_ONE,
|
||||
timeout=0.05, write_timeout=0,
|
||||
)
|
||||
ser.setDTR(True)
|
||||
ser.setRTS(True)
|
||||
except Exception as exc:
|
||||
self._raw_fh.close()
|
||||
self._raw_fh = None
|
||||
messagebox.showerror("Error", f"Cannot open {port}:\n{exc}")
|
||||
return
|
||||
|
||||
self._serial = ser
|
||||
self._stop_evt.clear()
|
||||
self._frame_count = 0
|
||||
self._analyzer_btn.configure(state="disabled")
|
||||
|
||||
self._reader_thread = threading.Thread(
|
||||
target=self._reader_loop,
|
||||
args=(ser, baud),
|
||||
daemon=True,
|
||||
)
|
||||
self._reader_thread.start()
|
||||
|
||||
self._status_var.set(f"Watching {port} @ {baud}")
|
||||
self._start_btn.configure(state="disabled")
|
||||
self._stop_btn.configure(state="normal", bg=RED)
|
||||
self._append(f"── Serial watch started {port} @ {baud} [{ts}] ──\n", "dim")
|
||||
self._append(f" Capture: {self._raw_path}\n", "dim")
|
||||
self._append(" Waiting for data…\n\n", "dim")
|
||||
|
||||
def _stop(self) -> None:
|
||||
self._stop_evt.set()
|
||||
if self._serial:
|
||||
try:
|
||||
self._serial.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._serial = None
|
||||
if self._raw_fh:
|
||||
self._raw_fh.close()
|
||||
self._raw_fh = None
|
||||
self._status_var.set("Stopped")
|
||||
self._start_btn.configure(state="normal")
|
||||
self._stop_btn.configure(state="disabled", bg=BG3)
|
||||
if self._raw_path and Path(self._raw_path).exists():
|
||||
self._analyzer_btn.configure(state="normal")
|
||||
self._append("\n── Watch stopped ──\n", "dim")
|
||||
|
||||
# ── reader thread ─────────────────────────────────────────────────────
|
||||
|
||||
def _reader_loop(self, ser, baud: int) -> None:
|
||||
parser = _S3FrameParser()
|
||||
rx_buf = bytearray()
|
||||
ack_ok = self._ack_ok_var.get()
|
||||
|
||||
# Monitor control lines in a sub-thread
|
||||
ctrl_stop = threading.Event()
|
||||
ctrl_thread = threading.Thread(
|
||||
target=self._ctrl_loop, args=(ser, ctrl_stop), daemon=True)
|
||||
ctrl_thread.start()
|
||||
|
||||
try:
|
||||
while not self._stop_evt.is_set():
|
||||
try:
|
||||
data = ser.read(4096)
|
||||
except Exception as exc:
|
||||
self._log_q.put((f"Read error: {exc}\n", "err"))
|
||||
break
|
||||
|
||||
if not data:
|
||||
continue
|
||||
|
||||
# Save raw bytes
|
||||
if self._raw_fh:
|
||||
try:
|
||||
self._raw_fh.write(data)
|
||||
self._raw_fh.flush()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Parse S3 frames
|
||||
for byte in data:
|
||||
result = parser.feed(bytes([byte]))
|
||||
if result:
|
||||
frames = result if isinstance(result, list) else [result]
|
||||
for f in frames:
|
||||
self._frame_count += 1
|
||||
name = _SW_KNOWN_SUBS.get(f.sub, f"UNK_0x{f.sub:02X}")
|
||||
chk = "✓" if f.checksum_valid else "✗ BAD_CHK"
|
||||
peek = f.data[:32].hex() + ("…" if len(f.data) > 32 else "")
|
||||
msg = (
|
||||
f"[{self._frame_count:04d}] "
|
||||
f"SUB=0x{f.sub:02X} ({name:<22}) "
|
||||
f"page=0x{f.page_key:04X} "
|
||||
f"data={len(f.data):4d}B {chk}\n"
|
||||
f" {peek}\n"
|
||||
)
|
||||
self._log_q.put((msg, "frame"))
|
||||
|
||||
# AT command handling for --ack-ok mode
|
||||
if ack_ok:
|
||||
rx_buf.extend(data)
|
||||
while b"\r" in rx_buf or b"\n" in rx_buf:
|
||||
for sep in (b"\r", b"\n"):
|
||||
idx = rx_buf.find(sep)
|
||||
if idx != -1:
|
||||
line_bytes = bytes(rx_buf[:idx])
|
||||
del rx_buf[:idx + 1]
|
||||
break
|
||||
else:
|
||||
break
|
||||
line_str = line_bytes.decode("latin1", errors="ignore").strip()
|
||||
if line_str.upper().startswith("AT"):
|
||||
self._log_q.put((f"AT: {line_str!r}\n", "at"))
|
||||
if not line_str.upper().startswith("ATDT"):
|
||||
try:
|
||||
ser.write(b"\r\nOK\r\n")
|
||||
ser.flush()
|
||||
self._log_q.put((f" → OK\n", "at"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
finally:
|
||||
ctrl_stop.set()
|
||||
ctrl_thread.join(timeout=0.5)
|
||||
# Signal the main thread that the reader ended naturally
|
||||
if not self._stop_evt.is_set():
|
||||
self._log_q.put(("<<done>>", ""))
|
||||
|
||||
def _ctrl_loop(self, ser, stop: threading.Event) -> None:
|
||||
prev = {}
|
||||
try:
|
||||
prev = dict(CTS=ser.cts, DSR=ser.dsr, DCD=ser.cd)
|
||||
try:
|
||||
prev["RI"] = ser.ri
|
||||
except Exception:
|
||||
prev["RI"] = None
|
||||
except Exception:
|
||||
return
|
||||
|
||||
while not stop.is_set():
|
||||
try:
|
||||
cur = dict(CTS=ser.cts, DSR=ser.dsr, DCD=ser.cd, RI=None)
|
||||
try:
|
||||
cur["RI"] = ser.ri
|
||||
except Exception:
|
||||
pass
|
||||
for name, val in cur.items():
|
||||
if val != prev.get(name):
|
||||
self._log_q.put((f"CTRL {name} → {val}\n", "ctrl"))
|
||||
prev[name] = val
|
||||
except Exception:
|
||||
break
|
||||
stop.wait(0.2)
|
||||
|
||||
# ── log view ──────────────────────────────────────────────────────────
|
||||
|
||||
def _poll_log_queue(self) -> None:
|
||||
try:
|
||||
while True:
|
||||
text, tag = self._log_q.get_nowait()
|
||||
if text == "<<done>>":
|
||||
self._stop()
|
||||
break
|
||||
self._append(text, tag)
|
||||
except queue.Empty:
|
||||
pass
|
||||
finally:
|
||||
self.after(80, self._poll_log_queue)
|
||||
|
||||
def _append(self, text: str, tag: str = "") -> None:
|
||||
self._log.configure(state="normal")
|
||||
if tag:
|
||||
self._log.insert(tk.END, text, tag)
|
||||
else:
|
||||
self._log.insert(tk.END, text)
|
||||
self._log.see(tk.END)
|
||||
self._log.configure(state="disabled")
|
||||
|
||||
def _clear_log(self) -> None:
|
||||
self._log.configure(state="normal")
|
||||
self._log.delete("1.0", tk.END)
|
||||
self._log.configure(state="disabled")
|
||||
|
||||
# ── send to analyzer ──────────────────────────────────────────────────
|
||||
|
||||
def _send_to_analyzer(self) -> None:
|
||||
if self._raw_path and self._on_capture_ready:
|
||||
self._on_capture_ready(self._raw_path)
|
||||
|
||||
|
||||
# Console panel (tk.Frame — lives inside a notebook tab)
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1504,6 +1896,12 @@ class SeismoLab(tk.Tk):
|
||||
)
|
||||
nb.add(self._console_panel, text=" Console ")
|
||||
|
||||
self._serial_watch_panel = SerialWatchPanel(
|
||||
nb,
|
||||
on_capture_ready=self._on_serial_capture_ready,
|
||||
)
|
||||
nb.add(self._serial_watch_panel, text=" Serial Watch ")
|
||||
|
||||
self._nb = nb
|
||||
self.protocol("WM_DELETE_WINDOW", self._on_close)
|
||||
|
||||
@@ -1522,8 +1920,14 @@ class SeismoLab(tk.Tk):
|
||||
self._analyzer_panel.s3_var.set(raw_s3_path)
|
||||
self._nb.select(1)
|
||||
|
||||
def _on_serial_capture_ready(self, raw_s3_path: str) -> None:
|
||||
"""Serial Watch capture finished → inject into Analyzer and switch tab."""
|
||||
self._analyzer_panel.s3_var.set(raw_s3_path)
|
||||
self._nb.select(1)
|
||||
|
||||
def _on_close(self) -> None:
|
||||
self._bridge_panel.stop_bridge()
|
||||
self._serial_watch_panel._stop()
|
||||
self.destroy()
|
||||
|
||||
|
||||
|
||||
+486
@@ -0,0 +1,486 @@
|
||||
"""
|
||||
sfm/database.py — SQLite persistence layer for seismo-relay.
|
||||
|
||||
Three tables, all keyed by unit serial number:
|
||||
|
||||
ach_sessions — one row per inbound ACH call-home
|
||||
events — one row per triggered waveform event (deduped by serial+timestamp)
|
||||
monitor_log — one row per monitoring interval (deduped by serial+start_time)
|
||||
|
||||
The DB file lives at:
|
||||
<output_dir>/seismo_relay.db (default: bridges/captures/seismo_relay.db)
|
||||
|
||||
Usage
|
||||
-----
|
||||
from sfm.database import SeismoDb
|
||||
|
||||
db = SeismoDb("bridges/captures/seismo_relay.db")
|
||||
|
||||
# Write a call-home session
|
||||
session_id = db.insert_ach_session(serial="BE11529", peer="1.2.3.4:51920",
|
||||
events_downloaded=3, monitor_entries=2,
|
||||
duration_seconds=47.3)
|
||||
|
||||
# Write events (silently skips duplicates)
|
||||
db.insert_events(events, serial="BE11529", session_id=session_id)
|
||||
|
||||
# Write monitor log entries
|
||||
db.insert_monitor_log(entries, session_id=session_id)
|
||||
|
||||
# Query
|
||||
rows = db.query_events(serial="BE11529", from_dt=datetime(...), to_dt=datetime(...))
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import sqlite3
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from minimateplus.models import Event, MonitorLogEntry
|
||||
|
||||
log = logging.getLogger("sfm.database")
|
||||
|
||||
# ── Schema ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
_SCHEMA = """
|
||||
PRAGMA journal_mode = WAL;
|
||||
PRAGMA foreign_keys = ON;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS ach_sessions (
|
||||
id TEXT PRIMARY KEY, -- UUID
|
||||
serial TEXT NOT NULL,
|
||||
session_time TEXT NOT NULL, -- ISO-8601 UTC
|
||||
peer TEXT, -- "ip:port"
|
||||
events_downloaded INTEGER NOT NULL DEFAULT 0,
|
||||
monitor_entries INTEGER NOT NULL DEFAULT 0,
|
||||
duration_seconds REAL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now'))
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_ach_sessions_serial ON ach_sessions(serial);
|
||||
CREATE INDEX IF NOT EXISTS idx_ach_sessions_time ON ach_sessions(session_time);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
id TEXT PRIMARY KEY, -- UUID
|
||||
serial TEXT NOT NULL,
|
||||
waveform_key TEXT NOT NULL, -- 8-hex device key (dedup field)
|
||||
session_id TEXT, -- FK → ach_sessions.id
|
||||
timestamp TEXT, -- ISO-8601 local time from device
|
||||
tran_ppv REAL, -- in/s
|
||||
vert_ppv REAL, -- in/s
|
||||
long_ppv REAL, -- in/s
|
||||
peak_vector_sum REAL, -- in/s
|
||||
mic_ppv REAL, -- psi or dB depending on setup
|
||||
project TEXT,
|
||||
client TEXT,
|
||||
operator TEXT,
|
||||
sensor_location TEXT,
|
||||
sample_rate INTEGER,
|
||||
record_type TEXT, -- "single_shot" | "continuous"
|
||||
false_trigger INTEGER NOT NULL DEFAULT 0, -- 0=no, 1=yes (manual flag)
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||
UNIQUE(serial, timestamp)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_serial ON events(serial);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_timestamp ON events(timestamp);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_session ON events(session_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS monitor_log (
|
||||
id TEXT PRIMARY KEY, -- UUID
|
||||
serial TEXT NOT NULL,
|
||||
waveform_key TEXT NOT NULL, -- 8-hex device key (dedup field)
|
||||
session_id TEXT, -- FK → ach_sessions.id
|
||||
start_time TEXT, -- ISO-8601
|
||||
stop_time TEXT, -- ISO-8601
|
||||
duration_seconds REAL,
|
||||
geo_threshold_ips REAL, -- in/s
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||
UNIQUE(serial, start_time)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_serial ON monitor_log(serial);
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_start ON monitor_log(start_time);
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_session ON monitor_log(session_id);
|
||||
"""
|
||||
|
||||
|
||||
# ── SeismoDb class ─────────────────────────────────────────────────────────────
|
||||
|
||||
class SeismoDb:
|
||||
"""
|
||||
Thin SQLite wrapper for seismo-relay persistence.
|
||||
|
||||
Thread-safe: each call opens, uses, and closes a connection with
|
||||
check_same_thread=False and WAL mode enabled. For the ACH server's
|
||||
single-writer / occasional-reader pattern this is more than sufficient.
|
||||
"""
|
||||
|
||||
def __init__(self, db_path: str | Path) -> None:
|
||||
self.db_path = Path(db_path)
|
||||
self.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._init_schema()
|
||||
log.info("SeismoDb initialised at %s", self.db_path)
|
||||
|
||||
# ── Internal helpers ───────────────────────────────────────────────────────
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
conn = sqlite3.connect(str(self.db_path), check_same_thread=False)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA journal_mode = WAL")
|
||||
conn.execute("PRAGMA foreign_keys = ON")
|
||||
return conn
|
||||
|
||||
def _init_schema(self) -> None:
|
||||
with self._connect() as conn:
|
||||
conn.executescript(_SCHEMA)
|
||||
self._migrate(conn)
|
||||
|
||||
def _migrate(self, conn: sqlite3.Connection) -> None:
|
||||
"""Apply in-place schema migrations for existing databases."""
|
||||
|
||||
# Migration 1: change events UNIQUE from (serial, waveform_key) [or any
|
||||
# waveform_key-based variant] to (serial, timestamp).
|
||||
# Rationale: device key counter resets to 01110000 after every erase, so
|
||||
# waveform_key is not a stable dedup field across erase cycles. The event
|
||||
# timestamp (from the device clock) is the correct natural key.
|
||||
row = conn.execute(
|
||||
"SELECT sql FROM sqlite_master WHERE type='table' AND name='events'"
|
||||
).fetchone()
|
||||
if row and "UNIQUE(serial, timestamp)" not in row[0]:
|
||||
log.info("_migrate: rebuilding events table — UNIQUE(serial, timestamp)")
|
||||
conn.executescript("""
|
||||
ALTER TABLE events RENAME TO events_old;
|
||||
|
||||
CREATE TABLE events (
|
||||
id TEXT PRIMARY KEY,
|
||||
serial TEXT NOT NULL,
|
||||
waveform_key TEXT NOT NULL,
|
||||
session_id TEXT,
|
||||
timestamp TEXT,
|
||||
tran_ppv REAL,
|
||||
vert_ppv REAL,
|
||||
long_ppv REAL,
|
||||
peak_vector_sum REAL,
|
||||
mic_ppv REAL,
|
||||
project TEXT,
|
||||
client TEXT,
|
||||
operator TEXT,
|
||||
sensor_location TEXT,
|
||||
sample_rate INTEGER,
|
||||
record_type TEXT,
|
||||
false_trigger INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||
UNIQUE(serial, timestamp)
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO events SELECT * FROM events_old;
|
||||
DROP TABLE events_old;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_events_serial ON events(serial);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_timestamp ON events(timestamp);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_session ON events(session_id);
|
||||
""")
|
||||
log.info("_migrate: events table rebuilt OK")
|
||||
|
||||
# Migration 2: change monitor_log UNIQUE from (serial, waveform_key) to
|
||||
# (serial, start_time) — same reasoning as events.
|
||||
row = conn.execute(
|
||||
"SELECT sql FROM sqlite_master WHERE type='table' AND name='monitor_log'"
|
||||
).fetchone()
|
||||
if row and "UNIQUE(serial, start_time)" not in row[0]:
|
||||
log.info("_migrate: rebuilding monitor_log table — UNIQUE(serial, start_time)")
|
||||
conn.executescript("""
|
||||
ALTER TABLE monitor_log RENAME TO monitor_log_old;
|
||||
|
||||
CREATE TABLE monitor_log (
|
||||
id TEXT PRIMARY KEY,
|
||||
serial TEXT NOT NULL,
|
||||
waveform_key TEXT NOT NULL,
|
||||
session_id TEXT,
|
||||
start_time TEXT,
|
||||
stop_time TEXT,
|
||||
duration_seconds REAL,
|
||||
geo_threshold_ips REAL,
|
||||
created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%SZ', 'now')),
|
||||
UNIQUE(serial, start_time)
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO monitor_log SELECT * FROM monitor_log_old;
|
||||
DROP TABLE monitor_log_old;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_serial ON monitor_log(serial);
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_start ON monitor_log(start_time);
|
||||
CREATE INDEX IF NOT EXISTS idx_monitor_log_session ON monitor_log(session_id);
|
||||
""")
|
||||
log.info("_migrate: monitor_log table rebuilt OK")
|
||||
|
||||
@staticmethod
|
||||
def _iso(dt: Optional[datetime.datetime]) -> Optional[str]:
|
||||
return dt.isoformat() if dt is not None else None
|
||||
|
||||
@staticmethod
|
||||
def _new_id() -> str:
|
||||
return str(uuid.uuid4())
|
||||
|
||||
# ── ACH sessions ──────────────────────────────────────────────────────────
|
||||
|
||||
def insert_ach_session(
|
||||
self,
|
||||
*,
|
||||
serial: str,
|
||||
peer: Optional[str] = None,
|
||||
events_downloaded: int = 0,
|
||||
monitor_entries: int = 0,
|
||||
duration_seconds: Optional[float] = None,
|
||||
session_time: Optional[datetime.datetime] = None,
|
||||
) -> str:
|
||||
"""Insert a new ACH session row. Returns the new session UUID."""
|
||||
sid = self._new_id()
|
||||
ts = self._iso(session_time or datetime.datetime.utcnow())
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO ach_sessions
|
||||
(id, serial, session_time, peer,
|
||||
events_downloaded, monitor_entries, duration_seconds)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(sid, serial, ts, peer,
|
||||
events_downloaded, monitor_entries, duration_seconds),
|
||||
)
|
||||
log.debug("ach_session inserted: %s serial=%s events=%d monitor=%d",
|
||||
sid, serial, events_downloaded, monitor_entries)
|
||||
return sid
|
||||
|
||||
def get_sessions(
|
||||
self,
|
||||
serial: Optional[str] = None,
|
||||
limit: int = 50,
|
||||
) -> list[dict]:
|
||||
"""Return recent ACH sessions, newest first."""
|
||||
with self._connect() as conn:
|
||||
if serial:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM ach_sessions WHERE serial=? "
|
||||
"ORDER BY session_time DESC LIMIT ?",
|
||||
(serial, limit),
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM ach_sessions ORDER BY session_time DESC LIMIT ?",
|
||||
(limit,),
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
# ── Events ────────────────────────────────────────────────────────────────
|
||||
|
||||
def insert_events(
|
||||
self,
|
||||
events: list[Event],
|
||||
*,
|
||||
serial: str,
|
||||
session_id: Optional[str] = None,
|
||||
) -> tuple[int, int]:
|
||||
"""
|
||||
Insert triggered events. Silently skips duplicates (serial+timestamp).
|
||||
Returns (inserted, skipped).
|
||||
"""
|
||||
inserted = skipped = 0
|
||||
with self._connect() as conn:
|
||||
for ev in events:
|
||||
key = ev._waveform_key.hex() if ev._waveform_key else None
|
||||
if key is None:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
ts = None
|
||||
if ev.timestamp:
|
||||
try:
|
||||
ts = datetime.datetime(
|
||||
ev.timestamp.year, ev.timestamp.month, ev.timestamp.day,
|
||||
ev.timestamp.hour, ev.timestamp.minute, ev.timestamp.second,
|
||||
).isoformat()
|
||||
except Exception:
|
||||
ts = str(ev.timestamp)
|
||||
|
||||
pv = ev.peak_values
|
||||
pi = ev.project_info
|
||||
|
||||
try:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO events
|
||||
(id, serial, waveform_key, session_id, timestamp,
|
||||
tran_ppv, vert_ppv, long_ppv, peak_vector_sum, mic_ppv,
|
||||
project, client, operator, sensor_location,
|
||||
sample_rate, record_type)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
self._new_id(), serial, key, session_id, ts,
|
||||
pv.tran if pv else None,
|
||||
pv.vert if pv else None,
|
||||
pv.long if pv else None,
|
||||
pv.peak_vector_sum if pv else None,
|
||||
pv.micl if pv else None,
|
||||
pi.project if pi else None,
|
||||
pi.client if pi else None,
|
||||
pi.operator if pi else None,
|
||||
pi.sensor_location if pi else None,
|
||||
ev.sample_rate,
|
||||
ev.record_type,
|
||||
),
|
||||
)
|
||||
inserted += 1
|
||||
except sqlite3.IntegrityError:
|
||||
skipped += 1
|
||||
|
||||
log.debug("insert_events serial=%s inserted=%d skipped=%d",
|
||||
serial, inserted, skipped)
|
||||
return inserted, skipped
|
||||
|
||||
def query_events(
|
||||
self,
|
||||
serial: Optional[str] = None,
|
||||
from_dt: Optional[datetime.datetime] = None,
|
||||
to_dt: Optional[datetime.datetime] = None,
|
||||
false_trigger: Optional[bool] = None,
|
||||
limit: int = 500,
|
||||
offset: int = 0,
|
||||
) -> list[dict]:
|
||||
"""Query events with optional filters. Returns newest first."""
|
||||
clauses: list[str] = []
|
||||
params: list = []
|
||||
|
||||
if serial:
|
||||
clauses.append("serial = ?")
|
||||
params.append(serial)
|
||||
if from_dt:
|
||||
clauses.append("timestamp >= ?")
|
||||
params.append(from_dt.isoformat())
|
||||
if to_dt:
|
||||
clauses.append("timestamp <= ?")
|
||||
params.append(to_dt.isoformat())
|
||||
if false_trigger is not None:
|
||||
clauses.append("false_trigger = ?")
|
||||
params.append(1 if false_trigger else 0)
|
||||
|
||||
where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
|
||||
params += [limit, offset]
|
||||
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
f"SELECT * FROM events {where} "
|
||||
f"ORDER BY timestamp DESC LIMIT ? OFFSET ?",
|
||||
params,
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
def set_false_trigger(self, event_id: str, value: bool) -> bool:
|
||||
"""Set or clear the false_trigger flag on an event. Returns True if found."""
|
||||
with self._connect() as conn:
|
||||
cur = conn.execute(
|
||||
"UPDATE events SET false_trigger=? WHERE id=?",
|
||||
(1 if value else 0, event_id),
|
||||
)
|
||||
return cur.rowcount > 0
|
||||
|
||||
# ── Monitor log ───────────────────────────────────────────────────────────
|
||||
|
||||
def insert_monitor_log(
|
||||
self,
|
||||
entries: list[MonitorLogEntry],
|
||||
*,
|
||||
session_id: Optional[str] = None,
|
||||
) -> tuple[int, int]:
|
||||
"""
|
||||
Insert monitor log entries. Silently skips duplicates (serial+start_time).
|
||||
Returns (inserted, skipped).
|
||||
"""
|
||||
inserted = skipped = 0
|
||||
with self._connect() as conn:
|
||||
for e in entries:
|
||||
try:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO monitor_log
|
||||
(id, serial, waveform_key, session_id,
|
||||
start_time, stop_time, duration_seconds,
|
||||
geo_threshold_ips)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
self._new_id(),
|
||||
e.serial or "",
|
||||
e.key,
|
||||
session_id,
|
||||
self._iso(e.start_time),
|
||||
self._iso(e.stop_time),
|
||||
e.duration_seconds,
|
||||
e.geo_threshold_ips,
|
||||
),
|
||||
)
|
||||
inserted += 1
|
||||
except sqlite3.IntegrityError:
|
||||
skipped += 1
|
||||
|
||||
log.debug("insert_monitor_log inserted=%d skipped=%d", inserted, skipped)
|
||||
return inserted, skipped
|
||||
|
||||
def query_monitor_log(
|
||||
self,
|
||||
serial: Optional[str] = None,
|
||||
from_dt: Optional[datetime.datetime] = None,
|
||||
to_dt: Optional[datetime.datetime] = None,
|
||||
limit: int = 500,
|
||||
offset: int = 0,
|
||||
) -> list[dict]:
|
||||
"""Query monitor log entries with optional filters. Returns newest first."""
|
||||
clauses: list[str] = []
|
||||
params: list = []
|
||||
|
||||
if serial:
|
||||
clauses.append("serial = ?")
|
||||
params.append(serial)
|
||||
if from_dt:
|
||||
clauses.append("start_time >= ?")
|
||||
params.append(from_dt.isoformat())
|
||||
if to_dt:
|
||||
clauses.append("start_time <= ?")
|
||||
params.append(to_dt.isoformat())
|
||||
|
||||
where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
|
||||
params += [limit, offset]
|
||||
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
f"SELECT * FROM monitor_log {where} "
|
||||
f"ORDER BY start_time DESC LIMIT ? OFFSET ?",
|
||||
params,
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
# ── Fleet overview ────────────────────────────────────────────────────────
|
||||
|
||||
def query_units(self) -> list[dict]:
|
||||
"""
|
||||
Return one row per known serial with summary stats:
|
||||
last_seen, total_events, total_monitor_entries.
|
||||
"""
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT
|
||||
s.serial,
|
||||
MAX(s.session_time) AS last_seen,
|
||||
SUM(s.events_downloaded) AS total_events,
|
||||
SUM(s.monitor_entries) AS total_monitor_entries,
|
||||
COUNT(*) AS total_sessions
|
||||
FROM ach_sessions s
|
||||
GROUP BY s.serial
|
||||
ORDER BY last_seen DESC
|
||||
"""
|
||||
).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
+274
-1
@@ -34,8 +34,11 @@ or:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
@@ -59,6 +62,7 @@ from minimateplus.protocol import ProtocolError
|
||||
from minimateplus.models import ComplianceConfig, DeviceInfo, Event, PeakValues, ProjectInfo, Timestamp
|
||||
from minimateplus.transport import TcpTransport, DEFAULT_TCP_PORT
|
||||
from sfm.cache import SFMCache, get_cache
|
||||
from sfm.database import SeismoDb
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
@@ -89,6 +93,151 @@ app.add_middleware(
|
||||
)
|
||||
|
||||
|
||||
# ── DB ────────────────────────────────────────────────────────────────────────
|
||||
# Shared SeismoDb instance. Path can be overridden by --db-path at startup,
|
||||
# or defaults to bridges/captures/seismo_relay.db relative to the repo root.
|
||||
|
||||
_DEFAULT_DB_PATH = Path(__file__).parent.parent / "bridges" / "captures" / "seismo_relay.db"
|
||||
_db: Optional[SeismoDb] = None
|
||||
|
||||
|
||||
def _get_db() -> SeismoDb:
|
||||
global _db
|
||||
if _db is None:
|
||||
_db = SeismoDb(_DEFAULT_DB_PATH)
|
||||
return _db
|
||||
|
||||
|
||||
# ── Live device cache ─────────────────────────────────────────────────────────
|
||||
# In-memory cache for live device data. Avoids re-dialing the device on every
|
||||
# request when the data hasn't changed.
|
||||
#
|
||||
# Keyed by conn_key ("tcp:host:port" or "serial:port:baud").
|
||||
# Does NOT persist across server restarts — this is purely an in-process cache
|
||||
# to reduce TCP round-trips and cellular data usage.
|
||||
#
|
||||
# Invalidation rules:
|
||||
# device_info — cached until POST /device/config marks it dirty
|
||||
# events — cached by (conn_key, device_event_count); re-fetched when
|
||||
# a quick count_events() probe shows new events on the device
|
||||
# monitor_status — 30-second TTL (changes frequently during monitoring)
|
||||
# waveforms — permanent (immutable once recorded; indexed by conn_key+idx)
|
||||
#
|
||||
# All endpoints accept ?force=true to bypass the cache and re-read from device.
|
||||
|
||||
_MONITOR_STATUS_TTL = 30.0 # seconds
|
||||
|
||||
|
||||
class _LiveCache:
|
||||
"""
|
||||
Thread-safe in-memory cache for live SFM device data.
|
||||
One singleton per server process.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
# conn_key → serialised device info dict
|
||||
self._device_info: dict[str, dict] = {}
|
||||
# conn_key → (device_event_count_when_cached, [event dicts])
|
||||
self._events: dict[str, tuple[int, list]] = {}
|
||||
# conn_key → (fetched_at_unix, status_dict)
|
||||
self._monitor_status: dict[str, tuple[float, dict]] = {}
|
||||
# conn_key → bool (True = re-read device on next /device/info)
|
||||
self._config_dirty: dict[str, bool] = {}
|
||||
# (conn_key, event_index) → waveform dict (permanent)
|
||||
self._waveforms: dict[tuple, dict] = {}
|
||||
|
||||
# ── Connection key ────────────────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def make_conn_key(
|
||||
host: Optional[str],
|
||||
tcp_port: int,
|
||||
port: Optional[str],
|
||||
baud: int,
|
||||
) -> str:
|
||||
if host:
|
||||
return f"tcp:{host}:{tcp_port}"
|
||||
return f"serial:{port}:{baud}"
|
||||
|
||||
# ── Device info ───────────────────────────────────────────────────────────
|
||||
|
||||
def get_device_info(self, conn_key: str) -> Optional[dict]:
|
||||
with self._lock:
|
||||
if self._config_dirty.get(conn_key):
|
||||
return None
|
||||
return self._device_info.get(conn_key)
|
||||
|
||||
def set_device_info(self, conn_key: str, info: dict) -> None:
|
||||
with self._lock:
|
||||
self._device_info[conn_key] = info
|
||||
self._config_dirty[conn_key] = False
|
||||
|
||||
# ── Events ────────────────────────────────────────────────────────────────
|
||||
|
||||
def get_events(self, conn_key: str, device_count: int) -> Optional[list]:
|
||||
"""
|
||||
Return cached events if the device's current event count matches what
|
||||
we had when we last fetched. Returns None (cache miss) otherwise.
|
||||
"""
|
||||
with self._lock:
|
||||
if self._config_dirty.get(conn_key):
|
||||
return None
|
||||
entry = self._events.get(conn_key)
|
||||
if entry is None:
|
||||
return None
|
||||
cached_count, events = entry
|
||||
return events if cached_count == device_count else None
|
||||
|
||||
def set_events(self, conn_key: str, device_count: int, events: list) -> None:
|
||||
with self._lock:
|
||||
self._events[conn_key] = (device_count, events)
|
||||
|
||||
# ── Monitor status ────────────────────────────────────────────────────────
|
||||
|
||||
def get_monitor_status(self, conn_key: str) -> Optional[dict]:
|
||||
with self._lock:
|
||||
entry = self._monitor_status.get(conn_key)
|
||||
if entry is None:
|
||||
return None
|
||||
fetched_at, status = entry
|
||||
if time.time() - fetched_at > _MONITOR_STATUS_TTL:
|
||||
return None
|
||||
return status
|
||||
|
||||
def set_monitor_status(self, conn_key: str, status: dict) -> None:
|
||||
with self._lock:
|
||||
self._monitor_status[conn_key] = (time.time(), status)
|
||||
|
||||
def invalidate_monitor_status(self, conn_key: str) -> None:
|
||||
with self._lock:
|
||||
self._monitor_status.pop(conn_key, None)
|
||||
|
||||
# ── Config dirty flag ─────────────────────────────────────────────────────
|
||||
|
||||
def mark_config_dirty(self, conn_key: str) -> None:
|
||||
"""
|
||||
Called after a successful POST /device/config write.
|
||||
Forces next /device/info and /device/events to re-read from the device.
|
||||
"""
|
||||
with self._lock:
|
||||
self._config_dirty[conn_key] = True
|
||||
self._events.pop(conn_key, None)
|
||||
|
||||
# ── Waveforms (permanent cache) ───────────────────────────────────────────
|
||||
|
||||
def get_waveform(self, conn_key: str, index: int) -> Optional[dict]:
|
||||
with self._lock:
|
||||
return self._waveforms.get((conn_key, index))
|
||||
|
||||
def set_waveform(self, conn_key: str, index: int, waveform: dict) -> None:
|
||||
with self._lock:
|
||||
self._waveforms[(conn_key, index)] = waveform
|
||||
|
||||
|
||||
_live_cache = _LiveCache()
|
||||
|
||||
|
||||
# ── Serialisers ────────────────────────────────────────────────────────────────
|
||||
# Plain dict helpers — avoids a Pydantic dependency in the library layer.
|
||||
|
||||
@@ -281,6 +430,12 @@ def webapp():
|
||||
return str(Path(__file__).parent / "sfm_webapp.html")
|
||||
|
||||
|
||||
@app.get("/waveform", response_class=FileResponse)
|
||||
def waveform_viewer():
|
||||
"""Serve the standalone waveform viewer."""
|
||||
return str(Path(__file__).parent / "waveform_viewer.html")
|
||||
|
||||
|
||||
@app.get("/device/info")
|
||||
def device_info(
|
||||
port: Optional[str] = Query(None, description="Serial port (e.g. COM5, /dev/ttyUSB0)"),
|
||||
@@ -365,6 +520,11 @@ def device_events(
|
||||
|
||||
Supply either *port* (serial) or *host* (TCP/modem).
|
||||
|
||||
**Caching:** a quick count_events() probe (~2s) is performed first. If the
|
||||
device's event count matches the cached count, the cached response is returned
|
||||
immediately without a full download. Pass ?force=true to skip this and always
|
||||
re-download.
|
||||
|
||||
Pass debug=true to include raw_record_hex in each event — useful for
|
||||
verifying field offsets against the protocol reference.
|
||||
|
||||
@@ -494,8 +654,16 @@ def device_events(
|
||||
cache.set_events(conn_key, serialised)
|
||||
cache.set_device_info(conn_key, _serialise_device_info(info))
|
||||
|
||||
serialised_info = _serialise_device_info(info)
|
||||
serialised_events = [_serialise_event(ev, debug=debug) for ev in events]
|
||||
|
||||
# Update cache (skip if debug=True — raw hex blobs shouldn't pollute the cache)
|
||||
if not debug:
|
||||
_live_cache.set_device_info(conn_key, serialised_info)
|
||||
_live_cache.set_events(conn_key, len(events), serialised_events)
|
||||
|
||||
return {
|
||||
"device": _serialise_device_info(info),
|
||||
"device": serialised_info,
|
||||
"event_count": len(events),
|
||||
"events": serialised,
|
||||
}
|
||||
@@ -738,6 +906,7 @@ def device_config(
|
||||
422 if neither port nor host is provided.
|
||||
"""
|
||||
changed = body.model_dump(exclude_none=True)
|
||||
conn_key = _live_cache.make_conn_key(host, tcp_port, port, baud)
|
||||
log.info("POST /device/config port=%s host=%s fields=%s", port, host, list(changed.keys()))
|
||||
|
||||
try:
|
||||
@@ -859,6 +1028,7 @@ def device_monitor_start(
|
||||
|
||||
Sends SUB 0x96 and waits for ack SUB 0x69.
|
||||
"""
|
||||
conn_key = _live_cache.make_conn_key(host, tcp_port, port, baud)
|
||||
with _build_client(port=port, baud=baud, host=host, tcp_port=tcp_port) as client:
|
||||
try:
|
||||
client.poll()
|
||||
@@ -884,6 +1054,7 @@ def device_monitor_stop(
|
||||
|
||||
Sends SUB 0x97 and waits for ack SUB 0x68.
|
||||
"""
|
||||
conn_key = _live_cache.make_conn_key(host, tcp_port, port, baud)
|
||||
with _build_client(port=port, baud=baud, host=host, tcp_port=tcp_port) as client:
|
||||
try:
|
||||
client.poll()
|
||||
@@ -929,6 +1100,108 @@ def cache_clear_device(
|
||||
return {"status": "cleared", "conn_key": conn_key, "deleted": counts}
|
||||
|
||||
|
||||
# ── DB read endpoints ─────────────────────────────────────────────────────────
|
||||
#
|
||||
# These endpoints expose the seismo-relay SQLite DB written by ach_server.py.
|
||||
# All queries are read-only. Terra-view calls these to build project event
|
||||
# views, unit history panels, and (eventually) vibration summary reports.
|
||||
|
||||
|
||||
@app.get("/db/units")
|
||||
def db_units() -> list[dict]:
|
||||
"""
|
||||
Return one row per known serial with summary stats:
|
||||
last_seen, total_events, total_monitor_entries, total_sessions.
|
||||
"""
|
||||
return _get_db().query_units()
|
||||
|
||||
|
||||
@app.get("/db/events")
|
||||
def db_events(
|
||||
serial: Optional[str] = Query(None, description="Filter by unit serial (e.g. BE11529)"),
|
||||
from_dt: Optional[str] = Query(None, description="ISO-8601 start datetime (inclusive)"),
|
||||
to_dt: Optional[str] = Query(None, description="ISO-8601 end datetime (inclusive)"),
|
||||
false_trigger: Optional[bool] = Query(None, description="Filter by false_trigger flag"),
|
||||
limit: int = Query(500, description="Max rows to return (default 500)"),
|
||||
offset: int = Query(0, description="Pagination offset"),
|
||||
) -> dict:
|
||||
"""
|
||||
Query triggered events from the DB.
|
||||
|
||||
Returns events newest-first. All filter params are optional.
|
||||
|
||||
Example:
|
||||
GET /db/events?serial=BE11529&from_dt=2026-04-01&limit=100
|
||||
"""
|
||||
from_parsed = datetime.datetime.fromisoformat(from_dt) if from_dt else None
|
||||
to_parsed = datetime.datetime.fromisoformat(to_dt) if to_dt else None
|
||||
|
||||
rows = _get_db().query_events(
|
||||
serial=serial,
|
||||
from_dt=from_parsed,
|
||||
to_dt=to_parsed,
|
||||
false_trigger=false_trigger,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
return {"count": len(rows), "events": rows}
|
||||
|
||||
|
||||
@app.patch("/db/events/{event_id}/false_trigger")
|
||||
def db_set_false_trigger(
|
||||
event_id: str,
|
||||
value: bool = Query(..., description="True to flag as false trigger, False to clear"),
|
||||
) -> dict:
|
||||
"""
|
||||
Set or clear the false_trigger flag on a single event.
|
||||
|
||||
Used by the terra-view event review UI.
|
||||
Returns 404 if the event_id is not found.
|
||||
"""
|
||||
found = _get_db().set_false_trigger(event_id, value)
|
||||
if not found:
|
||||
raise HTTPException(status_code=404, detail=f"Event {event_id} not found")
|
||||
return {"status": "ok", "event_id": event_id, "false_trigger": value}
|
||||
|
||||
|
||||
@app.get("/db/monitor_log")
|
||||
def db_monitor_log(
|
||||
serial: Optional[str] = Query(None, description="Filter by unit serial"),
|
||||
from_dt: Optional[str] = Query(None, description="ISO-8601 start datetime (inclusive)"),
|
||||
to_dt: Optional[str] = Query(None, description="ISO-8601 end datetime (inclusive)"),
|
||||
limit: int = Query(500, description="Max rows to return"),
|
||||
offset: int = Query(0, description="Pagination offset"),
|
||||
) -> dict:
|
||||
"""
|
||||
Query monitor log entries (continuous monitoring intervals) from the DB.
|
||||
|
||||
Returns entries newest-first.
|
||||
"""
|
||||
from_parsed = datetime.datetime.fromisoformat(from_dt) if from_dt else None
|
||||
to_parsed = datetime.datetime.fromisoformat(to_dt) if to_dt else None
|
||||
|
||||
rows = _get_db().query_monitor_log(
|
||||
serial=serial,
|
||||
from_dt=from_parsed,
|
||||
to_dt=to_parsed,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
return {"count": len(rows), "entries": rows}
|
||||
|
||||
|
||||
@app.get("/db/sessions")
|
||||
def db_sessions(
|
||||
serial: Optional[str] = Query(None, description="Filter by unit serial"),
|
||||
limit: int = Query(50, description="Max rows to return"),
|
||||
) -> dict:
|
||||
"""
|
||||
Query ACH call-home sessions from the DB, newest first.
|
||||
"""
|
||||
rows = _get_db().get_sessions(serial=serial, limit=limit)
|
||||
return {"count": len(rows), "sessions": rows}
|
||||
|
||||
|
||||
# ── Entry point ────────────────────────────────────────────────────────────────
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+676
-17
@@ -110,8 +110,7 @@
|
||||
.btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
|
||||
.btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
|
||||
|
||||
#connect-btn { background: var(--green); color: #fff; margin-left: auto; }
|
||||
#connect-btn:hover { background: var(--green-lt); }
|
||||
/* #connect-btn styles moved to #live-connect-bar block */
|
||||
|
||||
/* ── Device info bar ── */
|
||||
#device-bar {
|
||||
@@ -448,6 +447,196 @@
|
||||
font-size: 14px;
|
||||
margin-top: 40px;
|
||||
}
|
||||
|
||||
/* ── DB tabs (History / Units / Monitor Log / Sessions) ── */
|
||||
.db-tab-pane { padding: 0; flex-direction: column; overflow: hidden; }
|
||||
.db-tab-pane.active { display: flex; }
|
||||
|
||||
.db-toolbar {
|
||||
background: var(--surface);
|
||||
border-bottom: 1px solid var(--border2);
|
||||
padding: 8px 18px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
flex-wrap: wrap;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.db-toolbar label { color: var(--text-dim); font-size: 11px; white-space: nowrap; }
|
||||
.db-toolbar input[type="text"],
|
||||
.db-toolbar input[type="date"],
|
||||
.db-toolbar select { font-size: 12px; padding: 4px 8px; }
|
||||
.db-toolbar select#db-serial-filter { width: 120px; }
|
||||
.db-toolbar input.date-input { width: 130px; }
|
||||
.db-toolbar-spacer { flex: 1; }
|
||||
.db-count-badge {
|
||||
color: var(--text-mute);
|
||||
font-size: 11px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.db-scroll { flex: 1; overflow-y: auto; padding: 14px 18px; }
|
||||
|
||||
.db-table-wrap {
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
max-width: 100%;
|
||||
}
|
||||
table.db-table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
font-size: 12px;
|
||||
}
|
||||
table.db-table thead th {
|
||||
background: var(--surface2);
|
||||
color: var(--text-dim);
|
||||
font-size: 10px;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.06em;
|
||||
text-transform: uppercase;
|
||||
padding: 7px 12px;
|
||||
text-align: left;
|
||||
border-bottom: 1px solid var(--border);
|
||||
white-space: nowrap;
|
||||
}
|
||||
table.db-table tbody tr { border-bottom: 1px solid var(--border2); }
|
||||
table.db-table tbody tr:last-child { border-bottom: none; }
|
||||
table.db-table tbody tr:nth-child(even) { background: var(--surface); }
|
||||
table.db-table tbody tr:hover { background: var(--surface2); }
|
||||
table.db-table tbody td {
|
||||
padding: 7px 12px;
|
||||
color: var(--text);
|
||||
white-space: nowrap;
|
||||
font-family: monospace;
|
||||
font-size: 12px;
|
||||
}
|
||||
table.db-table tbody td.td-text {
|
||||
font-family: inherit;
|
||||
max-width: 180px;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
table.db-table tbody td.td-dim { color: var(--text-mute); }
|
||||
table.db-table tbody td.td-key { color: var(--blue-lt); }
|
||||
|
||||
/* PPV color tiers: green < 0.5, amber < 2.0, red ≥ 2.0 in/s */
|
||||
.ppv-ok { color: var(--green-lt); font-weight: 600; }
|
||||
.ppv-warn { color: var(--yellow); font-weight: 600; }
|
||||
.ppv-high { color: var(--red); font-weight: 600; }
|
||||
|
||||
.ft-badge {
|
||||
background: rgba(248,81,73,0.15);
|
||||
border: 1px solid rgba(248,81,73,0.4);
|
||||
border-radius: 4px;
|
||||
color: var(--red);
|
||||
font-family: inherit;
|
||||
font-size: 10px;
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.05em;
|
||||
padding: 1px 6px;
|
||||
}
|
||||
.ft-toggle-btn {
|
||||
background: none;
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
color: var(--text-dim);
|
||||
cursor: pointer;
|
||||
font-size: 11px;
|
||||
padding: 2px 8px;
|
||||
}
|
||||
.ft-toggle-btn:hover { border-color: var(--red); color: var(--red); }
|
||||
.ft-toggle-btn.flagged { border-color: var(--red); color: var(--red); background: rgba(248,81,73,0.1); }
|
||||
|
||||
.db-empty {
|
||||
color: var(--text-mute);
|
||||
font-size: 13px;
|
||||
padding: 40px 0;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
/* Units tab cards */
|
||||
.units-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(200px, 1fr));
|
||||
gap: 12px;
|
||||
max-width: 900px;
|
||||
}
|
||||
.unit-card {
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 14px 16px;
|
||||
cursor: pointer;
|
||||
}
|
||||
.unit-card:hover { border-color: var(--blue-lt); }
|
||||
.unit-card .uc-serial {
|
||||
font-size: 16px;
|
||||
font-weight: 700;
|
||||
font-family: monospace;
|
||||
color: var(--blue-lt);
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.unit-card .uc-stat {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.unit-card .uc-label { font-size: 11px; color: var(--text-mute); }
|
||||
.unit-card .uc-val { font-size: 12px; color: var(--text); font-family: monospace; }
|
||||
|
||||
/* ── Section switcher ── */
|
||||
.section-switcher {
|
||||
display: flex;
|
||||
gap: 3px;
|
||||
background: var(--bg);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 7px;
|
||||
padding: 3px;
|
||||
}
|
||||
.section-btn {
|
||||
border: none;
|
||||
border-radius: 5px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
padding: 4px 14px;
|
||||
transition: background 0.12s, color 0.12s;
|
||||
background: none;
|
||||
color: var(--text-dim);
|
||||
white-space: nowrap;
|
||||
}
|
||||
.section-btn:hover { color: var(--text); }
|
||||
.section-btn.active { background: var(--blue); color: #fff; }
|
||||
|
||||
/* ── Section containers ── */
|
||||
#section-live, #section-db {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex: 1;
|
||||
overflow: hidden;
|
||||
min-height: 0;
|
||||
}
|
||||
#section-db { display: none; }
|
||||
|
||||
/* ── Live connect bar (host/port/connect, live section only) ── */
|
||||
#live-connect-bar {
|
||||
background: var(--surface);
|
||||
border-bottom: 1px solid var(--border2);
|
||||
padding: 8px 18px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
flex-shrink: 0;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
#live-connect-bar label.hdr { color: var(--text-dim); font-size: 11px; }
|
||||
#live-connect-bar input[type="text"],
|
||||
#live-connect-bar input[type="number"] { font-size: 12px; padding: 5px 8px; }
|
||||
#live-connect-bar #dev-host { width: 150px; }
|
||||
#live-connect-bar #dev-port { width: 70px; }
|
||||
#connect-btn { margin-left: auto; background: var(--green); color: #fff; }
|
||||
#connect-btn:hover { background: var(--green-lt); }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -460,14 +649,26 @@
|
||||
<label class="hdr">API</label>
|
||||
<input type="text" id="api-base" />
|
||||
</div>
|
||||
<div class="hdr-group">
|
||||
<label class="hdr">Device host</label>
|
||||
<div class="hdr-sep"></div>
|
||||
<div class="section-switcher">
|
||||
<button class="section-btn active" onclick="switchSection('live')">Live Device</button>
|
||||
<button class="section-btn" onclick="switchSection('db')">Database</button>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
SECTION: Live Device
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="section-live">
|
||||
|
||||
<!-- ── Live connect bar ────────────────────────────────────────────── -->
|
||||
<div id="live-connect-bar">
|
||||
<label class="hdr">Host</label>
|
||||
<input type="text" id="dev-host" placeholder="e.g. 63.43.212.232" />
|
||||
<label class="hdr">Port</label>
|
||||
<input type="number" id="dev-port" value="9034" />
|
||||
</div>
|
||||
<button class="btn" id="connect-btn" onclick="connectUnit()">Connect</button>
|
||||
</header>
|
||||
</div>
|
||||
|
||||
<!-- ── Device info bar ─────────────────────────────────────────────── -->
|
||||
<div id="device-bar">
|
||||
@@ -533,11 +734,11 @@
|
||||
<!-- ── Status bar ─────────────────────────────────────────────────── -->
|
||||
<div id="status-bar">Ready — enter device host and click Connect.</div>
|
||||
|
||||
<!-- ── Tab bar ────────────────────────────────────────────────────── -->
|
||||
<div class="tab-bar">
|
||||
<button class="tab-btn active" onclick="switchTab('device')">Device</button>
|
||||
<button class="tab-btn" onclick="switchTab('events')">Events</button>
|
||||
<button class="tab-btn" onclick="switchTab('config')">Config</button>
|
||||
<!-- ── Live tab bar ───────────────────────────────────────────────── -->
|
||||
<div class="tab-bar" id="live-tab-bar">
|
||||
<button class="tab-btn active" data-tab="device" onclick="switchTab('device')">Device</button>
|
||||
<button class="tab-btn" data-tab="events" onclick="switchTab('events')">Events</button>
|
||||
<button class="tab-btn" data-tab="config" onclick="switchTab('config')">Config</button>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
@@ -676,8 +877,157 @@
|
||||
<span id="cfg-status"></span>
|
||||
</div>
|
||||
|
||||
</div><!-- end #tab-config -->
|
||||
|
||||
</div><!-- end #section-live -->
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
SECTION: Database
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="section-db">
|
||||
|
||||
<!-- ── Database tab bar ──────────────────────────────────────────── -->
|
||||
<div class="tab-bar" id="db-tab-bar">
|
||||
<button class="tab-btn active" data-tab="history" onclick="switchTab('history')">History</button>
|
||||
<button class="tab-btn" data-tab="units" onclick="switchTab('units')">Units</button>
|
||||
<button class="tab-btn" data-tab="monlog" onclick="switchTab('monlog')">Monitor Log</button>
|
||||
<button class="tab-btn" data-tab="sessions" onclick="switchTab('sessions')">Sessions</button>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: History (events from DB)
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-history" class="tab-pane db-tab-pane">
|
||||
<div class="db-toolbar">
|
||||
<label>Serial</label>
|
||||
<select id="hist-serial-filter" onchange="loadHistory()">
|
||||
<option value="">All units</option>
|
||||
</select>
|
||||
<label>From</label>
|
||||
<input type="date" class="date-input" id="hist-from" onchange="loadHistory()" />
|
||||
<label>To</label>
|
||||
<input type="date" class="date-input" id="hist-to" onchange="loadHistory()" />
|
||||
<label style="display:flex;align-items:center;gap:5px;cursor:pointer;">
|
||||
<input type="checkbox" id="hist-hide-ft" onchange="loadHistory()" />
|
||||
Hide false triggers
|
||||
</label>
|
||||
<div class="db-toolbar-spacer"></div>
|
||||
<button class="btn btn-ghost" onclick="loadHistory()">↻ Refresh</button>
|
||||
<span class="db-count-badge" id="hist-count"></span>
|
||||
</div>
|
||||
<div class="db-scroll" id="hist-scroll">
|
||||
<div class="db-empty" id="hist-empty" style="display:none">No events found.</div>
|
||||
<div class="db-table-wrap" id="hist-table-wrap" style="display:none">
|
||||
<table class="db-table" id="hist-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Timestamp</th>
|
||||
<th>Serial</th>
|
||||
<th>Tran (in/s)</th>
|
||||
<th>Vert (in/s)</th>
|
||||
<th>Long (in/s)</th>
|
||||
<th>PVS (in/s)</th>
|
||||
<th>Mic (dBL)</th>
|
||||
<th>Project</th>
|
||||
<th>Client</th>
|
||||
<th>Type</th>
|
||||
<th>Key</th>
|
||||
<th></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="hist-tbody"></tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: Units
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-units" class="tab-pane db-tab-pane">
|
||||
<div class="db-toolbar">
|
||||
<div class="db-toolbar-spacer"></div>
|
||||
<button class="btn btn-ghost" onclick="loadUnits()">↻ Refresh</button>
|
||||
<span class="db-count-badge" id="units-count"></span>
|
||||
</div>
|
||||
<div class="db-scroll">
|
||||
<div class="db-empty" id="units-empty" style="display:none">No units in database yet.</div>
|
||||
<div class="units-grid" id="units-grid"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: Monitor Log
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-monlog" class="tab-pane db-tab-pane">
|
||||
<div class="db-toolbar">
|
||||
<label>Serial</label>
|
||||
<select id="monlog-serial-filter" onchange="loadMonitorLog()">
|
||||
<option value="">All units</option>
|
||||
</select>
|
||||
<label>From</label>
|
||||
<input type="date" class="date-input" id="monlog-from" onchange="loadMonitorLog()" />
|
||||
<label>To</label>
|
||||
<input type="date" class="date-input" id="monlog-to" onchange="loadMonitorLog()" />
|
||||
<div class="db-toolbar-spacer"></div>
|
||||
<button class="btn btn-ghost" onclick="loadMonitorLog()">↻ Refresh</button>
|
||||
<span class="db-count-badge" id="monlog-count"></span>
|
||||
</div>
|
||||
<div class="db-scroll" id="monlog-scroll">
|
||||
<div class="db-empty" id="monlog-empty" style="display:none">No monitor log entries found.</div>
|
||||
<div class="db-table-wrap" id="monlog-table-wrap" style="display:none">
|
||||
<table class="db-table" id="monlog-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Start Time</th>
|
||||
<th>Stop Time</th>
|
||||
<th>Duration</th>
|
||||
<th>Serial</th>
|
||||
<th>Geo Threshold</th>
|
||||
<th>Key</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="monlog-tbody"></tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: Sessions
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-sessions" class="tab-pane db-tab-pane">
|
||||
<div class="db-toolbar">
|
||||
<label>Serial</label>
|
||||
<select id="sess-serial-filter" onchange="loadSessions()">
|
||||
<option value="">All units</option>
|
||||
</select>
|
||||
<div class="db-toolbar-spacer"></div>
|
||||
<button class="btn btn-ghost" onclick="loadSessions()">↻ Refresh</button>
|
||||
<span class="db-count-badge" id="sess-count"></span>
|
||||
</div>
|
||||
<div class="db-scroll" id="sess-scroll">
|
||||
<div class="db-empty" id="sess-empty" style="display:none">No ACH sessions recorded yet.</div>
|
||||
<div class="db-table-wrap" id="sess-table-wrap" style="display:none">
|
||||
<table class="db-table" id="sess-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Session Time</th>
|
||||
<th>Serial</th>
|
||||
<th>Peer</th>
|
||||
<th>Events DL'd</th>
|
||||
<th>Monitor Entries</th>
|
||||
<th>Duration (s)</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody id="sess-tbody"></tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div><!-- end #tab-sessions -->
|
||||
|
||||
</div><!-- end #section-db -->
|
||||
|
||||
<!-- ── Script ─────────────────────────────────────────────────────── -->
|
||||
<script>
|
||||
'use strict';
|
||||
@@ -720,22 +1070,53 @@ function deviceParams() {
|
||||
return `host=${encodeURIComponent(devHost())}&tcp_port=${devPort()}`;
|
||||
}
|
||||
|
||||
// ── Section switching ─────────────────────────────────────────────────────────
|
||||
let currentSection = 'live';
|
||||
|
||||
function switchSection(name) {
|
||||
currentSection = name;
|
||||
document.querySelectorAll('.section-btn').forEach(b => {
|
||||
b.classList.toggle('active', b.textContent.toLowerCase().startsWith(name === 'live' ? 'live' : 'data'));
|
||||
});
|
||||
document.getElementById('section-live').style.display = name === 'live' ? 'flex' : 'none';
|
||||
document.getElementById('section-db').style.display = name === 'db' ? 'flex' : 'none';
|
||||
|
||||
// Auto-load DB section on first visit
|
||||
if (name === 'db') {
|
||||
if (!histLoaded) loadHistory();
|
||||
if (!unitsLoaded) loadUnits();
|
||||
}
|
||||
}
|
||||
|
||||
// ── Tab switching ──────────────────────────────────────────────────────────────
|
||||
function switchTab(name) {
|
||||
document.querySelectorAll('.tab-btn').forEach((b, i) => {
|
||||
const names = ['device','events','config'];
|
||||
b.classList.toggle('active', names[i] === name);
|
||||
});
|
||||
// Activate the matching tab button within its own tab bar
|
||||
const btn = document.querySelector(`.tab-btn[data-tab="${name}"]`);
|
||||
if (btn) {
|
||||
btn.closest('.tab-bar').querySelectorAll('.tab-btn')
|
||||
.forEach(b => b.classList.remove('active'));
|
||||
btn.classList.add('active');
|
||||
}
|
||||
|
||||
// Hide all panes in both sections, then show the target
|
||||
document.querySelectorAll('.tab-pane').forEach(p => {
|
||||
p.classList.remove('active');
|
||||
if (p.style.display === 'flex') p.style.display = 'none';
|
||||
});
|
||||
const pane = document.getElementById(`tab-${name}`);
|
||||
if (pane.id === 'tab-events') {
|
||||
if (!pane) return;
|
||||
const needsFlex = pane.id === 'tab-events' || pane.classList.contains('db-tab-pane');
|
||||
if (needsFlex) {
|
||||
pane.style.display = 'flex';
|
||||
} else {
|
||||
pane.classList.add('active');
|
||||
}
|
||||
|
||||
// Auto-load DB tabs on first switch
|
||||
if (name === 'history') { if (!histLoaded) loadHistory(); }
|
||||
if (name === 'units') { if (!unitsLoaded) loadUnits(); }
|
||||
if (name === 'monlog') { if (!monlogLoaded) loadMonitorLog(); }
|
||||
if (name === 'sessions') { if (!sessLoaded) loadSessions(); }
|
||||
}
|
||||
|
||||
// ── Connect ────────────────────────────────────────────────────────────────────
|
||||
@@ -1261,6 +1642,283 @@ function renderWaveform(data) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── DB tabs ────────────────────────────────────────────────────────────────────
|
||||
let histLoaded = false;
|
||||
let unitsLoaded = false;
|
||||
let monlogLoaded = false;
|
||||
let sessLoaded = false;
|
||||
|
||||
// Shared serial filter options — populated from /db/units
|
||||
const _unitSerials = new Set();
|
||||
|
||||
function _ppvClass(v) {
|
||||
if (v == null) return '';
|
||||
if (v >= 2.0) return 'ppv-high';
|
||||
if (v >= 0.5) return 'ppv-warn';
|
||||
return 'ppv-ok';
|
||||
}
|
||||
function _ppvFmt(v) {
|
||||
return v != null ? v.toFixed(5) : '—';
|
||||
}
|
||||
function _fmtTs(ts) {
|
||||
if (!ts) return '—';
|
||||
// ts is ISO string; show date + time, strip trailing seconds if all zeros
|
||||
const d = new Date(ts);
|
||||
return d.toLocaleString();
|
||||
}
|
||||
function _fmtDur(sec) {
|
||||
if (sec == null) return '—';
|
||||
const h = Math.floor(sec / 3600);
|
||||
const m = Math.floor((sec % 3600) / 60);
|
||||
const s = Math.floor(sec % 60);
|
||||
if (h > 0) return `${h}h ${m}m ${s}s`;
|
||||
if (m > 0) return `${m}m ${s}s`;
|
||||
return `${s}s`;
|
||||
}
|
||||
|
||||
function _populateSerialDropdown(selectId, currentVal) {
|
||||
const sel = document.getElementById(selectId);
|
||||
const prev = currentVal ?? sel.value;
|
||||
sel.innerHTML = '<option value="">All units</option>';
|
||||
for (const sn of [..._unitSerials].sort()) {
|
||||
const opt = document.createElement('option');
|
||||
opt.value = sn; opt.textContent = sn;
|
||||
sel.appendChild(opt);
|
||||
}
|
||||
if (prev) sel.value = prev;
|
||||
}
|
||||
|
||||
async function _fetchUnits() {
|
||||
try {
|
||||
const r = await fetch(`${api()}/db/units`);
|
||||
if (!r.ok) return [];
|
||||
return await r.json();
|
||||
} catch { return []; }
|
||||
}
|
||||
|
||||
// ── History tab ────────────────────────────────────────────────────────────────
|
||||
async function loadHistory() {
|
||||
histLoaded = true;
|
||||
const serial = document.getElementById('hist-serial-filter').value;
|
||||
const from_dt = document.getElementById('hist-from').value;
|
||||
const to_dt = document.getElementById('hist-to').value;
|
||||
const hideFT = document.getElementById('hist-hide-ft').checked;
|
||||
|
||||
let url = `${api()}/db/events?limit=500`;
|
||||
if (serial) url += `&serial=${encodeURIComponent(serial)}`;
|
||||
if (from_dt) url += `&from_dt=${encodeURIComponent(from_dt)}`;
|
||||
if (to_dt) url += `&to_dt=${encodeURIComponent(to_dt + 'T23:59:59')}`;
|
||||
|
||||
let data;
|
||||
try {
|
||||
const r = await fetch(url);
|
||||
if (!r.ok) throw new Error(r.statusText);
|
||||
data = await r.json();
|
||||
} catch (e) {
|
||||
document.getElementById('hist-count').textContent = `Error: ${e.message}`;
|
||||
return;
|
||||
}
|
||||
|
||||
let events = data.events || [];
|
||||
if (hideFT) events = events.filter(ev => !ev.false_trigger);
|
||||
|
||||
// Update serial dropdowns with any new serials seen
|
||||
events.forEach(ev => { if (ev.serial) _unitSerials.add(ev.serial); });
|
||||
_populateSerialDropdown('hist-serial-filter');
|
||||
_populateSerialDropdown('monlog-serial-filter');
|
||||
_populateSerialDropdown('sess-serial-filter');
|
||||
|
||||
document.getElementById('hist-count').textContent = `${events.length} event${events.length !== 1 ? 's' : ''}`;
|
||||
const tbody = document.getElementById('hist-tbody');
|
||||
tbody.innerHTML = '';
|
||||
|
||||
if (events.length === 0) {
|
||||
document.getElementById('hist-empty').style.display = 'block';
|
||||
document.getElementById('hist-table-wrap').style.display = 'none';
|
||||
return;
|
||||
}
|
||||
document.getElementById('hist-empty').style.display = 'none';
|
||||
document.getElementById('hist-table-wrap').style.display = 'block';
|
||||
|
||||
for (const ev of events) {
|
||||
const tr = document.createElement('tr');
|
||||
const pvs = ev.peak_vector_sum;
|
||||
const maxPPV = Math.max(ev.tran_ppv ?? 0, ev.vert_ppv ?? 0, ev.long_ppv ?? 0);
|
||||
tr.innerHTML = `
|
||||
<td>${_fmtTs(ev.timestamp)}</td>
|
||||
<td class="td-key">${ev.serial ?? '—'}</td>
|
||||
<td class="${_ppvClass(ev.tran_ppv)}">${_ppvFmt(ev.tran_ppv)}</td>
|
||||
<td class="${_ppvClass(ev.vert_ppv)}">${_ppvFmt(ev.vert_ppv)}</td>
|
||||
<td class="${_ppvClass(ev.long_ppv)}">${_ppvFmt(ev.long_ppv)}</td>
|
||||
<td class="${_ppvClass(pvs)}">${_ppvFmt(pvs)}</td>
|
||||
<td class="td-dim">${ev.mic_ppv != null && ev.mic_ppv > 0 ? (20 * Math.log10(ev.mic_ppv / DBL_REF)).toFixed(1) + ' dBL' : '—'}</td>
|
||||
<td class="td-text">${ev.project ?? '—'}</td>
|
||||
<td class="td-text">${ev.client ?? '—'}</td>
|
||||
<td class="td-dim">${ev.record_type ?? '—'}</td>
|
||||
<td class="td-dim" style="font-size:10px">${ev.waveform_key ?? '—'}</td>
|
||||
<td>${ev.false_trigger ? '<span class="ft-badge">FALSE</span>' : `<button class="ft-toggle-btn" onclick="toggleFalseTrigger(${ev.id}, this)" title="Flag as false trigger">Flag</button>`}</td>
|
||||
`;
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleFalseTrigger(id, btn) {
|
||||
btn.disabled = true;
|
||||
try {
|
||||
const r = await fetch(`${api()}/db/events/${id}/false_trigger?value=true`, { method: 'PATCH' });
|
||||
if (!r.ok) throw new Error(r.statusText);
|
||||
btn.outerHTML = '<span class="ft-badge">FALSE</span>';
|
||||
} catch (e) {
|
||||
btn.disabled = false;
|
||||
alert(`Failed to flag: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Units tab ──────────────────────────────────────────────────────────────────
|
||||
async function loadUnits() {
|
||||
unitsLoaded = true;
|
||||
const units = await _fetchUnits();
|
||||
|
||||
units.forEach(u => { if (u.serial) _unitSerials.add(u.serial); });
|
||||
_populateSerialDropdown('hist-serial-filter');
|
||||
_populateSerialDropdown('monlog-serial-filter');
|
||||
_populateSerialDropdown('sess-serial-filter');
|
||||
|
||||
document.getElementById('units-count').textContent = `${units.length} unit${units.length !== 1 ? 's' : ''}`;
|
||||
const grid = document.getElementById('units-grid');
|
||||
grid.innerHTML = '';
|
||||
|
||||
if (units.length === 0) {
|
||||
document.getElementById('units-empty').style.display = 'block';
|
||||
return;
|
||||
}
|
||||
document.getElementById('units-empty').style.display = 'none';
|
||||
|
||||
for (const u of units) {
|
||||
const card = document.createElement('div');
|
||||
card.className = 'unit-card';
|
||||
card.title = 'Click to filter History by this unit';
|
||||
card.onclick = () => {
|
||||
_populateSerialDropdown('hist-serial-filter', u.serial);
|
||||
switchTab('history');
|
||||
loadHistory();
|
||||
};
|
||||
const lastSeen = u.last_seen ? new Date(u.last_seen).toLocaleDateString() : '—';
|
||||
card.innerHTML = `
|
||||
<div class="uc-serial">${u.serial}</div>
|
||||
<div class="uc-stat"><span class="uc-label">Events</span><span class="uc-val">${u.total_events ?? 0}</span></div>
|
||||
<div class="uc-stat"><span class="uc-label">Monitor entries</span><span class="uc-val">${u.total_monitor_entries ?? 0}</span></div>
|
||||
<div class="uc-stat"><span class="uc-label">Sessions</span><span class="uc-val">${u.total_sessions ?? 0}</span></div>
|
||||
<div class="uc-stat"><span class="uc-label">Last seen</span><span class="uc-val">${lastSeen}</span></div>
|
||||
`;
|
||||
grid.appendChild(card);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Monitor Log tab ────────────────────────────────────────────────────────────
|
||||
async function loadMonitorLog() {
|
||||
monlogLoaded = true;
|
||||
const serial = document.getElementById('monlog-serial-filter').value;
|
||||
const from_dt = document.getElementById('monlog-from').value;
|
||||
const to_dt = document.getElementById('monlog-to').value;
|
||||
|
||||
let url = `${api()}/db/monitor_log?`;
|
||||
if (serial) url += `serial=${encodeURIComponent(serial)}&`;
|
||||
if (from_dt) url += `from_dt=${encodeURIComponent(from_dt)}&`;
|
||||
if (to_dt) url += `to_dt=${encodeURIComponent(to_dt + 'T23:59:59')}&`;
|
||||
|
||||
let data;
|
||||
try {
|
||||
const r = await fetch(url);
|
||||
if (!r.ok) throw new Error(r.statusText);
|
||||
data = await r.json();
|
||||
} catch (e) {
|
||||
document.getElementById('monlog-count').textContent = `Error: ${e.message}`;
|
||||
return;
|
||||
}
|
||||
|
||||
const entries = data.entries || [];
|
||||
entries.forEach(e => { if (e.serial) _unitSerials.add(e.serial); });
|
||||
_populateSerialDropdown('hist-serial-filter');
|
||||
_populateSerialDropdown('monlog-serial-filter');
|
||||
_populateSerialDropdown('sess-serial-filter');
|
||||
|
||||
document.getElementById('monlog-count').textContent = `${entries.length} entr${entries.length !== 1 ? 'ies' : 'y'}`;
|
||||
const tbody = document.getElementById('monlog-tbody');
|
||||
tbody.innerHTML = '';
|
||||
|
||||
if (entries.length === 0) {
|
||||
document.getElementById('monlog-empty').style.display = 'block';
|
||||
document.getElementById('monlog-table-wrap').style.display = 'none';
|
||||
return;
|
||||
}
|
||||
document.getElementById('monlog-empty').style.display = 'none';
|
||||
document.getElementById('monlog-table-wrap').style.display = 'block';
|
||||
|
||||
for (const e of entries) {
|
||||
const tr = document.createElement('tr');
|
||||
tr.innerHTML = `
|
||||
<td>${_fmtTs(e.start_time)}</td>
|
||||
<td>${_fmtTs(e.stop_time)}</td>
|
||||
<td>${_fmtDur(e.duration_seconds)}</td>
|
||||
<td class="td-key">${e.serial ?? '—'}</td>
|
||||
<td>${e.geo_threshold_ips != null ? e.geo_threshold_ips.toFixed(4) + ' in/s' : '—'}</td>
|
||||
<td class="td-dim" style="font-size:10px">${e.key ?? '—'}</td>
|
||||
`;
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Sessions tab ───────────────────────────────────────────────────────────────
|
||||
async function loadSessions() {
|
||||
sessLoaded = true;
|
||||
const serial = document.getElementById('sess-serial-filter').value;
|
||||
|
||||
let url = `${api()}/db/sessions?limit=200`;
|
||||
if (serial) url += `&serial=${encodeURIComponent(serial)}`;
|
||||
|
||||
let data;
|
||||
try {
|
||||
const r = await fetch(url);
|
||||
if (!r.ok) throw new Error(r.statusText);
|
||||
data = await r.json();
|
||||
} catch (e) {
|
||||
document.getElementById('sess-count').textContent = `Error: ${e.message}`;
|
||||
return;
|
||||
}
|
||||
|
||||
const sessions = data.sessions || [];
|
||||
sessions.forEach(s => { if (s.serial) _unitSerials.add(s.serial); });
|
||||
_populateSerialDropdown('hist-serial-filter');
|
||||
_populateSerialDropdown('monlog-serial-filter');
|
||||
_populateSerialDropdown('sess-serial-filter');
|
||||
|
||||
document.getElementById('sess-count').textContent = `${sessions.length} session${sessions.length !== 1 ? 's' : ''}`;
|
||||
const tbody = document.getElementById('sess-tbody');
|
||||
tbody.innerHTML = '';
|
||||
|
||||
if (sessions.length === 0) {
|
||||
document.getElementById('sess-empty').style.display = 'block';
|
||||
document.getElementById('sess-table-wrap').style.display = 'none';
|
||||
return;
|
||||
}
|
||||
document.getElementById('sess-empty').style.display = 'none';
|
||||
document.getElementById('sess-table-wrap').style.display = 'block';
|
||||
|
||||
for (const s of sessions) {
|
||||
const tr = document.createElement('tr');
|
||||
tr.innerHTML = `
|
||||
<td>${_fmtTs(s.session_time)}</td>
|
||||
<td class="td-key">${s.serial ?? '—'}</td>
|
||||
<td class="td-dim">${s.peer ?? '—'}</td>
|
||||
<td>${s.events_downloaded ?? 0}</td>
|
||||
<td>${s.monitor_entries ?? 0}</td>
|
||||
<td>${s.duration_seconds != null ? s.duration_seconds.toFixed(1) : '—'}</td>
|
||||
`;
|
||||
tbody.appendChild(tr);
|
||||
}
|
||||
}
|
||||
|
||||
// ── Keyboard shortcuts ─────────────────────────────────────────────────────────
|
||||
document.addEventListener('keydown', e => {
|
||||
if (e.target.tagName === 'INPUT' || e.target.tagName === 'SELECT') return;
|
||||
@@ -1272,7 +1930,8 @@ document.addEventListener('keydown', e => {
|
||||
// hit localhost:8200, 10.0.0.44:8200, or anything else.
|
||||
document.getElementById('api-base').value = window.location.origin;
|
||||
|
||||
['api-base','dev-host','dev-port'].forEach(id => {
|
||||
// Press Enter in any live connect field to connect
|
||||
['dev-host','dev-port'].forEach(id => {
|
||||
document.getElementById(id)?.addEventListener('keydown', e => { if (e.key === 'Enter') connectUnit(); });
|
||||
});
|
||||
</script>
|
||||
|
||||
@@ -183,7 +183,7 @@
|
||||
<h1>SFM Waveform Viewer</h1>
|
||||
<div class="conn-group">
|
||||
<label>API</label>
|
||||
<input type="text" id="api-base" value="http://localhost:8200" style="width:180px" />
|
||||
<input type="text" id="api-base" style="width:180px" />
|
||||
</div>
|
||||
<div class="conn-group">
|
||||
<label>Device host</label>
|
||||
@@ -588,6 +588,9 @@
|
||||
}
|
||||
}
|
||||
|
||||
// Auto-detect API base from wherever this page was served from
|
||||
document.getElementById('api-base').value = window.location.origin;
|
||||
|
||||
// Allow Enter key on connection inputs to trigger connect
|
||||
['api-base', 'dev-host', 'dev-tcp-port'].forEach(id => {
|
||||
document.getElementById(id).addEventListener('keydown', e => {
|
||||
|
||||
Reference in New Issue
Block a user