From ef88240796b5dd12ae98ded12289952496cd3e12 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Mon, 13 Apr 2026 16:12:07 -0400 Subject: [PATCH] docs: update README to v0.12.0 Rewrites the v0.6.0 README to reflect current project state: ACH server, SQLite DB, SFM REST API with caching, monitor/erase, updated roadmap. Co-Authored-By: Claude Sonnet 4.6 --- README.md | 331 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 170 insertions(+), 161 deletions(-) diff --git a/README.md b/README.md index 2e91799..8c5fd6f 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,16 @@ -# seismo-relay `v0.6.0` +# seismo-relay `v0.12.0` A ground-up replacement for **Blastware** — Instantel's aging Windows-only software for managing MiniMate Plus seismographs. -Built in Python. Runs on Windows. Connects to instruments over direct RS-232 -or cellular modem (Sierra Wireless RV50 / RV55). +Built in Python. Runs on Windows, Linux, or macOS. Connects to instruments +over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55). -> **Status:** Active development. Full read pipeline working end-to-end: -> device info, compliance config (with geo thresholds), event download with -> true event-time metadata (project / client / operator / sensor location -> sourced from the device at record-time via SUB 5A). Write commands in progress. -> See [CHANGELOG.md](CHANGELOG.md) for version history. +> **Status:** Active development. Full read + write + erase + monitoring +> pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server +> handles inbound unit connections, downloads events, and persists everything +> to a SQLite database. SFM REST API exposes device control and DB queries. +> See [CHANGELOG.md](CHANGELOG.md) for full version history. --- @@ -21,26 +21,28 @@ seismo-relay/ ├── seismo_lab.py ← Main GUI (Bridge + Analyzer + Console tabs) │ ├── minimateplus/ ← MiniMate Plus client library -│ ├── transport.py ← SerialTransport and TcpTransport -│ ├── protocol.py ← DLE frame layer (read/write/parse) -│ ├── client.py ← High-level client (connect, get_config, etc.) -│ ├── framing.py ← Frame builder/parser primitives -│ └── models.py ← DeviceInfo, EventRecord, etc. +│ ├── transport.py ← SerialTransport, TcpTransport, SocketTransport +│ ├── protocol.py ← DLE frame layer, SUB command dispatch +│ ├── client.py ← High-level client (connect, get_events, push_config, …) +│ ├── framing.py ← Frame builders, DLE codec, S3FrameParser +│ └── models.py ← DeviceInfo, Event, ComplianceConfig, MonitorLogEntry, … │ -├── sfm/ ← SFM REST API server (FastAPI) -│ └── server.py ← /device/info, /device/events, /device/event +├── sfm/ ← SFM REST API server (FastAPI, port 8200) +│ ├── server.py ← All device + DB endpoints +│ ├── database.py ← SeismoDb — SQLite persistence layer +│ └── sfm_webapp.html ← Embedded web UI (served at /) │ ├── bridges/ -│ ├── s3-bridge/ -│ │ └── s3_bridge.py ← RS-232 serial bridge (capture tool) +│ ├── ach_server.py ← Inbound ACH call-home server (main production server) +│ ├── ach_mitm.py ← Transparent MITM proxy for capturing BW sessions +│ ├── s3-bridge/ ← RS-232 serial bridge (capture tool) │ ├── tcp_serial_bridge.py ← Local TCP↔serial bridge (bench testing) -│ ├── gui_bridge.py ← Standalone bridge GUI (legacy) +│ ├── gui_bridge.py ← Standalone bridge GUI │ └── raw_capture.py ← Simple raw capture tool │ ├── parsers/ -│ ├── s3_parser.py ← DLE frame extractor │ ├── s3_analyzer.py ← Session parser, differ, Claude export -│ ├── gui_analyzer.py ← Standalone analyzer GUI (legacy) +│ ├── gui_analyzer.py ← Standalone analyzer GUI │ └── frame_db.py ← SQLite frame database │ └── docs/ @@ -51,123 +53,88 @@ seismo-relay/ ## Quick start -### Seismo Lab (main GUI) +### ACH inbound server (production) -The all-in-one tool. Three tabs: **Bridge**, **Analyzer**, **Console**. +Listens for inbound unit call-homes, downloads all new events and monitor log +entries, and writes everything to `bridges/captures/seismo_relay.db`. +```bash +python bridges/ach_server.py --port 12345 --output bridges/captures/ ``` -python seismo_lab.py + +Point the unit's ACEmanager **Remote Host** to this machine's IP and **Remote Port** to `12345`. + +Options: +``` +--port N Listen port (default 12345) +--output DIR Capture directory (default bridges/captures/) +--allow-ip IP Allowlist an IP (repeat for multiple; default: accept all) +--max-events N Safety cap for first run (default: unlimited) +--clear-after-download Erase device memory after successful download +--verbose Debug logging ``` ### SFM REST server -Exposes MiniMate Plus commands as a REST API for integration with other systems. +Exposes device control and DB queries as a REST API. Proxied by terra-view. -``` -cd sfm -uvicorn server:app --reload +```bash +python sfm/server.py # default: 0.0.0.0:8200 +python -m uvicorn sfm.server:app --host 0.0.0.0 --port 8200 --reload ``` -**Endpoints:** +Open `http://localhost:8200` for the embedded web UI, or `http://localhost:8200/docs` +for the interactive API docs. + +### Seismo Lab GUI + +```bash +python seismo_lab.py +``` + +--- + +## SFM REST API + +### Live device endpoints + +Each call dials the device, does its work, and closes the connection. TCP +connections are retried once on `ProtocolError` to handle cold-boot timing. + +**Caching** — frequently-polled endpoints are cached in-process to avoid +redundant TCP round-trips: + +| Method | URL | Cache | +|--------|-----|-------| +| `GET` | `/device/info` | Indefinite; invalidated by `POST /device/config` | +| `GET` | `/device/events` | Count-probe fast path (~2s); full download only when new events detected | +| `GET` | `/device/event/{idx}/waveform` | Permanent per event index | +| `GET` | `/device/monitor/status` | 30-second TTL | +| `POST` | `/device/connect` | — | +| `POST` | `/device/config` | Writes compliance config; invalidates cache | +| `POST` | `/device/monitor/start` | Sends SUB 0x96 | +| `POST` | `/device/monitor/stop` | Sends SUB 0x97 | + +All cached endpoints accept `?force=true` to bypass the cache. + +Transport query params (supply one set): +``` +Serial: ?port=COM5&baud=38400 +TCP: ?host=1.2.3.4&tcp_port=12345 +``` + +### DB read endpoints + +Query the SQLite database written by `ach_server.py`. All read-only except +`PATCH /db/events/{id}/false_trigger`. | Method | URL | Description | |--------|-----|-------------| -| `GET` | `/device/info?port=COM5` | Device info via serial | -| `GET` | `/device/info?host=1.2.3.4&tcp_port=9034` | Device info via cellular modem | -| `GET` | `/device/events?port=COM5` | Event index | -| `GET` | `/device/event?port=COM5&index=0` | Single event record | - ---- - -## Seismo Lab tabs - -### Bridge tab - -Captures live RS-232 traffic between Blastware and the seismograph. Sits in -the middle as a transparent pass-through while logging everything to disk. - -``` -Blastware → COM4 (virtual) ↔ s3_bridge ↔ COM5 (physical) → MiniMate Plus -``` - -Set your COM ports and log directory, then hit **Start Bridge**. Use -**Add Mark** to annotate the capture at specific moments (e.g. "changed -trigger level"). When the bridge starts, the Analyzer tab automatically wires -up to the live files and starts updating in real time. - -### Analyzer tab - -Parses raw captures into DLE-framed protocol sessions, diffs consecutive -sessions to show exactly which bytes changed, and lets you query across all -historical captures via the built-in SQLite database. - -- **Inventory** — all frames in a session, click to drill in -- **Hex Dump** — full payload hex dump with changed-byte annotations -- **Diff** — byte-level before/after diff between sessions -- **Full Report** — plain text session report -- **Query DB** — search across all captures by SUB, direction, or byte value - -Use **Export for Claude** to generate a self-contained `.md` report for -AI-assisted field mapping. - -### Console tab - -Direct connection to a MiniMate Plus — no bridge, no Blastware. Useful for -diagnosing field units over cellular without a full capture session. - -**Connection:** choose Serial (COM port + baud) or TCP (IP + port for -cellular modem). - -**Commands:** -| Button | What it does | -|--------|-------------| -| POLL | Startup handshake — confirms unit is alive and identifies model | -| Serial # | Reads unit serial number | -| Full Config | Reads full 166-byte config block (firmware version, channel scales, etc.) | -| Event Index | Reads stored event list | - -Output is colour-coded: TX in blue, raw RX bytes in teal, decoded fields in -green, errors in red. **Save Log** writes a timestamped `.log` file to -`bridges/captures/`. **Send to Analyzer** injects the captured bytes into the -Analyzer tab for deeper inspection. - ---- - -## Connecting over cellular (RV50 / RV55 modems) - -Field units connect via Sierra Wireless RV50 or RV55 cellular modems. Use -TCP mode in the Console or SFM: - -``` -# Console tab -Transport: TCP -Host: -Port: 9034 ← Device Port in ACEmanager (call-up mode) -``` - -```python -# In code -from minimateplus.transport import TcpTransport -from minimateplus.client import MiniMateClient - -client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0) -info = client.connect() -``` - -### Required ACEmanager settings (Serial tab) - -These must match exactly — a single wrong setting causes the unit to beep -on connect but never respond: - -| Setting | Value | Why | -|---------|-------|-----| -| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate | -| Flow Control | `None` | Hardware flow control blocks unit TX if pins unconnected | -| **Quiet Mode** | **Enable** | **Critical.** Disabled → modem injects `RING`/`CONNECT` onto serial line, corrupting the S3 handshake | -| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency; `5` works but is sluggish | -| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame | -| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect | -| DB9 Serial Echo | `Disable` | Echo corrupts the data stream | +| `GET` | `/db/units` | All known serials with summary stats | +| `GET` | `/db/events` | Triggered events (filter by serial, date range, false_trigger) | +| `GET` | `/db/monitor_log` | Monitoring intervals | +| `GET` | `/db/sessions` | ACH call-home session history | +| `PATCH` | `/db/events/{id}/false_trigger?value=true` | Flag / unflag false triggers | --- @@ -175,25 +142,76 @@ on connect but never respond: ```python from minimateplus import MiniMateClient -from minimateplus.transport import SerialTransport, TcpTransport +from minimateplus.transport import TcpTransport # Serial client = MiniMateClient(port="COM5") # TCP (cellular modem) -client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0) +client = MiniMateClient(transport=TcpTransport("1.2.3.4", 12345), timeout=30.0) with client: - info = client.connect() # DeviceInfo — model, serial, firmware, compliance config - serial = client.get_serial() # Serial number string - config = client.get_config() # Full config block (bytes) - events = client.get_events() # List[EventRecord] with true event-time metadata + # Read + info = client.connect() # DeviceInfo — serial, firmware, compliance config + count = client.count_events() # Number of stored events + keys = client.list_event_keys() # Fast browse walk — event keys only, no download + events = client.get_events() # Full download: headers + peaks + metadata + monitor = client.get_monitor_status() # Battery, memory, is_monitoring flag + log = client.get_monitor_log_entries() # Monitoring intervals (partial 0x2C records) + + # Write + client.apply_config( + sample_rate=1024, + trigger_level_geo=0.5, + project="Bridge Inspection 2026", + client_name="City of Portland", + operator="B. Harrison", + ) + + # Control + client.start_monitoring() # SUB 0x96 + client.stop_monitoring() # SUB 0x97 + client.delete_all_events() # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2) ``` -`get_events()` runs the full download sequence per event: `1E → 0A → 0C → 5A → 1F`. -The SUB 5A bulk waveform stream is used to retrieve `client`, `operator`, and -`sensor_location` as they existed at record time — not backfilled from the current -compliance config. +`get_events()` runs the full per-event sequence: `1E → 0A → 0C → 5A → 1F`. +SUB 5A bulk stream provides `client`, `operator`, and `sensor_location` as they +existed at record time — not backfilled from the current compliance config. + +--- + +## Database + +`ach_server.py` writes to `bridges/captures/seismo_relay.db` (SQLite, WAL mode). +Three tables, all unit-keyed by serial number: + +| Table | Key | Contents | +|-------|-----|----------| +| `ach_sessions` | UUID | Per-call-home audit record: serial, peer IP, events_downloaded, duration | +| `events` | UUID, UNIQUE(serial, waveform_key) | Triggered events: timestamp, PPV per channel, project/client/operator strings, false_trigger flag | +| `monitor_log` | UUID, UNIQUE(serial, waveform_key) | Monitoring intervals: start/stop time, duration, geo threshold | + +Deduplication is by `(serial, waveform_key)` — repeat call-homes or re-runs +never produce duplicate rows. Post-erase key reuse is handled automatically +via the high-water mark in `ach_state.json`. + +--- + +## Connecting over cellular (RV50 / RV55) + +Field units connect via Sierra Wireless RV50 or RV55 cellular modems. + +### Required ACEmanager settings + +| Setting | Value | Why | +|---------|-------|-----| +| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate | +| Flow Control | `None` | Hardware FC blocks TX if pins unconnected | +| **Quiet Mode** | **Enable** | **Critical** — disabled injects `RING`/`CONNECT` onto serial, corrupting the S3 handshake | +| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency | +| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame | +| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect | +| DB9 Serial Echo | `Disable` | Echo corrupts the data stream | --- @@ -204,23 +222,10 @@ compliance config. | DLE | `0x10` | Data Link Escape | | STX | `0x02` | Start of frame | | ETX | `0x03` | End of frame | -| ACK | `0x41` (`'A'`) | Frame-start marker sent before every frame | +| ACK | `0x41` | Frame-start marker sent before every BW frame | | DLE stuffing | `10 10` on wire | Literal `0x10` in payload | -**S3-side frame** (seismograph → Blastware): `ACK DLE+STX [payload] CHK DLE+ETX` - -**De-stuffed payload header:** -``` -[0] CMD 0x10 = BW request, 0x00 = S3 response -[1] ? unknown (0x00 BW / 0x10 S3) -[2] SUB Command/response identifier ← the key field -[3] PAGE_HI Page address high byte -[4] PAGE_LO Page address low byte -[5+] DATA Payload content -``` - -**Response SUB rule:** `response_SUB = 0xFF - request_SUB` -Example: request SUB `0x08` (Event Index) → response SUB `0xF7` +**Response SUB rule:** `response_SUB = 0xFF - request_SUB` (no exceptions) Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/instantel_protocol_reference.md) @@ -228,32 +233,36 @@ Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/insta ## Requirements -``` +```bash pip install pyserial fastapi uvicorn ``` Python 3.10+. Tkinter is included with the standard Python installer on -Windows (make sure "tcl/tk and IDLE" is checked during install). +Windows (check "tcl/tk and IDLE" during install). --- ## Virtual COM ports (bridge capture) -The bridge needs two COM ports on the same PC — one that Blastware connects -to, and one wired to the seismograph. Use a virtual COM port pair -(**com0com** or **VSPD**) to give Blastware a port to talk to. - ``` Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus ``` +Use **com0com** or **VSPD** to create the virtual COM pair on Windows. + --- ## Roadmap -- [x] Event download — pull waveform records from the unit (`1E → 0A → 0C → 5A → 1F`) -- [x] True event-time metadata — project / client / operator / sensor location from SUB 5A -- [ ] Write commands — push config changes to the unit (compliance setup, channel config, trigger settings) -- [ ] ACH inbound server — accept call-home connections from field units -- [ ] Modem manager — push standard configs to RV50/RV55 fleet via Sierra Wireless API -- [ ] Full Blastware parity — complete read/write/download cycle without Blastware +- [x] Full read pipeline — device info, compliance config, event download with true event-time metadata +- [x] Write commands — push compliance config, trigger thresholds, project strings to device +- [x] Erase all events — confirmed erase sequence from live MITM capture +- [x] Monitor control — start/stop monitoring, read battery/memory/status +- [x] Monitor log entries — decode partial 0x2C records (continuous monitoring intervals) +- [x] ACH inbound server — accept call-home connections, download events, dedup by key +- [x] SQLite persistence — events, monitor log, and session history in `seismo_relay.db` +- [x] SFM REST API — device control + DB query endpoints, live device cache +- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing +- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first) +- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object +- [ ] Modem manager — push RV50/RV55 configs via Sierra Wireless API