docs: update README to v0.12.0

Rewrites the v0.6.0 README to reflect current project state: ACH server, SQLite DB, SFM REST API with caching, monitor/erase, updated roadmap. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 16:12:07 -04:00
parent 48d7e94c02
commit 034b3f044d
1 changed files with 170 additions and 161 deletions
@@ -1,16 +1,16 @@
-# seismo-relay  `v0.6.0`
+# seismo-relay  `v0.12.0`

 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing MiniMate Plus seismographs.

-Built in Python. Runs on Windows. Connects to instruments over direct RS-232
-or cellular modem (Sierra Wireless RV50 / RV55).
+Built in Python. Runs on Windows, Linux, or macOS. Connects to instruments
+over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).

-> **Status:** Active development. Full read pipeline working end-to-end:
-> device info, compliance config (with geo thresholds), event download with
-> true event-time metadata (project / client / operator / sensor location
-> sourced from the device at record-time via SUB 5A). Write commands in progress.
-> See [CHANGELOG.md](CHANGELOG.md) for version history.
+> **Status:** Active development. Full read + write + erase + monitoring
+> pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server
+> handles inbound unit connections, downloads events, and persists everything
+> to a SQLite database. SFM REST API exposes device control and DB queries.
+> See [CHANGELOG.md](CHANGELOG.md) for full version history.

 ---

@@ -21,26 +21,28 @@ seismo-relay/
 ├── seismo_lab.py              ← Main GUI (Bridge + Analyzer + Console tabs)
 │
 ├── minimateplus/              ← MiniMate Plus client library
-│   ├── transport.py           ←   SerialTransport and TcpTransport
-│   ├── protocol.py            ←   DLE frame layer (read/write/parse)
-│   ├── client.py              ←   High-level client (connect, get_config, etc.)
-│   ├── framing.py             ←   Frame builder/parser primitives
-│   └── models.py              ←   DeviceInfo, EventRecord, etc.
+│   ├── transport.py           ←   SerialTransport, TcpTransport, SocketTransport
+│   ├── protocol.py            ←   DLE frame layer, SUB command dispatch
+│   ├── client.py              ←   High-level client (connect, get_events, push_config, …)
+│   ├── framing.py             ←   Frame builders, DLE codec, S3FrameParser
+│   └── models.py              ←   DeviceInfo, Event, ComplianceConfig, MonitorLogEntry, …
 │
-├── sfm/                       ← SFM REST API server (FastAPI)
-│   └── server.py              ←   /device/info, /device/events, /device/event
+├── sfm/                       ← SFM REST API server (FastAPI, port 8200)
+│   ├── server.py              ←   All device + DB endpoints
+│   ├── database.py            ←   SeismoDb — SQLite persistence layer
+│   └── sfm_webapp.html        ←   Embedded web UI (served at /)
 │
 ├── bridges/
-│   ├── s3-bridge/
-│   │   └── s3_bridge.py       ←   RS-232 serial bridge (capture tool)
+│   ├── ach_server.py          ←   Inbound ACH call-home server (main production server)
+│   ├── ach_mitm.py            ←   Transparent MITM proxy for capturing BW sessions
+│   ├── s3-bridge/             ←   RS-232 serial bridge (capture tool)
 │   ├── tcp_serial_bridge.py   ←   Local TCP↔serial bridge (bench testing)
-│   ├── gui_bridge.py          ←   Standalone bridge GUI (legacy)
+│   ├── gui_bridge.py          ←   Standalone bridge GUI
 │   └── raw_capture.py         ←   Simple raw capture tool
 │
 ├── parsers/
-│   ├── s3_parser.py           ←   DLE frame extractor
 │   ├── s3_analyzer.py         ←   Session parser, differ, Claude export
-│   ├── gui_analyzer.py        ←   Standalone analyzer GUI (legacy)
+│   ├── gui_analyzer.py        ←   Standalone analyzer GUI
 │   └── frame_db.py            ←   SQLite frame database
 │
 └── docs/
@@ -51,123 +53,88 @@ seismo-relay/

 ## Quick start

-### Seismo Lab (main GUI)
+### ACH inbound server (production)

-The all-in-one tool. Three tabs: **Bridge**, **Analyzer**, **Console**.
+Listens for inbound unit call-homes, downloads all new events and monitor log
+entries, and writes everything to `bridges/captures/seismo_relay.db`.

+```bash
+python bridges/ach_server.py --port 12345 --output bridges/captures/
 ```
-python seismo_lab.py
+
+Point the unit's ACEmanager **Remote Host** to this machine's IP and **Remote Port** to `12345`.
+
+Options:
+```
+--port N            Listen port (default 12345)
+--output DIR        Capture directory (default bridges/captures/)
+--allow-ip IP       Allowlist an IP (repeat for multiple; default: accept all)
+--max-events N      Safety cap for first run (default: unlimited)
+--clear-after-download  Erase device memory after successful download
+--verbose           Debug logging
 ```

 ### SFM REST server

-Exposes MiniMate Plus commands as a REST API for integration with other systems.
+Exposes device control and DB queries as a REST API. Proxied by terra-view.

-```
-cd sfm
-uvicorn server:app --reload
+```bash
+python sfm/server.py               # default: 0.0.0.0:8200
+python -m uvicorn sfm.server:app --host 0.0.0.0 --port 8200 --reload
 ```

-**Endpoints:**
+Open `http://localhost:8200` for the embedded web UI, or `http://localhost:8200/docs`
+for the interactive API docs.
+
+### Seismo Lab GUI
+
+```bash
+python seismo_lab.py
+```
+
+---
+
+## SFM REST API
+
+### Live device endpoints
+
+Each call dials the device, does its work, and closes the connection. TCP
+connections are retried once on `ProtocolError` to handle cold-boot timing.
+
+**Caching** — frequently-polled endpoints are cached in-process to avoid
+redundant TCP round-trips:
+
+| Method | URL | Cache |
+|--------|-----|-------|
+| `GET` | `/device/info` | Indefinite; invalidated by `POST /device/config` |
+| `GET` | `/device/events` | Count-probe fast path (~2s); full download only when new events detected |
+| `GET` | `/device/event/{idx}/waveform` | Permanent per event index |
+| `GET` | `/device/monitor/status` | 30-second TTL |
+| `POST` | `/device/connect` | — |
+| `POST` | `/device/config` | Writes compliance config; invalidates cache |
+| `POST` | `/device/monitor/start` | Sends SUB 0x96 |
+| `POST` | `/device/monitor/stop` | Sends SUB 0x97 |
+
+All cached endpoints accept `?force=true` to bypass the cache.
+
+Transport query params (supply one set):
+```
+Serial:  ?port=COM5&baud=38400
+TCP:     ?host=1.2.3.4&tcp_port=12345
+```
+
+### DB read endpoints
+
+Query the SQLite database written by `ach_server.py`. All read-only except
+`PATCH /db/events/{id}/false_trigger`.

 | Method | URL | Description |
 |--------|-----|-------------|
-| `GET` | `/device/info?port=COM5` | Device info via serial |
-| `GET` | `/device/info?host=1.2.3.4&tcp_port=9034` | Device info via cellular modem |
-| `GET` | `/device/events?port=COM5` | Event index |
-| `GET` | `/device/event?port=COM5&index=0` | Single event record |
-
---
-
-## Seismo Lab tabs
-
-### Bridge tab
-
-Captures live RS-232 traffic between Blastware and the seismograph. Sits in
-the middle as a transparent pass-through while logging everything to disk.
-
-```
-Blastware → COM4 (virtual) ↔ s3_bridge ↔ COM5 (physical) → MiniMate Plus
-```
-
-Set your COM ports and log directory, then hit **Start Bridge**. Use
-**Add Mark** to annotate the capture at specific moments (e.g. "changed
-trigger level"). When the bridge starts, the Analyzer tab automatically wires
-up to the live files and starts updating in real time.
-
-### Analyzer tab
-
-Parses raw captures into DLE-framed protocol sessions, diffs consecutive
-sessions to show exactly which bytes changed, and lets you query across all
-historical captures via the built-in SQLite database.
-
- **Inventory** — all frames in a session, click to drill in
- **Hex Dump** — full payload hex dump with changed-byte annotations
- **Diff** — byte-level before/after diff between sessions
- **Full Report** — plain text session report
- **Query DB** — search across all captures by SUB, direction, or byte value
-
-Use **Export for Claude** to generate a self-contained `.md` report for
-AI-assisted field mapping.
-
-### Console tab
-
-Direct connection to a MiniMate Plus — no bridge, no Blastware. Useful for
-diagnosing field units over cellular without a full capture session.
-
-**Connection:** choose Serial (COM port + baud) or TCP (IP + port for
-cellular modem).
-
-**Commands:**
-| Button | What it does |
-|--------|-------------|
-| POLL | Startup handshake — confirms unit is alive and identifies model |
-| Serial # | Reads unit serial number |
-| Full Config | Reads full 166-byte config block (firmware version, channel scales, etc.) |
-| Event Index | Reads stored event list |
-
-Output is colour-coded: TX in blue, raw RX bytes in teal, decoded fields in
-green, errors in red. **Save Log** writes a timestamped `.log` file to
-`bridges/captures/`. **Send to Analyzer** injects the captured bytes into the
-Analyzer tab for deeper inspection.
-
---
-
-## Connecting over cellular (RV50 / RV55 modems)
-
-Field units connect via Sierra Wireless RV50 or RV55 cellular modems. Use
-TCP mode in the Console or SFM:
-
-```
-# Console tab
-Transport: TCP
-Host: <modem public IP>
-Port: 9034          ← Device Port in ACEmanager (call-up mode)
-```
-
-```python
-# In code
-from minimateplus.transport import TcpTransport
-from minimateplus.client import MiniMateClient
-
-client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
-info = client.connect()
-```
-
-### Required ACEmanager settings (Serial tab)
-
-These must match exactly — a single wrong setting causes the unit to beep
-on connect but never respond:
-
-| Setting | Value | Why |
-|---------|-------|-----|
-| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate |
-| Flow Control | `None` | Hardware flow control blocks unit TX if pins unconnected |
-| **Quiet Mode** | **Enable** | **Critical.** Disabled → modem injects `RING`/`CONNECT` onto serial line, corrupting the S3 handshake |
-| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency; `5` works but is sluggish |
-| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame |
-| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect |
-| DB9 Serial Echo | `Disable` | Echo corrupts the data stream |
+| `GET` | `/db/units` | All known serials with summary stats |
+| `GET` | `/db/events` | Triggered events (filter by serial, date range, false_trigger) |
+| `GET` | `/db/monitor_log` | Monitoring intervals |
+| `GET` | `/db/sessions` | ACH call-home session history |
+| `PATCH` | `/db/events/{id}/false_trigger?value=true` | Flag / unflag false triggers |

 ---

@@ -175,25 +142,76 @@ on connect but never respond:

 ```python
 from minimateplus import MiniMateClient
-from minimateplus.transport import SerialTransport, TcpTransport
+from minimateplus.transport import TcpTransport

 # Serial
 client = MiniMateClient(port="COM5")

 # TCP (cellular modem)
-client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0)
+client = MiniMateClient(transport=TcpTransport("1.2.3.4", 12345), timeout=30.0)

 with client:
-    info    = client.connect()        # DeviceInfo — model, serial, firmware, compliance config
-    serial  = client.get_serial()     # Serial number string
-    config  = client.get_config()     # Full config block (bytes)
-    events  = client.get_events()     # List[EventRecord] with true event-time metadata
+    # Read
+    info    = client.connect()               # DeviceInfo — serial, firmware, compliance config
+    count   = client.count_events()          # Number of stored events
+    keys    = client.list_event_keys()       # Fast browse walk — event keys only, no download
+    events  = client.get_events()            # Full download: headers + peaks + metadata
+    monitor = client.get_monitor_status()    # Battery, memory, is_monitoring flag
+    log     = client.get_monitor_log_entries()  # Monitoring intervals (partial 0x2C records)
+
+    # Write
+    client.apply_config(
+        sample_rate=1024,
+        trigger_level_geo=0.5,
+        project="Bridge Inspection 2026",
+        client_name="City of Portland",
+        operator="B. Harrison",
+    )
+
+    # Control
+    client.start_monitoring()                # SUB 0x96
+    client.stop_monitoring()                 # SUB 0x97
+    client.delete_all_events()              # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2)
 ```

-`get_events()` runs the full download sequence per event: `1E → 0A → 0C → 5A → 1F`.
-The SUB 5A bulk waveform stream is used to retrieve `client`, `operator`, and
-`sensor_location` as they existed at record time — not backfilled from the current
-compliance config.
+`get_events()` runs the full per-event sequence: `1E → 0A → 0C → 5A → 1F`.
+SUB 5A bulk stream provides `client`, `operator`, and `sensor_location` as they
+existed at record time — not backfilled from the current compliance config.
+
+---
+
+## Database
+
+`ach_server.py` writes to `bridges/captures/seismo_relay.db` (SQLite, WAL mode).
+Three tables, all unit-keyed by serial number:
+
+| Table | Key | Contents |
+|-------|-----|----------|
+| `ach_sessions` | UUID | Per-call-home audit record: serial, peer IP, events_downloaded, duration |
+| `events` | UUID, UNIQUE(serial, waveform_key) | Triggered events: timestamp, PPV per channel, project/client/operator strings, false_trigger flag |
+| `monitor_log` | UUID, UNIQUE(serial, waveform_key) | Monitoring intervals: start/stop time, duration, geo threshold |
+
+Deduplication is by `(serial, waveform_key)` — repeat call-homes or re-runs
+never produce duplicate rows. Post-erase key reuse is handled automatically
+via the high-water mark in `ach_state.json`.
+
+---
+
+## Connecting over cellular (RV50 / RV55)
+
+Field units connect via Sierra Wireless RV50 or RV55 cellular modems.
+
+### Required ACEmanager settings
+
+| Setting | Value | Why |
+|---------|-------|-----|
+| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate |
+| Flow Control | `None` | Hardware FC blocks TX if pins unconnected |
+| **Quiet Mode** | **Enable** | **Critical** — disabled injects `RING`/`CONNECT` onto serial, corrupting the S3 handshake |
+| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency |
+| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame |
+| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect |
+| DB9 Serial Echo | `Disable` | Echo corrupts the data stream |

 ---

@@ -204,23 +222,10 @@ compliance config.
 | DLE | `0x10` | Data Link Escape |
 | STX | `0x02` | Start of frame |
 | ETX | `0x03` | End of frame |
-| ACK | `0x41` (`'A'`) | Frame-start marker sent before every frame |
+| ACK | `0x41` | Frame-start marker sent before every BW frame |
 | DLE stuffing | `10 10` on wire | Literal `0x10` in payload |

-**S3-side frame** (seismograph → Blastware): `ACK DLE+STX [payload] CHK DLE+ETX`
-
-**De-stuffed payload header:**
-```
-[0] CMD        0x10 = BW request, 0x00 = S3 response
-[1] ?          unknown (0x00 BW / 0x10 S3)
-[2] SUB        Command/response identifier  ← the key field
-[3] PAGE_HI    Page address high byte
-[4] PAGE_LO    Page address low byte
-[5+] DATA      Payload content
-```
-
-**Response SUB rule:** `response_SUB = 0xFF - request_SUB`
-Example: request SUB `0x08` (Event Index) → response SUB `0xF7`
+**Response SUB rule:** `response_SUB = 0xFF - request_SUB` (no exceptions)

 Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/instantel_protocol_reference.md)

@@ -228,32 +233,36 @@ Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/insta

 ## Requirements

-```
+```bash
 pip install pyserial fastapi uvicorn
 ```

 Python 3.10+. Tkinter is included with the standard Python installer on
-Windows (make sure "tcl/tk and IDLE" is checked during install).
+Windows (check "tcl/tk and IDLE" during install).

 ---

 ## Virtual COM ports (bridge capture)

-The bridge needs two COM ports on the same PC — one that Blastware connects
-to, and one wired to the seismograph. Use a virtual COM port pair
-(**com0com** or **VSPD**) to give Blastware a port to talk to.
-
 ```
 Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus
 ```

+Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
+
 ---

 ## Roadmap

- [x] Event download — pull waveform records from the unit (`1E → 0A → 0C → 5A → 1F`)
- [x] True event-time metadata — project / client / operator / sensor location from SUB 5A
- [ ] Write commands — push config changes to the unit (compliance setup, channel config, trigger settings)
- [ ] ACH inbound server — accept call-home connections from field units
- [ ] Modem manager — push standard configs to RV50/RV55 fleet via Sierra Wireless API
- [ ] Full Blastware parity — complete read/write/download cycle without Blastware
+- [x] Full read pipeline — device info, compliance config, event download with true event-time metadata
+- [x] Write commands — push compliance config, trigger thresholds, project strings to device
+- [x] Erase all events — confirmed erase sequence from live MITM capture
+- [x] Monitor control — start/stop monitoring, read battery/memory/status
+- [x] Monitor log entries — decode partial 0x2C records (continuous monitoring intervals)
+- [x] ACH inbound server — accept call-home connections, download events, dedup by key
+- [x] SQLite persistence — events, monitor log, and session history in `seismo_relay.db`
+- [x] SFM REST API — device control + DB query endpoints, live device cache
+- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
+- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
+- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
+- [ ] Modem manager — push RV50/RV55 configs via Sierra Wireless API