diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..35efe7b --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,84 @@ +# Changelog + +All notable changes to seismo-relay are documented here. + +--- + +## v0.5.0 — 2026-03-31 + +### Added +- **Console tab in `seismo_lab.py`** — direct device connection without the bridge subprocess. + - Serial and TCP transport selectable via radio buttons. + - Four one-click commands: POLL, Serial #, Full Config, Event Index. + - Colour-coded scrolling output: TX (blue), RX raw hex (teal), parsed/decoded (green), errors (red). + - Save Log and Send to Analyzer buttons; logs auto-saved to `bridges/captures/console_.log`. + - Queue/`after(100)` pattern — no UI blocking or performance impact. +- **`minimateplus` package** — clean Python client library for the MiniMate Plus S3 protocol. + - `SerialTransport` and `TcpTransport` (for Sierra Wireless RV50/RV55 cellular modems). + - `MiniMateProtocol` — DLE frame parser/builder, two-step paged reads, checksum validation. + - `MiniMateClient` — high-level client: `connect()`, `get_serial()`, `get_config()`, `get_events()`. +- **TCP/cellular transport** (`TcpTransport`) — connect to field units via Sierra Wireless RV50/RV55 modems over cellular. + - `read_until_idle(idle_gap=1.5s)` to handle modem data-forwarding buffer delay. + - Confirmed working end-to-end: TCP → RV50/RV55 → RS-232 → MiniMate Plus. +- **`bridges/tcp_serial_bridge.py`** — local TCP-to-serial bridge for bench testing `TcpTransport` without a cellular modem. +- **SFM REST server** (`sfm/server.py`) — FastAPI server with device info, event list, and event record endpoints over both serial and TCP. + +### Fixed +- `protocol.py` `startup()` was using a hardcoded `POLL_RECV_TIMEOUT = 10.0` constant, ignoring the configurable `self._recv_timeout`. Fixed to use `self._recv_timeout` throughout. +- `sfm/server.py` now retries once on `ProtocolError` for TCP connections to handle cold-boot timing on first connect. + +### Protocol / Documentation +- **Sierra Wireless RV50/RV55 modem config** — confirmed required ACEmanager settings: Quiet Mode = Enable, Data Forwarding Timeout = 1, TCP Connect Response Delay = 0. Quiet Mode disabled causes modem to inject `RING\r\nCONNECT\r\n` onto the serial line, breaking the S3 handshake. +- **Calibration year** confirmed at SUB FE (Full Config) destuffed payload offset 0x56–0x57 (uint16 BE). `0x07E7` = 2023, `0x07E9` = 2025. +- **`"Operating System"` boot string** — 16-byte UART boot message captured on cold-start before unit enters DLE-framed mode. Parser handles correctly by scanning for DLE+STX. +- RV50/RV55 sends `RING`/`CONNECT` over TCP to the calling client even with Quiet Mode enabled — this is normal behaviour, parser discards it. + +--- + +## v0.4.0 — 2026-03-12 + +### Added +- **`seismo_lab.py`** — combined Bridge + Analyzer GUI. Single window with two tabs; bridge start auto-wires live mode in the Analyzer. +- **`frame_db.py`** — SQLite frame database. Captures accumulate over time; Query DB tab searches across all sessions. +- **`bridges/s3-bridge/proxy.py`** — bridge proxy module. +- Large BW→S3 write frame checksum algorithm confirmed and implemented (`SUM8` of payload `[2:-1]` skipping `0x10` bytes, plus constant `0x10`, mod 256). +- SUB `A4` identified as composite container frame with embedded inner frames; `_extract_a4_inner_frames()` and `_diff_a4_payloads()` reduce diff noise from 2300 → 17 meaningful entries. + +### Fixed +- BAD CHK false positives on BW POLL frames — BW frame terminator `03 41` was being included in the de-stuffed payload. Fixed to strip correctly. +- Aux Trigger read location confirmed at SUB FE offset `0x0109`. + +--- + +## v0.3.0 — 2026-03-09 + +### Added +- Record time confirmed at SUB E5 page2 offset `+0x28` as float32 BE. +- Trigger Sample Width confirmed at BW→S3 write frame SUB `0x82`, destuffed payload offset `[22]`. +- Mode-gating documented: several settings only appear on the wire when the appropriate mode is active. + +### Fixed +- `0x082A` mystery resolved — fixed-size E5 payload length (2090 bytes), not a record-time field. + +--- + +## v0.2.0 — 2026-03-01 + +### Added +- Channel config float layout fully confirmed: trigger level, alarm level, and unit string per channel (IEEE 754 BE floats). +- Blastware `.set` file format decoded — little-endian binary struct mirroring the wire payload. +- Operator manual (716U0101 Rev 15) added as cross-reference source. + +--- + +## v0.1.0 — 2026-02-26 + +### Added +- Initial `s3_bridge.py` serial bridge — transparent RS-232 tap between Blastware and MiniMate Plus. +- `s3_parser.py` — deterministic DLE state machine frame extractor. +- `s3_analyzer.py` — session parser, frame differ, Claude export. +- `gui_bridge.py` and `gui_analyzer.py` — Tkinter GUIs. +- DLE framing confirmed: `DLE+STX` / `DLE+ETX`, `0x41` = ACK (not STX), DLE stuffing rule. +- Response SUB rule confirmed: `response_SUB = 0xFF - request_SUB`. +- Year `0x07CB` = 1995 confirmed as MiniMate factory RTC default. +- Full write command family documented (SUBs `68`–`83`). diff --git a/README.md b/README.md index 3b52ec4..fddacad 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,14 @@ -# seismo-relay +# seismo-relay `v0.5.0` -Tools for capturing and reverse-engineering the RS-232 serial protocol between -**Blastware** software and **Instantel MiniMate Plus** seismographs. +A ground-up replacement for **Blastware** — Instantel's aging Windows-only +software for managing MiniMate Plus seismographs. -Built for Windows, stdlib-only (plus `pyserial` for the bridge). +Built in Python. Runs on Windows. Connects to instruments over direct RS-232 +or cellular modem (Sierra Wireless RV50 / RV55). + +> **Status:** Active development. Core read pipeline working (device info, +> config, event index). Event download and write commands in progress. +> See [CHANGELOG.md](CHANGELOG.md) for version history. --- @@ -11,217 +16,177 @@ Built for Windows, stdlib-only (plus `pyserial` for the bridge). ``` seismo-relay/ +├── seismo_lab.py ← Main GUI (Bridge + Analyzer + Console tabs) +│ +├── minimateplus/ ← MiniMate Plus client library +│ ├── transport.py ← SerialTransport and TcpTransport +│ ├── protocol.py ← DLE frame layer (read/write/parse) +│ ├── client.py ← High-level client (connect, get_config, etc.) +│ ├── framing.py ← Frame builder/parser primitives +│ └── models.py ← DeviceInfo, EventRecord, etc. +│ +├── sfm/ ← SFM REST API server (FastAPI) +│ └── server.py ← /device/info, /device/events, /device/event +│ ├── bridges/ │ ├── s3-bridge/ -│ │ └── s3_bridge.py ← The serial bridge (core capture tool) -│ ├── gui_bridge.py ← Tkinter GUI wrapper for s3_bridge -│ └── raw_capture.py ← Simpler raw-only capture tool -└── parsers/ - ├── s3_parser.py ← Low-level DLE frame extractor - ├── s3_analyzer.py ← Protocol analyzer (sessions, diffs, exports) - ├── gui_analyzer.py ← Tkinter GUI for the analyzer - └── frame_db.py ← SQLite frame database +│ │ └── s3_bridge.py ← RS-232 serial bridge (capture tool) +│ ├── tcp_serial_bridge.py ← Local TCP↔serial bridge (bench testing) +│ ├── gui_bridge.py ← Standalone bridge GUI (legacy) +│ └── raw_capture.py ← Simple raw capture tool +│ +├── parsers/ +│ ├── s3_parser.py ← DLE frame extractor +│ ├── s3_analyzer.py ← Session parser, differ, Claude export +│ ├── gui_analyzer.py ← Standalone analyzer GUI (legacy) +│ └── frame_db.py ← SQLite frame database +│ +└── docs/ + └── instantel_protocol_reference.md ← Reverse-engineered protocol spec ``` --- -## How it all fits together +## Quick start -The workflow has two phases: **capture**, then **analyze**. +### Seismo Lab (main GUI) + +The all-in-one tool. Three tabs: **Bridge**, **Analyzer**, **Console**. ``` - Blastware PC - │ - Virtual COM (e.g. COM4) - │ - s3_bridge.py ←─── sits in the middle, forwards all bytes both ways - │ writes raw_bw.bin and raw_s3.bin - Physical COM (e.g. COM5) - │ - MiniMate Plus seismograph +python seismo_lab.py ``` -After capturing, you point the analyzer at the two `.bin` files to inspect -what happened. +### SFM REST server + +Exposes MiniMate Plus commands as a REST API for integration with other systems. + +``` +cd sfm +uvicorn server:app --reload +``` + +**Endpoints:** + +| Method | URL | Description | +|--------|-----|-------------| +| `GET` | `/device/info?port=COM5` | Device info via serial | +| `GET` | `/device/info?host=1.2.3.4&tcp_port=9034` | Device info via cellular modem | +| `GET` | `/device/events?port=COM5` | Event index | +| `GET` | `/device/event?port=COM5&index=0` | Single event record | --- -## Part 1 — The Bridge +## Seismo Lab tabs -### `s3_bridge.py` — Serial bridge +### Bridge tab -Transparently forwards bytes between Blastware and the seismograph while -logging everything to disk. Blastware operates normally and has no idea the -bridge is there. +Captures live RS-232 traffic between Blastware and the seismograph. Sits in +the middle as a transparent pass-through while logging everything to disk. -**Run it:** ``` -python bridges/s3-bridge/s3_bridge.py --bw COM4 --s3 COM5 --logdir captures/ +Blastware → COM4 (virtual) ↔ s3_bridge ↔ COM5 (physical) → MiniMate Plus ``` -**Key flags:** -| Flag | Default | Description | -|------|---------|-------------| -| `--bw` | required | COM port connected to Blastware | -| `--s3` | required | COM port connected to the seismograph | -| `--baud` | 38400 | Baud rate (match your device) | -| `--logdir` | `.` | Where to write log/bin files | -| `--raw-bw` | off | Also write a flat raw file for BW→S3 traffic | -| `--raw-s3` | off | Also write a flat raw file for S3→BW traffic | +Set your COM ports and log directory, then hit **Start Bridge**. Use +**Add Mark** to annotate the capture at specific moments (e.g. "changed +trigger level"). When the bridge starts, the Analyzer tab automatically wires +up to the live files and starts updating in real time. -**Output files (in `--logdir`):** -- `s3_session_.bin` — structured binary log with timestamps - and direction tags (record format: `[type:1][ts_us:8][len:4][payload]`) -- `s3_session_.log` — human-readable hex dump (text) -- `raw_bw.bin` — flat BW→S3 byte stream (if `--raw-bw` used) -- `raw_s3.bin` — flat S3→BW byte stream (if `--raw-s3` used) +### Analyzer tab -> The analyzer needs `raw_bw.bin` + `raw_s3.bin`. Always use `--raw-bw` and -> `--raw-s3` when capturing. +Parses raw captures into DLE-framed protocol sessions, diffs consecutive +sessions to show exactly which bytes changed, and lets you query across all +historical captures via the built-in SQLite database. -**Interactive commands** (type while bridge is running): -- `m` + Enter → prompts for a label and inserts a MARK record into the log -- `q` + Enter → quit +- **Inventory** — all frames in a session, click to drill in +- **Hex Dump** — full payload hex dump with changed-byte annotations +- **Diff** — byte-level before/after diff between sessions +- **Full Report** — plain text session report +- **Query DB** — search across all captures by SUB, direction, or byte value + +Use **Export for Claude** to generate a self-contained `.md` report for +AI-assisted field mapping. + +### Console tab + +Direct connection to a MiniMate Plus — no bridge, no Blastware. Useful for +diagnosing field units over cellular without a full capture session. + +**Connection:** choose Serial (COM port + baud) or TCP (IP + port for +cellular modem). + +**Commands:** +| Button | What it does | +|--------|-------------| +| POLL | Startup handshake — confirms unit is alive and identifies model | +| Serial # | Reads unit serial number | +| Full Config | Reads full 166-byte config block (firmware version, channel scales, etc.) | +| Event Index | Reads stored event list | + +Output is colour-coded: TX in blue, raw RX bytes in teal, decoded fields in +green, errors in red. **Save Log** writes a timestamped `.log` file to +`bridges/captures/`. **Send to Analyzer** injects the captured bytes into the +Analyzer tab for deeper inspection. --- -### `gui_bridge.py` — Bridge GUI +## Connecting over cellular (RV50 / RV55 modems) -A simple point-and-click wrapper around `s3_bridge.py`. Easier than the -command line if you don't want to type flags every time. +Field units connect via Sierra Wireless RV50 or RV55 cellular modems. Use +TCP mode in the Console or SFM: ``` -python bridges/gui_bridge.py +# Console tab +Transport: TCP +Host: +Port: 9034 ← Device Port in ACEmanager (call-up mode) ``` -Set your COM ports, log directory, and tick the raw tap checkboxes before -hitting **Start**. The **Add Mark** button lets you annotate the capture -at any point (e.g. "changed record time to 13s"). +```python +# In code +from minimateplus.transport import TcpTransport +from minimateplus.client import MiniMateClient + +client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034)) +info = client.connect() +``` + +### Required ACEmanager settings (Serial tab) + +These must match exactly — a single wrong setting causes the unit to beep +on connect but never respond: + +| Setting | Value | Why | +|---------|-------|-----| +| Configure Serial Port | `38400,8N1` | Must match MiniMate baud rate | +| Flow Control | `None` | Hardware flow control blocks unit TX if pins unconnected | +| **Quiet Mode** | **Enable** | **Critical.** Disabled → modem injects `RING`/`CONNECT` onto serial line, corrupting the S3 handshake | +| Data Forwarding Timeout | `1` (= 0.1 s) | Lower latency; `5` works but is sluggish | +| TCP Connect Response Delay | `0` | Non-zero silently drops the first POLL frame | +| TCP Idle Timeout | `2` (minutes) | Prevents premature disconnect | +| DB9 Serial Echo | `Disable` | Echo corrupts the data stream | --- -## Part 2 — The Analyzer +## minimateplus library -After capturing, you have `raw_bw.bin` (bytes Blastware sent) and `raw_s3.bin` -(bytes the seismograph replied with). The analyzer parses these into protocol -frames, groups them into sessions, and helps you figure out what each byte means. +```python +from minimateplus import MiniMateClient +from minimateplus.transport import SerialTransport, TcpTransport -### What's a "session"? +# Serial +client = MiniMateClient(port="COM5") -Each time you open the settings dialog in Blastware and click Apply/OK, that's -one session — a complete read/modify/write cycle. The bridge detects session -boundaries by watching for the final write-confirm packet (SUB `0x74`). - -Each session contains a sequence of request/response frame pairs: -- Blastware sends a **request** (BW→S3): "give me your config block" -- The seismograph sends a **response** (S3→BW): here it is -- At the end, Blastware sends the modified settings back in a series of write packets - -The analyzer lines these up and diffs consecutive sessions to show you exactly -which bytes changed. - ---- - -### `gui_analyzer.py` — Analyzer GUI +# TCP (cellular modem) +client = MiniMateClient(transport=TcpTransport("1.2.3.4", 9034), timeout=30.0) +with client: + info = client.connect() # DeviceInfo — model, serial, firmware + serial = client.get_serial() # Serial number string + config = client.get_config() # Full config block (bytes) + events = client.get_events() # Event index ``` -python parsers/gui_analyzer.py -``` - -This is the main tool. It has five tabs: - -#### Toolbar -- **S3 raw / BW raw** — browse to your `raw_s3.bin` and `raw_bw.bin` files -- **Analyze** — parse and load the captures -- **Live: OFF/ON** — watch the files grow in real time while the bridge is running -- **Export for Claude** — generate a self-contained `.md` report for AI-assisted analysis - -#### Inventory tab -Shows all frames in the selected session — direction, SUB command, page, -length, and checksum status. Click any frame in the left tree to drill in. - -#### Hex Dump tab -Full hex dump of the selected frame's payload. If the frame had changed bytes -vs the previous session, those are listed below the dump with before/after values -and field names where known. - -#### Diff tab -Side-by-side byte-level diff between the current session and the previous one. -Only SUBs (command types) that actually changed are shown. - -#### Full Report tab -Raw text version of the session report — useful for copying into notes. - -#### Query DB tab -Search across all your captured sessions using the built-in database. - ---- - -### `s3_analyzer.py` — Analyzer (command line) - -If you prefer the terminal: - -``` -python parsers/s3_analyzer.py --s3 raw_s3.bin --bw raw_bw.bin -``` - -**Flags:** -| Flag | Description | -|------|-------------| -| `--s3` | Path to raw_s3.bin | -| `--bw` | Path to raw_bw.bin | -| `--live` | Tail files in real time (poll mode) | -| `--export` | Also write a `claude_export_.md` file | -| `--outdir` | Where to write `.report` files (default: same folder as input) | -| `--poll` | Live mode poll interval in seconds (default: 0.05) | - -Writes one `.report` file per session and prints a summary to the console. - ---- - -## The Frame Database - -Every time you click **Analyze**, the frames are automatically saved to a -SQLite database at: - -``` -C:\Users\\.seismo_lab\frames.db -``` - -This accumulates captures over time so you can query across sessions and dates. - -### Query DB tab - -Use the filter bar to search: -- **Capture** — narrow to a specific capture (timestamp shown) -- **Dir** — BW (requests) or S3 (responses) only -- **SUB** — filter by command type (e.g. `0xF7` = EVENT_INDEX_RESPONSE) -- **Offset** — filter to frames that have a specific byte offset -- **Value** — combined with Offset: "show frames where byte 85 = 0x0A" - -Click any result row, then use the **Byte interpretation** panel at the bottom -to see what that offset's bytes look like as uint8, int8, uint16 BE/LE, -uint32 BE/LE, and float32 BE/LE simultaneously. - -This is the main tool for mapping unknown fields — if you change one setting in -Blastware, capture before and after, then query for frames where that offset -moved, you can pin down exactly which byte controls what. - ---- - -## Export for Claude - -The **Export for Claude** button (orange, in the toolbar) generates a single -`.md` file containing: - -1. Protocol background and known field map -2. Capture summary (session count, frame counts, what changed) -3. Per-diff tables — before/after bytes for every changed offset, with field - names where known -4. Full hex dumps of all frames in the baseline session - -Paste this file into a Claude conversation to get help mapping unknown fields, -interpreting data structures, or understanding sequences. --- @@ -232,47 +197,55 @@ interpreting data structures, or understanding sequences. | DLE | `0x10` | Data Link Escape | | STX | `0x02` | Start of frame | | ETX | `0x03` | End of frame | -| ACK | `0x41` | Frame start marker (BW side) | +| ACK | `0x41` (`'A'`) | Frame-start marker sent before every frame | | DLE stuffing | `10 10` on wire | Literal `0x10` in payload | -**S3-side frame** (seismograph → Blastware): `DLE STX [payload] DLE ETX` -**BW-side frame** (Blastware → seismograph): `ACK STX [payload] ETX` +**S3-side frame** (seismograph → Blastware): `ACK DLE+STX [payload] CHK DLE+ETX` -**De-stuffed payload header** (first 5 bytes after de-stuffing): +**De-stuffed payload header:** ``` [0] CMD 0x10 = BW request, 0x00 = S3 response -[1] ? 0x00 (BW) or 0x10 (S3) +[1] ? unknown (0x00 BW / 0x10 S3) [2] SUB Command/response identifier ← the key field -[3] OFFSET_HI Page address high byte -[4] OFFSET_LO Page address low byte +[3] PAGE_HI Page address high byte +[4] PAGE_LO Page address low byte [5+] DATA Payload content ``` **Response SUB rule:** `response_SUB = 0xFF - request_SUB` -Example: request SUB `0x08` → response SUB `0xF7` +Example: request SUB `0x08` (Event Index) → response SUB `0xF7` + +Full protocol documentation: [`docs/instantel_protocol_reference.md`](docs/instantel_protocol_reference.md) --- ## Requirements ``` -pip install pyserial +pip install pyserial fastapi uvicorn ``` -Python 3.10+. Everything else is stdlib (Tkinter, sqlite3, struct, hashlib). - -Tkinter is included with the standard Python installer on Windows. If it's -missing, reinstall Python and make sure "tcl/tk and IDLE" is checked. +Python 3.10+. Tkinter is included with the standard Python installer on +Windows (make sure "tcl/tk and IDLE" is checked during install). --- -## Virtual COM ports +## Virtual COM ports (bridge capture) -The bridge needs two COM ports on the same PC — one that Blastware connects to, -and one wired to the actual seismograph. On Windows, use a virtual COM port pair -(e.g. **com0com** or **VSPD**) to give Blastware a port to talk to while the -bridge sits in the middle. +The bridge needs two COM ports on the same PC — one that Blastware connects +to, and one wired to the seismograph. Use a virtual COM port pair +(**com0com** or **VSPD**) to give Blastware a port to talk to. ``` -Blastware → COM4 (virtual) ↔ s3_bridge ↔ COM5 (physical) → MiniMate +Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus ``` + +--- + +## Roadmap + +- [ ] Event download — pull waveform records from the unit (SUBs `1E` → `0A` → `0C` → `5A`) +- [ ] Write commands — push config changes to the unit (compliance setup, channel config, trigger settings) +- [ ] ACH inbound server — accept call-home connections from field units +- [ ] Modem manager — push standard configs to RV50/RV55 fleet via Sierra Wireless API +- [ ] Full Blastware parity — complete read/write/download cycle without Blastware