serversdown 9afa3484f4 feat(cache): implement integrity checks for cached events and waveforms
- Added `waveform_key` and `event_timestamp` columns to `CachedEvent` and `CachedWaveform` for integrity verification.
- Implemented logic to flush the cache when a mismatch in (waveform_key, event_timestamp) is detected during event and waveform updates.
- Enhanced `set_events` and `set_waveform` methods to check for mismatches and trigger cache eviction as necessary.
- Introduced a new `LiveCache` class to manage in-memory caching of live device data, separating it from the server logic for better testability.
- Added tests to verify the correctness of cache invalidation logic, particularly for post-erase key reuse scenarios.
- Updated web application to include a "Force refresh" toggle, allowing users to bypass the cache and re-fetch data from the device.
2026-05-07 04:42:00 +00:00
2026-05-05 20:39:47 -04:00
2026-05-05 20:48:58 -04:00

seismo-relay v0.14.3

A ground-up replacement for Blastware — Instantel's aging Windows-only software for managing MiniMate Plus seismographs.

Built in Python. Runs on Windows, Linux, or macOS. Connects to instruments over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).

Status: Active development. Full read + write + erase + monitoring pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server handles inbound unit connections, downloads events, and persists everything to a SQLite database. SFM REST API exposes device control and DB queries. As of v0.14.3 (2026-05-05): SUB 5A bulk waveform protocol is verified byte-perfect against Blastware captures across 2-sec, 3-sec, and 10-sec events. Generated .G10 / .AB0 files open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots. See CHANGELOG.md for full version history.


What's in here

seismo-relay/
├── seismo_lab.py              ← Main GUI (Bridge + Analyzer + Download + Console tabs)
│
├── minimateplus/              ← MiniMate Plus client library
│   ├── transport.py           ←   SerialTransport, TcpTransport, SocketTransport
│   ├── protocol.py            ←   DLE frame layer, SUB command dispatch
│   ├── client.py              ←   High-level client (connect, get_events, delete_all_events, push_config, get_call_home_config, …)
│   ├── framing.py             ←   Frame builders, DLE codec, S3FrameParser
│   ├── models.py              ←   DeviceInfo, Event, ComplianceConfig, MonitorLogEntry, CallHomeConfig, …
│   └── blastware_file.py      ←   Write events to Blastware-compatible .AB0 files
│
├── sfm/                       ← SFM REST API server (FastAPI, port 8200)
│   ├── server.py              ←   Live device endpoints + DB query endpoints + caching
│   ├── database.py            ←   SeismoDb — SQLite persistence (events, monitor_log, ach_sessions, sessions table)
│   └── sfm_webapp.html        ←   Embedded web UI with Call Home config tab
│
├── bridges/
│   ├── ach_server.py          ←   Inbound ACH call-home server (main production server)
│   ├── ach_mitm.py            ←   Transparent MITM proxy for capturing BW sessions
│   ├── s3-bridge/             ←   RS-232 serial bridge (capture tool)
│   ├── tcp_serial_bridge.py   ←   Local TCP↔serial bridge (bench testing)
│   ├── gui_bridge.py          ←   Standalone bridge GUI with raw capture checkboxes
│   └── raw_capture.py         ←   Simple raw capture tool
│
├── parsers/
│   ├── s3_analyzer.py         ←   Session parser, differ, Claude export
│   ├── gui_analyzer.py        ←   Standalone analyzer GUI
│   └── frame_db.py            ←   SQLite frame database
│
└── docs/
    └── instantel_protocol_reference.md  ← Reverse-engineered protocol spec

Quick start

ACH inbound server (production)

Listens for inbound unit call-homes, downloads all new events and monitor log entries, and writes everything to bridges/captures/seismo_relay.db.

python bridges/ach_server.py --port 12345 --output bridges/captures/

Point the unit's ACEmanager Remote Host to this machine's IP and Remote Port to 12345.

Options:

--port N            Listen port (default 12345)
--output DIR        Capture directory (default bridges/captures/)
--allow-ip IP       Allowlist an IP (repeat for multiple; default: accept all)
--max-events N      Safety cap for first run (default: unlimited)
--clear-after-download  Erase device memory after successful download
--verbose           Debug logging

SFM REST server

Exposes device control and DB queries as a REST API. Proxied by terra-view.

python sfm/server.py               # default: 0.0.0.0:8200
python -m uvicorn sfm.server:app --host 0.0.0.0 --port 8200 --reload

Open http://localhost:8200 for the embedded web UI, or http://localhost:8200/docs for the interactive API docs.

Seismo Lab GUI

python seismo_lab.py

SFM REST API

Live device endpoints

Each call dials the device, does its work, and closes the connection. TCP connections are retried once on ProtocolError to handle cold-boot timing.

In-memory caching — frequently-polled endpoints avoid redundant TCP round-trips via a thread-safe _LiveCache (plain Python dict + threading.Lock):

Method URL Cache Strategy
GET /device/info Indefinite; invalidated by POST /device/config
GET /device/events Count-probe fast path (~2s); full download only when new events detected
GET /device/event/{idx}/waveform Permanent per event index
GET /device/monitor/status 30-second TTL; invalidated by monitor start/stop
GET /device/call_home Fresh read from device (not cached)
POST /device/connect
POST /device/config Writes compliance config; invalidates info + events cache
POST /device/config/project Patches project/client/operator/sensor_location strings
POST /device/monitor/start Sends SUB 0x96; immediately evicts status cache
POST /device/monitor/stop Sends SUB 0x97; immediately evicts status cache
POST /device/call_home Reads, patches specified fields, writes back to device

Cache bypass — All cached endpoints accept ?force=true to skip the cache and force a fresh read from the device.

Cache statsGET /cache/stats returns hit/miss counts and TTL info; DELETE /cache/device clears the device cache immediately.

Transport query params (supply one set):

Serial:  ?port=COM5&baud=38400
TCP:     ?host=1.2.3.4&tcp_port=12345

DB read endpoints

Query the SQLite database written by ach_server.py. All read-only except PATCH /db/events/{id}/false_trigger.

Method URL Description
GET /db/units All known serials with summary stats
GET /db/events Triggered events (filter by serial, date range, false_trigger)
GET /db/monitor_log Monitoring intervals
GET /db/sessions ACH call-home session history
PATCH /db/events/{id}/false_trigger?value=true Flag / unflag false triggers

minimateplus library

from minimateplus import MiniMateClient
from minimateplus.transport import TcpTransport

# Serial
client = MiniMateClient(port="COM5")

# TCP (cellular modem)
client = MiniMateClient(transport=TcpTransport("1.2.3.4", 12345), timeout=30.0)

with client:
    # Read
    info     = client.connect()               # DeviceInfo — serial, firmware, compliance config
    count    = client.count_events()          # Number of stored events
    keys     = client.list_event_keys()       # Fast browse walk — event keys only, no download
    events   = client.get_events()            # Full download: headers + peaks + metadata
    monitor  = client.get_monitor_status()    # Battery, memory, is_monitoring flag
    log      = client.get_monitor_log_entries()  # Monitoring intervals (partial 0x2C records)
    ach_cfg  = client.get_call_home_config()  # Auto Call Home settings (SUB 0x2C)

    # Write
    client.apply_config(
        sample_rate=1024,
        recording_mode="Continuous",         # Single Shot / Continuous / Histogram / Histogram+Continuous
        histogram_interval_sec=15,            # 2, 5, 15, 60, 300, 900
        trigger_level_geo=0.5,
        geo_range="Normal",                   # Normal (10.000 in/s) / Sensitive (1.25 in/s)
        project="Bridge Inspection 2026",
        client_name="City of Portland",
        operator="B. Harrison",
    )
    
    client.set_call_home_config(
        auto_call_home_enabled=True,
        after_event_recorded=True,
        at_specified_times=True,
        time1_hour=18, time1_min=30,          # 6:30 PM
        time2_hour=6, time2_min=0,            # 6:00 AM
    )

    # Control
    client.start_monitoring()                # SUB 0x96
    client.stop_monitoring()                 # SUB 0x97
    client.delete_all_events()              # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2)

get_events() runs the full per-event sequence: 1E → 0A → 1E(arm token=0xFE) → 0C → 1F(arm) → POLL×3 → 5A → 1F(browse). SUB 5A bulk stream walks chunks bounded by the end_offset extracted from the STRT record at byte 17 of the probe response — no over-reading, no chunk-count cap. Project / client / operator / sensor location strings come from the dedicated metadata pages at counter 0x1002 and 0x1004, read once per session (they reflect the compliance setup at session start, not per individual event).


Database

ach_server.py writes to bridges/captures/seismo_relay.db (SQLite, WAL mode) using the SeismoDb persistence layer. Four tables, all unit-keyed by serial number:

Table Key Contents
ach_sessions UUID Per-call-home audit record: serial, timestamp, peer IP, events_downloaded, monitor_entries, duration_seconds
events UUID, UNIQUE(serial, waveform_key) Triggered events: timestamp, Tran/Vert/Long/VectorSum/Mic PPV, project/client/operator/sensor_location strings, sample_rate, record_type, false_trigger flag
monitor_log UUID, UNIQUE(serial, waveform_key) Monitoring intervals: serial, waveform_key, start_time, stop_time, duration_seconds, geo_threshold_ips
events.false_trigger Boolean flag PATCH endpoint to mark/unmark false triggers for review

Deduplication is by (serial, waveform_key) — repeat call-homes or re-runs never produce duplicate rows. Post-erase key reuse is handled automatically via the high-water mark in ach_state.json. Key-based state tracking allows correct handling of device erasures (external or post-download).


Connecting over cellular (RV50 / RV55)

Field units connect via Sierra Wireless RV50 or RV55 cellular modems.

Required ACEmanager settings

Setting Value Why
Configure Serial Port 38400,8N1 Must match MiniMate baud rate
Flow Control None Hardware FC blocks TX if pins unconnected
Quiet Mode Enable Critical — disabled injects RING/CONNECT onto serial, corrupting the S3 handshake
Data Forwarding Timeout 1 (= 0.1 s) Lower latency
TCP Connect Response Delay 0 Non-zero silently drops the first POLL frame
TCP Idle Timeout 2 (minutes) Prevents premature disconnect
DB9 Serial Echo Disable Echo corrupts the data stream

Protocol quick-reference

Term Value Meaning
DLE 0x10 Data Link Escape
STX 0x02 Start of frame
ETX 0x03 End of frame
ACK 0x41 Frame-start marker sent before every BW frame
DLE stuffing 10 10 on wire Literal 0x10 in payload

Response SUB rule: response_SUB = 0xFF - request_SUB (no exceptions)

Full protocol documentation: docs/instantel_protocol_reference.md


Compliance Config Features

The REST API and web UI expose full control over device compliance settings:

  • Recording Mode (Single Shot / Continuous / Histogram / Histogram+Continuous)
  • Sample Rate (1024 / 2048 / 4096 sps)
  • Record Time (float, seconds)
  • Histogram Interval (2s, 5s, 15s, 1m, 5m, 15m) — when recording mode includes histogram
  • Geo Trigger Levels (float, in/s per channel)
  • Geo Maximum Range (Normal 10.000 in/s / Sensitive 1.250 in/s per channel)
  • Project / Client / Operator / Sensor Location (ASCII strings)

Auto Call Home config:

  • Auto Call Home Enable (bool)
  • Dial String (read-only; 40-byte ASCII)
  • Trigger on Event (bool)
  • Scheduled Call-Ins (two time slots with HH:MM each)
  • Retry Settings (count, delay, connection timeout, warm-up time)

Requirements

pip install pyserial fastapi uvicorn

Python 3.10+. Tkinter is included with the standard Python installer on Windows (check "tcl/tk and IDLE" during install).


Virtual COM ports (bridge capture)

Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus

Use com0com or VSPD to create the virtual COM pair on Windows.


Key Features

Device support:

  • Full read/write/erase pipelines
  • Compliance config (recording mode, sample rate, histogram interval, geo sensitivity, project strings)
  • Auto Call Home config (read/write ACH settings, dial string, time slots, retries)
  • Monitor control (start/stop, status polling, battery/memory)
  • Monitor log entries (continuous monitoring intervals without full waveform download)

Data persistence:

  • SQLite database (seismo_relay.db) with 4 tables: ach_sessions, events, monitor_log, plus false_trigger flag
  • Deduplication by waveform key (handles re-runs and repeat call-homes)
  • Post-erase key-reuse detection (tracks high-water mark)
  • Session state (ach_state.json) with downloaded keys and max key

REST API:

  • Live device endpoints with in-memory caching (_LiveCache)
  • Cache statistics (/cache/stats) and manual invalidation (/cache/device)
  • DB query endpoints (units, events, monitor_log, sessions, false_trigger PATCH)
  • Call Home config read/write endpoints
  • Blastware file download endpoint (/device/event/{index}/blastware_file)

File output (v0.7+, byte-perfect as of v0.14.3):

  • Blastware-compatible .AB0 / .G10 file generation (waveform + metadata)
  • Multi-channel waveform decode from SUB 5A bulk stream
  • Second-resolution timestamp encoding in Blastware filename
  • Byte-perfect against BW reference captures (verified across 2-sec / 3-sec / 10-sec event durations, both event 0 and event N continuation events)
  • STRT-bounded chunk walk + correct event-N probe counter + partial DLE stuffing of 0x10 in 5A params (the four fixes that landed in v0.14.0v0.14.3)

Capture tools:

  • Serial-to-TCP bridge with raw BW/S3 capture (s3_bridge.py, defaults to auto-capture)
  • GUI bridge with raw capture checkboxes (gui_bridge.py)
  • ACH inbound server with bidirectional capture (ach_server.py saves raw_tx + raw_rx)
  • Transparent TCP MITM proxy for live BW session capture (ach_mitm.py)

Analysis tools:

  • s3_analyzer.py — session parser, frame differ, Claude export
  • gui_analyzer.py — standalone analyzer GUI
  • frame_db.py — SQLite frame database for capture analysis

seismo_lab.py GUI:

  • Bridge tab — Serial/TCP mode selector with raw capture options
  • Analyzer tab — BW/S3 capture playback and differencing
  • Download tab — Live wire-byte capture during event download
  • Console tab — Logging and diagnostics

Roadmap (Future)

  • Verify 30-sec event download — body may exceed 0xFFFF and force the device into a different end_key encoding (none of 2/3/10-sec test cases hit this boundary)
  • Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
  • Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
  • Compliance config encoder — build raw write payloads from a ComplianceConfig object
  • Modem manager — push RV50/RV55 configs via Sierra Wireless API
  • Histogram mode recording support (5A stream analysis for mode 0x03)
  • Call Home dial_string write support (requires DLE escaping for embedded control characters)
S
Description
Program to talk to, listen to, and control various devices such as seismographs.
Readme 13 MiB
Languages
Python 88.5%
HTML 10.7%
Shell 0.7%