seismo-relay v0.19.0

A ground-up replacement for Blastware — Instantel's aging Windows-only software for managing seismographs. Supports both the MiniMate Plus (Series III) and the Micromate (Series IV / "Thor") families: Series III via the live RS-232 / TCP wire protocol and Blastware ACH file ingest; Series IV currently via Thor TXT-paired IDF file ingest, with the binary codec on the roadmap.

Built in Python. Runs on Windows, Linux, or macOS. Connects to instruments over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).

Status: Active development. Full read + write + erase + monitoring pipeline working end-to-end over TCP/cellular. ACH Auto Call Home server handles inbound unit connections, downloads events, and persists everything to a SQLite database. SFM REST API exposes device control and DB queries. As of v0.14.3 (2026-05-05): SUB 5A bulk waveform protocol is verified byte-perfect against Blastware captures across 2-sec, 3-sec, and 10-sec events. Generated .G10 / .AB0 files open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots. v0.16.0 (2026-05-11) adds BW ASCII report ingestion to /db/import/blastware_file — paired with series3-watcher v1.5.0, every Blastware ACH event lands in SeismoDb with device-authoritative peaks, project metadata, sensor self-check, and ZC/Time-of-Peak data, without depending on the still-undecoded waveform body codec. v0.18.0 (2026-05-19) adds Thor / Micromate Series IV ingest at /db/import/idf_file — paired with thor-watcher v0.3.0, every .IDFH / .IDFW event file (plus its .txt sidecar) lands in SeismoDb the same way BW events do. See docs/idf_protocol_reference.md for the IDF format reference and reverse-engineering plan. v0.19.0 (2026-05-20) separates Series III and Series IV at the code level: new micromate/ package alongside minimateplus/, new events.device_family DB column ("series3" / "series4") so the UI and storage layer dispatch deterministically instead of sniffing filenames. Self-applying migration backfills existing rows from the binary filename extension. See CHANGELOG.md for full version history.

---

What's in here

seismo-relay/
├── seismo_lab.py              ← Main GUI (Bridge + Analyzer + Download + Console tabs)
│
├── minimateplus/              ← Series III (MiniMate Plus) client library
│   ├── transport.py           ←   SerialTransport, TcpTransport, SocketTransport
│   ├── protocol.py            ←   DLE frame layer, SUB command dispatch
│   ├── client.py              ←   High-level client (connect, get_events, delete_all_events, push_config, get_call_home_config, …)
│   ├── framing.py             ←   Frame builders, DLE codec, S3FrameParser
│   ├── models.py              ←   DeviceInfo, Event, ComplianceConfig, MonitorLogEntry, CallHomeConfig, …
│   ├── bw_ascii_report.py     ←   Parse BW per-event ASCII reports (.TXT sidecars)
│   ├── event_file_io.py       ←   Read BW binaries, write .sfm.json sidecars
│   └── blastware_file.py      ←   Write events to Blastware-compatible .AB0 files
│
├── micromate/                 ← Series IV (Micromate / Thor) client library (NEW v0.19)
│   ├── models.py              ←   IdfEvent, IdfReport, IdfPeaks, IdfProjectInfo, IdfSensorCheck (mic in native dB(L))
│   ├── idf_ascii_report.py    ←   Parse Thor .IDFW.txt / .IDFH.txt event sidecars
│   └── idf_file.py            ←   Stub for the .IDFW / .IDFH binary codec (reverse-engineering pending)
│
├── sfm/                       ← SFM REST API server (FastAPI, port 8200)
│   ├── server.py              ←   Live device endpoints + DB query + ingest endpoints + caching
│   ├── database.py            ←   SeismoDb — SQLite persistence (events, monitor_log, ach_sessions)
│   ├── waveform_store.py      ←   On-disk store for BW + IDF event binaries + .sfm.json sidecars
│   └── sfm_webapp.html        ←   Embedded web UI with Call Home config tab
│
├── bridges/
│   ├── ach_server.py          ←   Inbound ACH call-home server (main production server)
│   ├── ach_mitm.py            ←   Transparent MITM proxy for capturing BW sessions
│   ├── s3-bridge/             ←   RS-232 serial bridge (capture tool)
│   ├── tcp_serial_bridge.py   ←   Local TCP↔serial bridge (bench testing)
│   ├── gui_bridge.py          ←   Standalone bridge GUI with raw capture checkboxes
│   └── raw_capture.py         ←   Simple raw capture tool
│
├── parsers/
│   ├── s3_analyzer.py         ←   Session parser, differ, Claude export
│   ├── gui_analyzer.py        ←   Standalone analyzer GUI
│   └── frame_db.py            ←   SQLite frame database
│
└── docs/
    ├── instantel_protocol_reference.md  ← Series III protocol spec (the Rosetta Stone)
    └── idf_protocol_reference.md         ← Series IV (Thor IDF) format reference + codec RE plan

---

Quick start

ACH inbound server (production)

Listens for inbound unit call-homes, downloads all new events and monitor log entries, and writes everything to bridges/captures/seismo_relay.db.

python bridges/ach_server.py --port 12345 --output bridges/captures/

Point the unit's ACEmanager Remote Host to this machine's IP and Remote Port to 12345.

Options:

--port N            Listen port (default 12345)
--output DIR        Capture directory (default bridges/captures/)
--allow-ip IP       Allowlist an IP (repeat for multiple; default: accept all)
--max-events N      Safety cap for first run (default: unlimited)
--clear-after-download  Erase device memory after successful download
--verbose           Debug logging

SFM REST server

Exposes device control and DB queries as a REST API. Proxied by terra-view.

python sfm/server.py               # default: 0.0.0.0:8200
python -m uvicorn sfm.server:app --host 0.0.0.0 --port 8200 --reload

Open http://localhost:8200 for the embedded web UI, or http://localhost:8200/docs for the interactive API docs.

Seismo Lab GUI

python seismo_lab.py

---

SFM REST API

Live device endpoints

Each call dials the device, does its work, and closes the connection. TCP connections are retried once on ProtocolError to handle cold-boot timing.

In-memory caching — frequently-polled endpoints avoid redundant TCP round-trips via a thread-safe _LiveCache (plain Python dict + threading.Lock):

Method	URL	Cache Strategy
GET	/device/info	Indefinite; invalidated by POST /device/config
GET	/device/events	Count-probe fast path (~2s); full download only when new events detected
GET	/device/event/{idx}/waveform	Permanent per event index
GET	/device/monitor/status	30-second TTL; invalidated by monitor start/stop
GET	/device/call_home	Fresh read from device (not cached)
POST	/device/connect	—
POST	/device/config	Writes compliance config; invalidates info + events cache
POST	/device/config/project	Patches project/client/operator/sensor_location strings
POST	/device/monitor/start	Sends SUB 0x96; immediately evicts status cache
POST	/device/monitor/stop	Sends SUB 0x97; immediately evicts status cache
POST	/device/call_home	Reads, patches specified fields, writes back to device

Cache bypass — All cached endpoints accept ?force=true to skip the cache and force a fresh read from the device.

Cache stats — GET /cache/stats returns hit/miss counts and TTL info; DELETE /cache/device clears the device cache immediately.

Transport query params (supply one set):

Serial:  ?port=COM5&baud=38400
TCP:     ?host=1.2.3.4&tcp_port=12345

DB read endpoints

Query the SQLite database written by ach_server.py. All read-only except PATCH /db/events/{id}/false_trigger.

Method	URL	Description
GET	/db/units	All known serials with summary stats
GET	/db/events	Triggered events (filter by serial, date range, false_trigger). Response rows include device_family ("series3" / "series4") so clients dispatch on unit type without sniffing filenames.
GET	/db/monitor_log	Monitoring intervals
GET	/db/sessions	ACH call-home session history
PATCH	/db/events/{id}/false_trigger?value=true	Flag / unflag false triggers

File ingest endpoints

Used by watcher daemons to push field-collected event files into the SFM DB

• waveform store. Both accept multipart uploads of binary event files optionally paired with their ASCII sidecar reports; both dedup by (serial, timestamp) and UPSERT device-authoritative fields on re-import.

Method	URL	Description
POST	/db/import/blastware_file	Series III: .AB0* / .N00 binaries + paired _ASCII.TXT. Source: series3-watcher.
POST	/db/import/idf_file	Series IV: .IDFH / .IDFW binaries + paired .IDFW.txt / .IDFH.txt. Source: thor-watcher.

---

minimateplus library

from minimateplus import MiniMateClient
from minimateplus.transport import TcpTransport

# Serial
client = MiniMateClient(port="COM5")

# TCP (cellular modem)
client = MiniMateClient(transport=TcpTransport("1.2.3.4", 12345), timeout=30.0)

with client:
    # Read
    info     = client.connect()               # DeviceInfo — serial, firmware, compliance config
    count    = client.count_events()          # Number of stored events
    keys     = client.list_event_keys()       # Fast browse walk — event keys only, no download
    events   = client.get_events()            # Full download: headers + peaks + metadata
    monitor  = client.get_monitor_status()    # Battery, memory, is_monitoring flag
    log      = client.get_monitor_log_entries()  # Monitoring intervals (partial 0x2C records)
    ach_cfg  = client.get_call_home_config()  # Auto Call Home settings (SUB 0x2C)

    # Write
    client.apply_config(
        sample_rate=1024,
        recording_mode="Continuous",         # Single Shot / Continuous / Histogram / Histogram+Continuous
        histogram_interval_sec=15,            # 2, 5, 15, 60, 300, 900
        trigger_level_geo=0.5,
        geo_range="Normal",                   # Normal (10.000 in/s) / Sensitive (1.25 in/s)
        project="Bridge Inspection 2026",
        client_name="City of Portland",
        operator="B. Harrison",
    )
    
    client.set_call_home_config(
        auto_call_home_enabled=True,
        after_event_recorded=True,
        at_specified_times=True,
        time1_hour=18, time1_min=30,          # 6:30 PM
        time2_hour=6, time2_min=0,            # 6:00 AM
    )

    # Control
    client.start_monitoring()                # SUB 0x96
    client.stop_monitoring()                 # SUB 0x97
    client.delete_all_events()              # Erase all (SUB 0xA3 → 0x1C → 0x06 → 0xA2)

get_events() runs the full per-event sequence: 1E → 0A → 1E(arm token=0xFE) → 0C → 1F(arm) → POLL×3 → 5A → 1F(browse). SUB 5A bulk stream walks chunks bounded by the end_offset extracted from the STRT record at byte 17 of the probe response — no over-reading, no chunk-count cap. Project / client / operator / sensor location strings come from the dedicated metadata pages at counter 0x1002 and 0x1004, read once per session (they reflect the compliance setup at session start, not per individual event).

---

micromate library

Series IV / Thor support, sibling to minimateplus. Currently scoped to offline-file ingest from Thor's TXT exporter; live-device protocol is deferred until the binary codec is cracked.

from micromate import IdfEvent, parse_idf_report

# Parse a .IDFW.txt / .IDFH.txt sidecar (1014 example files round-trip cleanly)
text = open("UM11719_20231219162723.IDFW.txt").read()
report_dict = parse_idf_report(text)        # permissive dict

# Wrap into a typed event using the device-native binary filename
event = IdfEvent.from_report(report_dict, "UM11719_20231219162723.IDFW")

event.serial                     # "UM11719"
event.kind                       # "Waveform" or "Histogram"
event.peaks.transverse_ips       # 0.0251  (in/s, native unit)
event.peaks.mic_pspl_dbl         # 99.4    (dB(L), Thor's native mic unit — NOT psi)
event.project_info.project       # "UPMC Presby-Loc 3-Level1-1R Elevator Rm"
event.sensor_check.tran          # True (passed self-check)
event.firmware_version           # "Micromate ISEE 11.0AK"
event.calibration_text           # "November 22, 2023 by Instantel"

# Bridge to the existing minimateplus.Event shape for the DB / sidecar paths
# (waveform_key is a 16-byte sha256 prefix when ingesting from a binary file)
bridged_event = event.to_minimateplus_event(waveform_key=b"\x00" * 16)

The binary codec (.IDFW / .IDFH event files themselves) is on the roadmap — see docs/idf_protocol_reference.md for everything known so far, the two observed file signatures, and the reverse-engineering plan. The micromate/idf_file.py stub is where read_idf_file() will land.

---

Database

ach_server.py and the file-ingest endpoints write to bridges/captures/seismo_relay.db (SQLite, WAL mode) via the SeismoDb persistence layer. Three tables, all unit-keyed by serial number:

Table	Key	Contents
ach_sessions	UUID	Per-call-home audit record: serial, timestamp, peer IP, events_downloaded, monitor_entries, duration_seconds
events	UUID, UNIQUE(serial, timestamp)	Triggered events: timestamp, Tran/Vert/Long/VectorSum/Mic PPV, project/client/operator/sensor_location strings, sample_rate, record_type, false_trigger flag, device_family ("series3" / "series4"), blastware_filename (binary at-rest in waveforms/), sidecar references
monitor_log	UUID, UNIQUE(serial, start_time)	Monitoring intervals: serial, waveform_key, start_time, stop_time, duration_seconds, geo_threshold_ips

Deduplication is by (serial, timestamp) — the device clock is the stable natural key. Repeat call-homes or re-runs UPSERT the row in place, refreshing every device-authoritative field (peaks, project strings, sample_rate, file references) so the latest writer wins. false_trigger and device_family are preserved across UPSERTs. Earlier versions used (serial, waveform_key) for dedup, but the device's event-key counter resets to 0x01110000 after every erase, so timestamps are the correct dedup field. Migration handles the transition transparently on first startup.

device_family (added v0.19.0) discriminates Series III from Series IV at the SQL level. Set by every import path; the UI dispatches on it to render mic units correctly (Series III: psi → dBL conversion; Series IV: native dBL passthrough). Existing rows are backfilled at first startup of v0.19.0+ by sniffing the binary filename extension.

The on-disk waveform store lives at bridges/captures/waveforms/<serial>/ and holds the original event binaries (BW .AB0* / .N00 for Series III, .IDFH / .IDFW for Series IV) plus their .sfm.json review/metadata sidecars. Series III events also produce .a5.pkl source-frame pickles and .h5 clean-waveform exports; Series IV doesn't yet (pending codec).

---

Connecting over cellular (RV50 / RV55)

Field units connect via Sierra Wireless RV50 or RV55 cellular modems.

Required ACEmanager settings

Setting	Value	Why
Configure Serial Port	38400,8N1	Must match MiniMate baud rate
Flow Control	None	Hardware FC blocks TX if pins unconnected
Quiet Mode	Enable	Critical — disabled injects RING/CONNECT onto serial, corrupting the S3 handshake
Data Forwarding Timeout	1 (= 0.1 s)	Lower latency
TCP Connect Response Delay	0	Non-zero silently drops the first POLL frame
TCP Idle Timeout	2 (minutes)	Prevents premature disconnect
DB9 Serial Echo	Disable	Echo corrupts the data stream

---

Protocol quick-reference

Term	Value	Meaning
DLE	0x10	Data Link Escape
STX	0x02	Start of frame
ETX	0x03	End of frame
ACK	0x41	Frame-start marker sent before every BW frame
DLE stuffing	10 10 on wire	Literal 0x10 in payload

Response SUB rule: response_SUB = 0xFF - request_SUB (no exceptions)

Full protocol documentation: docs/instantel_protocol_reference.md

---

Compliance Config Features

The REST API and web UI expose full control over device compliance settings:

• Recording Mode (Single Shot / Continuous / Histogram / Histogram+Continuous)
• Sample Rate (1024 / 2048 / 4096 sps)
• Record Time (float, seconds)
• Histogram Interval (2s, 5s, 15s, 1m, 5m, 15m) — when recording mode includes histogram
• Geo Trigger Levels (float, in/s per channel)
• Geo Maximum Range (Normal 10.000 in/s / Sensitive 1.250 in/s per channel)
• Project / Client / Operator / Sensor Location (ASCII strings)

Auto Call Home config:

• Auto Call Home Enable (bool)
• Dial String (read-only; 40-byte ASCII)
• Trigger on Event (bool)
• Scheduled Call-Ins (two time slots with HH:MM each)
• Retry Settings (count, delay, connection timeout, warm-up time)

---

Requirements

pip install pyserial fastapi uvicorn

Python 3.10+. Tkinter is included with the standard Python installer on Windows (check "tcl/tk and IDLE" during install).

---

Virtual COM ports (bridge capture)

Blastware → COM4 (virtual) ↔ s3_bridge.py ↔ COM5 (physical) → MiniMate Plus

Use com0com or VSPD to create the virtual COM pair on Windows.

---

Key Features

Series III (MiniMate Plus) device support:

• Full read/write/erase pipelines over RS-232 or TCP/cellular
• Compliance config (recording mode, sample rate, histogram interval, geo sensitivity, project strings)
• Auto Call Home config (read/write ACH settings, dial string, time slots, retries)
• Monitor control (start/stop, status polling, battery/memory)
• Monitor log entries (continuous monitoring intervals without full waveform download)
• Blastware file ingest at /db/import/blastware_file (paired with series3-watcher)

Series IV (Micromate / Thor) device support:

• Thor IDF file ingest at /db/import/idf_file (paired with thor-watcher, v0.18.0+)
• Native IdfEvent / IdfReport typed models — mic in dB(L), full title strings, sensor self-check, calibration, firmware version
• Parser verified against 1,014 paired .txt sidecars in thor-watcher/example-data/
• Binary .IDFW / .IDFH codec — pending (see Roadmap + docs/idf_protocol_reference.md)
• Live-device protocol — pending codec

Data persistence:

• SQLite database (seismo_relay.db) with events, monitor_log, ach_sessions tables
• Per-row device_family column ("series3" / "series4") for clean UI / unit-of-measurement dispatch (v0.19.0+)
• Deduplication by (serial, timestamp) — natural key handles post-erase counter resets
• UPSERT on re-import refreshes every device-authoritative field (peaks, project, sample_rate); preserves operator review state (false_trigger)
• Post-erase key-reuse detection (tracks high-water mark in ach_state.json)

REST API:

• Live device endpoints with in-memory caching (_LiveCache)
• Cache statistics (/cache/stats) and manual invalidation (/cache/device)
• DB query endpoints (units, events, monitor_log, sessions, false_trigger PATCH)
• Call Home config read/write endpoints
• Blastware file download endpoint (/device/event/{index}/blastware_file)
• Import endpoints for both device families (/db/import/blastware_file, /db/import/idf_file)

File output (v0.7+, byte-perfect as of v0.14.3):

• Blastware-compatible .AB0 / .G10 file generation (waveform + metadata)
• Multi-channel waveform decode from SUB 5A bulk stream
• Second-resolution timestamp encoding in Blastware filename
• Byte-perfect against BW reference captures (verified across 2-sec / 3-sec / 10-sec event durations, both event 0 and event N continuation events)
• STRT-bounded chunk walk + correct event-N probe counter + partial DLE stuffing of 0x10 in 5A params (the four fixes that landed in v0.14.0–v0.14.3)

Capture tools:

• Serial-to-TCP bridge with raw BW/S3 capture (s3_bridge.py, defaults to auto-capture)
• GUI bridge with raw capture checkboxes (gui_bridge.py)
• ACH inbound server with bidirectional capture (ach_server.py saves raw_tx + raw_rx)
• Transparent TCP MITM proxy for live BW session capture (ach_mitm.py)

Analysis tools:

• s3_analyzer.py — session parser, frame differ, Claude export
• gui_analyzer.py — standalone analyzer GUI
• frame_db.py — SQLite frame database for capture analysis

seismo_lab.py GUI:

• Bridge tab — Serial/TCP mode selector with raw capture options
• Analyzer tab — BW/S3 capture playback and differencing
• Download tab — Live wire-byte capture during event download
• Console tab — Logging and diagnostics

Roadmap (Future)

High-impact (unblocks product features)

• Series III waveform body codec reverse-engineering. The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in docs/instantel_protocol_reference.md). Structural framing is ~50% decoded on branch claude/codec-re-cBGNe (tagged-block walker, segment counters); per-byte sample mapping is still open. Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to _peaks_from_samples() saturation noise. Workaround: pair every BW-imported event with its _ASCII.TXT so the device-authoritative peaks land in the DB regardless of codec.
• Series IV (Thor IDF) binary codec reverse-engineering. .IDFH / .IDFW files are currently stored opaquely by WaveformStore.save_imported_idf, with all metadata sourced from the paired .txt sidecar. This works because thor-watcher forwards both files together, but operators who haven't enabled Thor's TXT exporter get rows with NULL peaks. Cracking the binary closes that gap and unlocks waveform display. Starting-point reference at docs/idf_protocol_reference.md — two observed file signatures (1,012 newer-firmware files + 2 old files whose layout matches the Series III STRT-record format), suggested first-session plan (~2-4 hrs), 1,014 paired binary+txt files available as ground truth in thor-watcher/example-data/. Code seam ready at micromate/idf_file.py.
• In-app waveform viewer accuracy. Depends on Series III codec decode. Plot.v1 JSON pipeline + viewer skeleton already exist; will start showing real waveforms automatically once _decode_a5_waveform produces correct samples. Series IV waveforms come online when the IDF codec lands.
• Series IV live-device support. Once the IDF binary is decoded, extend micromate/ with transport.py / framing.py / protocol.py / client.py mirroring the minimateplus/ package layout — depends on capturing Thor's wire protocol (TCP / RS-232 captures TBD).
• Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing.
• Vibration summary reports — highest legit PPV per project → Word doc (false-trigger filtering first).

BW ASCII report parser enhancements (built in v0.16.0)

• Histogram-specific structural fields. Current parser handles the shared fields (PPV, ZC Freq, sensor self-check, project) but silently drops histogram-only fields: Histogram Start/Stop Time, Histogram Start/Stop Date, Number of Intervals, Interval Size, per-channel Peak Time + Peak Date (absolute timestamps rather than the waveform's Time of Peak relative seconds).
• Histogram interval bin-table parsing. Trailing 792-row table (per-interval Peak/Freq per channel + MicL) in histogram TXTs is unparsed. Probably too big for the sidecar JSON; may want a separate .histogram.h5 companion file.
• >100 Hz value parsing. Histogram TXTs use >100 Hz for out-of-range ZC freq; current _parse_number() returns None for these (loses information).

Ingestion gaps

• MLG forwarding. series3-watcher forwards event binaries + their _ASCII.TXT reports, but skips .MLG per-unit monitor log files entirely. Adding an POST /db/import/mlg_file endpoint + watcher scan path would populate monitor_log for non-ACH-routed units (coverage queries, "was this unit monitoring on date X" lookups).
• 0C-record raw bytes persistence in the sidecar. Currently on branch claude/codec-re-cBGNe as commit a187124; cherry-pick if useful as a standalone fix. Preserves the 210-byte 0C record under extensions.raw_records.waveform_record_b64 so future field-offset analysis (Peak Acceleration / Time of Peak / etc. — the fields BW computes client-side from samples) can run offline.

Operational

• series3-watcher file archive manager — 90-day-old events moved to <watch_folder>_archive/<year>/<month>/ subfolders. Plan drafted in claude/codec-re-cBGNe's plan-mode session; awaiting a 5-minute test on whether Blastware UI walks subfolders before any code lands (determines layout: in-place subfolders vs sibling archive).
• Compliance config encoder — build raw write payloads from a ComplianceConfig object.
• Modem manager — push RV50/RV55 configs via Sierra Wireless API.
• Call Home dial_string write support (requires DLE escaping for embedded control characters).
• Histogram mode recording support (5A stream analysis for mode 0x03 — separate from histogram ASCII parsing above).

Test coverage

• Verify 30-sec event download — body may exceed 0xFFFF and force the device into a different end_key encoding (none of the 2/3/10-sec test cases hit this boundary).
• Histogram mode (0x03) write via SFM — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state.

Lower-priority cleanups

• Compliance write anchor-9 cleanup — when changing recording_mode via SFM, a spurious 0x10 may persist after Histogram→other mode transitions. Doesn't affect device operation but differs from BW's byte-perfect output.
• Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring).
• Call Home — map time slots 3/4 offsets; confirm modem_power_relay_enabled.
• RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (--restart-monitoring flag deferred).