docs: three-tier architecture model + strategic roadmap

CLAUDE.md gains an Architecture section near the top describing the
canonical three-tier mental model:

  - SFM: device-side, live connections, /device/* endpoints
  - SDM: data-side, DB + waveform store + /db/* endpoints (currently
    living under sfm/ for historical reasons; rename deferred)
  - Codec library: pure data-interpretation, used by both tiers

Future code should be placed and named according to this model even
though the directory layout doesn't fully reflect it yet.  Decision
rule for where new code goes is documented inline.

README.md's Roadmap section gains two strategic-direction subsections:

  - "Strategic direction" — frames the suite-of-components vision and
    notes that BW ACH + Thor IDF call-home remain the data movers;
    seismo-relay's value is on the receiving and processing side.
  - "Terra-View ↔ SFM device control" — the long-term vision where
    Terra-View can launch into SFM device-control surfaces (operator
    notices missing unit → clicks "Connect to Device" → live view in
    browser).  Includes concrete implementation checklist (auth,
    embedded live-monitor view, action history, series IV live
    support).

The existing tactical roadmap items remain unchanged below.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-22 18:38:00 +00:00
parent ed6982c512
commit 49a524d0d4
2 changed files with 144 additions and 0 deletions
+78
View File
@@ -8,6 +8,84 @@ When new information about the protocol is discovered, please update the instant
--- ---
## Architecture: three-tier conceptual model
seismo-relay is a **suite of cooperating components**, not a single app.
The three tiers below are the canonical mental model — the current
directory layout doesn't fully reflect them yet (some of what is
conceptually SDM lives under `sfm/` today), but new code should be
placed and named according to this model.
### 1. SFM — the device-side (active connection to physical units)
Replaces Blastware's *talk-to-the-meter* role. Lives where a connection
to a physical seismograph is open.
In scope:
- `minimateplus/{transport,framing,protocol,client}.py` — wire protocol
- `seismo_lab.py` — diagnostic GUI (a thick client for SFM)
- The `/device/*` HTTP endpoints in `sfm/server.py`
`/device/info`, `/device/events`, `/device/monitor/*`, `/device/call_home`,
etc. Anything that opens a connection at the moment of the request.
- Future: a Thor / Micromate live client (mirror `minimateplus/`)
- Future: a control surface Terra-View can launch into — see the
README's Roadmap.
Does NOT own a database. Outputs `Event` objects. Has a "spun up when
needed" runtime profile rather than "always on".
### 2. SDM — the data-side (storage, ingest, and serving)
The new name for the receiving-and-storing role. Originally called SFM
because the FastAPI service started life as a thin device proxy, but
the actual role has migrated heavily toward data management. **For now
the directory remains `sfm/`** — renaming requires touching ~30-50
files in seismo-relay + ~10-15 in terra-view + a Docker volume
migration; deferred until the codebase is quiet enough to do it as a
clean refactor.
In scope:
- `sfm/database.py` (`SeismoDb`)
- `sfm/waveform_store.py`, `sfm/event_hdf5.py`
- The `/db/*` HTTP endpoints — `events`, `units`, `monitor_log`,
`sessions`, `false_trigger` mutations
- The `/db/import/*` ingest endpoints — `blastware_file` (series3),
`idf_file` (series4); anything that receives events FROM somewhere
- `scripts/backfill_sidecars.py`, `scripts/check_bw_report_preservation.py`,
and similar data-maintenance tools
- The `.sfm.json` sidecars and `.h5` files in the waveform store
- The shape that Terra-View consumes (Terra-View should never need to
reach into SFM/device-side endpoints to populate its UI)
Always-on, scaled for storage/serving, has the DB and waveform store.
### 3. Codec library — pure data interpretation (used by both sides)
Neither SFM nor SDM — a shared library both depend on.
In scope:
- `minimateplus/{waveform_codec,histogram_codec,event_file_io,bw_ascii_report,blastware_file}.py`
- `micromate/{idf_ascii_report,idf_file}.py`
These modules take bytes (off the wire on the SFM side, or from a
forwarded file on the SDM side) and return `Event` objects. They
should not import from `sfm/`, must not touch a DB, and have no I/O
beyond reading files passed as arguments. Keep them pure — both
tiers can then depend on them without circularity.
### Practical consequences
When deciding where new code goes, ask:
- *Does it need a connection to a device?* → SFM
- *Does it operate on stored events / sidecars / DB rows?* → SDM
- *Does it interpret bytes into structured data, with no I/O of its own?* → codec lib
Terra-View is downstream of SDM for data, and (per the roadmap) will
eventually invoke into SFM's device-control endpoints to provide a
"connect to unit" experience.
---
## Project layout ## Project layout
``` ```
+66
View File
@@ -459,6 +459,72 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
## Roadmap (Future) ## Roadmap (Future)
### Strategic direction — where this is going
seismo-relay is being built as a **suite of cooperating components**
that together replace and improve on Blastware's role. Three logical
tiers:
1. **SFM** (device-side) — owns the active connection to a physical
unit. Today: `minimateplus/`, `/device/*` HTTP endpoints,
`seismo_lab.py`. Future: live Thor / Micromate support.
2. **SDM** (data-side) — owns the database, waveform store, ingest
pipelines, and the read-API that Terra-View consumes. Today this
code lives under `sfm/` for historical reasons; the role has
migrated and the eventual rename is on the long-tail cleanup list.
3. **Codec library** — pure data-interpretation: `minimateplus/*_codec.py`,
`bw_ascii_report.py`, `micromate/idf_*.py`. Used by both SFM and
SDM, depends on neither.
Terra-View is downstream of SDM for fleet listings, event detail, etc.
The long-term vision adds a **second link** from Terra-View → SFM for
direct device interaction (see below).
The codec work in this repo isn't trying to replace BW's network
layer — BW's ACH file forwarding and Thor's IDF call-home are
battle-tested. The value is in the receiving and processing side: turn
the stream of binary+ASCII pairs into something users can search,
filter, alert on, and report from.
### Terra-View ↔ SFM device control (the long-term vision)
Today Terra-View only reads from SDM (event listings, dashboards,
project reports). When a unit goes missing — operator notices in the
Terra-View dashboard — there's no way to *do* anything from the UI.
The path of least resistance is to RDP into a Windows box and open
Blastware, which defeats the purpose of having Terra-View.
Target experience:
- Operator notices a unit in Terra-View dashboard hasn't called in.
- Clicks unit detail → "Connect to Device" button.
- Terra-View opens an embedded view (modal or side-panel) that talks
to SFM's `/device/*` endpoints over the network.
- Live view: device clock, battery, memory, current monitor status.
- Actions: start/stop monitoring, push compliance config changes, pull
fresh events, run a sensor self-check, change call-home settings.
- Audit log: every connect / action recorded in SDM for the unit
history.
Implementation steps (concrete):
- [ ] **SFM authentication & authorization layer.** Today `/device/*`
endpoints are unauthenticated — anyone on the network can call
them. Need at minimum a token-based auth, ideally with a "who
can connect to which units" mapping. Hard prerequisite for
letting Terra-View users into the control surface.
- [ ] **Terra-View "Connect to Device" entry point** on the unit
detail page. Renders only when unit has connection info on file
and the user has permission.
- [ ] **Embedded live-monitor view** in Terra-View — equivalent to
`seismo_lab.py`'s Bridge tab, but in the browser. Polls SFM's
`/device/monitor/status` on an interval; sends start/stop via
`/device/monitor/{start,stop}`.
- [ ] **Action history** — every connect / push / action call records
a row in `unit_history`, viewable on the unit detail page.
- [ ] **Series IV live-device support in SFM** — currently `/device/*`
only supports MiniMate Plus. Blocks "Connect to Device" for
Thor units until done. Depends on Thor wire-protocol capture
and a `micromate/` parallel of the `minimateplus/` modules.
### High-impact (unblocks product features) ### High-impact (unblocks product features)
- [ ] **Series III waveform body codec reverse-engineering.** The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`). Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open. Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise. Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec. - [ ] **Series III waveform body codec reverse-engineering.** The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`). Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open. Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise. Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec.