docs: three-tier architecture model + strategic roadmap
CLAUDE.md gains an Architecture section near the top describing the
canonical three-tier mental model:
- SFM: device-side, live connections, /device/* endpoints
- SDM: data-side, DB + waveform store + /db/* endpoints (currently
living under sfm/ for historical reasons; rename deferred)
- Codec library: pure data-interpretation, used by both tiers
Future code should be placed and named according to this model even
though the directory layout doesn't fully reflect it yet. Decision
rule for where new code goes is documented inline.
README.md's Roadmap section gains two strategic-direction subsections:
- "Strategic direction" — frames the suite-of-components vision and
notes that BW ACH + Thor IDF call-home remain the data movers;
seismo-relay's value is on the receiving and processing side.
- "Terra-View ↔ SFM device control" — the long-term vision where
Terra-View can launch into SFM device-control surfaces (operator
notices missing unit → clicks "Connect to Device" → live view in
browser). Includes concrete implementation checklist (auth,
embedded live-monitor view, action history, series IV live
support).
The existing tactical roadmap items remain unchanged below.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,84 @@ When new information about the protocol is discovered, please update the instant
|
||||
|
||||
---
|
||||
|
||||
## Architecture: three-tier conceptual model
|
||||
|
||||
seismo-relay is a **suite of cooperating components**, not a single app.
|
||||
The three tiers below are the canonical mental model — the current
|
||||
directory layout doesn't fully reflect them yet (some of what is
|
||||
conceptually SDM lives under `sfm/` today), but new code should be
|
||||
placed and named according to this model.
|
||||
|
||||
### 1. SFM — the device-side (active connection to physical units)
|
||||
|
||||
Replaces Blastware's *talk-to-the-meter* role. Lives where a connection
|
||||
to a physical seismograph is open.
|
||||
|
||||
In scope:
|
||||
- `minimateplus/{transport,framing,protocol,client}.py` — wire protocol
|
||||
- `seismo_lab.py` — diagnostic GUI (a thick client for SFM)
|
||||
- The `/device/*` HTTP endpoints in `sfm/server.py` —
|
||||
`/device/info`, `/device/events`, `/device/monitor/*`, `/device/call_home`,
|
||||
etc. Anything that opens a connection at the moment of the request.
|
||||
- Future: a Thor / Micromate live client (mirror `minimateplus/`)
|
||||
- Future: a control surface Terra-View can launch into — see the
|
||||
README's Roadmap.
|
||||
|
||||
Does NOT own a database. Outputs `Event` objects. Has a "spun up when
|
||||
needed" runtime profile rather than "always on".
|
||||
|
||||
### 2. SDM — the data-side (storage, ingest, and serving)
|
||||
|
||||
The new name for the receiving-and-storing role. Originally called SFM
|
||||
because the FastAPI service started life as a thin device proxy, but
|
||||
the actual role has migrated heavily toward data management. **For now
|
||||
the directory remains `sfm/`** — renaming requires touching ~30-50
|
||||
files in seismo-relay + ~10-15 in terra-view + a Docker volume
|
||||
migration; deferred until the codebase is quiet enough to do it as a
|
||||
clean refactor.
|
||||
|
||||
In scope:
|
||||
- `sfm/database.py` (`SeismoDb`)
|
||||
- `sfm/waveform_store.py`, `sfm/event_hdf5.py`
|
||||
- The `/db/*` HTTP endpoints — `events`, `units`, `monitor_log`,
|
||||
`sessions`, `false_trigger` mutations
|
||||
- The `/db/import/*` ingest endpoints — `blastware_file` (series3),
|
||||
`idf_file` (series4); anything that receives events FROM somewhere
|
||||
- `scripts/backfill_sidecars.py`, `scripts/check_bw_report_preservation.py`,
|
||||
and similar data-maintenance tools
|
||||
- The `.sfm.json` sidecars and `.h5` files in the waveform store
|
||||
- The shape that Terra-View consumes (Terra-View should never need to
|
||||
reach into SFM/device-side endpoints to populate its UI)
|
||||
|
||||
Always-on, scaled for storage/serving, has the DB and waveform store.
|
||||
|
||||
### 3. Codec library — pure data interpretation (used by both sides)
|
||||
|
||||
Neither SFM nor SDM — a shared library both depend on.
|
||||
|
||||
In scope:
|
||||
- `minimateplus/{waveform_codec,histogram_codec,event_file_io,bw_ascii_report,blastware_file}.py`
|
||||
- `micromate/{idf_ascii_report,idf_file}.py`
|
||||
|
||||
These modules take bytes (off the wire on the SFM side, or from a
|
||||
forwarded file on the SDM side) and return `Event` objects. They
|
||||
should not import from `sfm/`, must not touch a DB, and have no I/O
|
||||
beyond reading files passed as arguments. Keep them pure — both
|
||||
tiers can then depend on them without circularity.
|
||||
|
||||
### Practical consequences
|
||||
|
||||
When deciding where new code goes, ask:
|
||||
- *Does it need a connection to a device?* → SFM
|
||||
- *Does it operate on stored events / sidecars / DB rows?* → SDM
|
||||
- *Does it interpret bytes into structured data, with no I/O of its own?* → codec lib
|
||||
|
||||
Terra-View is downstream of SDM for data, and (per the roadmap) will
|
||||
eventually invoke into SFM's device-control endpoints to provide a
|
||||
"connect to unit" experience.
|
||||
|
||||
---
|
||||
|
||||
## Project layout
|
||||
|
||||
```
|
||||
|
||||
@@ -459,6 +459,72 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
|
||||
|
||||
## Roadmap (Future)
|
||||
|
||||
### Strategic direction — where this is going
|
||||
|
||||
seismo-relay is being built as a **suite of cooperating components**
|
||||
that together replace and improve on Blastware's role. Three logical
|
||||
tiers:
|
||||
|
||||
1. **SFM** (device-side) — owns the active connection to a physical
|
||||
unit. Today: `minimateplus/`, `/device/*` HTTP endpoints,
|
||||
`seismo_lab.py`. Future: live Thor / Micromate support.
|
||||
2. **SDM** (data-side) — owns the database, waveform store, ingest
|
||||
pipelines, and the read-API that Terra-View consumes. Today this
|
||||
code lives under `sfm/` for historical reasons; the role has
|
||||
migrated and the eventual rename is on the long-tail cleanup list.
|
||||
3. **Codec library** — pure data-interpretation: `minimateplus/*_codec.py`,
|
||||
`bw_ascii_report.py`, `micromate/idf_*.py`. Used by both SFM and
|
||||
SDM, depends on neither.
|
||||
|
||||
Terra-View is downstream of SDM for fleet listings, event detail, etc.
|
||||
The long-term vision adds a **second link** from Terra-View → SFM for
|
||||
direct device interaction (see below).
|
||||
|
||||
The codec work in this repo isn't trying to replace BW's network
|
||||
layer — BW's ACH file forwarding and Thor's IDF call-home are
|
||||
battle-tested. The value is in the receiving and processing side: turn
|
||||
the stream of binary+ASCII pairs into something users can search,
|
||||
filter, alert on, and report from.
|
||||
|
||||
### Terra-View ↔ SFM device control (the long-term vision)
|
||||
|
||||
Today Terra-View only reads from SDM (event listings, dashboards,
|
||||
project reports). When a unit goes missing — operator notices in the
|
||||
Terra-View dashboard — there's no way to *do* anything from the UI.
|
||||
The path of least resistance is to RDP into a Windows box and open
|
||||
Blastware, which defeats the purpose of having Terra-View.
|
||||
|
||||
Target experience:
|
||||
- Operator notices a unit in Terra-View dashboard hasn't called in.
|
||||
- Clicks unit detail → "Connect to Device" button.
|
||||
- Terra-View opens an embedded view (modal or side-panel) that talks
|
||||
to SFM's `/device/*` endpoints over the network.
|
||||
- Live view: device clock, battery, memory, current monitor status.
|
||||
- Actions: start/stop monitoring, push compliance config changes, pull
|
||||
fresh events, run a sensor self-check, change call-home settings.
|
||||
- Audit log: every connect / action recorded in SDM for the unit
|
||||
history.
|
||||
|
||||
Implementation steps (concrete):
|
||||
- [ ] **SFM authentication & authorization layer.** Today `/device/*`
|
||||
endpoints are unauthenticated — anyone on the network can call
|
||||
them. Need at minimum a token-based auth, ideally with a "who
|
||||
can connect to which units" mapping. Hard prerequisite for
|
||||
letting Terra-View users into the control surface.
|
||||
- [ ] **Terra-View "Connect to Device" entry point** on the unit
|
||||
detail page. Renders only when unit has connection info on file
|
||||
and the user has permission.
|
||||
- [ ] **Embedded live-monitor view** in Terra-View — equivalent to
|
||||
`seismo_lab.py`'s Bridge tab, but in the browser. Polls SFM's
|
||||
`/device/monitor/status` on an interval; sends start/stop via
|
||||
`/device/monitor/{start,stop}`.
|
||||
- [ ] **Action history** — every connect / push / action call records
|
||||
a row in `unit_history`, viewable on the unit detail page.
|
||||
- [ ] **Series IV live-device support in SFM** — currently `/device/*`
|
||||
only supports MiniMate Plus. Blocks "Connect to Device" for
|
||||
Thor units until done. Depends on Thor wire-protocol capture
|
||||
and a `micromate/` parallel of the `minimateplus/` modules.
|
||||
|
||||
### High-impact (unblocks product features)
|
||||
|
||||
- [ ] **Series III waveform body codec reverse-engineering.** The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`). Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open. Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise. Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec.
|
||||
|
||||
Reference in New Issue
Block a user