seismo-relay/docs/sfm_tool_status.md

SFM — where it actually stands as a tool

Status as of 2026-09-20 (v0.31.0). This is the honest assessment, not the roadmap — README.md § Roadmap covers where it is going. Expect this file to go stale; re-date it when you revise it.

---

The framing

SFM is three different things wearing one name, at three very different levels of maturity:

	what it is	maturity
The codec library	minimateplus/, micromate/ — bytes in, Event out	Production. Verified per-sample at scale.
SDM — the data side	the DB, waveform store, /db/*, ingest	Production. Terra-View depends on it daily.
SFM — the device side	/device/*, live connections to units	Emergency-grade. Works, but manual, unauthenticated, and thinly tested.
The lab	seismo_lab.py, scratch/, the Inspector	Research artifacts. Useful, not products.

Brian's own description — "right now it's an emergency tool and a research project" — is accurate, and it applies specifically to the device side. The data side is not an emergency tool; it has been carrying production for months.

Most confusion about "is SFM reliable?" comes from answering for the wrong tier.

---

1. What you can rely on

Production-grade — trust it

• Series-3 decode. 14,338 / 14,338 files decode per-sample exact against preserved Blastware ASCII exports, 45 units, files back to 2018.
• Series-4 (Thor) decode. 1,057,536 / 1,057,536 geo samples exact against Thor's own CSV exports; production IDFW 575/575 with zero truncations.
• Histogram decode. 1,211 / 1,211 production histograms exact, including 842,442 per-interval frequency comparisons with zero mismatches.
• The ingest path. /db/import/blastware_file and /db/import/idf_file fed by the watchers — this is how prod actually gets its data, and it has been running unattended for months.
• /db/* read API. Always-on, consumed by Terra-View for every fleet listing, event detail and report.
• The waveform store — .h5 + .sfm.json sidecars + retained raw binaries, with operator review state preserved across regeneration.
• bridges/ach_server.py — speaks the full BW protocol to calling units. Proven in the field, including as a rescue tool (see the runbook).

Emergency-grade — works, but you are the error handling

• /device/* live endpoints. They do what they say. But they are synchronous, unauthenticated, and a single cellular download can exceed the 60 s timeouts that sit in front of them.
• The rescue ladder (rescue, stop_monitoring_*, events/erase). Each has worked in a real incident — but each has been used a handful of times, by one person, with the runbook open.
• The standalone webapp. Perfectly usable, and as of v0.31.0 the cheap probes and rescue actions are reachable without curl. No auth of any kind.

Research artifacts — useful, not products

• seismo_lab.py — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB / Inspector). Desktop-only, single-user, no tests.
• scratch/ — the verification harnesses (verify_against_ascii.py, verify_thor_against_csv.py) and the offset detector (offset_scan3.py). These produced the numbers the production claims rest on, so they matter — but they are analysis scripts, not maintained code.
• docs/offset_investigation.md — an open investigation, not a feature.

---

2. What to use when

you want to…	use	notes
Know if a unit is monitoring / its battery / memory	GET /device/monitor/status?force=true	~2 s
Know whether ACH is on	GET /device/call_home	~2 s. Not /device/events.
See how full a unit's buffer is	GET /device/events/storage_range	~2 s, no chain walk
Stop a runaway unit	Diagnostics tab → Stop Monitoring	see the runbook first
Reach a unit that will not answer	point its modem at an ach_server and answer its call	runbook Method A — do not race it
List a unit's stored events	Events tab → Load events	slow, and broken past 64 KB (below)
Get event data into the DB	the watcher → /db/import/* path	not the live walk

The single most useful habit: the cheap probes are cheap and the event walk is not. Reaching for /device/events to answer a yes/no question about a unit is the mistake that motivated the v0.31.0 webapp changes.

---

3. Known issues

issue	impact	status
5A walk dies once a unit's buffer crosses 64 KB	/device/events 500s; event body never downloads	Known, documented in CLAUDE.md. Needs a BW capture of a spanning event to fix properly.
No auth on SFM at all	21 /device/* endpoints, including destructive ones, open to anything that reaches the port	Design agreed (Terra-View as authenticated jump host); not built.
Swagger try-it-out is live on destructive endpoints	POST /device/events/erase is one click away at :8200/docs	Partially mitigated: the webapp's erase now requires typing the serial. /docs itself is unguarded.
SUB 0x08 lifetime counter reads 0	/device/events/index returns a meaningless number	Suspected field-offset bug. Surfaced in the UI as "unreliable".
Long device operations are synchronous	60 s timeouts in routers/sfm.py and the reverse proxy; a full download exceeds both	Known design constraint. Must be POST-starts-job / GET-polls before any remote lab.
backfill_sidecars.py --force silently inserts DB rows	store files with no DB row get one; the dry-run does not report the count	Known. Avoid --force — TOOL_VERSION gates regeneration anyway.
14 sensitive-range files show an exact 8× discrepancy	10.0 / 1.25 — a units problem, not a decode problem	Open, not blocking.
16 failing tests on dev	15 need gitignored fixture bundles; 1 is real (sc["peak_values"]["transverse"] returns None where 0.0 is expected)	The real one shipped in v0.31.0.

---

4. What stands between this and a real tool

Roughly in dependency order — each unblocks the ones below it.

1. Authentication. Everything else is gated on this. SFM has none, and the modem IP whitelist gives zero protection because SFM is the whitelisted origin. The agreed design delegates rather than builds: Terra-View becomes the authenticated jump host (/api/sfm/* already inherits deny-by-default operator auth), and the 8200:8200 publish is dropped so Terra-View is the only door.

2. Async long operations. POST starts a job, GET polls. Retrofitting this after building a remote lab on top of synchronous endpoints would be far worse than designing for it now.

3. Confirm-guards on the remaining destructive endpoints. Auth answers who, not did you mean it. The webapp's erase is guarded; the other seven destructive POSTs and /docs are not.

4. The 5A page-boundary fix. Until this lands, live event download is unreliable on exactly the units most likely to need attention — the ones that have been recording heavily. Wants a Blastware capture of an event spanning a page boundary before the chunk-addressing half is trustworthy.

5. A live Thor / Micromate client. The device side is MiniMate-only. Series-4 units can only be read from forwarded files, so half the fleet has no live path at all.

6. Test coverage that runs from a clean checkout. 15 of 16 current failures are missing fixture bundles. A test suite that cannot go green on a fresh clone cannot gate anything.

7. The SDM rename. Cosmetic relative to the above, but the longer sfm/ holds the data-side code the more the tiers blur. ~30–50 files here, ~10–15 in Terra-View, plus a Docker volume migration. Do it when the codebase is quiet.

---

The short version

The data side is a real tool already. The device side is a set of sharp instruments that work in the hands of the person who wrote them, with the runbook open. The gap between those two states is mostly auth, async, and guardrails — not protocol work. The protocol is the part that is actually finished.