serversdown/seismo-relay
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

seismo-relay v0.19.0 — device-family separation + micromate/ package

Browse Source 
Tighten the Series III / Series IV boundary so UI and storage dispatch
on a clean signal instead of sniffing filenames or applying magnitude
heuristics.

Phase 1 — events.device_family column ("series3" | "series4"):
  self-applying migration with filename-based backfill of existing rows
  (1,132 backfilled on prod 2026-05-20); plumbed through every import
  path (BW endpoint, IDF endpoint, ACH server, BW CLI, sidecar
  backfill); UPSERT preserves via COALESCE; UI dispatches on it.

Phase 2 — extract micromate/ package alongside minimateplus/:
  native IdfEvent / IdfReport / IdfPeaks / IdfProjectInfo /
  IdfSensorCheck (mic in dB(L), not pseudo-psi); moved
  idf_ascii_report.py from sfm/ to micromate/; refactored
  save_imported_idf to use IdfEvent and bridge to minimateplus.Event at
  the SQL-insert boundary; idf_file.py stub for the future binary codec.

Phase 3 prep — docs/idf_protocol_reference.md captures the two
observed Thor binary header signatures (1,012 newer-firmware files vs
2 old files whose layout is byte-for-byte BW-STRT-compatible), file-size
hints suggesting int8 sample encoding, open questions in dependency
order, and a concrete first-session plan for cracking the codec.

Also rolled in the v0.18.1 hotfixes that motivated this work:
  - idf_ascii_report parser now handles "<0.005 in/s" (below-threshold)
    and "N/A" markers without leaving raw strings in numeric DB columns.
  - sfm_webapp.html: defensive _ppvFmt / mic formatter so future
    data-shape drift can't kill the whole events table render.

All 1,014 example-data sidecars round-trip through the new package.
See CHANGELOG.md for full notes.
This commit is contained in:
serversdown
2026-05-20 15:19:49 +00:00
parent e95ac692ee
commit ecc935482b
11 changed files with 966 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+27
View File
@@ -4,6 +4,33 @@ All notable changes to seismo-relay are documented here.
---
## v0.19.0 — 2026-05-20
The "device-family separation" release. Tightens the boundary between Series III (MiniMate Plus / Blastware) and Series IV (Micromate / Thor) so the UI and storage layer dispatch deterministically by family instead of sniffing filename extensions or magnitude heuristics.
### Added — Phase 1: `device_family` column on `events`
- **`events.device_family TEXT`** — new column carrying `"series3"` or `"series4"`. Populated by every import path (`/db/import/blastware_file`, `/db/import/idf_file`, ACH server, BW CLI, sidecar backfill script). Returned through `/db/events` since `query_events` uses `SELECT *`.
- **Self-applying migration** — on startup, `ALTER TABLE ... ADD COLUMN` lands the new column; a follow-on `UPDATE` backfills existing rows from the binary filename extension (`.IDFH`/`.IDFW` → `series4`, everything else → `series3`). No manual SQL needed.
- **UPSERT preserves family** — re-imports without an explicit family don't blank existing rows (`COALESCE(?, device_family)`).
- **UI dispatches on the column** — `sfm_webapp.html` events-table mic formatter now branches on `ev.device_family === 'series4'` (Thor stores native dB(L); BW stores psi). Modal uses `source.kind === 'idf-import'` from the sidecar (sidecars don't carry the DB column). Source-files section labels changed from "BW filename / BW filesize / BW sha256" to format-neutral "Event file / File size / File sha256".
### Added — Phase 2: `micromate/` package alongside `minimateplus/`
- **`micromate/`** — new sibling package for the Thor / Micromate Series IV device. Currently scoped to offline-file ingest; live-device support (TCP transport, framing, protocol, client) will land here when reverse-engineering happens.
- `micromate/idf_ascii_report.py` — moved from `sfm/idf_ascii_report.py`. No behaviour change.
- `micromate/models.py` — typed `IdfReport`, `IdfEvent`, `IdfPeaks`, `IdfProjectInfo`, `IdfSensorCheck`. Stores mic in native `mic_pspl_dbl` (dB(L)) instead of the pseudo-psi shoehorn that the BW-shaped model uses. `IdfEvent.from_report()` constructs from a parsed dict + filename; `IdfEvent.to_minimateplus_event(waveform_key)` bridges to the existing sidecar / DB-insert machinery.
- `micromate/idf_file.py` — placeholder for the binary codec (`.IDFH` / `.IDFW`). Stubbed `read_idf_file()` raises `NotImplementedError`; documents the planned reverse-engineering path.
- **`WaveformStore.save_imported_idf`** refactored to use the native `IdfEvent` and bridge at the SQL-insert boundary. Cleaner separation of "parse a Thor event" (in `micromate/`) from "store it on disk + write a sidecar" (in `sfm/waveform_store.py`).
- **Tests** — `tests/test_idf_ascii_report.py` imports updated to `micromate.idf_ascii_report`. All 1,014 example-data sidecars round-trip through `IdfEvent.from_report()` without errors.
### Companion releases
- **thor-watcher** unaffected — it talks to the relay over HTTP only. No version bump needed.
- **terra-view** unaffected today; can use `device_family` in its event-detail rendering when convenient.
---
## v0.18.0 — 2026-05-19
The "Thor / Series IV ingest adapter" release. Seismo-relay can now accept event files from Instantel Micromate Series IV (Thor) units alongside the existing MiniMate Plus (Series III) Blastware pipeline.