serversdown/seismo-relay
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

release: v0.21.0 — Thor / Series IV codec + Thor→BW adapter

Browse Source 
Documents two commits that landed on dev since v0.20.0:

  9b71ead  series 4 codec work, initial decode success
           micromate/idf_file.read_idf_file() decodes both IDFW
           (waveform; 87-99% sample fidelity reusing
           decode_waveform_v2 at offset 0x0f1f) and IDFH (histogram;
           dedicated segment-based decoder, all 859 corpus files
           decode, 181,071 intervals total).

  9fd52dd  feat: add thor report generation, pdf generation
           micromate/idf_to_bw_report.py adapter projects parsed
           Thor data into the bw_report sidecar shape so Thor
           events flow through sfm/report_pdf.py without a
           separate renderer.  Wired into save_imported_idf.

Net effect: a Thor event ingested via /db/import/idf_file now
lands with the same fidelity as a BW event, gets a per-event PDF
on demand, and renders in Terra-View's modal chart using the same
plotting code as a BW event.

Roadmap items closed:
- Binary .IDFW / .IDFH codec (was pending)
- Series IV (Thor IDF) binary codec reverse-engineering

Companion: Terra-View v0.13.0 ships in parallel and closes Phase 1
of the SFM integration.  No API changes in seismo-relay for that
piece — Terra-View just consumes existing endpoints better.

Bumps:
- pyproject.toml 0.20.0 → 0.21.0
- minimateplus.event_file_io.TOOL_VERSION 0.20.0 → 0.21.0
  (any subsequent backfill_sidecars.py --force will re-stamp
  existing sidecars; expected + harmless)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
serversdown
2026-05-29 19:25:44 +00:00
parent 9fd52ddabb
commit e42956a20b
5 changed files with 51 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -4
View File
@@ -1,4 +1,4 @@
# seismo-relay `v0.20.0`
# seismo-relay `v0.21.0`
A ground-up replacement for **Blastware** — Instantel's aging Windows-only
software for managing seismographs. Supports both the **MiniMate Plus
@@ -45,6 +45,15 @@ over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).
> `scripts/backfill_sidecars.py --reparse-txt` lets parser fixes be
> applied retroactively to existing events without re-forwarding,
> using the `.TXT` files preserved at ingest time.
> **v0.21.0 (2026-05-29)** is the Thor / Series IV decoder release —
> `micromate/idf_file.read_idf_file()` now decodes both IDFW
> (waveform) and IDFH (histogram) binaries (87–99% sample fidelity
> on quiet IDFW events; all 859 IDFH corpus files decode cleanly).
> A new `micromate/idf_to_bw_report.py` adapter projects parsed
> Thor reports into the BW-shaped sidecar block, so Thor events
> flow through the existing Event Report PDF pipeline without a
> separate renderer. Terra-View v0.13.0 ships in parallel and
> closes Phase 1 of the SFM integration — see its CHANGELOG.
> See [CHANGELOG.md](CHANGELOG.md) for full version history.
---
@@ -68,7 +77,8 @@ seismo-relay/
├── micromate/ ← Series IV (Micromate / Thor) client library (NEW v0.19)
│ ├── models.py ← IdfEvent, IdfReport, IdfPeaks, IdfProjectInfo, IdfSensorCheck (mic in native dB(L))
│ ├── idf_ascii_report.py ← Parse Thor .IDFW.txt / .IDFH.txt event sidecars
│ └── idf_file.py ← Stub for the .IDFW / .IDFH binary codec (reverse-engineering pending)
│ ├── idf_file.py ← Binary codec for .IDFW + .IDFH (v0.21.0+)
│ └── idf_to_bw_report.py ← Adapter projecting Thor IDF into the BW report shape (v0.21.0+)
│
├── sfm/ ← SFM REST API server (FastAPI, port 8200)
│ ├── server.py ← Live device endpoints + DB query + ingest endpoints + caching
@@ -425,7 +435,7 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
- [x] Thor IDF file ingest at `/db/import/idf_file` (paired with `thor-watcher`, v0.18.0+)
- [x] Native `IdfEvent` / `IdfReport` typed models — mic in dB(L), full title strings, sensor self-check, calibration, firmware version
- [x] Parser verified against 1,014 paired `.txt` sidecars in `thor-watcher/example-data/`
- [ ] Binary `.IDFW` / `.IDFH` codec — pending (see Roadmap + [`docs/idf_protocol_reference.md`](docs/idf_protocol_reference.md))
- [x] Binary `.IDFW` / `.IDFH` codec — ✅ v0.21.0. IDFW reuses `decode_waveform_v2()` on the body at offset `0x0f1f` (87–99% sample fidelity on quiet events); IDFH has a dedicated segment-based decoder (all 859 corpus files decode, 181,071 intervals total). See `micromate/idf_file.py` + `docs/idf_protocol_reference.md`.
- [ ] Live-device protocol — pending codec
**Data persistence:**
@@ -538,7 +548,7 @@ Implementation steps (concrete):
### High-impact (unblocks product features)
- [ ] **Series III waveform body codec reverse-engineering.** The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`). Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open. Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise. Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec.
- [ ] **Series IV (Thor IDF) binary codec reverse-engineering.** `.IDFH` / `.IDFW` files are currently stored opaquely by `WaveformStore.save_imported_idf`, with all metadata sourced from the paired `.txt` sidecar. This works because thor-watcher forwards both files together, but operators who haven't enabled Thor's TXT exporter get rows with NULL peaks. Cracking the binary closes that gap and unlocks waveform display. Starting-point reference at [`docs/idf_protocol_reference.md`](docs/idf_protocol_reference.md) — two observed file signatures (1,012 newer-firmware files + 2 old files whose layout matches the Series III STRT-record format), suggested first-session plan (~2-4 hrs), 1,014 paired binary+txt files available as ground truth in `thor-watcher/example-data/`. Code seam ready at `micromate/idf_file.py`.
- [x] **Series IV (Thor IDF) binary codec reverse-engineering.** ✅ v0.21.0 — `micromate/idf_file.read_idf_file()` decodes both IDFW (waveform body at offset `0x0f1f`, reusing `decode_waveform_v2()`; 87–99% sample fidelity on quiet events) and IDFH (dedicated segment-based decoder: all 859 corpus files decode, 181,071 intervals, peaks within ~1.8% of sidecar values). `WaveformStore.save_imported_idf` now also projects parsed Thor data into a `bw_report` block via `micromate/idf_to_bw_report.py` so Thor events render in the existing Event Report PDF pipeline without a separate renderer.
- [ ] **In-app waveform viewer accuracy.** Depends on Series III codec decode. Plot.v1 JSON pipeline + viewer skeleton already exist; will start showing real waveforms automatically once `_decode_a5_waveform` produces correct samples. Series IV waveforms come online when the IDF codec lands.
- [ ] **Series IV live-device support.** Once the IDF binary is decoded, extend `micromate/` with `transport.py` / `framing.py` / `protocol.py` / `client.py` mirroring the `minimateplus/` package layout — depends on capturing Thor's wire protocol (TCP / RS-232 captures TBD).
- [ ] **Terra-view integration** — seismo-relay router, unit detail page, VISON-style event listing.