release: v0.21.0 — Thor / Series IV codec + Thor→BW adapter
Documents two commits that landed on dev since v0.20.0:9b71eadseries 4 codec work, initial decode success micromate/idf_file.read_idf_file() decodes both IDFW (waveform; 87-99% sample fidelity reusing decode_waveform_v2 at offset 0x0f1f) and IDFH (histogram; dedicated segment-based decoder, all 859 corpus files decode, 181,071 intervals total).9fd52ddfeat: add thor report generation, pdf generation micromate/idf_to_bw_report.py adapter projects parsed Thor data into the bw_report sidecar shape so Thor events flow through sfm/report_pdf.py without a separate renderer. Wired into save_imported_idf. Net effect: a Thor event ingested via /db/import/idf_file now lands with the same fidelity as a BW event, gets a per-event PDF on demand, and renders in Terra-View's modal chart using the same plotting code as a BW event. Roadmap items closed: - Binary .IDFW / .IDFH codec (was pending) - Series IV (Thor IDF) binary codec reverse-engineering Companion: Terra-View v0.13.0 ships in parallel and closes Phase 1 of the SFM integration. No API changes in seismo-relay for that piece — Terra-View just consumes existing endpoints better. Bumps: - pyproject.toml 0.20.0 → 0.21.0 - minimateplus.event_file_io.TOOL_VERSION 0.20.0 → 0.21.0 (any subsequent backfill_sidecars.py --force will re-stamp existing sidecars; expected + harmless) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,40 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
---
|
||||
|
||||
## v0.21.0 — 2026-05-29
|
||||
|
||||
The "Thor / Series IV codec" release. Two big pieces landed: (1) the IDF binary codec actually decodes now, both IDFW and IDFH, and (2) a Thor→BW adapter lets Thor events flow through the existing Series III Event Report PDF pipeline. Combined effect: a Thor event ingested via `/db/import/idf_file` now lands in the DB with the same fidelity as a Blastware event, gets a per-event PDF on demand, and renders in Terra-View's modal chart with the same plotting code as a BW event.
|
||||
|
||||
### Added — Thor IDF binary codec (`micromate/idf_file.read_idf_file`)
|
||||
|
||||
- **IDFW (waveform)** — body sits at fixed file offset `0x0f1f`; reuses the verified `decode_waveform_v2()` walker from `minimateplus.waveform_codec`. Sample fidelity is **87–99% byte-exact** against the ASCII-sidecar reference values on quiet events; loud events hit the same walker-stops-early limitation as the BW codec on `SP0/SS0/SV0`-style events.
|
||||
- **IDFH (histogram)** — dedicated segment-based decoder for the Thor histogram body format: `[len_be][0a 00 00 00][00 NN][05 3f]` framing plus N × 72-byte interval records (4 × 16-byte per-channel min/max/halfp). **All 859 Thor IDFH corpus files decode**, totalling **181,071 intervals**; per-channel peaks match the sidecar within **~1.8% (ADC quantization)**.
|
||||
- **BW-aliased binary detection** — a small number of corpus files (e.g. `BE9439_*.IDFW/IDFH`) are actually Series III Blastware binaries that share the IDF filename convention by accident. `read_idf_file()` detects them via their BW `STRT` signature and raises `NotImplementedError` pointing the caller at `read_blastware_file()` instead of trying to decode them as IDF.
|
||||
- Full field layouts in `docs/idf_protocol_reference.md`; supporting analysis scripts in `analysis_idf/` (decode validators, per-file detail dumps, corpus accuracy reports).
|
||||
|
||||
### Added — Thor → BW report adapter (`micromate/idf_to_bw_report.py`)
|
||||
|
||||
- **`build_bw_report_from_idf(report_dict, binary_md=, intervals=, is_histogram=)`** projects a parsed Thor `IdfReport` plus binary-extracted metadata plus decoded IDFH intervals into the `bw_report`-shaped dict that `sfm.report_pdf.gather_report_data` consumes. No need to duplicate the renderer — Thor data is ~95% the same metric set as BW; the adapter handles the field-name mapping (`MicPSPL` → `pspl_dbl`, `>100` sentinel → `zc_freq_above_range`, free-form `Calibration : Nov 22, 2023 by Instantel` → `calibration_date` + `calibration_by`, etc.).
|
||||
- For IDFH events the adapter derives `histogram.interval_times` by stepping `IntervalSize` from `HistogramStartTime`, matching what the BW pipeline expects from a histogram-mode event.
|
||||
- **Wired into `WaveformStore.save_imported_idf`** — every Thor event ingested via `/db/import/idf_file` now gets a `bw_report` block in its sidecar in addition to the existing `extensions.idf_report` (the raw parsed Thor payload). Falls back gracefully (PDF renders from DB-only fields) if the adapter raises — logged as a warning rather than failing the ingest.
|
||||
|
||||
### Companion releases
|
||||
|
||||
- **Terra-View v0.13.0** ships in parallel — closes Phase 1 of the SFM integration. The shared event-detail modal now renders the SFM event story (Chart.js waveform/histogram chart, inline PDF preview, `.TXT` download, FT/reviewer/notes review form) without operators needing to bounce to the standalone SFM webapp on port 8200. Uses only existing seismo-relay endpoints — no API changes here, just better consumption.
|
||||
|
||||
### Migration / Operations
|
||||
|
||||
No DB migration needed. Existing Thor events already in the store don't automatically pick up the new `bw_report` block — they'd need a re-ingest (post the IDF binary + paired `.TXT` back to `/db/import/idf_file`) for the adapter to run. Alternatively, run `scripts/backfill_sidecars.py --reparse-txt` after a small adapter change (the script currently only re-runs the BW ASCII parser; extending it to handle Thor would be a small follow-up).
|
||||
|
||||
```bash
|
||||
cd /home/serversdown/terra-view
|
||||
docker compose build sfm && docker compose up -d sfm
|
||||
```
|
||||
|
||||
The bumped `TOOL_VERSION = "0.21.0"` in `minimateplus/event_file_io.py` means any subsequent `backfill_sidecars.py --force` pass will re-write sidecars with the new version stamp; that's expected and harmless.
|
||||
|
||||
---
|
||||
|
||||
## v0.20.0 — 2026-05-28
|
||||
|
||||
The "PDF + parser polish" release. Closes out the Event-Report PDF iteration started in v0.17.x: histogram layouts now render correctly against BW reference PDFs, the ASCII parser handles the real-world edge cases production events were tripping over (OORANGE, `>100 Hz`, histogram timestamps), and the `.TXT` preservation rollout lets parser fixes be applied retroactively to ingested events. Adds server-wide timezone support so operator-visible timestamps no longer drift into UTC. Rolls up the substantial "pre-v0.20" body of work that had accumulated under `[Unreleased]` (PDF generation, histogram codec fix, histogram parser fields, `.TXT` preservation, backfill safety) — see the trailing "pre-v0.20.0 work" section below for the full list.
|
||||
|
||||
Reference in New Issue
Block a user