feat(backfill): regenerate .h5 with /sensor_check; TOOL_VERSION 0.31.0
Complete the sensor-check standardization: existing events need their .h5
regenerated to gain the v2 /sensor_check group.
* backfill_thor_events.py attaches the decoded series-4 traces
(micromate.sensor_check) on its own IDF decode path, mirroring
save_imported_idf, so regenerated Thor .h5 files get the group. Series-3
backfill needs no change — it re-decodes via read_blastware_file, which now
attaches the traces itself.
* TOOL_VERSION 0.30.0 → 0.31.0 so the standard backfill regenerates every
event (no --force): the tool now produces the /sensor_check group. Purely
additive — no decoded value changes.
* CHANGELOG (Unreleased): sensor-check now series-3 + series-4, standardized
into the .h5 (schema v2), with the ⚠ backfill note; FFT + compliance stay
no-backfill (they read existing .h5 samples).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
This commit is contained in:
Claude Opus 4.8
+38
-20
@@ -7,14 +7,25 @@ All notable changes to seismo-relay are documented here.
|
||||
## Unreleased
|
||||
|
||||
**Blastware Event/FFT-Report parity — the FFT, the USBM compliance chart, and
|
||||
the sensor self-check.** Three analyses Blastware derives from event data,
|
||||
reverse-engineered against BE12844 (MiniMate Plus) reports and reproduced in
|
||||
seismo-relay: the compliance chart and the sensor-check strip now render on
|
||||
the event-report PDF, and the FFT reproduces Blastware's FFT Report. All three
|
||||
are additive and read from data already on disk — the `.h5` samples and the
|
||||
retained raw BW binary — so there is **no `.h5`/DB change, no migration, and no
|
||||
backfill**: a report regenerated for an existing event simply gains the new
|
||||
panels.
|
||||
the sensor self-check (series-3 *and* series-4).** Analyses Blastware/Thor
|
||||
derive from event data, reverse-engineered against BE12844 (MiniMate Plus) and
|
||||
UM (Thor) events and reproduced in seismo-relay: the compliance chart and the
|
||||
sensor-check strip render on the event-report PDF, and the FFT reproduces
|
||||
Blastware's FFT Report.
|
||||
|
||||
The FFT and the compliance chart are additive and read from the existing `.h5`
|
||||
samples — no `.h5` or DB change for those. The **sensor self-check** now lives
|
||||
in the standardized `.h5` (schema **v2**, a new `/sensor_check` group) so SFM
|
||||
serves it device-agnostically rather than decoding at report time.
|
||||
|
||||
⚠ **The sensor-check needs a backfill.** Existing `.h5` files are schema v1 and
|
||||
carry no `/sensor_check` group, so their reports show no sensor-check strip
|
||||
until regenerated. `TOOL_VERSION` is bumped to **0.31.0**, so the standard
|
||||
backfill regenerates every event and picks up the traces with **no `--force`**:
|
||||
`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
|
||||
backfill — this rides along) and the series-3 sidecar/shape backfill for
|
||||
MiniMate events. Purely additive — no decoded value changes, and v1 `.h5` files
|
||||
read fine until then (empty strip). DB backup first, as always.
|
||||
|
||||
### Added
|
||||
- **Blastware-compatible channel FFT (`waveform_fft`).** Reproduces Blastware's
|
||||
@@ -35,18 +46,25 @@ panels.
|
||||
reference PDF. A technical breakdown of the curve is in
|
||||
`docs/ri8507_compliance_curve.md`.
|
||||
|
||||
- **Sensor self-check waveforms decoded and drawn (`minimateplus.sensor_check`).**
|
||||
The "Sensor Check" traces Blastware shows to the right of the waveform panel
|
||||
live in the series-3 binary's trailing block as four length-prefixed records
|
||||
(`0x3c`–`0x3f`) using the same delta-block codec as the main waveform:
|
||||
Tran/Vert/Long geophone ring-downs (the transducer's damped impulse response —
|
||||
resonant frequency + overswing/damping) and a MicL pulse train (the mic's
|
||||
known-signal gain check). `gather_report_data` decodes them from the retained
|
||||
BW binary at report time; the report renders them as a strip flush against the
|
||||
waveform panel plus the **Sensor Check → Frequency / Overswing Ratio** sub-rows
|
||||
in the stats table. Verified against the reports on all 7 oracle events (mic
|
||||
zero-crossing frequency = 20.1 Hz exact; geophone ring-downs consistent
|
||||
~7.5 Hz with overswing ~3.5). Tests in `tests/test_sensor_check.py`.
|
||||
- **Sensor self-check waveforms decoded and drawn — both series.** The little
|
||||
"Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
|
||||
response — plus a MicL pulse train, the mic's known-signal gain check) are the
|
||||
unit's proof its sensors were healthy when it recorded the event.
|
||||
- **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
|
||||
the binary's trailing block, same delta-block codec as the main waveform.
|
||||
Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
|
||||
geophone ring-downs ~7.5 Hz, overswing ~3.5).
|
||||
- **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
|
||||
fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
|
||||
raw int16 traces; three-channel (mic-disabled) units carry only the three
|
||||
geophones. Validated by shape + cross-event consistency.
|
||||
- **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
|
||||
decoder attaches the traces to the event at decode, the writer persists
|
||||
them, and `gather_report_data` reads them back — so SFM renders the strip
|
||||
(flush against the waveform panel) plus the **Sensor Check → Frequency /
|
||||
Overswing Ratio** sub-rows without knowing the source instrument.
|
||||
- Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
|
||||
`tests/test_event_hdf5_sensor_check.py`.
|
||||
|
||||
- **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
|
||||
binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
|
||||
|
||||
@@ -50,7 +50,7 @@ SIDECAR_KIND = "sfm.event"
|
||||
# bumped without a `pip install` re-run — leading to confusing stale
|
||||
# version stamps in sidecars. Bump this constant and CHANGELOG.md
|
||||
# together at release time.
|
||||
TOOL_VERSION = "0.30.0"
|
||||
TOOL_VERSION = "0.31.0" # +/sensor_check group (schema v2); gates the backfill regen
|
||||
|
||||
try:
|
||||
# Best-effort: prefer the installed metadata when it's NEWER than the
|
||||
|
||||
@@ -305,6 +305,11 @@ def main(argv=None) -> int:
|
||||
default=0,
|
||||
)
|
||||
ev.total_samples = ev.total_samples or n_samp
|
||||
# Sensor self-check traces from the IDFW fixed
|
||||
# header, so regenerated .h5 files gain the v2
|
||||
# /sensor_check group (mirrors save_imported_idf).
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
|
||||
|
||||
event_hdf5.write_event_hdf5(
|
||||
hdf5_path, ev,
|
||||
|
||||
Reference in New Issue
Block a user