feat(backfill): regenerate .h5 with /sensor_check; TOOL_VERSION 0.31.0
Complete the sensor-check standardization: existing events need their .h5
regenerated to gain the v2 /sensor_check group.
* backfill_thor_events.py attaches the decoded series-4 traces
(micromate.sensor_check) on its own IDF decode path, mirroring
save_imported_idf, so regenerated Thor .h5 files get the group. Series-3
backfill needs no change — it re-decodes via read_blastware_file, which now
attaches the traces itself.
* TOOL_VERSION 0.30.0 → 0.31.0 so the standard backfill regenerates every
event (no --force): the tool now produces the /sensor_check group. Purely
additive — no decoded value changes.
* CHANGELOG (Unreleased): sensor-check now series-3 + series-4, standardized
into the .h5 (schema v2), with the ⚠ backfill note; FFT + compliance stay
no-backfill (they read existing .h5 samples).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
This commit is contained in:
+38
-20
@@ -7,14 +7,25 @@ All notable changes to seismo-relay are documented here.
|
|||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
**Blastware Event/FFT-Report parity — the FFT, the USBM compliance chart, and
|
**Blastware Event/FFT-Report parity — the FFT, the USBM compliance chart, and
|
||||||
the sensor self-check.** Three analyses Blastware derives from event data,
|
the sensor self-check (series-3 *and* series-4).** Analyses Blastware/Thor
|
||||||
reverse-engineered against BE12844 (MiniMate Plus) reports and reproduced in
|
derive from event data, reverse-engineered against BE12844 (MiniMate Plus) and
|
||||||
seismo-relay: the compliance chart and the sensor-check strip now render on
|
UM (Thor) events and reproduced in seismo-relay: the compliance chart and the
|
||||||
the event-report PDF, and the FFT reproduces Blastware's FFT Report. All three
|
sensor-check strip render on the event-report PDF, and the FFT reproduces
|
||||||
are additive and read from data already on disk — the `.h5` samples and the
|
Blastware's FFT Report.
|
||||||
retained raw BW binary — so there is **no `.h5`/DB change, no migration, and no
|
|
||||||
backfill**: a report regenerated for an existing event simply gains the new
|
The FFT and the compliance chart are additive and read from the existing `.h5`
|
||||||
panels.
|
samples — no `.h5` or DB change for those. The **sensor self-check** now lives
|
||||||
|
in the standardized `.h5` (schema **v2**, a new `/sensor_check` group) so SFM
|
||||||
|
serves it device-agnostically rather than decoding at report time.
|
||||||
|
|
||||||
|
⚠ **The sensor-check needs a backfill.** Existing `.h5` files are schema v1 and
|
||||||
|
carry no `/sensor_check` group, so their reports show no sensor-check strip
|
||||||
|
until regenerated. `TOOL_VERSION` is bumped to **0.31.0**, so the standard
|
||||||
|
backfill regenerates every event and picks up the traces with **no `--force`**:
|
||||||
|
`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
|
||||||
|
backfill — this rides along) and the series-3 sidecar/shape backfill for
|
||||||
|
MiniMate events. Purely additive — no decoded value changes, and v1 `.h5` files
|
||||||
|
read fine until then (empty strip). DB backup first, as always.
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- **Blastware-compatible channel FFT (`waveform_fft`).** Reproduces Blastware's
|
- **Blastware-compatible channel FFT (`waveform_fft`).** Reproduces Blastware's
|
||||||
@@ -35,18 +46,25 @@ panels.
|
|||||||
reference PDF. A technical breakdown of the curve is in
|
reference PDF. A technical breakdown of the curve is in
|
||||||
`docs/ri8507_compliance_curve.md`.
|
`docs/ri8507_compliance_curve.md`.
|
||||||
|
|
||||||
- **Sensor self-check waveforms decoded and drawn (`minimateplus.sensor_check`).**
|
- **Sensor self-check waveforms decoded and drawn — both series.** The little
|
||||||
The "Sensor Check" traces Blastware shows to the right of the waveform panel
|
"Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
|
||||||
live in the series-3 binary's trailing block as four length-prefixed records
|
response — plus a MicL pulse train, the mic's known-signal gain check) are the
|
||||||
(`0x3c`–`0x3f`) using the same delta-block codec as the main waveform:
|
unit's proof its sensors were healthy when it recorded the event.
|
||||||
Tran/Vert/Long geophone ring-downs (the transducer's damped impulse response —
|
- **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
|
||||||
resonant frequency + overswing/damping) and a MicL pulse train (the mic's
|
the binary's trailing block, same delta-block codec as the main waveform.
|
||||||
known-signal gain check). `gather_report_data` decodes them from the retained
|
Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
|
||||||
BW binary at report time; the report renders them as a strip flush against the
|
geophone ring-downs ~7.5 Hz, overswing ~3.5).
|
||||||
waveform panel plus the **Sensor Check → Frequency / Overswing Ratio** sub-rows
|
- **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
|
||||||
in the stats table. Verified against the reports on all 7 oracle events (mic
|
fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
|
||||||
zero-crossing frequency = 20.1 Hz exact; geophone ring-downs consistent
|
raw int16 traces; three-channel (mic-disabled) units carry only the three
|
||||||
~7.5 Hz with overswing ~3.5). Tests in `tests/test_sensor_check.py`.
|
geophones. Validated by shape + cross-event consistency.
|
||||||
|
- **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
|
||||||
|
decoder attaches the traces to the event at decode, the writer persists
|
||||||
|
them, and `gather_report_data` reads them back — so SFM renders the strip
|
||||||
|
(flush against the waveform panel) plus the **Sensor Check → Frequency /
|
||||||
|
Overswing Ratio** sub-rows without knowing the source instrument.
|
||||||
|
- Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
|
||||||
|
`tests/test_event_hdf5_sensor_check.py`.
|
||||||
|
|
||||||
- **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
|
- **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
|
||||||
binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
|
binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
|
||||||
|
|||||||
@@ -50,7 +50,7 @@ SIDECAR_KIND = "sfm.event"
|
|||||||
# bumped without a `pip install` re-run — leading to confusing stale
|
# bumped without a `pip install` re-run — leading to confusing stale
|
||||||
# version stamps in sidecars. Bump this constant and CHANGELOG.md
|
# version stamps in sidecars. Bump this constant and CHANGELOG.md
|
||||||
# together at release time.
|
# together at release time.
|
||||||
TOOL_VERSION = "0.30.0"
|
TOOL_VERSION = "0.31.0" # +/sensor_check group (schema v2); gates the backfill regen
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Best-effort: prefer the installed metadata when it's NEWER than the
|
# Best-effort: prefer the installed metadata when it's NEWER than the
|
||||||
|
|||||||
@@ -305,6 +305,11 @@ def main(argv=None) -> int:
|
|||||||
default=0,
|
default=0,
|
||||||
)
|
)
|
||||||
ev.total_samples = ev.total_samples or n_samp
|
ev.total_samples = ev.total_samples or n_samp
|
||||||
|
# Sensor self-check traces from the IDFW fixed
|
||||||
|
# header, so regenerated .h5 files gain the v2
|
||||||
|
# /sensor_check group (mirrors save_imported_idf).
|
||||||
|
from micromate.sensor_check import decode_idf_sensor_check
|
||||||
|
ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
|
||||||
|
|
||||||
event_hdf5.write_event_hdf5(
|
event_hdf5.write_event_hdf5(
|
||||||
hdf5_path, ev,
|
hdf5_path, ev,
|
||||||
|
|||||||
Reference in New Issue
Block a user