Feat/sensor check h5 #39
Assignees
serversdown
Clear assignees
serversdown
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: serversdown/seismo-relay#39
Reference in New Issue
Block a user
The Thor/Micromate (series-4) IDFW binary carries the sensor self-check in its fixed-header region (before the waveform body), as up to four records tagged 01 0e 3c/3d/3e/3f — the SAME channel ids as series-3 (Tran/Vert/Long/MicL). Unlike series-3's delta-coded trailing block, series-4 stores each trace as a raw int16-BE array after an 18-byte record header (2-byte sample count at offset +8). Three-channel (mic-disabled) units carry only 3c/3d/3e. New micromate/sensor_check.py: decode_idf_sensor_check(raw) locates the record chain (id-ordered marker run, so a stray body match can't chain) and reads each trace's int16 samples → {Tran,Vert,Long[,MicL]: [counts]}, or {} when absent. Reverse-engineered + validated against 4 UM oracle events (added as fixtures): clean geophone ring-downs on all, mic pulse trains on the 4-channel units, correctly no MicL on the two 3-channel units. Validated by shape + cross-event consistency (no Thor report strip to exact-match, unlike series-3's BW reports). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzfMake the sensor self-check a first-class part of the standardized decoded event so SFM stops decoding it at report time — device-agnostic, per the store's decoder→standardized-.h5→SFM model. * Event gains a `sensor_check` field; both decoders attach the traces where they set raw_samples — series-3 in event_file_io.read_blastware_file (minimateplus.sensor_check), series-4 in waveform_store's IDF path (micromate.sensor_check). Covers ingest and backfill (both re-decode). * event_hdf5 bumps schema_version 1→2 and writes an optional /sensor_check group (raw counts, int32, per channel present). read_event_hdf5 returns it; plot_json_from_hdf5 carries it as a top-level key. Old v1 files still read cleanly (no group → None), so nothing breaks before the backfill. * gather_report_data reads sensor_check_waveforms from the .h5 and drops the report-time series-3 decode — the report no longer reaches into a decoder, and a series-4 event now lights up the same strip automatically. Stored as raw counts (a shape diagnostic, rendered fit-to-box): the per-series count scale differs and a physical mic unit is ill-defined, so conversion would add complexity for no display benefit — easy to add later if a numeric use appears. Tests: .h5 roundtrip + backward-compat + plot_json + real series-3 decode attaches to the Event. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzfComplete the sensor-check standardization: existing events need their .h5 regenerated to gain the v2 /sensor_check group. * backfill_thor_events.py attaches the decoded series-4 traces (micromate.sensor_check) on its own IDF decode path, mirroring save_imported_idf, so regenerated Thor .h5 files get the group. Series-3 backfill needs no change — it re-decodes via read_blastware_file, which now attaches the traces itself. * TOOL_VERSION 0.30.0 → 0.31.0 so the standard backfill regenerates every event (no --force): the tool now produces the /sensor_check group. Purely additive — no decoded value changes. * CHANGELOG (Unreleased): sensor-check now series-3 + series-4, standardized into the .h5 (schema v2), with the ⚠ backfill note; FFT + compliance stay no-backfill (they read existing .h5 samples). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzfmanually merged due to document conflict
Pull request closed