Merge feat/sensor-check-h5 into dev
Sensor self-check standardized into the .h5 (schema v2, /sensor_check group), decoded for both series-3 and series-4, plus the Thor backfill script. CHANGELOG resolved per the convention adopted today: the incoming Unreleased preamble was dropped rather than reconciled — no preamble under Unreleased, the theme gets written at release time — and its load-bearing half was folded into ### Migration, which said "None" and is now false. That block now states the real cost: .h5 schema v1 -> v2, TOOL_VERSION 0.31.0 so the standard backfill picks the traces up with no --force, and ~2 h on the NAS. The FFT, the compliance chart and the ach_server rescue flags still owe nothing. The branch's rewritten "Sensor self-check — both series" entry merged cleanly and is kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
This commit is contained in:
+34
-17
@@ -46,18 +46,25 @@ All notable changes to seismo-relay are documented here.
|
||||
reference PDF. A technical breakdown of the curve is in
|
||||
`docs/ri8507_compliance_curve.md`.
|
||||
|
||||
- **Sensor self-check waveforms decoded and drawn (`minimateplus.sensor_check`).**
|
||||
The "Sensor Check" traces Blastware shows to the right of the waveform panel
|
||||
live in the series-3 binary's trailing block as four length-prefixed records
|
||||
(`0x3c`–`0x3f`) using the same delta-block codec as the main waveform:
|
||||
Tran/Vert/Long geophone ring-downs (the transducer's damped impulse response —
|
||||
resonant frequency + overswing/damping) and a MicL pulse train (the mic's
|
||||
known-signal gain check). `gather_report_data` decodes them from the retained
|
||||
BW binary at report time; the report renders them as a strip flush against the
|
||||
waveform panel plus the **Sensor Check → Frequency / Overswing Ratio** sub-rows
|
||||
in the stats table. Verified against the reports on all 7 oracle events (mic
|
||||
zero-crossing frequency = 20.1 Hz exact; geophone ring-downs consistent
|
||||
~7.5 Hz with overswing ~3.5). Tests in `tests/test_sensor_check.py`.
|
||||
- **Sensor self-check waveforms decoded and drawn — both series.** The little
|
||||
"Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
|
||||
response — plus a MicL pulse train, the mic's known-signal gain check) are the
|
||||
unit's proof its sensors were healthy when it recorded the event.
|
||||
- **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
|
||||
the binary's trailing block, same delta-block codec as the main waveform.
|
||||
Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
|
||||
geophone ring-downs ~7.5 Hz, overswing ~3.5).
|
||||
- **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
|
||||
fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
|
||||
raw int16 traces; three-channel (mic-disabled) units carry only the three
|
||||
geophones. Validated by shape + cross-event consistency.
|
||||
- **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
|
||||
decoder attaches the traces to the event at decode, the writer persists
|
||||
them, and `gather_report_data` reads them back — so SFM renders the strip
|
||||
(flush against the waveform panel) plus the **Sensor Check → Frequency /
|
||||
Overswing Ratio** sub-rows without knowing the source instrument.
|
||||
- Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
|
||||
`tests/test_event_hdf5_sensor_check.py`.
|
||||
|
||||
- **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
|
||||
binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
|
||||
@@ -77,11 +84,21 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
### Migration
|
||||
|
||||
**None.** Every change here is additive and reads from data already on disk —
|
||||
the `.h5` samples and the retained raw BW binary. No `.h5`/DB change, no
|
||||
schema change, no migration, no backfill, and **no `TOOL_VERSION` bump**: a
|
||||
report regenerated for an existing event simply gains the new panels, and the
|
||||
`ach_server` rescue flags don't touch the codec.
|
||||
⚠ **The sensor-check needs a backfill.** Existing `.h5` files are schema v1
|
||||
and carry no `/sensor_check` group, so their reports show no sensor-check strip
|
||||
until regenerated. `TOOL_VERSION` is bumped to **0.31.0**, so the standard
|
||||
backfill regenerates every event and picks up the traces with **no `--force`**:
|
||||
`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
|
||||
backfill — this rides along) and the series-3 sidecar/shape backfill for
|
||||
MiniMate events. Purely additive — no decoded value changes, and v1 `.h5`
|
||||
files read fine until then (empty strip). DB backup first, as always.
|
||||
|
||||
⚠ Budget **~2 h on the NAS** — ~1.5 files/sec there versus ~85/sec on the dev
|
||||
box (gzip-4 in `sfm/event_hdf5.py` against a Synology CPU).
|
||||
|
||||
Everything else in this release owes nothing: the FFT, the USBM compliance
|
||||
chart and the `ach_server` rescue flags are additive and read data already on
|
||||
disk — no schema change, no DB migration.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
r"""Decode the Thor / Micromate (series-4) sensor self-check waveforms from an
|
||||
IDFW event binary.
|
||||
|
||||
Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events. The IDFW
|
||||
binary carries the sensor self-check in its fixed-header region (before the
|
||||
waveform body), as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
|
||||
channel ids as the series-3 MiniMate Plus (Tran / Vert / Long / MicL), which is
|
||||
the physical self-test:
|
||||
|
||||
* 3c / 3d / 3e = Tran / Vert / Long geophone ring-downs (a damped impulse
|
||||
response — resonant frequency + damping).
|
||||
* 3f = MicL pulse train (the mic's known-signal gain check). Absent
|
||||
on three-channel (mic-disabled) units.
|
||||
|
||||
Record framing (per record)::
|
||||
|
||||
01 0e [id:1] [flags:3] [count:2 BE] [pad:10] [int16-BE samples × count]
|
||||
\___ 18-byte header ___/
|
||||
|
||||
Unlike series-3's delta-coded trailing block, series-4 stores each trace as a
|
||||
raw int16 big-endian array. ``count`` (the 2-byte field at header offset +8)
|
||||
is the sample count; the record is padded to a fixed stride after that.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
from typing import Dict, List
|
||||
|
||||
# Record id → channel. Same ids/order as series-3 (minimateplus.sensor_check).
|
||||
_ID_TO_CHANNEL = {0x3C: "Tran", 0x3D: "Vert", 0x3E: "Long", 0x3F: "MicL"}
|
||||
_CHAIN_IDS = (0x3C, 0x3D, 0x3E, 0x3F)
|
||||
|
||||
_MARKER = b"\x01\x0e" # precedes the 1-byte channel id
|
||||
_HEADER_LEN = 18 # bytes from the marker start to the first sample
|
||||
_COUNT_OFF = 8 # 2-byte BE sample count, from the marker start
|
||||
_MAX_COUNT = 4000 # sanity cap (traces are ~70-200 samples)
|
||||
|
||||
|
||||
def _find_chain(raw: bytes):
|
||||
"""Locate the sensor-check record chain. Returns a list of
|
||||
``(offset, id, count)`` for the first run of markers whose ids run
|
||||
3c, 3d, 3e[, 3f] in order, or ``[]``.
|
||||
|
||||
Records are padded to a fixed stride, so the next marker is not at
|
||||
``header + count*2``; instead collect every ``01 0e [id]`` marker with a
|
||||
sane count and take the first id-ordered run. Validating the id sequence
|
||||
(not a lone ``01 0e 3c``) keeps a stray marker in the waveform body from
|
||||
matching — the real chain sits in the fixed header, ahead of the body.
|
||||
"""
|
||||
n = len(raw)
|
||||
markers = []
|
||||
for p in range(n - _HEADER_LEN):
|
||||
if raw[p:p + 2] == _MARKER and raw[p + 2] in _ID_TO_CHANNEL:
|
||||
count = int.from_bytes(raw[p + _COUNT_OFF:p + _COUNT_OFF + 2], "big")
|
||||
if 0 < count <= _MAX_COUNT:
|
||||
markers.append((p, raw[p + 2], count))
|
||||
|
||||
for i, (off, rid, _c) in enumerate(markers):
|
||||
if rid != 0x3C:
|
||||
continue
|
||||
run = [markers[i]]
|
||||
for m in markers[i + 1:]:
|
||||
if len(run) < len(_CHAIN_IDS) and m[1] == _CHAIN_IDS[len(run)]:
|
||||
run.append(m)
|
||||
else:
|
||||
break
|
||||
if len(run) >= 3: # 3-channel (mic-disabled) units are valid
|
||||
return run
|
||||
return []
|
||||
|
||||
|
||||
def decode_idf_sensor_check(raw: bytes) -> Dict[str, List[int]]:
|
||||
"""Decode the sensor self-check traces from a Thor/Micromate IDFW binary.
|
||||
|
||||
Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}`` in
|
||||
raw int16 ADC counts (MicL omitted on 3-channel units), or ``{}`` if the
|
||||
binary carries no sensor-check chain (a non-IDF file, or an IDFH histogram).
|
||||
"""
|
||||
chain = _find_chain(raw)
|
||||
if not chain:
|
||||
return {}
|
||||
out: Dict[str, List[int]] = {}
|
||||
for off, rid, count in chain:
|
||||
start = off + _HEADER_LEN
|
||||
blob = raw[start:start + count * 2]
|
||||
if len(blob) < count * 2:
|
||||
continue
|
||||
out[_ID_TO_CHANNEL[rid]] = list(struct.unpack(">%dh" % count, blob))
|
||||
return out
|
||||
@@ -50,7 +50,7 @@ SIDECAR_KIND = "sfm.event"
|
||||
# bumped without a `pip install` re-run — leading to confusing stale
|
||||
# version stamps in sidecars. Bump this constant and CHANGELOG.md
|
||||
# together at release time.
|
||||
TOOL_VERSION = "0.30.0"
|
||||
TOOL_VERSION = "0.31.0" # +/sensor_check group (schema v2); gates the backfill regen
|
||||
|
||||
try:
|
||||
# Best-effort: prefer the installed metadata when it's NEWER than the
|
||||
@@ -960,6 +960,11 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
project=project, client=client, operator=user, sensor_location=seisloc,
|
||||
)
|
||||
ev.raw_samples = samples
|
||||
# Sensor self-check traces from the binary's trailing block (waveform
|
||||
# events only; returns {} for histograms / when absent). Carried on the
|
||||
# Event so the .h5 writer persists them device-agnostically.
|
||||
from minimateplus.sensor_check import decode_sensor_check
|
||||
ev.sensor_check = decode_sensor_check(raw) or None
|
||||
# Only compute peaks from samples when we actually have samples.
|
||||
# For events the codec couldn't decode (histogram-mode bodies, until
|
||||
# the §7.6.2 histogram codec is wired in), samples is an empty dict
|
||||
|
||||
@@ -544,6 +544,15 @@ class Event:
|
||||
pretrig_samples: Optional[int] = None # from STRT record: pre-trigger sample count
|
||||
rectime_seconds: Optional[int] = None # from STRT record: record duration (seconds)
|
||||
|
||||
# Sensor self-check traces keyed by channel label — the short diagnostic
|
||||
# waveforms the unit records when it pulses each sensor before monitoring
|
||||
# (geophone ring-downs + a mic pulse train). Decoded from the binary by
|
||||
# the per-series decoder (minimateplus.sensor_check / micromate.sensor_check)
|
||||
# and carried here so the .h5 writer can persist them device-agnostically.
|
||||
# Raw ADC counts; the source series' scale differs but the trace is a
|
||||
# shape diagnostic (rendered fit-to-box). None when absent.
|
||||
sensor_check: Optional[dict] = None # {"Tran": [...], ..., "MicL": [...]}
|
||||
|
||||
# ── Debug / introspection ─────────────────────────────────────────────────
|
||||
# Raw 210-byte waveform record bytes, set when debug mode is active.
|
||||
# Exposed by the SFM server via ?debug=true so field layouts can be verified.
|
||||
|
||||
@@ -305,6 +305,11 @@ def main(argv=None) -> int:
|
||||
default=0,
|
||||
)
|
||||
ev.total_samples = ev.total_samples or n_samp
|
||||
# Sensor self-check traces from the IDFW fixed
|
||||
# header, so regenerated .h5 files gain the v2
|
||||
# /sensor_check group (mirrors save_imported_idf).
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
|
||||
|
||||
event_hdf5.write_event_hdf5(
|
||||
hdf5_path, ev,
|
||||
|
||||
+44
-4
@@ -12,8 +12,11 @@ Layout written to `<filename>.h5`:
|
||||
├─ samples_int16/ (optional)
|
||||
│ ├─ Tran (int16, raw ADC counts) shape: (N,)
|
||||
│ └─ ... per channel (only when present in the source)
|
||||
├─ sensor_check/ (optional, schema v2+)
|
||||
│ ├─ Tran (int32, raw counts) shape: (M,) M ≪ N
|
||||
│ └─ ... per channel present in the source (MicL absent on 3-channel units)
|
||||
└─ root attrs (event metadata):
|
||||
schema_version int = 1
|
||||
schema_version int = 2
|
||||
kind str = "sfm.event.hdf5"
|
||||
serial str
|
||||
waveform_key str (8-hex)
|
||||
@@ -64,7 +67,7 @@ from minimateplus.models import Event
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
SCHEMA_VERSION = 2 # v2 adds the optional /sensor_check group
|
||||
HDF5_KIND = "sfm.event.hdf5"
|
||||
|
||||
# Geophone full-scale velocity per range (in/s). Confirmed in CLAUDE.md
|
||||
@@ -270,6 +273,22 @@ def write_event_hdf5(
|
||||
)
|
||||
igrp.attrs["mic_psi_per_count"] = float(mic_factor)
|
||||
|
||||
# /sensor_check — optional short diagnostic self-check traces (schema
|
||||
# v2+). Raw ADC counts (a shape diagnostic; the per-series count scale
|
||||
# differs, and the renderer fits each trace to its box). Only channels
|
||||
# the decoder found are written — 3-channel units carry no MicL.
|
||||
sc = event.sensor_check or {}
|
||||
if sc:
|
||||
scgrp = f.create_group("sensor_check")
|
||||
for ch in ("Tran", "Vert", "Long", "MicL"):
|
||||
vals = sc.get(ch)
|
||||
if vals:
|
||||
scgrp.create_dataset(
|
||||
ch, data=np.asarray(vals, dtype=np.int32),
|
||||
compression="gzip", compression_opts=4, shuffle=True,
|
||||
)
|
||||
scgrp.attrs["units"] = "raw_counts"
|
||||
|
||||
import os
|
||||
os.replace(tmp, path)
|
||||
|
||||
@@ -334,6 +353,16 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
|
||||
if mic_attr is not None:
|
||||
mic_psi = float(mic_attr)
|
||||
|
||||
# /sensor_check — optional (schema v2+); absent on older files.
|
||||
sensor_check = None
|
||||
scgrp = f.get("sensor_check")
|
||||
if scgrp is not None:
|
||||
sensor_check = {}
|
||||
for ch in ("Tran", "Vert", "Long", "MicL"):
|
||||
ds = scgrp.get(ch)
|
||||
if ds is not None:
|
||||
sensor_check[ch] = np.asarray(ds[()])
|
||||
|
||||
return {
|
||||
"schema_version": sv,
|
||||
"kind": attrs.get("kind"),
|
||||
@@ -341,6 +370,7 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
|
||||
"samples": samples,
|
||||
"samples_int16": samples_int16,
|
||||
"mic_psi_per_count": mic_psi,
|
||||
"sensor_check": sensor_check,
|
||||
}
|
||||
|
||||
|
||||
@@ -431,11 +461,16 @@ def plot_json_from_hdf5(
|
||||
event_id: Optional[str] = None,
|
||||
index: Optional[int] = None,
|
||||
) -> dict:
|
||||
"""Build a `sfm.plot.v1` JSON dict from a stored .h5 file."""
|
||||
"""Build a `sfm.plot.v1` JSON dict from a stored .h5 file.
|
||||
|
||||
The dict also carries a top-level ``sensor_check`` key (the raw self-check
|
||||
traces as ``{ch: [int]}``, or None) beyond the plot schema, so report
|
||||
generation can read the traces from the same single .h5 load.
|
||||
"""
|
||||
data = read_event_hdf5(path)
|
||||
a = data["attrs"]
|
||||
s = data["samples"]
|
||||
return _build_plot_dict(
|
||||
out = _build_plot_dict(
|
||||
n_samples=len(s["Tran"]) if "Tran" in s else 0,
|
||||
sample_rate=int(a.get("sample_rate", 1024) or 1024),
|
||||
pretrig_samples=int(a.get("pretrig_samples", 0) or 0),
|
||||
@@ -463,6 +498,11 @@ def plot_json_from_hdf5(
|
||||
event_id=event_id,
|
||||
index=index,
|
||||
)
|
||||
scd = data.get("sensor_check")
|
||||
out["sensor_check"] = (
|
||||
{ch: v.tolist() for ch, v in scd.items()} if scd else None
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _build_plot_dict(
|
||||
|
||||
+11
-16
@@ -121,9 +121,11 @@ class ReportData:
|
||||
t0_ms: Optional[float] = None
|
||||
dt_ms: Optional[float] = None
|
||||
|
||||
# Sensor self-check traces — {ch: [samples]} in raw decode units, decoded
|
||||
# from the binary's trailing block (see minimateplus.sensor_check). The
|
||||
# little waveforms BW draws in its "Sensor Check" strip. Empty when absent.
|
||||
# Sensor self-check traces — {ch: [samples]} in raw counts, read from the
|
||||
# standardized .h5 (/sensor_check group, schema v2+) where the per-series
|
||||
# decoder stored them at ingest. The little diagnostic waveforms BW draws
|
||||
# in its "Sensor Check" strip. Empty when absent (pre-v2 .h5, histogram,
|
||||
# or 3-channel unit's MicL).
|
||||
sensor_check_waveforms: dict = field(default_factory=dict)
|
||||
|
||||
# Record-type discriminator
|
||||
@@ -294,22 +296,15 @@ def gather_report_data(
|
||||
rd.pretrig_samples = ta.get("pretrig_samples")
|
||||
rd.t0_ms = ta.get("t0_ms")
|
||||
rd.dt_ms = ta.get("dt_ms")
|
||||
# Sensor self-check traces — read from the standardized .h5 (schema
|
||||
# v2+). Device-agnostic: whichever decoder produced the event
|
||||
# stored them at ingest, so SFM reads them here without knowing or
|
||||
# caring about the source instrument series. Empty on pre-v2 files
|
||||
# (until backfilled) and on 3-channel / histogram events.
|
||||
rd.sensor_check_waveforms = wf.get("sensor_check") or {}
|
||||
except Exception as exc:
|
||||
log.warning("gather_report_data: hdf5 read failed: %s", exc)
|
||||
|
||||
# ── Sensor self-check traces — decoded from the retained raw binary ──
|
||||
# The .h5 holds only the main waveform; the sensor-check traces live in the
|
||||
# binary's trailing block, so decode them straight from the kept BW file.
|
||||
# Waveform events only (histograms have no sensor-check strip).
|
||||
if not rd.is_histogram:
|
||||
try:
|
||||
from minimateplus.sensor_check import decode_sensor_check
|
||||
bw_path, _a5 = store.paths_for(serial, filename)
|
||||
if bw_path.exists():
|
||||
rd.sensor_check_waveforms = decode_sensor_check(bw_path.read_bytes())
|
||||
except Exception as exc:
|
||||
log.warning("gather_report_data: sensor-check decode failed: %s", exc)
|
||||
|
||||
# ── Histogram aggregation ──
|
||||
# Codec emits ~N per-block samples (typically 1/sec); BW reports
|
||||
# one bar per configured interval (1 min / 5 min / etc.). When
|
||||
|
||||
@@ -662,6 +662,11 @@ class WaveformStore:
|
||||
ev.raw_samples = idf_samples
|
||||
n_samples = max((len(idf_samples.get(ch, [])) for ch in ("Tran", "Vert", "Long", "MicL")), default=0)
|
||||
ev.total_samples = ev.total_samples or n_samples
|
||||
# Sensor self-check traces from the IDFW fixed header (waveform
|
||||
# events only; {} on histograms / when absent). Carried on the
|
||||
# bridged Event so the .h5 writer persists them like series-3.
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
ev.sensor_check = decode_idf_sensor_check(idf_bytes) or None
|
||||
|
||||
# For IDFH histograms there are no per-sample waveform arrays — the
|
||||
# device stores one peak ADC count per interval per channel. Synthesise
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,71 @@
|
||||
"""The event .h5 carries the sensor self-check traces (schema v2).
|
||||
|
||||
The sensor check is decoded by the per-series decoder and attached to the
|
||||
standardized Event, so the .h5 writer persists it device-agnostically and SFM
|
||||
reads it back without knowing which instrument produced it. Old v1 files (no
|
||||
sensor_check group) must still read cleanly.
|
||||
"""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from minimateplus.models import Event
|
||||
from minimateplus.event_file_io import read_blastware_file
|
||||
from sfm import event_hdf5
|
||||
|
||||
S3_FIX = Path(__file__).parent / "fixtures" / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"
|
||||
|
||||
|
||||
def _write(ev, **kw):
|
||||
d = Path(tempfile.mkdtemp())
|
||||
p = d / "e.h5"
|
||||
event_hdf5.write_event_hdf5(p, ev, serial="BE12844", **kw)
|
||||
return p
|
||||
|
||||
|
||||
def test_sensor_check_roundtrips_through_hdf5():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, -3], "Vert": [0, 1], "Long": [2], "MicL": [5, -5]}
|
||||
ev.sample_rate = 1024
|
||||
sc = {"Tran": [0, -990, -500, -100], "Vert": [0, -980, -480],
|
||||
"Long": [0, -986, -470], "MicL": [0, -1800, 1800, -1800]}
|
||||
ev.sensor_check = sc
|
||||
|
||||
r = event_hdf5.read_event_hdf5(_write(ev))
|
||||
assert r["schema_version"] == 2
|
||||
assert set(r["sensor_check"]) == {"Tran", "Vert", "Long", "MicL"}
|
||||
for ch, vals in sc.items():
|
||||
assert r["sensor_check"][ch].tolist() == vals
|
||||
|
||||
|
||||
def test_plot_json_carries_sensor_check():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, 3]}
|
||||
ev.sample_rate = 1024
|
||||
ev.sensor_check = {"Tran": [0, -990, -500], "Vert": [0, -980],
|
||||
"Long": [0, -986]} # 3-channel: no MicL
|
||||
pj = event_hdf5.plot_json_from_hdf5(_write(ev))
|
||||
assert pj["sensor_check"] is not None
|
||||
assert "MicL" not in pj["sensor_check"]
|
||||
assert pj["sensor_check"]["Tran"] == [0, -990, -500]
|
||||
|
||||
|
||||
def test_event_without_sensor_check_still_reads_as_v2():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, 3]}
|
||||
ev.sample_rate = 1024
|
||||
r = event_hdf5.read_event_hdf5(_write(ev))
|
||||
assert r["schema_version"] == 2
|
||||
assert r["sensor_check"] is None
|
||||
assert event_hdf5.plot_json_from_hdf5(_write(ev))["sensor_check"] is None
|
||||
|
||||
|
||||
def test_series3_decode_populates_event_sensor_check():
|
||||
# The real series-3 decoder attaches the traces to the Event, so the
|
||||
# ingest/backfill .h5 write picks them up with no extra plumbing.
|
||||
ev = read_blastware_file(S3_FIX)
|
||||
assert ev.sensor_check is not None
|
||||
assert set(ev.sensor_check) == {"Tran", "Vert", "Long", "MicL"}
|
||||
tran = np.asarray(ev.sensor_check["Tran"], dtype=float)
|
||||
assert tran.min() < -800 # the geophone ring-down deflection
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Series-4 (Thor / Micromate IDFW) sensor self-check waveform decode.
|
||||
|
||||
Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events. The IDFW
|
||||
binary carries the sensor self-check in its fixed-header region (before the
|
||||
waveform body) as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
|
||||
channel ids as series-3 (Tran/Vert/Long/MicL). Unlike series-3's delta-coded
|
||||
trailing block, series-4 stores each trace as a raw int16-BE array after an
|
||||
18-byte record header whose sample count is a 2-byte field at offset +8.
|
||||
|
||||
Three-channel (mic-disabled) Thor units carry only 3c/3d/3e — no MicL record.
|
||||
|
||||
Validated by shape (geophone ring-down / mic pulse train) and cross-event
|
||||
consistency, since there's no Thor Event-Report strip to exact-match against.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
|
||||
FIXDIR = Path(__file__).parent / "fixtures" / "thor-idf-sc"
|
||||
EVENTS = sorted(p.name for p in FIXDIR.glob("*.IDFW"))
|
||||
|
||||
|
||||
def _decode(name):
|
||||
return decode_idf_sensor_check((FIXDIR / name).read_bytes())
|
||||
|
||||
|
||||
def test_geo_channels_present_and_ringdown_shaped():
|
||||
# Every IDFW event has the three geophone self-checks; each is a large
|
||||
# one-sided deflection (~15000 raw counts) that rings back — the geophone's
|
||||
# damped impulse response.
|
||||
for name in EVENTS:
|
||||
sc = _decode(name)
|
||||
for ch in ("Tran", "Vert", "Long"):
|
||||
assert ch in sc, f"{name} missing {ch}"
|
||||
tr = np.asarray(sc[ch], dtype=float)
|
||||
tr = tr - tr[:4].mean() # reference to the pre-trigger baseline
|
||||
assert 40 <= len(tr) <= 300, f"{name}:{ch} n={len(tr)}"
|
||||
assert tr.min() < -8000, f"{name}:{ch} min {tr.min()}"
|
||||
# deflects one way and rings back toward / past the baseline
|
||||
assert tr.max() < abs(tr.min()), f"{name}:{ch} not one-sided"
|
||||
|
||||
|
||||
def test_mic_present_only_on_four_channel_units():
|
||||
# UM11719 / UM12947 record a mic; UM13981 / UM20147 are 3-channel
|
||||
# (mic-disabled) units and carry no MicL self-check.
|
||||
got = {name: ("MicL" in _decode(name)) for name in EVENTS}
|
||||
assert any(got.values()), "expected at least one 4-channel unit"
|
||||
assert not all(got.values()), "expected at least one 3-channel unit"
|
||||
for name, has_mic in got.items():
|
||||
if has_mic:
|
||||
tr = np.asarray(_decode(name)["MicL"], dtype=float)
|
||||
tr = tr - tr[:4].mean()
|
||||
# mic self-check is a bipolar pulse train — swings both ways, wide range
|
||||
assert tr.max() > 5000 and tr.min() < -5000, f"{name} mic not bipolar"
|
||||
|
||||
|
||||
def test_channel_ids_and_order():
|
||||
# ids decode to the canonical channel names, geo always in Tran/Vert/Long order
|
||||
sc = _decode(EVENTS[0])
|
||||
assert [c for c in ("Tran", "Vert", "Long") if c in sc] == ["Tran", "Vert", "Long"]
|
||||
|
||||
|
||||
def test_returns_empty_on_non_idf_input():
|
||||
assert decode_idf_sensor_check(b"not an IDF file") == {}
|
||||
assert decode_idf_sensor_check(b"") == {}
|
||||
Reference in New Issue
Block a user