Compare commits
25
Commits
+121
@@ -4,6 +4,127 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
---
|
||||
|
||||
## v0.31.0 — 2026-09-18
|
||||
|
||||
**Report parity, and a second way to rescue a runaway unit.** Two threads.
|
||||
|
||||
The first closes out Blastware Event/FFT-Report parity: the FFT, the USBM
|
||||
RI8507 compliance chart and the sensor self-check now render on the event
|
||||
report, reverse-engineered against BE12844 (MiniMate Plus) and UM (Thor)
|
||||
events. The sensor check is decoded for **both** series and standardized into
|
||||
the `.h5` (schema **v2**, a new `/sensor_check` group), so SFM serves it
|
||||
device-agnostically rather than decoding at report time. The Inspector — an
|
||||
annotated hex reader for series-3 binaries — is what made the trailing-block
|
||||
structure findable, and it earned its keep by *ruling out* a stored FFT block
|
||||
and proving Blastware computes it from the samples.
|
||||
|
||||
The second came out of a field emergency. BE12599's connector fault drove its
|
||||
Tran channel to its trigger level, so the unit recorded back-to-back and dialed
|
||||
the office ACH server every ~75 s, unreachable the whole time.
|
||||
`bridges/ach_server.py` gained `--stop-monitoring` / `--disable-ach` /
|
||||
`--rescue`, which **invert** the recovery: instead of racing a Stop into the
|
||||
gaps between dial-outs, point the modem's Destination at our own ACH server and
|
||||
answer the call. Proven in production the same night — the stop landed on the
|
||||
first call-in and held. See `docs/runbooks/wedged_unit_recovery.md`.
|
||||
|
||||
⚠ **This release owes prod a backfill** — see Migration below.
|
||||
|
||||
### Added
|
||||
- **Rescue-on-connect for `bridges/ach_server.py`** — `--stop-monitoring`
|
||||
(SUB 0x97), `--disable-ach` (SUB 0x2C read → 0x7E write → 0x7F confirm) and
|
||||
`--rescue` (both). They fire immediately after the startup handshake and
|
||||
**before** the event walk, so a unit that is recording back-to-back on a
|
||||
stuck-triggered geophone is quieted as early in the session as possible.
|
||||
Each action is independently guarded — a failure does not abort the download
|
||||
— and the outcome is written to `rescue.json` in the session directory.
|
||||
|
||||
This inverts the `docs/runbooks/wedged_unit_recovery.md` approach. That
|
||||
runbook reaches the unit *inbound* and clears the modem's Destination Address
|
||||
to stop it dialing. When the device is instead wedged mid-modem-init — ALEOS
|
||||
logs `tcpmode trying to send to invalid socket` and re-runs `Initialize Auto
|
||||
answer` every ~75 s, orphaning any held inbound session — inbound cannot win.
|
||||
Pointing the modem's Destination at an `ach_server` and letting the unit call
|
||||
*us* gives a device-initiated session the modem bridges properly.
|
||||
|
||||
⚠ Prefer `--stop-monitoring` alone on first contact. `--disable-ach` stops
|
||||
the unit calling, which is the only channel to a unit in this state; stopping
|
||||
the recording ends the call-home loop on its own when ACH is
|
||||
"after event recorded".
|
||||
|
||||
- **Blastware-compatible channel FFT (`waveform_fft`).** Reproduces Blastware's
|
||||
FFT Report: DC-removed, no window, zero-padded to 4096 (0.25 Hz bins at
|
||||
1024 sps), single-sided `2/N` amplitude. Matches Blastware's dominant
|
||||
frequency to the exact bin and the amplitude to report precision across all
|
||||
28 channels of the 7-event BE12844 oracle set. `channel_spectrum()` /
|
||||
`dominant_frequency()`; tests in `tests/test_waveform_fft.py`.
|
||||
|
||||
- **USBM RI8507 / OSMRE compliance chart on the event-report PDF
|
||||
(`sfm/compliance.py`).** The velocity-vs-frequency blasting-compliance
|
||||
scatter Blastware draws in the upper-right of its Event Report: each channel's
|
||||
significant cycles as `(frequency, peak velocity)` points (zero-crossing
|
||||
method, so each channel's cloud tops out at its PPV) plotted against the
|
||||
RI8507 Drywall (0.75 in/s) and plaster (0.50 in/s) limit curves, drawn
|
||||
continuous (constant-displacement bounds meeting the plateaus — no vertical
|
||||
steps). Sized and positioned to match a Blastware report, measured off the
|
||||
reference PDF. A technical breakdown of the curve is in
|
||||
`docs/ri8507_compliance_curve.md`.
|
||||
|
||||
- **Sensor self-check waveforms decoded and drawn — both series.** The little
|
||||
"Sensor Check" traces (geophone ring-downs — the transducer's damped impulse
|
||||
response — plus a MicL pulse train, the mic's known-signal gain check) are the
|
||||
unit's proof its sensors were healthy when it recorded the event.
|
||||
- **Series-3** (`minimateplus.sensor_check`): four records (`0x3c`–`0x3f`) in
|
||||
the binary's trailing block, same delta-block codec as the main waveform.
|
||||
Verified against all 7 BE12844 reports (mic zero-crossing = 20.1 Hz exact;
|
||||
geophone ring-downs ~7.5 Hz, overswing ~3.5).
|
||||
- **Series-4** (`micromate.sensor_check`): the same self-test in the Thor IDFW
|
||||
fixed header — four `01 0e 3c/3d/3e/3f` records (same channel ids) storing
|
||||
raw int16 traces; three-channel (mic-disabled) units carry only the three
|
||||
geophones. Validated by shape + cross-event consistency.
|
||||
- **Standardized into the `.h5`** (`/sensor_check`, schema v2): each series'
|
||||
decoder attaches the traces to the event at decode, the writer persists
|
||||
them, and `gather_report_data` reads them back — so SFM renders the strip
|
||||
(flush against the waveform panel) plus the **Sensor Check → Frequency /
|
||||
Overswing Ratio** sub-rows without knowing the source instrument.
|
||||
- Tests: `tests/test_sensor_check.py`, `tests/test_sensor_check_idf.py`,
|
||||
`tests/test_event_hdf5_sensor_check.py`.
|
||||
|
||||
- **Inspector tab in `seismo_lab.py` — annotated hex reader for series-3
|
||||
binaries (`minimateplus/binary_annotate.py`).** Tiles a raw Blastware file
|
||||
into labeled spans (header / STRT / body record-chain / trailing metadata +
|
||||
calibration + sensor-check records / footer) so a binary can be combed by eye.
|
||||
|
||||
### Fixed
|
||||
- **Event-report waveform panel — stacked-lane y-tick collision.** The lanes
|
||||
touch, so each lane's bottom `-1.0` overprinted the next lane's top `1.0` at
|
||||
the shared boundary. Prune the extreme ticks so each lane shows clean interior
|
||||
ticks only.
|
||||
- **Event-report header — serial+firmware line ran off the page.** The long
|
||||
`BE##### V ##.##-#.## MiniMate Plus` string overflowed the right margin;
|
||||
tighter right-column indent + BW's slightly smaller header size so it fits.
|
||||
|
||||
---
|
||||
|
||||
### Migration
|
||||
|
||||
⚠ **The sensor-check needs a backfill.** Existing `.h5` files are schema v1
|
||||
and carry no `/sensor_check` group, so their reports show no sensor-check strip
|
||||
until regenerated. `TOOL_VERSION` is bumped to **0.31.0**, so the standard
|
||||
backfill regenerates every event and picks up the traces with **no `--force`**:
|
||||
`scripts/backfill_thor_events.py` for series-4 (it already owed a v0.30.0 Thor
|
||||
backfill — this rides along) and the series-3 sidecar/shape backfill for
|
||||
MiniMate events. Purely additive — no decoded value changes, and v1 `.h5`
|
||||
files read fine until then (empty strip). DB backup first, as always.
|
||||
|
||||
⚠ Budget **~2 h on the NAS** — ~1.5 files/sec there versus ~85/sec on the dev
|
||||
box (gzip-4 in `sfm/event_hdf5.py` against a Synology CPU).
|
||||
|
||||
Everything else in this release owes nothing: the FFT, the USBM compliance
|
||||
chart and the `ach_server` rescue flags are additive and read data already on
|
||||
disk — no schema change, no DB migration.
|
||||
|
||||
---
|
||||
|
||||
## v0.30.0 — 2026-09-12
|
||||
|
||||
**The series-4 correctness release** — the Thor / Micromate counterpart to
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
|
||||
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.30.0**.
|
||||
(Sierra Wireless RV50 / RV55). Current version: **v0.31.0**.
|
||||
|
||||
Stack-level context — which repo owns what, and how the three project versions
|
||||
pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
|
||||
@@ -61,6 +61,24 @@ Read this first when picking the project back up.
|
||||
4th-decimal tick and are Thor's own rounding — no single linear LSB can
|
||||
reproduce every printed value (the constraints are infeasible by 7e-5
|
||||
relative), so do NOT retune `_GEO_LSB_IPS`.
|
||||
- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
|
||||
`parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
|
||||
discarding the key's page byte. An event starting at `0x0111F2A2` and ending
|
||||
at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
|
||||
start. The chunk loop then exits before fetching anything and TERM computes
|
||||
a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
|
||||
`/device/events` walk 500s. Reproduced on BE12599 (2026-09-19), which had
|
||||
78 KB stored and had rolled into page `0x12`.
|
||||
**Why it hid so long:** every 5A capture the walk was verified against came
|
||||
from a freshly-erased BE11529 — all three confirmed TERM examples in
|
||||
`framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`. Prod is
|
||||
unaffected: it ingests complete files via BW ACH, never this walk.
|
||||
**Fixing it has two layers** — the arithmetic (`if end < start: end +=
|
||||
0x10000`) stops the crash and bounds the loop correctly; carrying the page
|
||||
byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
|
||||
over) needs a BW capture of a spanning event first. Do not ship layer one
|
||||
alone without a loud truncation warning — a silently short event is the
|
||||
failure mode this codec has been bitten by repeatedly.
|
||||
- **Open, not blocking:** 14 sensitive-range files show an exact 8x
|
||||
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
|
||||
inserts DB rows for store files that have none (one-time per store) and the
|
||||
@@ -89,6 +107,47 @@ When new information about the protocol is discovered, please update the instant
|
||||
|
||||
---
|
||||
|
||||
## Changelog & release convention
|
||||
|
||||
**Feature branches do NOT touch `CHANGELOG.md`. Write the entry on `dev`, as
|
||||
part of finishing the merge, under `## Unreleased`. Cut the version on `dev` in a
|
||||
dedicated release commit when you are ready to ship to `main`.**
|
||||
|
||||
- **The changelog is written on `dev`, never on a feature branch.** With
|
||||
several branches in flight they all edit the same few lines at the top of
|
||||
the file and conflict every time. Writing it once, after the merge, also
|
||||
lets it describe what actually *landed* — including anything that changed
|
||||
during conflict resolution.
|
||||
- ⚠ **The merge is not finished until `## Unreleased` is updated.** Same sitting,
|
||||
not "later" — that is the one failure mode of writing it after the fact.
|
||||
Reconstruct from the branch's own commit messages:
|
||||
`git log --oneline dev..<branch>` before you merge, or
|
||||
`git log --oneline <merge-base>..<branch>` after.
|
||||
- **No preamble under `## Unreleased`** — just the `### Added` / `### Changed` /
|
||||
`### Fixed` lists. The themed opening paragraph gets written at release
|
||||
time, when the whole release is visible and can be named honestly. A theme
|
||||
written when the first item landed is stale by the third.
|
||||
- ⚠ **State the operational consequence** on any entry touching the codec, the
|
||||
waveform store, or the DB — **including when it is "none."** "requires
|
||||
`backfill_sidecars.py` + `backfill_event_shape.py`, ~2 h on the NAS",
|
||||
"`TOOL_VERSION` bumped", "no schema change, no migration". Silence is
|
||||
ambiguous; "none" is information. This repo's changelog is how future-you
|
||||
learns whether a deploy costs two hours.
|
||||
- **Releases are cut on judgement, not on a schedule or a merge.** `Unreleased`
|
||||
is the staging area for whatever is going into the next release; when enough
|
||||
has accumulated to be worth shipping, it gets a number and a date. Nothing
|
||||
about a merge to `dev` triggers a release.
|
||||
- **Cutting a release** is its own `chore(release): vX.Y.Z — <theme>` commit on
|
||||
`dev`, renaming `## Unreleased` → `## vX.Y.Z — YYYY-MM-DD` and touching:
|
||||
`CHANGELOG.md`, `pyproject.toml`, the version line in `CLAUDE.md` and
|
||||
`README.md`, and `minimateplus/event_file_io.py` (`TOOL_VERSION`) **when the
|
||||
codec changed** — that constant gates `.h5` regeneration.
|
||||
- **`main` carries only released versions.** No `## Unreleased` section there;
|
||||
it lands via the `dev` → `main` PR. `main` lagging `dev` by a version is
|
||||
normal.
|
||||
|
||||
---
|
||||
|
||||
## Architecture: three-tier conceptual model
|
||||
|
||||
seismo-relay is a **suite of cooperating components**, not a single app.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# seismo-relay `v0.30.0`
|
||||
# seismo-relay `v0.31.0`
|
||||
|
||||
A ground-up replacement for **Blastware** — Instantel's aging Windows-only
|
||||
software for managing seismographs. Supports both the **MiniMate Plus
|
||||
@@ -496,6 +496,11 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
|
||||
|
||||
## Roadmap (Future)
|
||||
|
||||
> **Where it stands *today*** — an honest per-capability maturity assessment,
|
||||
> what to rely on, known issues, and the gap to a real tool:
|
||||
> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md). This section covers
|
||||
> where it is *going*.
|
||||
|
||||
### Strategic direction — where this is going
|
||||
|
||||
seismo-relay is being built as a **suite of cooperating components**
|
||||
|
||||
@@ -177,6 +177,8 @@ class AchSession:
|
||||
store: "WaveformStore",
|
||||
clear_after_download: bool = False,
|
||||
restart_monitoring: bool = False,
|
||||
rescue_stop_monitoring: bool = False,
|
||||
rescue_disable_ach: bool = False,
|
||||
force_redownload: bool = False,
|
||||
) -> None:
|
||||
self.sock = sock
|
||||
@@ -190,6 +192,9 @@ class AchSession:
|
||||
self.store = store
|
||||
self.clear_after_download = clear_after_download
|
||||
self.restart_monitoring = restart_monitoring
|
||||
# Rescue actions for a runaway unit — fired before the event walk.
|
||||
self.rescue_stop_monitoring = rescue_stop_monitoring
|
||||
self.rescue_disable_ach = rescue_disable_ach
|
||||
# `force_redownload` tells this session to ignore ach_state and
|
||||
# re-download every event currently on the device, regardless of any
|
||||
# (key, timestamp) match. Useful as a manual override when state has
|
||||
@@ -290,6 +295,41 @@ class AchSession:
|
||||
root_logger.addHandler(fh)
|
||||
|
||||
try:
|
||||
# ── Step 1.5: rescue actions ──────────────────────────────────────
|
||||
# Fired BEFORE the event walk so a runaway unit is quieted as early
|
||||
# in the session as possible. A unit whose geophone sits above the
|
||||
# trigger threshold records back-to-back and, with ACH set to "after
|
||||
# event recorded", re-dials every time — saturating its own firmware
|
||||
# so it never services inbound requests. See
|
||||
# docs/runbooks/wedged_unit_recovery.md.
|
||||
#
|
||||
# Each action is independently guarded: a failure here must not
|
||||
# abort the download that follows.
|
||||
if self.rescue_stop_monitoring or self.rescue_disable_ach:
|
||||
rescue: dict = {"peer": self.peer, "ts": ts}
|
||||
|
||||
if self.rescue_stop_monitoring:
|
||||
log.info("Step 1.5: RESCUE — stop monitoring (SUB 0x97)")
|
||||
try:
|
||||
client.stop_monitoring()
|
||||
rescue["stop_monitoring"] = "ok"
|
||||
log.info(" stop monitoring OK — device should stop recording")
|
||||
except Exception as exc:
|
||||
rescue["stop_monitoring"] = f"failed: {exc}"
|
||||
log.error(" stop monitoring FAILED: %s", exc)
|
||||
|
||||
if self.rescue_disable_ach:
|
||||
log.info("Step 1.5: RESCUE — disable auto call home (SUB 0x2C/0x7E/0x7F)")
|
||||
try:
|
||||
client.set_call_home_config(auto_call_home_enabled=False)
|
||||
rescue["disable_ach"] = "ok"
|
||||
log.info(" disable ACH OK — unit should stop calling home")
|
||||
except Exception as exc:
|
||||
rescue["disable_ach"] = f"failed: {exc}"
|
||||
log.error(" disable ACH FAILED: %s", exc)
|
||||
|
||||
_save_json(session_dir / "rescue.json", rescue)
|
||||
|
||||
# ── Step 2: device info ───────────────────────────────────────────
|
||||
device_info = None
|
||||
if not self.events_only:
|
||||
@@ -747,6 +787,13 @@ def serve(args: argparse.Namespace) -> None:
|
||||
print(f" Max events per session: {max_ev if max_ev else 'unlimited'}")
|
||||
print(f" Clear device after download: {'YES' if args.clear_after_download else 'no'}")
|
||||
print(f" Restart monitoring after download: {'YES' if args.restart_monitoring else 'no'}")
|
||||
_stop_mon = args.stop_monitoring or args.rescue
|
||||
_dis_ach = args.disable_ach or args.rescue
|
||||
print(f" RESCUE stop monitoring on connect: {'YES' if _stop_mon else 'no'}")
|
||||
print(f" RESCUE disable auto call home: {'YES' if _dis_ach else 'no'}")
|
||||
if _stop_mon and args.restart_monitoring:
|
||||
print(" !! --restart-monitoring will re-start the unit after download,")
|
||||
print(" undoing --stop-monitoring. Drop one of them.")
|
||||
print(f" Force re-download all (ignore state): {'YES' if args.force_redownload_all else 'no'}")
|
||||
print(f"{'='*60}")
|
||||
print(f"\n Point your test unit's ACEmanager call-home settings to:")
|
||||
@@ -788,6 +835,8 @@ def serve(args: argparse.Namespace) -> None:
|
||||
store=store,
|
||||
clear_after_download=args.clear_after_download,
|
||||
restart_monitoring=args.restart_monitoring,
|
||||
rescue_stop_monitoring=args.stop_monitoring or args.rescue,
|
||||
rescue_disable_ach=args.disable_ach or args.rescue,
|
||||
force_redownload=args.force_redownload_all,
|
||||
)
|
||||
t = threading.Thread(target=session.run, daemon=True, name=f"ach-{peer}")
|
||||
@@ -862,6 +911,32 @@ def parse_args() -> argparse.Namespace:
|
||||
"DCD on disconnect — without this the unit stays idle after a call-home."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--stop-monitoring",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help=(
|
||||
"RESCUE: send SUB 0x97 (stop monitoring) immediately after the "
|
||||
"handshake, before any event download. Use on a unit that is "
|
||||
"recording back-to-back because of a stuck-triggered geophone."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--disable-ach",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help=(
|
||||
"RESCUE: disable Auto Call Home on the device (SUB 0x2C read → "
|
||||
"0x7E write → 0x7F confirm) immediately after the handshake. The "
|
||||
"unit stops dialing out until ACH is explicitly re-enabled."
|
||||
),
|
||||
)
|
||||
p.add_argument(
|
||||
"--rescue",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Shorthand for --stop-monitoring --disable-ach.",
|
||||
)
|
||||
p.add_argument(
|
||||
"--clear-after-download",
|
||||
action="store_true",
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
# USBM RI8507 / OSMRE Blasting Compliance Curve — Reference
|
||||
|
||||
Reference for the **velocity-vs-frequency blasting compliance chart** Blastware
|
||||
draws on its Event Report ("USBM RI8507 And OSMRE"), and how seismo-relay
|
||||
reproduces it. Implemented in [`sfm/compliance.py`](../sfm/compliance.py); the
|
||||
spectral (FFT) side lives in [`waveform_fft.py`](../waveform_fft.py).
|
||||
|
||||
Reverse-engineered 2026-09-14 against 7 BE12844 (MiniMate Plus) events, each
|
||||
with a Blastware Event Report + FFT Report as ground truth. Curve values from
|
||||
USBM RI8507 Appendix B and 30 CFR 816.67.
|
||||
|
||||
---
|
||||
|
||||
## What it is
|
||||
|
||||
Two closely-related sources for the same limit curve:
|
||||
|
||||
- **USBM RI8507** — Bureau of Mines *Report of Investigations 8507* (Siskind
|
||||
et al., 1980), *"Structure Response and Damage Produced by Ground Vibration
|
||||
From Surface Mine Blasting."* The curve is **Figure B-1**, Appendix B
|
||||
("Alternative Blasting Level Criteria"), p.73–74.
|
||||
- **OSMRE / OSM** — the Office of Surface Mining Reclamation and Enforcement
|
||||
codified it as **30 CFR 816.67, Figure 1**. "CFR" = the U.S. Code of Federal
|
||||
Regulations. Same curve, regulatory force.
|
||||
|
||||
The chart plots each geophone channel's significant vibration cycles as
|
||||
`(frequency, peak velocity)` points against this limit. A point **below** the
|
||||
line passes; **above** fails.
|
||||
|
||||
---
|
||||
|
||||
## The limit curve
|
||||
|
||||
A structure has a resonance band (~4–12 Hz for whole structures) where it is
|
||||
most vulnerable, so the safe velocity is **lower** at those frequencies and
|
||||
**higher** away from them. The curve captures this by alternating two kinds of
|
||||
bound:
|
||||
|
||||
- **Constant-velocity** segments — a flat horizontal line at a fixed PPV.
|
||||
- **Constant-displacement** segments — a fixed peak *displacement* `d`. For
|
||||
simple harmonic motion, peak velocity `v = 2πf·d`, so on a velocity-vs-
|
||||
frequency **log-log** plot this is a straight line of slope +1 (velocity rises
|
||||
with frequency). This is why the low- and high-frequency bounds are sloped.
|
||||
|
||||
### Two lines — structure type
|
||||
|
||||
RI8507 gives two lines for two interior-wall constructions (Table 13, p.67):
|
||||
|
||||
| line | construction | plateau PPV |
|
||||
|---|---|---|
|
||||
| **Drywall** (solid) | modern gypsum wallboard | **0.75 in/s** |
|
||||
| **Plaster** (dashed) | older plaster on wood lath | **0.50 in/s** |
|
||||
|
||||
Plaster-on-lath is more damage-prone, hence the lower limit. You apply **one**
|
||||
line depending on the monitored structure.
|
||||
|
||||
### The four segments (Figure B-1, p.74)
|
||||
|
||||
Going low → high frequency, each line is:
|
||||
|
||||
1. **Ultimate low-frequency bound** — constant displacement **0.030 in**
|
||||
(`v = 2πf·0.030`). Only relevant below ~4 Hz.
|
||||
2. **Plateau** — constant velocity **0.75** (Drywall) / **0.50** (plaster) in/s.
|
||||
3. **Rising diagonal** — constant displacement **0.008 in** (`v = 2πf·0.008`),
|
||||
climbing from the plateau up to the high-frequency cap.
|
||||
4. **High-frequency cap** — constant velocity **2.0 in/s** above ~40 Hz.
|
||||
|
||||
The segments are drawn **continuous**: each bound is used over the frequency
|
||||
range where it is the binding (lowest) limit, and consecutive bounds meet where
|
||||
they are equal — so there are no vertical steps. Transition frequencies come
|
||||
straight from the values (`f = V / (2π·d)`):
|
||||
|
||||
| transition | formula | Drywall | Plaster |
|
||||
|---|---|---|---|
|
||||
| 0.030 in → plateau | `V_mid / (2π·0.030)` | 3.98 Hz | 2.65 Hz |
|
||||
| plateau → 0.008 in | `V_mid / (2π·0.008)` | 14.92 Hz | 9.95 Hz |
|
||||
| 0.008 in → 2.0 in/s | `2.0 / (2π·0.008)` | 39.79 Hz | 39.79 Hz |
|
||||
|
||||
Because both lines share the same **0.008 in** rising diagonal, above ~15 Hz
|
||||
they lie on the *same* line (both reach 2.0 in/s at ~40 Hz) — RI8507's literal
|
||||
construction merges them there. Blastware renders the dashed line as a separate
|
||||
parallel diagonal, but that is cosmetic: above ~15 Hz both structure types carry
|
||||
the identical limit, so compliance is unaffected.
|
||||
|
||||
> ⚠ RI8507's *Table 13* is a simpler two-range criterion with a **sharp
|
||||
> discontinuity at 40 Hz** (flat plateau, then a jump to 2.0). Figure B-1 is the
|
||||
> **smoothed** version that adds the 0.008 in transition — that is the one drawn
|
||||
> on reports and implemented here.
|
||||
|
||||
---
|
||||
|
||||
## The compliance scatter (the points)
|
||||
|
||||
The cloud is **not** the FFT spectrum. It is a per-cycle, time-domain measure by
|
||||
the **zero-crossing method** (`channel_compliance_points`):
|
||||
|
||||
- Split the channel's waveform at its zero crossings.
|
||||
- Each half-cycle contributes one point: **frequency** `= 1 / (2 · half-period)`
|
||||
(from the samples between the two crossings), **velocity** `= peak |amplitude|`
|
||||
in that half-cycle.
|
||||
|
||||
This yields ~90–110 points per channel, and — by construction — each channel's
|
||||
**highest** point equals that channel's PPV. Verified against Blastware: the
|
||||
cloud shape, density, and ceiling all match.
|
||||
|
||||
### Why not the FFT?
|
||||
|
||||
A broadband blast spreads its energy across many FFT bins, so no single bin
|
||||
reaches the time-domain peak — the FFT amplitudes come out ~10× below the
|
||||
compliance-chart velocities. The compliance chart is a *per-cycle peak* view;
|
||||
the **FFT** is a separate analysis (Blastware's *FFT Report*), reproduced by
|
||||
[`waveform_fft.py`](../waveform_fft.py) and used for the dominant-frequency
|
||||
readout and the #10 FFT view — not for this scatter.
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
- `sfm/compliance.py`
|
||||
- `limit_at(freq, curve)` — the limit PPV at a frequency (`curve` = `"Drywall"`
|
||||
or `"Plaster"`); curves are data in `_CURVES`, so more standards can be added.
|
||||
- `channel_compliance_points(samples, sps)` — the zero-crossing scatter.
|
||||
- `draw_compliance_chart(ax, channels, sps)` — matplotlib rendering (both
|
||||
limit lines + per-channel scatter, Blastware's tick scales and channel
|
||||
markers: Tran `+` red, Vert `×` green, Long `o` blue).
|
||||
- Tests: `tests/test_compliance.py`.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- USBM **RI8507** (Siskind, Stagg, Kopp, Dowding, 1980), Appendix B / Figure B-1,
|
||||
p.73–74; Table 13, p.67. (`ref-stuff/usbm-ri8507-ground_vibration.pdf`.)
|
||||
- **30 CFR 816.67**, "Use of explosives: Control of adverse effects," Figure 1 —
|
||||
<https://www.ecfr.gov/current/title-30/chapter-VII/subchapter-K/part-816/section-816.67>
|
||||
@@ -1,6 +1,7 @@
|
||||
# Runbook — Recovering a wedged unit stuck in a call-home loop
|
||||
|
||||
**Original incident:** BE9558H at `166.246.130.1:9034`, recovered 2026-05-17.
|
||||
**Incidents:** BE9558H at `166.246.130.1:9034`, 2026-05-17 (Method B) ·
|
||||
BE12599 at `166.246.64.226:9034`, 2026-09-16 (Method A).
|
||||
|
||||
A field unit with a stuck-triggered geophone (or any hardware fault causing
|
||||
constant event triggering) will record events back-to-back, and if Auto Call
|
||||
@@ -14,6 +15,33 @@ This runbook describes how to break the loop and recover control.
|
||||
|
||||
---
|
||||
|
||||
## ⚠ Two cures for one disease — intercept first
|
||||
|
||||
Both incidents below are the **same failure**: a geophone offset crosses the
|
||||
trigger level, the unit records back-to-back, ACH set to "after event
|
||||
recorded" dials continuously, and the unit becomes unreachable because its
|
||||
modem is in client mode almost all of the time.
|
||||
|
||||
There are two ways to get a Stop Monitoring command into it.
|
||||
|
||||
| | **A — intercept the call** (preferred) | **B — catch it between calls** (original) |
|
||||
|---|---|---|
|
||||
| Idea | Be the server it dials. Point the modem's Destination at our own ACH server and answer it. | Clear the Destination so it stops dialing, then race a Stop into the gap. |
|
||||
| Needs inbound? | **No — the unit calls us** | Yes: working inbound TCP to the modem |
|
||||
| Determinism | Deterministic — it dials every ~75 s, we only have to be listening | A race. BE9558H took ~7 h of attempts before one landed. |
|
||||
| Tool | `bridges/ach_server.py --stop-monitoring` | `scripts/slow_drip.sh` |
|
||||
| Proven on | BE12599, 2026-09-16 | BE9558H, 2026-05-17 |
|
||||
|
||||
**Method A is the standard procedure now.** The unit won't answer us because
|
||||
it is on the phone — so stop dialing it and be the one it calls. It rings,
|
||||
we pick up, take its data, and tell it to stop calling here.
|
||||
|
||||
Method B is kept because it is proven, and because A needs a listener the
|
||||
modem can actually reach (public IP + forwarded port). When you have that,
|
||||
don't race it — intercept it.
|
||||
|
||||
---
|
||||
|
||||
## Symptoms
|
||||
|
||||
- Terra-View / SFM `/device/info` either hangs or fails on `count_events()`.
|
||||
@@ -31,9 +59,85 @@ If you see *all* of these, the unit is in this exact failure mode.
|
||||
|
||||
---
|
||||
|
||||
## Quick reference — how to recover
|
||||
## Method A (preferred) — intercept the call
|
||||
|
||||
You need **ACEmanager access** to the unit's modem.
|
||||
You need **ACEmanager access** and a host the modem can dial: public IP with
|
||||
the listener's port forwarded to it.
|
||||
|
||||
### A1 — start the listener BEFORE touching the modem
|
||||
|
||||
```bash
|
||||
cd /home/serversdown/seismo-relay
|
||||
tmux new -s rescue
|
||||
.venv/bin/python -u bridges/ach_server.py --port 12345 \
|
||||
-o bridges/captures/<unit>-diag --stop-monitoring -v
|
||||
```
|
||||
|
||||
⚠ **Listener first, always.** A Destination pointed at a dead port is the
|
||||
worst state available — the device still dials, the modem still flips to
|
||||
client mode, inbound stays blocked, and nothing gets delivered.
|
||||
|
||||
Do **not** add `--events-only` (it silently breaks dedup — see gotchas), and
|
||||
do **not** add `--disable-ach` yet (see A4).
|
||||
|
||||
### A2 — point the modem at it
|
||||
|
||||
ACEmanager → **Serial → Port Configuration**:
|
||||
|
||||
| Field | Set to |
|
||||
|---|---|
|
||||
| **Destination Address** | the listener's public IP |
|
||||
| **Destination Port** | the listener's port (e.g. `12345`) |
|
||||
|
||||
Apply. The modem auto-dials its Destination whenever serial data arrives
|
||||
while the serial port is closed — so the unit's own retry cycle now lands on
|
||||
you instead of nowhere.
|
||||
|
||||
### A3 — answer, and stop the bleeding
|
||||
|
||||
Within ~75 s you should see a call-in. `--stop-monitoring` fires SUB 0x97 at
|
||||
step 1.5 — after the handshake, **before** the event walk — so the recording
|
||||
halts at the earliest possible moment in the session. Confirm via
|
||||
`rescue.json` in the session directory:
|
||||
|
||||
```json
|
||||
{"peer": "166.246.64.226:60921", "stop_monitoring": "ok"}
|
||||
```
|
||||
|
||||
That is the bleeding stopped. Everything after this is cleanup.
|
||||
|
||||
### A4 — drain the backlog, THEN disable ACH
|
||||
|
||||
⚠ **Order matters, and it is counter-intuitive.** Stopping monitoring also
|
||||
removes your call-in trigger: ACH fires on "after event recorded", so with
|
||||
recording stopped the unit has no reason to dial again. The backlog sitting
|
||||
in its memory does **not** re-arm it.
|
||||
|
||||
So if the stored events are worth keeping — and on a fault unit they usually
|
||||
are, they're the evidence — drain them across however many call-ins it takes
|
||||
*before* you silence it. Only then add `--disable-ach` (or use
|
||||
`scripts/rescue_device.sh <host> <port> --no-erase`).
|
||||
|
||||
If the unit has gone quiet and you still need it, cycling the modem produces
|
||||
a call-in, and a unit with a scheduled daily call will dial at its configured
|
||||
time regardless.
|
||||
|
||||
### A5 — restore the Destination, and confirm you did
|
||||
|
||||
Put `Destination Address` back to `0.0.0.0` (or the office Instantel ACH
|
||||
server) once you are finished, and only stop the listener after that is done.
|
||||
|
||||
### A6 — do NOT re-enable ACH until the hardware fault is repaired
|
||||
|
||||
Otherwise the loop restarts the moment monitoring resumes and you run this
|
||||
runbook again.
|
||||
|
||||
---
|
||||
|
||||
## Method B (fallback) — catch it between calls
|
||||
|
||||
The original 2026-05 procedure. Use when you cannot stand up a listener the
|
||||
modem can reach. You need **ACEmanager access** to the unit's modem.
|
||||
|
||||
### Step 1: stop the modem's mode-flipping
|
||||
|
||||
@@ -253,3 +357,223 @@ service).
|
||||
|
||||
Total time from "i was wondering if its possible to" first attempt to
|
||||
recovery: ~7 hours of intermittent debugging across one evening.
|
||||
|
||||
---
|
||||
|
||||
# Second incident — BE12599, 2026-09-16/17
|
||||
|
||||
**Unit:** BE12599 at `166.246.64.226:9034`, RV50, job *I-80 North Fork Bridge
|
||||
— Abut 1 West* (Fay Company). Same job as BE9558H, which is a coincidence.
|
||||
|
||||
**Fault:** the connector fault documented in `docs/offset_investigation.md`
|
||||
§8e progressed until the Tran pedestal reached **0.400 in/s** — its trigger
|
||||
level. Constant triggering → constant recording → ACH "after event recorded"
|
||||
→ continuous dialing. Same disease as BE9558H.
|
||||
|
||||
**Same disease, inverted cure.** Method B's Step 1 *did* work — clearing the
|
||||
Destination stopped the dial-outs, confirmed in the ALEOS log. It was Step 2
|
||||
that didn't land, and rather than keep racing we turned the rescue around:
|
||||
gave the unit a different server to call, and answered it.
|
||||
|
||||
Total time ≈ 5 h, of which ~90 min went to two red herrings documented below.
|
||||
Much of the rest was rediscovering the May procedure, which is why the
|
||||
"two cures" table now sits at the top of this file.
|
||||
|
||||
---
|
||||
|
||||
## Turn on ALEOS_SERIAL debug FIRST
|
||||
|
||||
This is the single highest-value diagnostic and it should be step zero on any
|
||||
future incident. ACEmanager → **Admin → Log → ALEOS_SERIAL log level →
|
||||
DEBUG**, then view the serial log.
|
||||
|
||||
It is the only thing that tells you what the *device* is actually saying.
|
||||
Everything before we did this was guesswork.
|
||||
|
||||
## What the log showed — the unit is on the phone
|
||||
|
||||
Every ~75 seconds, verbatim:
|
||||
|
||||
```
|
||||
ALEOS_SERIAL_HIF: 29 byte(s) in buffer: 'ATQ1^MATE0^MATS0=2^M^MRADIO RING^M'
|
||||
ALEOS_SERIAL_HMC: TCP recvhost fd 65535 len 29 state TCPMode::kClosed
|
||||
ALEOS_SERIAL_HMC: tcpmode trying to send to invalid socket
|
||||
ALEOS_SERIAL_HMC: Connect to IP: 0.0.0.0 Port 0
|
||||
ALEOS_SERIAL_HMC: Initialize Auto answer on port 9034
|
||||
ALEOS_SERIAL_HMC: Cannot connect to 0.0.0.0
|
||||
```
|
||||
|
||||
Read that carefully:
|
||||
|
||||
- `ATQ1` (quiet) / `ATE0` (echo off) / `ATS0=2` (auto-answer after 2 rings).
|
||||
**There is no `ATD`.** The device is not dialing — it is trying to
|
||||
*configure* its modem.
|
||||
- The modem's serial port is in TCP data mode, so it never interprets these
|
||||
as AT commands. It treats them as payload and tries to ship them to a TCP
|
||||
socket that does not exist.
|
||||
- The device therefore never receives `OK`, never progresses, and **retries
|
||||
the identical 29 bytes forever**.
|
||||
|
||||
**While it is in this state it is busy placing a call, not listening for
|
||||
us.** This is almost certainly what BE9558H was doing too — we simply never
|
||||
turned on ALEOS_SERIAL debug in May to look. It is not a different disease;
|
||||
it is the same one, seen properly for the first time.
|
||||
|
||||
It is also the argument for Method A in one picture: the unit is mid-dial
|
||||
every ~75 s, and our inbound Stop has to thread the gaps between those
|
||||
attempts. Give it somewhere to dial and the problem inverts into a
|
||||
deterministic one.
|
||||
|
||||
### Why `slow_drip` lied
|
||||
|
||||
`slow_drip` returned the *success* signature except for the one field that
|
||||
mattered:
|
||||
|
||||
```json
|
||||
{"duration_s":120.0,"drips_sent":38,"bytes_sent":920,
|
||||
"bytes_received":0,"send_error":null}
|
||||
```
|
||||
|
||||
Full duration, no broken pipe — but zero bytes back. Cause is in the log
|
||||
above: each 75 s cycle re-runs `Initialize Auto answer on port 9034`, which
|
||||
orphans the held session (`data in for unknown reason 3 removing from
|
||||
select`, `OnMsg recv error: 107 - Transport endpoint is not connected`). Our
|
||||
local TCP stayed open so `sendall` never raised — but the modem stopped
|
||||
bridging after the first re-init, so every drip after that went into a socket
|
||||
nobody was reading.
|
||||
|
||||
⚠ **`send_error: null` + full duration is NOT success. Only
|
||||
`bytes_received > 0` is success.**
|
||||
|
||||
⚠ **In fairness to slow_drip: it got exactly one attempt here**, run ~90 s
|
||||
after a modem reboot, with a dead session visible in the log at 20:19:17 in
|
||||
that same window. BE9558H took hours of attempts before one landed. Method B
|
||||
was not ruled out on BE12599 so much as abandoned in favour of something that
|
||||
doesn't need luck.
|
||||
|
||||
---
|
||||
|
||||
## ⚠ Two red herrings that cost ~90 minutes
|
||||
|
||||
### 1. The trusted-IP whitelist (this was the real reason inbound never worked)
|
||||
|
||||
The RV50s run with **Security → Trusted IPs (Friends List) enabled**. A
|
||||
source IP that is not on the list is dropped **silently** — inbound presents
|
||||
as `Connection error: timed out`, never a refusal.
|
||||
|
||||
Brian's dev-box public IP is **dynamic** and had changed, so `tmi-dev` was no
|
||||
longer whitelisted. Every inbound attempt failed identically across four
|
||||
different modem and device states, which looked exactly like the BE9558H
|
||||
mode-flipping symptom and sent us chasing modem configuration for over an
|
||||
hour.
|
||||
|
||||
**Check this before diagnosing anything else.** Note that SFM in Docker
|
||||
egresses via the *host's public IP*, not its LAN IP.
|
||||
|
||||
### 2. A 502 from SFM does not mean TCP connected
|
||||
|
||||
`sfm/server.py` raises **502 for both** failure classes:
|
||||
|
||||
```python
|
||||
raise HTTPException(status_code=502, detail=f"Protocol error: {exc}")
|
||||
raise HTTPException(status_code=502, detail=f"Connection error: {exc}")
|
||||
```
|
||||
|
||||
We read an early 502 as "TCP connected, modem bridged, device mute" and built
|
||||
a whole theory on it. It was almost certainly a connect timeout.
|
||||
**Always read the `detail` string** — "connect failed" and "device didn't
|
||||
answer" are completely different problems and the status code will not
|
||||
separate them.
|
||||
|
||||
---
|
||||
|
||||
## What actually worked — invert the direction
|
||||
|
||||
The key observation is in the log above:
|
||||
|
||||
> `TCP recvhost ... state TCPMode::kClosed` → `Connect to IP: 0.0.0.0 Port 0`
|
||||
|
||||
**The modem auto-dials its Destination whenever serial data arrives while
|
||||
closed.** So instead of fighting for inbound, give it somewhere to dial:
|
||||
point `Destination Address` at our own `ach_server` and the device's own
|
||||
75-second attempts become **device-initiated sessions the modem bridges
|
||||
correctly**. No race, no contention, worst case a 75-second wait.
|
||||
|
||||
### Procedure
|
||||
|
||||
1. **Run the rescue server** on a host the modem can reach (public IP +
|
||||
forwarded port):
|
||||
|
||||
```bash
|
||||
cd /home/serversdown/seismo-relay
|
||||
.venv/bin/python -u bridges/ach_server.py --port 12345 \
|
||||
-o bridges/captures/<unit>-diag --stop-monitoring -v
|
||||
```
|
||||
|
||||
2. **Point the modem at it** — ACEmanager → Serial → Port Configuration →
|
||||
`Destination Address` = your public IP, `Destination Port` = 12345.
|
||||
|
||||
3. **Wait for the call-in.** `--stop-monitoring` fires SUB 0x97 at step 1.5,
|
||||
after the handshake and *before* the event walk. Confirm via
|
||||
`rescue.json` in the session directory:
|
||||
|
||||
```json
|
||||
{"peer": "166.246.64.226:60921", "stop_monitoring": "ok"}
|
||||
```
|
||||
|
||||
4. **Restore the modem's Destination** once you are done, then finish the
|
||||
device side (disable ACH, erase) through whichever channel works.
|
||||
|
||||
On BE12599 the first call-in landed at 20:58:11 and reported
|
||||
`stop_monitoring: ok`; a second at 20:58:20 confirmed it. `is_monitoring:
|
||||
false` was still true **6½ hours later** — the fix is durable.
|
||||
|
||||
---
|
||||
|
||||
## Hard-won gotchas (do not re-derive)
|
||||
|
||||
- **Never leave the Destination pointed at a host with nothing listening.**
|
||||
That is the worst state available: the device still dials, the modem still
|
||||
flips, inbound stays blocked, and nothing is delivered. An 8-minute gap
|
||||
with the listener down produced a spurious inbound timeout that cost
|
||||
another round of misdiagnosis.
|
||||
|
||||
- **Stopping monitoring removes your call-in channel.** ACH is "after event
|
||||
recorded"; no new events means no new dials. The backlog sitting in memory
|
||||
does *not* re-arm it. After a successful stop the unit goes quiet and you
|
||||
need the modem cycled (works — produced a call-in), the scheduled daily call
|
||||
(BE12599 calls at **05:00:14 device-local**, per §8e), or working inbound.
|
||||
**Plan the order before you fire the stop.**
|
||||
|
||||
- **`--events-only` silently breaks dedup.** It skips the device-info step,
|
||||
so the serial is never read; `ach_state.json` then keys on
|
||||
`peer:ephemeral_port`, which is unique per connection. Every session looks
|
||||
like a new unit, starts from key 0, and re-downloads the same event. Four
|
||||
sessions on BE12599 downloaded the identical event four times and made zero
|
||||
progress on the backlog. Events also file as `serial=UNKNOWN` with a
|
||||
`M000…` BW filename (serial_numeric 0) instead of `N599…`.
|
||||
**Do not use `--events-only` when you intend to download anything.**
|
||||
|
||||
- **`/device/events/index` reported `lifetime_count: 0`** on a unit with years
|
||||
of history. Suspected decode bug in the SUB 0x08 field offset — do not
|
||||
trust that number. The 88-byte payload is preserved in the `raw_hex` field
|
||||
if someone wants to chase it.
|
||||
|
||||
- **Memory used cross-checks the event keys exactly:**
|
||||
`last_key − buffer_start = memory_total − memory_free`. On BE12599:
|
||||
`0x011230ec − 0x01110000 = 78,060` and `983,028 − 904,968 = 78,060`.
|
||||
Useful sanity check that you are reading the keys right.
|
||||
|
||||
---
|
||||
|
||||
## Final state (2026-09-17 ~01:30 local)
|
||||
|
||||
- `is_monitoring: false`, held 6½ hours
|
||||
- Battery 6.76 V
|
||||
- Memory 78,060 / 983,028 bytes used (8%)
|
||||
- `first_key 01121728`, `last_key 011230ec` — ~6.6 KB of addressable event
|
||||
chain, roughly 3 events
|
||||
- ACH still **enabled** — to be disabled after the backlog is preserved
|
||||
- Modem Destination still pointed at tmi-dev — to be restored
|
||||
- ⚠ **Do not re-enable ACH until the connector is serviced.** Tran is still
|
||||
sitting at 0.400 and the loop restarts the moment monitoring resumes.
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
# SFM — where it actually stands as a tool
|
||||
|
||||
**Status as of 2026-09-20 (v0.31.0).** This is the honest assessment, not the
|
||||
roadmap — `README.md § Roadmap` covers where it is *going*. Expect this file to
|
||||
go stale; re-date it when you revise it.
|
||||
|
||||
---
|
||||
|
||||
## The framing
|
||||
|
||||
SFM is **three different things wearing one name**, at three very different
|
||||
levels of maturity:
|
||||
|
||||
| | what it is | maturity |
|
||||
|---|---|---|
|
||||
| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
|
||||
| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
|
||||
| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
|
||||
| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
|
||||
|
||||
Brian's own description — *"right now it's an emergency tool and a research
|
||||
project"* — is accurate, and it applies specifically to the **device side**.
|
||||
The data side is not an emergency tool; it has been carrying production for
|
||||
months.
|
||||
|
||||
Most confusion about "is SFM reliable?" comes from answering for the wrong
|
||||
tier.
|
||||
|
||||
---
|
||||
|
||||
## 1. What you can rely on
|
||||
|
||||
### Production-grade — trust it
|
||||
|
||||
- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
|
||||
preserved Blastware ASCII exports, 45 units, files back to 2018.
|
||||
- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
|
||||
Thor's own CSV exports; production IDFW 575/575 with zero truncations.
|
||||
- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
|
||||
842,442 per-interval frequency comparisons with zero mismatches.
|
||||
- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
|
||||
fed by the watchers — this is how prod actually gets its data, and it has
|
||||
been running unattended for months.
|
||||
- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
|
||||
listing, event detail and report.
|
||||
- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
|
||||
binaries, with operator review state preserved across regeneration.
|
||||
- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
|
||||
Proven in the field, including as a rescue tool (see the runbook).
|
||||
|
||||
### Emergency-grade — works, but you are the error handling
|
||||
|
||||
- **`/device/*` live endpoints.** They do what they say. But they are
|
||||
synchronous, unauthenticated, and a single cellular download can exceed the
|
||||
60 s timeouts that sit in front of them.
|
||||
- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
|
||||
Each has worked in a real incident — but each has been used a handful of
|
||||
times, by one person, with the runbook open.
|
||||
- **The standalone webapp.** Perfectly usable, and as of v0.31.0 the cheap
|
||||
probes and rescue actions are reachable without curl. No auth of any kind.
|
||||
|
||||
### Research artifacts — useful, not products
|
||||
|
||||
- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
|
||||
Inspector). Desktop-only, single-user, no tests.
|
||||
- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
|
||||
`verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
|
||||
These produced the numbers the production claims rest on, so they matter —
|
||||
but they are analysis scripts, not maintained code.
|
||||
- **`docs/offset_investigation.md`** — an open investigation, not a feature.
|
||||
|
||||
---
|
||||
|
||||
## 2. What to use when
|
||||
|
||||
| you want to… | use | notes |
|
||||
|---|---|---|
|
||||
| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
|
||||
| Know whether ACH is on | `GET /device/call_home` | ~2 s. **Not** `/device/events`. |
|
||||
| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
|
||||
| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
|
||||
| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
|
||||
| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
|
||||
| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
|
||||
|
||||
The single most useful habit: **the cheap probes are cheap and the event walk
|
||||
is not.** Reaching for `/device/events` to answer a yes/no question about a
|
||||
unit is the mistake that motivated the v0.31.0 webapp changes.
|
||||
|
||||
---
|
||||
|
||||
## 3. Known issues
|
||||
|
||||
| issue | impact | status |
|
||||
|---|---|---|
|
||||
| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`. Needs a BW capture of a spanning event to fix properly. |
|
||||
| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
|
||||
| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial. `/docs` itself is unguarded. |
|
||||
| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug. Surfaced in the UI as "unreliable". |
|
||||
| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint. Must be POST-starts-job / GET-polls before any remote lab. |
|
||||
| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known. Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
|
||||
| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
|
||||
| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
|
||||
|
||||
---
|
||||
|
||||
## 4. What stands between this and a real tool
|
||||
|
||||
Roughly in dependency order — each unblocks the ones below it.
|
||||
|
||||
**1. Authentication.** Everything else is gated on this. SFM has none, and
|
||||
the modem IP whitelist gives zero protection because SFM *is* the whitelisted
|
||||
origin. The agreed design delegates rather than builds: Terra-View becomes the
|
||||
authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
|
||||
auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
|
||||
|
||||
**2. Async long operations.** POST starts a job, GET polls. Retrofitting this
|
||||
after building a remote lab on top of synchronous endpoints would be far worse
|
||||
than designing for it now.
|
||||
|
||||
**3. Confirm-guards on the remaining destructive endpoints.** Auth answers
|
||||
*who*, not *did you mean it*. The webapp's erase is guarded; the other seven
|
||||
destructive POSTs and `/docs` are not.
|
||||
|
||||
**4. The 5A page-boundary fix.** Until this lands, live event download is
|
||||
unreliable on exactly the units most likely to need attention — the ones that
|
||||
have been recording heavily. Wants a Blastware capture of an event spanning a
|
||||
page boundary before the chunk-addressing half is trustworthy.
|
||||
|
||||
**5. A live Thor / Micromate client.** The device side is MiniMate-only.
|
||||
Series-4 units can only be read from forwarded files, so half the fleet has no
|
||||
live path at all.
|
||||
|
||||
**6. Test coverage that runs from a clean checkout.** 15 of 16 current
|
||||
failures are missing fixture bundles. A test suite that cannot go green on a
|
||||
fresh clone cannot gate anything.
|
||||
|
||||
**7. The SDM rename.** Cosmetic relative to the above, but the longer `sfm/`
|
||||
holds the data-side code the more the tiers blur. ~30–50 files here, ~10–15 in
|
||||
Terra-View, plus a Docker volume migration. Do it when the codebase is quiet.
|
||||
|
||||
---
|
||||
|
||||
## The short version
|
||||
|
||||
The **data side is a real tool already**. The **device side is a set of sharp
|
||||
instruments** that work in the hands of the person who wrote them, with the
|
||||
runbook open. The gap between those two states is mostly **auth, async, and
|
||||
guardrails** — not protocol work. The protocol is the part that is actually
|
||||
finished.
|
||||
@@ -0,0 +1,134 @@
|
||||
# Plan — "Rescue Listener": a first-class tool for the inverted rescue
|
||||
|
||||
**Status:** proposal, not started. Written 2026-09-17 ~01:40 local, straight
|
||||
off the BE12599 incident. Open questions at the bottom need Brian's answer
|
||||
before anything is built.
|
||||
|
||||
**Background:** `docs/runbooks/wedged_unit_recovery.md`, "Second incident —
|
||||
BE12599". The manual version of this worked; this plan is about making it a
|
||||
tool instead of a sequence of remembered steps at 1 AM.
|
||||
|
||||
---
|
||||
|
||||
## The problem, stated plainly
|
||||
|
||||
When a unit is wedged in the BE12599 mode — geophone offset above trigger,
|
||||
recording back-to-back, ACH dialing constantly, device stuck repeating an AT
|
||||
modem-init string and therefore **deaf to S3 over inbound** — the only channel
|
||||
that works is the one the *device* opens.
|
||||
|
||||
Recovering it currently means:
|
||||
|
||||
1. Remember that `bridges/ach_server.py` exists and takes the right flags
|
||||
2. Start it by hand on a box the modem can reach, with a public port forwarded
|
||||
3. Go into ACEmanager and repoint the modem's Destination
|
||||
4. Watch a terminal for a call-in
|
||||
5. Read `rescue.json` to find out whether it worked
|
||||
6. Go back into ACEmanager and repoint the modem to where it belongs
|
||||
7. **Not forget step 6**, because leaving the Destination pointed at a dead
|
||||
listener is worse than never having started
|
||||
|
||||
That is six manual steps and one landmine, executed under pressure while a
|
||||
unit floods the office server.
|
||||
|
||||
## What the tool should be
|
||||
|
||||
**A "rescue listener" an operator can start for one unit, which handles
|
||||
whatever that unit says when it calls in, and refuses to go away until the
|
||||
operator confirms the modem has been pointed back.**
|
||||
|
||||
Lifecycle:
|
||||
|
||||
1. **Start** — operator names the target unit and starts a rescue listener.
|
||||
The tool reports the exact address/port to enter in ACEmanager, plus the
|
||||
actions it will take.
|
||||
2. **Operator repoints the modem** to that address.
|
||||
3. **Wait** — listener sits there. Live status: "waiting for call-in",
|
||||
elapsed, last-seen.
|
||||
4. **Act** — on call-in, run the configured rescue actions automatically,
|
||||
in a safe order, each independently guarded. Report per-action outcome.
|
||||
5. **Hold** — the listener **stays up** and keeps reporting, because the
|
||||
modem is still pointed at it.
|
||||
6. **Confirm & stop** — the operator explicitly confirms the Destination has
|
||||
been restored (to `0.0.0.0`, or to the office Instantel ACH server).
|
||||
Only then does the listener shut down.
|
||||
|
||||
Step 6 is the whole point of making this a tool. It is the step that is
|
||||
easiest to skip and most expensive to skip.
|
||||
|
||||
## Default action set
|
||||
|
||||
Ordered deliberately — see "order matters" below.
|
||||
|
||||
| # | Action | Default | Why |
|
||||
|---|---|---|---|
|
||||
| 1 | **Stop monitoring** (SUB 0x97) | ✅ on | Halts recording; ends the trigger→record→dial loop at its source. Already implemented as `--stop-monitoring`. |
|
||||
| 2 | **Drain events** to a diagnostics store | ⚙ configurable | The backlog is usually evidence, not garbage — see the BE12599 offset investigation. Must NOT land in the prod SFM DB. |
|
||||
| 3 | **Disable ACH** (SUB 0x2C/0x7E/0x7F) | ❌ off by default | Stops the dialing — **and stops your only channel**. Opt-in, and ideally gated on step 1 having succeeded. |
|
||||
| 4 | **Erase events** | ❌ off by default | Destructive. Only after a verified drain. |
|
||||
|
||||
### Order matters — the lesson from BE12599
|
||||
|
||||
Stopping monitoring *removes the call-in trigger*. ACH fires on "after event
|
||||
recorded"; with recording stopped, the unit has no reason to dial again, even
|
||||
though the backlog is still sitting in its memory. So a naive
|
||||
"stop + disable + erase, all at once" rescue can silence the unit before
|
||||
you've collected anything, leaving you with no channel and a device full of
|
||||
evidence.
|
||||
|
||||
The tool should either sequence around this or warn loudly about it. My
|
||||
instinct is: **stop monitoring immediately** (it's the bleeding), then drain
|
||||
across however many call-ins it takes, and treat disable-ACH/erase as a
|
||||
separate, explicit "finish" action once the operator is satisfied.
|
||||
|
||||
## Where it should live — open question, with a proposal
|
||||
|
||||
The natural tier is **SFM** (device-side, per the three-tier model in
|
||||
CLAUDE.md). But the rescue listener must be reachable *from the cellular
|
||||
network*, which is a deployment constraint SFM's usual profile doesn't have.
|
||||
|
||||
**Proposal worth considering:** run it at the office, beside the real Instantel
|
||||
ACH server, on a **different port** (e.g. 12346 while Instantel holds 12345).
|
||||
Then the ACEmanager change is a **port change, not an IP change** — smaller,
|
||||
faster, less to get wrong, and trivially reversible. It also means the office
|
||||
public IP (already stable and known) is the destination, rather than whatever
|
||||
Brian's dynamic home IP happens to be that week.
|
||||
|
||||
The tmi-dev approach used on BE12599 worked, but required a router forward and
|
||||
ran into the dynamic-IP problem in the same session.
|
||||
|
||||
## Open questions
|
||||
|
||||
1. **Where does it run?** Office beside Instantel ACH (port swap), SFM on the
|
||||
NAS, or ad-hoc on tmi-dev? Affects everything else.
|
||||
2. **What drives it?** Terra-View admin page (fits "operator UI"), an SFM
|
||||
endpoint pair (`POST /device/rescue_listener/start` + `/stop` + `/status`),
|
||||
or a CLI wrapper? A long-lived listener doesn't fit the request/response
|
||||
endpoint shape well — probably needs a background task with a status poll.
|
||||
3. **How does it identify the unit?** It can't know the serial until the
|
||||
device calls in and the handshake reads it. Allowlist by modem IP? Accept
|
||||
anything and report what showed up?
|
||||
4. **Where do drained events go?** A per-incident diagnostics store
|
||||
(`bridges/captures/<unit>-diag`) seems right — explicitly *not* the prod
|
||||
SFM DB. Does that store need to be a first-class thing with its own
|
||||
retention, or is a directory fine?
|
||||
5. **How is "confirm the modem is repointed" verified?** Operator attestation
|
||||
(a button), or can we actually probe it? If the listener stops seeing
|
||||
call-ins that's weak evidence; if inbound to the unit starts working that's
|
||||
stronger.
|
||||
6. **Multi-unit?** One listener per incident, or one listener that handles any
|
||||
unit that dials in? Probably the former for safety.
|
||||
7. **Timeout / abandonment policy.** If nobody ever confirms, does it run
|
||||
forever? Alert after N hours?
|
||||
|
||||
## What already exists
|
||||
|
||||
- `bridges/ach_server.py` — the listener itself, with `--stop-monitoring`,
|
||||
`--disable-ach`, `--rescue` (added on `feat/ach-rescue-on-connect`, commit
|
||||
`9f1050b`), `--clear-after-download`, `--max-events`, `--allow-ip`.
|
||||
- Per-session `rescue.json` recording per-action outcomes.
|
||||
- Isolated per-output-dir SQLite + waveform store, so a diagnostics capture is
|
||||
already separate from prod by construction.
|
||||
|
||||
So the gap is not protocol work — it's lifecycle, operator surface, and the
|
||||
confirmation gate. Most of the risk is in questions 1 and 2.
|
||||
@@ -0,0 +1,89 @@
|
||||
r"""Decode the Thor / Micromate (series-4) sensor self-check waveforms from an
|
||||
IDFW event binary.
|
||||
|
||||
Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events. The IDFW
|
||||
binary carries the sensor self-check in its fixed-header region (before the
|
||||
waveform body), as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
|
||||
channel ids as the series-3 MiniMate Plus (Tran / Vert / Long / MicL), which is
|
||||
the physical self-test:
|
||||
|
||||
* 3c / 3d / 3e = Tran / Vert / Long geophone ring-downs (a damped impulse
|
||||
response — resonant frequency + damping).
|
||||
* 3f = MicL pulse train (the mic's known-signal gain check). Absent
|
||||
on three-channel (mic-disabled) units.
|
||||
|
||||
Record framing (per record)::
|
||||
|
||||
01 0e [id:1] [flags:3] [count:2 BE] [pad:10] [int16-BE samples × count]
|
||||
\___ 18-byte header ___/
|
||||
|
||||
Unlike series-3's delta-coded trailing block, series-4 stores each trace as a
|
||||
raw int16 big-endian array. ``count`` (the 2-byte field at header offset +8)
|
||||
is the sample count; the record is padded to a fixed stride after that.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
from typing import Dict, List
|
||||
|
||||
# Record id → channel. Same ids/order as series-3 (minimateplus.sensor_check).
|
||||
_ID_TO_CHANNEL = {0x3C: "Tran", 0x3D: "Vert", 0x3E: "Long", 0x3F: "MicL"}
|
||||
_CHAIN_IDS = (0x3C, 0x3D, 0x3E, 0x3F)
|
||||
|
||||
_MARKER = b"\x01\x0e" # precedes the 1-byte channel id
|
||||
_HEADER_LEN = 18 # bytes from the marker start to the first sample
|
||||
_COUNT_OFF = 8 # 2-byte BE sample count, from the marker start
|
||||
_MAX_COUNT = 4000 # sanity cap (traces are ~70-200 samples)
|
||||
|
||||
|
||||
def _find_chain(raw: bytes):
|
||||
"""Locate the sensor-check record chain. Returns a list of
|
||||
``(offset, id, count)`` for the first run of markers whose ids run
|
||||
3c, 3d, 3e[, 3f] in order, or ``[]``.
|
||||
|
||||
Records are padded to a fixed stride, so the next marker is not at
|
||||
``header + count*2``; instead collect every ``01 0e [id]`` marker with a
|
||||
sane count and take the first id-ordered run. Validating the id sequence
|
||||
(not a lone ``01 0e 3c``) keeps a stray marker in the waveform body from
|
||||
matching — the real chain sits in the fixed header, ahead of the body.
|
||||
"""
|
||||
n = len(raw)
|
||||
markers = []
|
||||
for p in range(n - _HEADER_LEN):
|
||||
if raw[p:p + 2] == _MARKER and raw[p + 2] in _ID_TO_CHANNEL:
|
||||
count = int.from_bytes(raw[p + _COUNT_OFF:p + _COUNT_OFF + 2], "big")
|
||||
if 0 < count <= _MAX_COUNT:
|
||||
markers.append((p, raw[p + 2], count))
|
||||
|
||||
for i, (off, rid, _c) in enumerate(markers):
|
||||
if rid != 0x3C:
|
||||
continue
|
||||
run = [markers[i]]
|
||||
for m in markers[i + 1:]:
|
||||
if len(run) < len(_CHAIN_IDS) and m[1] == _CHAIN_IDS[len(run)]:
|
||||
run.append(m)
|
||||
else:
|
||||
break
|
||||
if len(run) >= 3: # 3-channel (mic-disabled) units are valid
|
||||
return run
|
||||
return []
|
||||
|
||||
|
||||
def decode_idf_sensor_check(raw: bytes) -> Dict[str, List[int]]:
|
||||
"""Decode the sensor self-check traces from a Thor/Micromate IDFW binary.
|
||||
|
||||
Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}`` in
|
||||
raw int16 ADC counts (MicL omitted on 3-channel units), or ``{}`` if the
|
||||
binary carries no sensor-check chain (a non-IDF file, or an IDFH histogram).
|
||||
"""
|
||||
chain = _find_chain(raw)
|
||||
if not chain:
|
||||
return {}
|
||||
out: Dict[str, List[int]] = {}
|
||||
for off, rid, count in chain:
|
||||
start = off + _HEADER_LEN
|
||||
blob = raw[start:start + count * 2]
|
||||
if len(blob) < count * 2:
|
||||
continue
|
||||
out[_ID_TO_CHANNEL[rid]] = list(struct.unpack(">%dh" % count, blob))
|
||||
return out
|
||||
@@ -50,7 +50,7 @@ SIDECAR_KIND = "sfm.event"
|
||||
# bumped without a `pip install` re-run — leading to confusing stale
|
||||
# version stamps in sidecars. Bump this constant and CHANGELOG.md
|
||||
# together at release time.
|
||||
TOOL_VERSION = "0.30.0"
|
||||
TOOL_VERSION = "0.31.0" # +/sensor_check group (schema v2); gates the backfill regen
|
||||
|
||||
try:
|
||||
# Best-effort: prefer the installed metadata when it's NEWER than the
|
||||
@@ -960,6 +960,11 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
project=project, client=client, operator=user, sensor_location=seisloc,
|
||||
)
|
||||
ev.raw_samples = samples
|
||||
# Sensor self-check traces from the binary's trailing block (waveform
|
||||
# events only; returns {} for histograms / when absent). Carried on the
|
||||
# Event so the .h5 writer persists them device-agnostically.
|
||||
from minimateplus.sensor_check import decode_sensor_check
|
||||
ev.sensor_check = decode_sensor_check(raw) or None
|
||||
# Only compute peaks from samples when we actually have samples.
|
||||
# For events the codec couldn't decode (histogram-mode bodies, until
|
||||
# the §7.6.2 histogram codec is wired in), samples is an empty dict
|
||||
|
||||
@@ -544,6 +544,15 @@ class Event:
|
||||
pretrig_samples: Optional[int] = None # from STRT record: pre-trigger sample count
|
||||
rectime_seconds: Optional[int] = None # from STRT record: record duration (seconds)
|
||||
|
||||
# Sensor self-check traces keyed by channel label — the short diagnostic
|
||||
# waveforms the unit records when it pulses each sensor before monitoring
|
||||
# (geophone ring-downs + a mic pulse train). Decoded from the binary by
|
||||
# the per-series decoder (minimateplus.sensor_check / micromate.sensor_check)
|
||||
# and carried here so the .h5 writer can persist them device-agnostically.
|
||||
# Raw ADC counts; the source series' scale differs but the trace is a
|
||||
# shape diagnostic (rendered fit-to-box). None when absent.
|
||||
sensor_check: Optional[dict] = None # {"Tran": [...], ..., "MicL": [...]}
|
||||
|
||||
# ── Debug / introspection ─────────────────────────────────────────────────
|
||||
# Raw 210-byte waveform record bytes, set when debug mode is active.
|
||||
# Exposed by the SFM server via ?debug=true so field layouts can be verified.
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
r"""Decode the Blastware sensor self-check waveforms from a series-3 event binary.
|
||||
|
||||
Reverse-engineered 2026-09-15 against 7 BE12844 (MiniMate Plus) oracle events.
|
||||
After the main waveform record-chain and the trailing metadata / per-channel
|
||||
calibration records, the binary carries four length-prefixed records tagged
|
||||
0x3c-0x3f: the sensor self-check traces the unit records when it pulses each
|
||||
sensor before monitoring. Blastware draws these as the little waveforms in the
|
||||
"Sensor Check" strip on the right of the Event Report.
|
||||
|
||||
* 0x3c / 0x3d / 0x3e = Tran / Vert / Long geophone ring-downs (a damped
|
||||
oscillation at the geophone's resonance, ~7-8 Hz at 1024 sps).
|
||||
* 0x3f = MicL, a pulse train at the mic self-test frequency
|
||||
(~20 Hz), whose zero-crossing frequency is BW's mic "Channel Test" freq.
|
||||
|
||||
Record framing (per record, all four chained by their length prefix)::
|
||||
|
||||
[len:2 BE][id:1][00 00][Nchan:1][12-byte header][delta stream][40 02][6B]
|
||||
\_________________ payload (len bytes) _______________________________/
|
||||
|
||||
The delta stream is ``payload[20 : len-8]`` (the ``40 02`` terminator sits at
|
||||
``len-8``, followed by 6 trailing bytes). It uses the exact same 10/20/30/00
|
||||
delta-block tags as the main waveform codec
|
||||
(:mod:`minimateplus.waveform_codec`), decoded here from an implicit anchor of 0
|
||||
— so the traces come out in the same 16-count raw units as the main waveform
|
||||
(LSB = 0.005 in/s at Normal range for the geophones).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Dict, List
|
||||
|
||||
from minimateplus.waveform_codec import walk_body
|
||||
|
||||
# Record id → channel. Order mirrors the trailing per-channel calibration
|
||||
# records (Tran / Vert / Long / MicL), confirmed against BW's sensor-check
|
||||
# frequencies on all 7 oracle events.
|
||||
_ID_TO_CHANNEL = {0x3C: "Tran", 0x3D: "Vert", 0x3E: "Long", 0x3F: "MicL"}
|
||||
_CHAIN_IDS = (0x3C, 0x3D, 0x3E, 0x3F)
|
||||
|
||||
_HEADER_LEN = 20 # payload bytes before the delta stream
|
||||
_TRAILER_LEN = 8 # 40 02 terminator + 6 trailing bytes after the stream
|
||||
|
||||
|
||||
def _s4(nib: int) -> int:
|
||||
"""Sign-extend a 4-bit nibble delta."""
|
||||
return nib - 16 if nib >= 8 else nib
|
||||
|
||||
|
||||
def _i8(byte: int) -> int:
|
||||
"""Sign-extend an 8-bit int delta."""
|
||||
return byte - 256 if byte >= 128 else byte
|
||||
|
||||
|
||||
def _decode_delta_stream(buf: bytes) -> List[int]:
|
||||
"""Accumulate a 10/20/30/00 delta-block stream from an anchor of 0,
|
||||
stopping at the 0x40 terminator.
|
||||
|
||||
Mirrors the block semantics in
|
||||
:func:`minimateplus.waveform_codec.decode_waveform_v2` (fully decoded &
|
||||
byte-exact as of 2026-05-11); see that module for the format details.
|
||||
"""
|
||||
out: List[int] = []
|
||||
cur = 0
|
||||
for blk in walk_body(buf, 0):
|
||||
fam = blk.tag_hi & 0xF0
|
||||
if fam == 0x10:
|
||||
# nibble deltas, high nibble first
|
||||
for byte in blk.data:
|
||||
for nib in ((byte >> 4) & 0xF, byte & 0xF):
|
||||
cur += _s4(nib)
|
||||
out.append(cur)
|
||||
elif fam == 0x20:
|
||||
# int8 deltas
|
||||
for byte in blk.data:
|
||||
cur += _i8(byte)
|
||||
out.append(cur)
|
||||
elif fam == 0x30:
|
||||
# 12-bit signed deltas, packed as tag_lo/4 groups of 6 bytes
|
||||
for g in range(blk.tag_lo // 4):
|
||||
grp = blk.data[g * 6:(g + 1) * 6]
|
||||
if len(grp) < 6:
|
||||
break
|
||||
high_word = (grp[0] << 8) | grp[1]
|
||||
for k in range(4):
|
||||
nib = (high_word >> (12 - 4 * k)) & 0xF
|
||||
v = (nib << 8) | grp[2 + k]
|
||||
if v >= 0x800:
|
||||
v -= 0x1000
|
||||
cur += v
|
||||
out.append(cur)
|
||||
elif fam == 0x00:
|
||||
# RLE zero-delta run (wide form carries the high nibble in the tag)
|
||||
run = ((blk.tag_hi & 0x0F) << 8) | blk.tag_lo
|
||||
out.extend([cur] * run)
|
||||
elif fam == 0x40:
|
||||
# segment / record terminator
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
def _find_chain(body: bytes):
|
||||
"""Locate the four length-prefixed sensor-check records.
|
||||
|
||||
Returns a list of ``(offset, id, length)`` or ``None``. The chain is
|
||||
validated by walking the ids 0x3c → 0x3d → 0x3e → 0x3f via their own length
|
||||
prefixes, so a stray 0x3c byte in the waveform data cannot match.
|
||||
"""
|
||||
for p in range(len(body) - 6):
|
||||
if body[p + 2] == 0x3C and body[p + 3] == 0 and body[p + 4] == 0:
|
||||
q = p
|
||||
recs = []
|
||||
ok = True
|
||||
for expect in _CHAIN_IDS:
|
||||
if q + 3 > len(body) or body[q + 2] != expect:
|
||||
ok = False
|
||||
break
|
||||
length = int.from_bytes(body[q:q + 2], "big")
|
||||
recs.append((q, expect, length))
|
||||
q = q + 2 + length
|
||||
if ok and len(recs) == 4:
|
||||
return recs
|
||||
return None
|
||||
|
||||
|
||||
def decode_sensor_check(raw: bytes) -> Dict[str, List[int]]:
|
||||
"""Decode the four sensor self-check traces from a series-3 event binary.
|
||||
|
||||
Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}`` in
|
||||
raw decode units (same 16-count LSB as the main waveform), or ``{}`` if the
|
||||
binary carries no sensor-check block (a histogram event, a non-series-3
|
||||
file, or a unit/firmware that doesn't store it).
|
||||
"""
|
||||
strt = raw.find(b"STRT")
|
||||
if strt < 0 or len(raw) < strt + 21 + 26:
|
||||
return {}
|
||||
body = raw[strt + 21: len(raw) - 26]
|
||||
chain = _find_chain(body)
|
||||
if not chain:
|
||||
return {}
|
||||
out: Dict[str, List[int]] = {}
|
||||
for off, rid, length in chain:
|
||||
payload = body[off + 2: off + 2 + length]
|
||||
if len(payload) < _HEADER_LEN + _TRAILER_LEN:
|
||||
continue
|
||||
stream = payload[_HEADER_LEN: length - _TRAILER_LEN]
|
||||
out[_ID_TO_CHANNEL[rid]] = _decode_delta_stream(stream)
|
||||
return out
|
||||
+1
-1
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "seismo-relay"
|
||||
version = "0.30.0"
|
||||
version = "0.31.0"
|
||||
description = "Python client and REST server for MiniMate Plus seismographs"
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
|
||||
@@ -305,6 +305,11 @@ def main(argv=None) -> int:
|
||||
default=0,
|
||||
)
|
||||
ev.total_samples = ev.total_samples or n_samp
|
||||
# Sensor self-check traces from the IDFW fixed
|
||||
# header, so regenerated .h5 files gain the v2
|
||||
# /sensor_check group (mirrors save_imported_idf).
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
ev.sensor_check = decode_idf_sensor_check(binary_bytes) or None
|
||||
|
||||
event_hdf5.write_event_hdf5(
|
||||
hdf5_path, ev,
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
"""USBM RI8507 / OSMRE blasting compliance chart.
|
||||
|
||||
Renders the velocity-vs-frequency compliance scatter Blastware draws on its Event
|
||||
Report: each channel's significant waveform cycles as ``(frequency, peak
|
||||
velocity)`` points on log-log axes against the regulatory limit curve(s). A point
|
||||
below the curve passes; above fails.
|
||||
|
||||
Two pieces, kept separate so both can be reused/extended:
|
||||
* ``limit_at`` / ``limit_curve`` — the regulatory limit curve(s), as data.
|
||||
* ``channel_compliance_points`` — the per-cycle (freq, velocity) scatter, by
|
||||
the zero-crossing method (matches Blastware: each channel's cloud tops out
|
||||
at that channel's PPV).
|
||||
|
||||
Limit curves (USBM RI8507 Figure B-1 / OSM 30 CFR 816.67), drawn CONTINUOUS — a
|
||||
constant-displacement bound (sloped, ``v = 2πf·d``) meets a constant-velocity
|
||||
plateau at the frequency where they're equal, so there are no vertical steps
|
||||
(matching how Blastware draws it). Two lines:
|
||||
* **Drywall** (modern gypsum board) — 0.75 in/s plateau (solid).
|
||||
* **Plaster** on wood lath (older homes) — 0.50 in/s plateau (dashed).
|
||||
Both use a 0.030 in low-frequency displacement bound and rise through a 0.010 in
|
||||
displacement bound to a 2.0 in/s high-frequency plateau. Values from USBM RI8507
|
||||
(Appendix B) / 30 CFR 816.67; ⚠ confirm the exact shape against a Blastware
|
||||
report before trusting for compliance.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from typing import Dict, Sequence, Tuple
|
||||
|
||||
import numpy as np
|
||||
from matplotlib.ticker import FixedLocator, NullLocator
|
||||
|
||||
# curve name → (low-freq "ultimate" displacement in, mid velocity plateau in/s,
|
||||
# high-freq displacement in, high-freq velocity plateau in/s).
|
||||
# RI8507 Fig B-1 (p.74): ultimate max displacement 0.030 in (< ~4 Hz), plateau
|
||||
# 0.75 (Drywall) / 0.50 (plaster), rising diagonal at 0.008 in displacement up to
|
||||
# a 2.0 in/s plateau reached at ~40 Hz.
|
||||
_CURVES: Dict[str, Tuple[float, float, float, float]] = {
|
||||
"Drywall": (0.030, 0.75, 0.008, 2.00),
|
||||
"Plaster": (0.030, 0.50, 0.008, 2.00),
|
||||
}
|
||||
# how each curve is stroked on the chart
|
||||
_CURVE_STYLE = {"Drywall": {"ls": "-", "lw": 1.0}, "Plaster": {"ls": "--", "lw": 0.9}}
|
||||
|
||||
STANDARDS = tuple(_CURVES)
|
||||
|
||||
# Blastware's channel markers/colours on the compliance chart.
|
||||
_CHANNEL_STYLE = {
|
||||
"Tran": ("+", "#d62728"), # red +
|
||||
"Vert": ("x", "#2ca02c"), # green x
|
||||
"Long": ("o", "#1f77b4"), # blue o
|
||||
}
|
||||
|
||||
|
||||
def limit_at(freq_hz: float, curve: str = "Drywall") -> float:
|
||||
"""Max allowed PPV (in/s) at ``freq_hz`` for ``curve`` (continuous)."""
|
||||
d_low, v_mid, d_high, v_high = _CURVES[curve]
|
||||
f = max(freq_hz, 1.0)
|
||||
f_a = v_mid / (2.0 * math.pi * d_low) # disp_low → vel_mid
|
||||
f_b = v_mid / (2.0 * math.pi * d_high) # vel_mid → disp_high
|
||||
f_c = v_high / (2.0 * math.pi * d_high) # disp_high → vel_high
|
||||
if f <= f_a:
|
||||
return 2.0 * math.pi * f * d_low
|
||||
if f <= f_b:
|
||||
return v_mid
|
||||
if f <= f_c:
|
||||
return 2.0 * math.pi * f * d_high
|
||||
return v_high
|
||||
|
||||
|
||||
def limit_curve(curve: str = "Drywall", fmin: float = 1.0, fmax: float = 100.0, n: int = 400):
|
||||
"""(freqs, limits) sampled across the band for plotting one curve."""
|
||||
freqs = np.logspace(np.log10(fmin), np.log10(fmax), n)
|
||||
return freqs, np.array([limit_at(f, curve) for f in freqs])
|
||||
|
||||
|
||||
def channel_compliance_points(
|
||||
samples: Sequence[float], sps: float, fmin: float = 1.0, fmax: float = 100.0,
|
||||
vmin: float = 0.0,
|
||||
) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""Per-cycle (frequency, peak velocity) scatter for one channel.
|
||||
|
||||
Zero-crossing method: split the trace at sign changes; each half-cycle
|
||||
contributes one point at ``(1/(2·half_period), max|amplitude|)``. Matches
|
||||
Blastware — the cloud's ceiling is the channel PPV. ``samples`` must be in the
|
||||
velocity unit you want plotted (in/s). Points outside ``[fmin, fmax]`` or at
|
||||
or below ``vmin`` are dropped.
|
||||
"""
|
||||
x = np.asarray(samples, dtype=float)
|
||||
if x.size < 3:
|
||||
return np.empty(0), np.empty(0)
|
||||
zc = np.where(np.diff(np.signbit(x)))[0]
|
||||
freqs, vels = [], []
|
||||
for a, b in zip(zc[:-1], zc[1:]):
|
||||
half_period = (b - a) / sps
|
||||
if half_period <= 0:
|
||||
continue
|
||||
freqs.append(1.0 / (2.0 * half_period))
|
||||
vels.append(float(np.abs(x[a:b + 1]).max()))
|
||||
f = np.array(freqs)
|
||||
v = np.array(vels)
|
||||
keep = (f >= fmin) & (f <= fmax) & (v > vmin)
|
||||
return f[keep], v[keep]
|
||||
|
||||
|
||||
def draw_compliance_chart(ax, channels: Dict[str, Sequence[float]], sps: float) -> None:
|
||||
"""Draw the compliance chart (both limit curves + per-channel scatter)."""
|
||||
for name, style in _CURVE_STYLE.items():
|
||||
cf, cv = limit_curve(name)
|
||||
ax.plot(cf, cv, color="#333", zorder=3, **style)
|
||||
|
||||
for ch, (marker, color) in _CHANNEL_STYLE.items():
|
||||
samples = channels.get(ch)
|
||||
if samples is None or len(samples) == 0:
|
||||
continue
|
||||
f, v = channel_compliance_points(samples, sps)
|
||||
ax.scatter(f, v, marker=marker, s=12, c=color, linewidths=0.7, zorder=4, label=ch)
|
||||
|
||||
ax.set_xscale("log")
|
||||
ax.set_yscale("log")
|
||||
ax.set_xlim(1, 100)
|
||||
ax.set_ylim(0.0394, 10)
|
||||
ax.set_box_aspect(1) # square plot box (log-log compliance charts are square)
|
||||
xt = [1, 2, 5, 10, 20, 50, 100]
|
||||
yt = [0.0394, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10]
|
||||
ax.xaxis.set_major_locator(FixedLocator(xt)); ax.xaxis.set_minor_locator(NullLocator())
|
||||
ax.yaxis.set_major_locator(FixedLocator(yt)); ax.yaxis.set_minor_locator(NullLocator())
|
||||
ax.set_xticklabels([str(v) for v in xt])
|
||||
ax.set_yticklabels([("%g" % v) for v in yt])
|
||||
ax.set_xlabel("Frequency (Hz)", fontsize=7)
|
||||
ax.set_ylabel("Velocity (in/s)", fontsize=7)
|
||||
ax.tick_params(labelsize=6)
|
||||
ax.grid(True, which="both", ls=":", lw=0.4, color="#ccc")
|
||||
+44
-4
@@ -12,8 +12,11 @@ Layout written to `<filename>.h5`:
|
||||
├─ samples_int16/ (optional)
|
||||
│ ├─ Tran (int16, raw ADC counts) shape: (N,)
|
||||
│ └─ ... per channel (only when present in the source)
|
||||
├─ sensor_check/ (optional, schema v2+)
|
||||
│ ├─ Tran (int32, raw counts) shape: (M,) M ≪ N
|
||||
│ └─ ... per channel present in the source (MicL absent on 3-channel units)
|
||||
└─ root attrs (event metadata):
|
||||
schema_version int = 1
|
||||
schema_version int = 2
|
||||
kind str = "sfm.event.hdf5"
|
||||
serial str
|
||||
waveform_key str (8-hex)
|
||||
@@ -64,7 +67,7 @@ from minimateplus.models import Event
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
SCHEMA_VERSION = 2 # v2 adds the optional /sensor_check group
|
||||
HDF5_KIND = "sfm.event.hdf5"
|
||||
|
||||
# Geophone full-scale velocity per range (in/s). Confirmed in CLAUDE.md
|
||||
@@ -270,6 +273,22 @@ def write_event_hdf5(
|
||||
)
|
||||
igrp.attrs["mic_psi_per_count"] = float(mic_factor)
|
||||
|
||||
# /sensor_check — optional short diagnostic self-check traces (schema
|
||||
# v2+). Raw ADC counts (a shape diagnostic; the per-series count scale
|
||||
# differs, and the renderer fits each trace to its box). Only channels
|
||||
# the decoder found are written — 3-channel units carry no MicL.
|
||||
sc = event.sensor_check or {}
|
||||
if sc:
|
||||
scgrp = f.create_group("sensor_check")
|
||||
for ch in ("Tran", "Vert", "Long", "MicL"):
|
||||
vals = sc.get(ch)
|
||||
if vals:
|
||||
scgrp.create_dataset(
|
||||
ch, data=np.asarray(vals, dtype=np.int32),
|
||||
compression="gzip", compression_opts=4, shuffle=True,
|
||||
)
|
||||
scgrp.attrs["units"] = "raw_counts"
|
||||
|
||||
import os
|
||||
os.replace(tmp, path)
|
||||
|
||||
@@ -334,6 +353,16 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
|
||||
if mic_attr is not None:
|
||||
mic_psi = float(mic_attr)
|
||||
|
||||
# /sensor_check — optional (schema v2+); absent on older files.
|
||||
sensor_check = None
|
||||
scgrp = f.get("sensor_check")
|
||||
if scgrp is not None:
|
||||
sensor_check = {}
|
||||
for ch in ("Tran", "Vert", "Long", "MicL"):
|
||||
ds = scgrp.get(ch)
|
||||
if ds is not None:
|
||||
sensor_check[ch] = np.asarray(ds[()])
|
||||
|
||||
return {
|
||||
"schema_version": sv,
|
||||
"kind": attrs.get("kind"),
|
||||
@@ -341,6 +370,7 @@ def read_event_hdf5(path: Union[str, Path]) -> dict:
|
||||
"samples": samples,
|
||||
"samples_int16": samples_int16,
|
||||
"mic_psi_per_count": mic_psi,
|
||||
"sensor_check": sensor_check,
|
||||
}
|
||||
|
||||
|
||||
@@ -431,11 +461,16 @@ def plot_json_from_hdf5(
|
||||
event_id: Optional[str] = None,
|
||||
index: Optional[int] = None,
|
||||
) -> dict:
|
||||
"""Build a `sfm.plot.v1` JSON dict from a stored .h5 file."""
|
||||
"""Build a `sfm.plot.v1` JSON dict from a stored .h5 file.
|
||||
|
||||
The dict also carries a top-level ``sensor_check`` key (the raw self-check
|
||||
traces as ``{ch: [int]}``, or None) beyond the plot schema, so report
|
||||
generation can read the traces from the same single .h5 load.
|
||||
"""
|
||||
data = read_event_hdf5(path)
|
||||
a = data["attrs"]
|
||||
s = data["samples"]
|
||||
return _build_plot_dict(
|
||||
out = _build_plot_dict(
|
||||
n_samples=len(s["Tran"]) if "Tran" in s else 0,
|
||||
sample_rate=int(a.get("sample_rate", 1024) or 1024),
|
||||
pretrig_samples=int(a.get("pretrig_samples", 0) or 0),
|
||||
@@ -463,6 +498,11 @@ def plot_json_from_hdf5(
|
||||
event_id=event_id,
|
||||
index=index,
|
||||
)
|
||||
scd = data.get("sensor_check")
|
||||
out["sensor_check"] = (
|
||||
{ch: v.tolist() for ch, v in scd.items()} if scd else None
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _build_plot_dict(
|
||||
|
||||
+158
-31
@@ -121,6 +121,13 @@ class ReportData:
|
||||
t0_ms: Optional[float] = None
|
||||
dt_ms: Optional[float] = None
|
||||
|
||||
# Sensor self-check traces — {ch: [samples]} in raw counts, read from the
|
||||
# standardized .h5 (/sensor_check group, schema v2+) where the per-series
|
||||
# decoder stored them at ingest. The little diagnostic waveforms BW draws
|
||||
# in its "Sensor Check" strip. Empty when absent (pre-v2 .h5, histogram,
|
||||
# or 3-channel unit's MicL).
|
||||
sensor_check_waveforms: dict = field(default_factory=dict)
|
||||
|
||||
# Record-type discriminator
|
||||
record_type: Optional[str] = None
|
||||
is_histogram: bool = False
|
||||
@@ -246,6 +253,8 @@ def gather_report_data(
|
||||
"peak_accel_g": ch.get("peak_accel_g"),
|
||||
"peak_disp_in": ch.get("peak_disp_in"),
|
||||
"sensor_check": sc_ch.get("result"),
|
||||
"sc_freq_hz": sc_ch.get("freq_hz"),
|
||||
"sc_ratio": sc_ch.get("ratio"),
|
||||
"peak_date": peak_date,
|
||||
"peak_time": peak_time,
|
||||
})
|
||||
@@ -287,6 +296,12 @@ def gather_report_data(
|
||||
rd.pretrig_samples = ta.get("pretrig_samples")
|
||||
rd.t0_ms = ta.get("t0_ms")
|
||||
rd.dt_ms = ta.get("dt_ms")
|
||||
# Sensor self-check traces — read from the standardized .h5 (schema
|
||||
# v2+). Device-agnostic: whichever decoder produced the event
|
||||
# stored them at ingest, so SFM reads them here without knowing or
|
||||
# caring about the source instrument series. Empty on pre-v2 files
|
||||
# (until backfilled) and on 3-channel / histogram events.
|
||||
rd.sensor_check_waveforms = wf.get("sensor_check") or {}
|
||||
except Exception as exc:
|
||||
log.warning("gather_report_data: hdf5 read failed: %s", exc)
|
||||
|
||||
@@ -396,9 +411,34 @@ def _render_waveform_layout(fig, rd: ReportData) -> None:
|
||||
ax_stats = fig.add_subplot(gs[2]); ax_stats.axis("off")
|
||||
_draw_channel_stats_waveform(ax_stats, rd)
|
||||
|
||||
_draw_compliance_panel(fig, rd)
|
||||
_draw_waveform_subplot(fig, gs[3], rd)
|
||||
|
||||
|
||||
# Compliance-chart placement, in figure fractions. Measured directly off a
|
||||
# Blastware Event Report PDF (ref-stuff/n844lqhbzt0w_bw_pdf.pdf) so the chart
|
||||
# matches BW's size and position: it spans from just under the header down
|
||||
# through the stats band, hard against the right page margin. The left edge
|
||||
# leaves room for the y-axis tick labels + "Velocity (in/s)" title, which the
|
||||
# compacted stats table (see _draw_channel_stats_waveform) is sized to clear.
|
||||
_COMPLIANCE_BOX = (0.489, 0.502, 0.951, 0.867) # x0, y0, x1, y1
|
||||
|
||||
|
||||
def _draw_compliance_panel(fig, rd: ReportData) -> None:
|
||||
"""Large USBM RI8507 compliance chart in the upper-right, sized and
|
||||
positioned to match Blastware's Event Report (see _COMPLIANCE_BOX)."""
|
||||
x0, y0, x1, y1 = _COMPLIANCE_BOX
|
||||
fig.text((x0 + x1) / 2, y1 + 0.006, "USBM RI8507 And OSMRE", fontsize=9,
|
||||
weight="bold", color="#333", ha="center", va="bottom")
|
||||
if rd.channels and rd.sample_rate_sps:
|
||||
from sfm.compliance import draw_compliance_chart
|
||||
ax = fig.add_axes([x0, y0, x1 - x0, y1 - y0])
|
||||
draw_compliance_chart(ax, rd.channels, rd.sample_rate_sps)
|
||||
else:
|
||||
fig.text((x0 + x1) / 2, (y0 + y1) / 2, "(no waveform data)", fontsize=8,
|
||||
color="#bbb", ha="center", va="center", style="italic")
|
||||
|
||||
|
||||
def _render_histogram_layout(fig, rd: ReportData) -> None:
|
||||
"""Histogram layout: header / mic-only / per-channel stats / bar plot.
|
||||
|
||||
@@ -477,11 +517,11 @@ def _split_iso_to_date_time(iso: Optional[str]) -> tuple[Optional[str], Optional
|
||||
return (None, None)
|
||||
|
||||
|
||||
def _kv(ax, x, y, label, value, *, label_w=0.18):
|
||||
def _kv(ax, x, y, label, value, *, label_w=0.18, fontsize=8):
|
||||
"""Render a 'Label Value' row at axes-coordinates (x, y)."""
|
||||
ax.text(x, y, label, fontsize=8, color="#555", ha="left", va="top",
|
||||
ax.text(x, y, label, fontsize=fontsize, color="#555", ha="left", va="top",
|
||||
transform=ax.transAxes)
|
||||
ax.text(x + label_w, y, _fmt(value), fontsize=8, ha="left", va="top",
|
||||
ax.text(x + label_w, y, _fmt(value), fontsize=fontsize, ha="left", va="top",
|
||||
transform=ax.transAxes, family="monospace")
|
||||
|
||||
|
||||
@@ -544,14 +584,17 @@ def _draw_header_columns(ax, rows_left, rd: ReportData) -> None:
|
||||
("File Name", rd.file_name),
|
||||
("Post Event Notes", rd.post_event_notes),
|
||||
]
|
||||
# fontsize 7.5 (BW's header is a touch smaller than our body text) + a
|
||||
# tighter right-column value indent so the long serial+firmware line
|
||||
# ("BE##### V ##.##-#.## MiniMate Plus") fits without running off the page.
|
||||
y = 0.95
|
||||
dy = 0.095
|
||||
for label, value in rows_left:
|
||||
_kv(ax, 0.0, y, label, value, label_w=0.18)
|
||||
_kv(ax, 0.0, y, label, value, label_w=0.18, fontsize=7.5)
|
||||
y -= dy
|
||||
y = 0.95
|
||||
for label, value in rows_right:
|
||||
_kv(ax, 0.55, y, label, value, label_w=0.20)
|
||||
_kv(ax, 0.55, y, label, value, label_w=0.14, fontsize=7.5)
|
||||
y -= dy
|
||||
|
||||
|
||||
@@ -574,19 +617,14 @@ def _draw_mic_and_usbm(ax, rd: ReportData) -> None:
|
||||
transform=ax.transAxes, va="top")
|
||||
rows = _mic_rows(rd)
|
||||
y = 0.80
|
||||
# Tighter label indent + slightly smaller font so the long "Channel Test
|
||||
# Passed (Freq = … Amp = … mv)" line clears the enlarged compliance chart's
|
||||
# left edge (_COMPLIANCE_BOX) instead of running behind it.
|
||||
for label, value in rows:
|
||||
_kv(ax, 0.0, y, label, value, label_w=0.18)
|
||||
_kv(ax, 0.0, y, label, value, label_w=0.13, fontsize=7)
|
||||
y -= 0.15
|
||||
|
||||
# USBM chart placeholder — upper-right. Real piecewise compliance
|
||||
# curves are a separate work item; for now this just shows the title
|
||||
# + a "see report" message so the layout is correct.
|
||||
ax.text(0.72, 0.97, "USBM RI8507 And OSMRE",
|
||||
fontsize=9, weight="bold", color="#333", ha="center", va="top",
|
||||
transform=ax.transAxes)
|
||||
ax.text(0.72, 0.50, "[compliance chart\ncoming soon]",
|
||||
fontsize=8, color="#bbb", ha="center", va="center",
|
||||
transform=ax.transAxes, style="italic")
|
||||
# The USBM compliance chart is drawn as its own large square panel spanning
|
||||
# the mic + stats rows on the right — see _draw_compliance_panel().
|
||||
|
||||
|
||||
def _mic_rows(rd: ReportData) -> list[tuple[str, Optional[str]]]:
|
||||
@@ -636,8 +674,18 @@ def _draw_channel_stats_waveform(ax, rd: ReportData) -> None:
|
||||
("Peak Acceleration", "peak_accel_g", "g"),
|
||||
("Peak Displacement", "peak_disp_in", "in"),
|
||||
("Sensor Check", "sensor_check", ""),
|
||||
# Sensor-check sub-rows (indented under "Sensor Check", like BW): the
|
||||
# geophone ring-down frequency + overswing ratio from the self-check.
|
||||
(" Frequency", "sc_freq_hz", "Hz"),
|
||||
(" Overswing Ratio", "sc_ratio", ""),
|
||||
]
|
||||
_draw_stats_table(ax, rd, rows_spec)
|
||||
# Compacted to the left half so the enlarged compliance chart (BW-sized,
|
||||
# right against the page margin) has room — see _COMPLIANCE_BOX.
|
||||
_draw_stats_table(
|
||||
ax, rd, rows_spec,
|
||||
bbox_width=0.42, fontsize=7.5,
|
||||
col_widths=[0.185, 0.065, 0.065, 0.065, 0.040],
|
||||
)
|
||||
_draw_pvs_summary(ax, rd, n_data_rows=len(rows_spec))
|
||||
|
||||
|
||||
@@ -698,19 +746,39 @@ def _draw_pvs_summary(
|
||||
table_bottom_y = getattr(ax, "_stats_table_bottom", -0.10)
|
||||
pvs_y = table_bottom_y - 0.04 # small gap below the table border
|
||||
|
||||
# Centered for visual balance — looks intentional rather than offset.
|
||||
# The original BW-replica had a "NA: Not Applicable" caption below
|
||||
# this line; dropped because we use "—" for missing values and the
|
||||
# legend was always squished against the PVS line.
|
||||
# Centered under the stats table for visual balance — looks intentional
|
||||
# rather than offset. When the table is compacted (waveform layout), it
|
||||
# occupies only the left portion of the axes, so center on the table's
|
||||
# width rather than the full axes (which would push the line under the
|
||||
# compliance chart). The original BW-replica had a "NA: Not Applicable"
|
||||
# caption below this line; dropped because we use "—" for missing values.
|
||||
table_w = getattr(ax, "_stats_table_width", 0.80)
|
||||
if table_w < 0.79:
|
||||
# Compacted (waveform) layout: left-align under the table, one point
|
||||
# smaller, so the line clears the enlarged compliance chart's
|
||||
# bottom-left tick labels on the right.
|
||||
ax.text(0.0, pvs_y, line, fontsize=8, weight="bold",
|
||||
ha="left", va="top", transform=ax.transAxes)
|
||||
else:
|
||||
ax.text(0.5, pvs_y, line, fontsize=9, weight="bold",
|
||||
ha="center", va="top", transform=ax.transAxes)
|
||||
|
||||
|
||||
def _draw_stats_table(ax, rd: ReportData, rows_spec: list[tuple[str, str, str]]) -> None:
|
||||
def _draw_stats_table(
|
||||
ax, rd: ReportData, rows_spec: list[tuple[str, str, str]],
|
||||
*, bbox_width: float = 0.80, fontsize: float = 8,
|
||||
col_widths: Optional[list[float]] = None,
|
||||
) -> None:
|
||||
"""Render a per-channel stats table (Tran/Vert/Long).
|
||||
|
||||
rows_spec: list of (label, field_name_in_channel_stats, unit_string)
|
||||
|
||||
``bbox_width`` / ``col_widths`` / ``fontsize`` let a caller compact the
|
||||
table (the waveform layout packs it into the left half to clear the
|
||||
compliance chart; the histogram layout keeps the wider defaults).
|
||||
"""
|
||||
if col_widths is None:
|
||||
col_widths = [0.28, 0.14, 0.14, 0.14, 0.10]
|
||||
headers = ["", "Tran", "Vert", "Long", ""]
|
||||
ch_lookup = {c["name"]: c for c in rd.channel_stats}
|
||||
|
||||
@@ -726,6 +794,8 @@ def _draw_stats_table(ax, rd: ReportData, rows_spec: list[tuple[str, str, str]])
|
||||
if field == "zc_freq_hz":
|
||||
prefix = ">" if ch_rec.get("zc_freq_above_range") else ""
|
||||
return f"{prefix}{val:.0f}"
|
||||
if field in ("sc_freq_hz", "sc_ratio"):
|
||||
return f"{val:.1f}" # BW shows 1 decimal (7.5 Hz, 3.6)
|
||||
return f"{val:.3f}"
|
||||
return str(val)
|
||||
|
||||
@@ -750,16 +820,17 @@ def _draw_stats_table(ax, rd: ReportData, rows_spec: list[tuple[str, str, str]])
|
||||
table_bottom = 1.0 - table_height
|
||||
tbl = ax.table(
|
||||
cellText=table_data,
|
||||
colWidths=[0.28, 0.14, 0.14, 0.14, 0.10],
|
||||
colWidths=col_widths,
|
||||
cellLoc="left", edges="open",
|
||||
bbox=[0.0, table_bottom, 0.80, table_height],
|
||||
bbox=[0.0, table_bottom, bbox_width, table_height],
|
||||
)
|
||||
tbl.auto_set_font_size(False)
|
||||
tbl.set_fontsize(8)
|
||||
tbl.set_fontsize(fontsize)
|
||||
for j in range(5):
|
||||
tbl[(0, j)].set_text_props(weight="bold", color="#555")
|
||||
# Stash the bottom Y so _draw_pvs_summary can position itself below.
|
||||
# Stash the bottom Y + width so _draw_pvs_summary can position itself.
|
||||
ax._stats_table_bottom = table_bottom
|
||||
ax._stats_table_width = bbox_width
|
||||
|
||||
|
||||
def _channel_axis_color(ch: str) -> str:
|
||||
@@ -769,9 +840,25 @@ def _channel_axis_color(ch: str) -> str:
|
||||
def _draw_waveform_subplot(fig, gridspec_cell, rd: ReportData) -> None:
|
||||
"""4-channel stacked waveform plot — Instantel printout order
|
||||
(MicL on top, Tran on bottom), shared x-axis in SECONDS, trigger
|
||||
triangle markers at t=0, '0.0' baseline label on right of each."""
|
||||
inner = gridspec_cell.subgridspec(4, 1, hspace=0.0)
|
||||
triangle markers at t=0, '0.0' baseline label on right of each.
|
||||
|
||||
When sensor self-check traces are present (rd.sensor_check_waveforms), a
|
||||
narrow "Sensor Check" strip of per-channel mini-plots is drawn to the right,
|
||||
aligned to the lanes — matching Blastware's Event Report.
|
||||
"""
|
||||
from matplotlib.ticker import MaxNLocator
|
||||
|
||||
order = ["MicL", "Long", "Vert", "Tran"]
|
||||
has_sc = bool(rd.sensor_check_waveforms)
|
||||
if has_sc:
|
||||
# main lanes + a narrow sensor-check strip column, flush against the
|
||||
# main panel (BW shares the border — no gap), with the "0.0" baseline
|
||||
# labels moved to the right of the strip. Proportions match BW's
|
||||
# Event Report (main ~0.75 / strip ~0.10 of the panel width).
|
||||
inner = gridspec_cell.subgridspec(4, 2, width_ratios=[1.0, 0.13],
|
||||
wspace=0.0, hspace=0.0)
|
||||
else:
|
||||
inner = gridspec_cell.subgridspec(4, 1, hspace=0.0)
|
||||
sr = rd.sample_rate_sps or 1024
|
||||
# Convert ms-based time axis to seconds for the x-axis
|
||||
dt_s = (rd.dt_ms or (1000.0 / sr)) / 1000.0
|
||||
@@ -790,9 +877,12 @@ def _draw_waveform_subplot(fig, gridspec_cell, rd: ReportData) -> None:
|
||||
_geo_amax = _a
|
||||
geo_shared = max(_geo_amax * 1.10, GEO_FLOOR_INS)
|
||||
|
||||
main_axes = []
|
||||
sc_axes = []
|
||||
last_idx = len(order) - 1
|
||||
for i, ch in enumerate(order):
|
||||
ax = fig.add_subplot(inner[i])
|
||||
ax = fig.add_subplot(inner[i, 0] if has_sc else inner[i])
|
||||
main_axes.append(ax)
|
||||
values = rd.channels.get(ch) or []
|
||||
times = [t0_s + j * dt_s for j in range(len(values))]
|
||||
|
||||
@@ -810,7 +900,10 @@ def _draw_waveform_subplot(fig, gridspec_cell, rd: ReportData) -> None:
|
||||
# Channel label on the LEFT (matches BW)
|
||||
ax.set_ylabel(ch, fontsize=8, rotation=0, ha="right", va="center",
|
||||
color=_channel_axis_color(ch), weight="bold", labelpad=14)
|
||||
# "0.0" on the RIGHT (BW convention)
|
||||
# "0.0" baseline label on the RIGHT (BW convention). With the sensor-
|
||||
# check strip attached, it goes to the right of the STRIP (drawn below);
|
||||
# otherwise just outside the main lane.
|
||||
if not has_sc:
|
||||
ax.text(1.005, 0.5, "0.0", transform=ax.transAxes,
|
||||
fontsize=7, color="#555", va="center", ha="left")
|
||||
|
||||
@@ -827,12 +920,46 @@ def _draw_waveform_subplot(fig, gridspec_cell, rd: ReportData) -> None:
|
||||
else:
|
||||
ax.tick_params(axis="x", labelsize=7)
|
||||
ax.tick_params(axis="y", labelsize=6)
|
||||
# Stacked lanes touch, so the top/bottom y-tick labels of adjacent lanes
|
||||
# would overprint at the shared boundary. Prune the extreme ticks so
|
||||
# each boundary shows clean interior ticks (0.5 / 0.0 / -0.5) only.
|
||||
ax.yaxis.set_major_locator(MaxNLocator(nbins=4, prune="both"))
|
||||
|
||||
# Sensor self-check mini-plot in the right strip (aligned to this lane).
|
||||
if has_sc:
|
||||
scx = fig.add_subplot(inner[i, 1])
|
||||
sc_axes.append(scx)
|
||||
sc_vals = rd.sensor_check_waveforms.get(ch) or []
|
||||
if sc_vals:
|
||||
_col = _channel_axis_color(ch)
|
||||
# Faint zero baseline (BW draws the channel baseline through the
|
||||
# strip) — reference for the one-sided geophone ring-downs.
|
||||
scx.axhline(0.0, color=_col, linewidth=0.3, alpha=0.4)
|
||||
scx.plot(range(len(sc_vals)), sc_vals, color=_col, linewidth=0.5)
|
||||
# Fit the trace to the box (BW-style) rather than a symmetric
|
||||
# scale: the geo self-checks are one-sided dips, so a symmetric
|
||||
# scale would strand them in the bottom half with an empty top.
|
||||
_lo, _hi = min(sc_vals), max(sc_vals)
|
||||
_pad = 0.10 * ((_hi - _lo) or 1.0)
|
||||
scx.set_ylim(_lo - _pad, _hi + _pad)
|
||||
scx.set_xticks([]); scx.set_yticks([])
|
||||
for _s in scx.spines.values():
|
||||
_s.set_linewidth(0.4); _s.set_color("#999")
|
||||
# "0.0" baseline label to the RIGHT of the strip (BW convention)
|
||||
scx.text(1.10, 0.5, "0.0", transform=scx.transAxes,
|
||||
fontsize=7, color="#555", va="center", ha="left")
|
||||
|
||||
# Trigger triangle marker ▼ above the top channel at t=0
|
||||
top_ax = fig.axes[-4] # MicL is the first added in this gridspec
|
||||
top_ax = main_axes[0] # MicL
|
||||
top_ax.plot([0], [top_ax.get_ylim()[1]], marker="v", color="black",
|
||||
markersize=8, clip_on=False, zorder=10)
|
||||
|
||||
# "Sensor Check" caption under the strip (BW convention)
|
||||
if has_sc and sc_axes:
|
||||
pos = sc_axes[-1].get_position()
|
||||
fig.text((pos.x0 + pos.x1) / 2, pos.y0 - 0.012, "Sensor Check",
|
||||
fontsize=7, color="#555", ha="center", va="top")
|
||||
|
||||
# Compute scale-per-division for the footer (10 divs across the chart)
|
||||
# and find peak geo amplitude for the geo amp/div setting.
|
||||
total_s = times[-1] - times[0] if values else 0
|
||||
|
||||
+295
-20
@@ -108,6 +108,12 @@
|
||||
color: var(--text);
|
||||
}
|
||||
.btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
|
||||
.btn-danger { background: var(--red); color: #fff; }
|
||||
.btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
|
||||
.diag-result { display:block; margin-top:6px; font-size:12px; opacity:.85;
|
||||
white-space:pre-wrap; word-break:break-word; }
|
||||
.diag-result.ok { color: var(--green); }
|
||||
.diag-result.error { color: var(--red); }
|
||||
.btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
|
||||
|
||||
/* #connect-btn styles moved to #live-connect-bar block */
|
||||
@@ -910,6 +916,7 @@
|
||||
<button class="tab-btn" data-tab="events" onclick="switchTab('events')">Events</button>
|
||||
<button class="tab-btn" data-tab="config" onclick="switchTab('config')">Config</button>
|
||||
<button class="tab-btn" data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
|
||||
<button class="tab-btn" data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
|
||||
</div>
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
@@ -938,6 +945,10 @@
|
||||
<div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
|
||||
|
||||
<div class="event-toolbar">
|
||||
<button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
|
||||
title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
|
||||
⟳ Load events
|
||||
</button>
|
||||
<button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
|
||||
<button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
|
||||
title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
|
||||
@@ -1205,6 +1216,77 @@
|
||||
|
||||
</div><!-- end #tab-call-home -->
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
TAB: Diagnostics
|
||||
═══════════════════════════════════════════════════════════════════ -->
|
||||
<div id="tab-diagnostics" class="tab-pane">
|
||||
|
||||
<div class="cfg-grid">
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Device State</div>
|
||||
<div class="hint" style="margin-bottom:10px">
|
||||
Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
|
||||
</div>
|
||||
<div class="dev-table" id="diag-table"></div>
|
||||
<div class="cfg-actions" style="margin-top:12px">
|
||||
<button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
|
||||
<span id="diag-status"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Actions</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Stop Monitoring</label>
|
||||
<button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
|
||||
<div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
|
||||
<span class="diag-result" id="diag-stop-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Disable Auto Call Home</label>
|
||||
<button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
|
||||
<div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
|
||||
<span class="diag-result" id="diag-ach-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Erase All Events</label>
|
||||
<input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
|
||||
oninput="diagCheckEraseConfirm()" autocomplete="off" />
|
||||
<button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
|
||||
<div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
|
||||
<span class="diag-result" id="diag-erase-result"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="cfg-section">
|
||||
<div class="cfg-section-title">Unresponsive Unit</div>
|
||||
<div class="hint" style="margin-bottom:10px">
|
||||
The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
|
||||
to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
|
||||
<code>ach_server</code> and answer its call — before racing it with these.
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
|
||||
<button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
|
||||
<div class="hint">Success is <code>bytes_received > 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
|
||||
<span class="diag-result" id="diag-drip-result"></span>
|
||||
</div>
|
||||
|
||||
<div class="cfg-field">
|
||||
<label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
|
||||
<button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
|
||||
<span class="diag-result" id="diag-blind-result"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div><!-- end #tab-diagnostics -->
|
||||
|
||||
</div><!-- end #section-live -->
|
||||
|
||||
<!-- ════════════════════════════════════════════════════════════════
|
||||
@@ -1361,6 +1443,8 @@
|
||||
// ── State ──────────────────────────────────────────────────────────────────────
|
||||
let unitInfo = null;
|
||||
let eventList = [];
|
||||
let storageInfo = null; // /device/events/storage_range — cheap, read on connect
|
||||
let eventsLoaded = false; // the event chain walk is opt-in; see loadEventList()
|
||||
let currentEvent = 0;
|
||||
let charts = {};
|
||||
let geoAdcScale = 6.206;
|
||||
@@ -1458,6 +1542,7 @@ function switchTab(name) {
|
||||
if (name === 'units') { if (!unitsLoaded) loadUnits(); }
|
||||
if (name === 'monlog') { if (!monlogLoaded) loadMonitorLog(); }
|
||||
if (name === 'sessions') { if (!sessLoaded) loadSessions(); }
|
||||
if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
|
||||
}
|
||||
|
||||
// ── Connect ────────────────────────────────────────────────────────────────────
|
||||
@@ -1478,18 +1563,13 @@ async function connectUnit() {
|
||||
btn.disabled = false; btn.textContent = 'Connect'; return;
|
||||
}
|
||||
|
||||
setStatus('Fetching event list…', 'loading');
|
||||
try {
|
||||
const r = await fetch(`${api()}/device/events?${deviceParams()}`);
|
||||
if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
|
||||
const evData = await r.json();
|
||||
eventList = evData.events || [];
|
||||
// Merge compliance from /device/events response (it re-reads it)
|
||||
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
|
||||
} catch (e) {
|
||||
setStatus(`Event fetch failed: ${e.message}`, 'error');
|
||||
btn.disabled = false; btn.textContent = 'Reconnect'; return;
|
||||
}
|
||||
// Connecting deliberately does NOT walk the event chain. That walk reads
|
||||
// every event header over the cellular link and can take minutes — or fail
|
||||
// outright on a unit whose buffer has wrapped past 0xFFFF. Use the ~2 s
|
||||
// probes instead; the event list is opt-in via loadEventList().
|
||||
eventList = []; eventsLoaded = false;
|
||||
setStatus('Reading device state…', 'loading');
|
||||
storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
|
||||
|
||||
populateDeviceBar();
|
||||
populateDeviceTab();
|
||||
@@ -1498,11 +1578,9 @@ async function connectUnit() {
|
||||
|
||||
document.getElementById('device-bar').style.display = 'flex';
|
||||
document.getElementById('monitor-panel').style.display = 'flex';
|
||||
document.getElementById('load-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('save-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('download-btn').disabled = eventList.length === 0;
|
||||
document.getElementById('prev-btn').disabled = true;
|
||||
document.getElementById('next-btn').disabled = eventList.length <= 1;
|
||||
setEventButtonsEnabled();
|
||||
document.getElementById('load-events-btn').disabled = false;
|
||||
setDiagButtonsEnabled(true);
|
||||
document.getElementById('cfg-read-btn').disabled = false;
|
||||
document.getElementById('cfg-write-btn').disabled = false;
|
||||
document.getElementById('ch-read-btn').disabled = false;
|
||||
@@ -1510,7 +1588,9 @@ async function connectUnit() {
|
||||
|
||||
btn.disabled = false; btn.textContent = 'Reconnect';
|
||||
|
||||
setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
|
||||
setStatus(storageInfo && storageInfo.is_empty
|
||||
? 'Connected — no events stored.'
|
||||
: 'Connected. Event list not loaded (Events → Load events).', 'ok');
|
||||
|
||||
// Fetch monitor status in background (non-blocking)
|
||||
refreshMonitorStatus().catch(() => {});
|
||||
@@ -1522,6 +1602,48 @@ async function connectUnit() {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Shared fetch helper ────────────────────────────────────────────────────────
|
||||
async function fetchJson(path, opts) {
|
||||
const sep = path.includes('?') ? '&' : '?';
|
||||
const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
|
||||
const body = await r.json().catch(() => ({}));
|
||||
if (!r.ok) throw new Error(body.detail || r.statusText);
|
||||
return body;
|
||||
}
|
||||
|
||||
function setEventButtonsEnabled() {
|
||||
const n = eventList.length;
|
||||
document.getElementById('load-btn').disabled = n === 0;
|
||||
document.getElementById('save-btn').disabled = n === 0;
|
||||
document.getElementById('download-btn').disabled = n === 0;
|
||||
document.getElementById('prev-btn').disabled = true;
|
||||
document.getElementById('next-btn').disabled = n <= 1;
|
||||
}
|
||||
|
||||
// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
|
||||
async function loadEventList() {
|
||||
if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
|
||||
const btn = document.getElementById('load-events-btn');
|
||||
btn.disabled = true;
|
||||
setStatus('Walking the event chain — this can take a while…', 'loading');
|
||||
try {
|
||||
const evData = await fetchJson('/device/events');
|
||||
eventList = evData.events || [];
|
||||
eventsLoaded = true;
|
||||
// /device/events re-reads compliance; fold it in.
|
||||
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
|
||||
} catch (e) {
|
||||
setStatus(`Event fetch failed: ${e.message}`, 'error');
|
||||
btn.disabled = false; return;
|
||||
}
|
||||
populateDeviceBar();
|
||||
populateDeviceTab();
|
||||
populateEventChips();
|
||||
setEventButtonsEnabled();
|
||||
btn.disabled = false;
|
||||
setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
|
||||
}
|
||||
|
||||
// ── Device bar ─────────────────────────────────────────────────────────────────
|
||||
function populateDeviceBar() {
|
||||
qs('di-serial').textContent = unitInfo.serial || '—';
|
||||
@@ -1530,7 +1652,7 @@ function populateDeviceBar() {
|
||||
qs('di-sr').textContent = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
|
||||
qs('di-rt').textContent = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
|
||||
qs('di-trig').textContent = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
|
||||
qs('di-count').textContent = eventList.length;
|
||||
qs('di-count').textContent = eventsLoaded ? eventList.length : '—';
|
||||
qs('di-project').textContent = cc.project || '—';
|
||||
qs('di-client').textContent = cc.client || '—';
|
||||
qs('di-operator').textContent = cc.operator || '—';
|
||||
@@ -1660,7 +1782,8 @@ function populateDeviceTab() {
|
||||
{ label:'DSP', value: unitInfo.dsp_version || '—' },
|
||||
{ label:'Model', value: unitInfo.model || '—' },
|
||||
{ label:'Manufacturer', value: unitInfo.manufacturer || '—' },
|
||||
{ label:'Stored Events', value: eventList.length },
|
||||
{ label:'Stored Events', value: eventsLoaded ? eventList.length : 'not loaded' },
|
||||
{ label:'Storage Used', value: storageUsedLabel() },
|
||||
];
|
||||
for (const {label, value} of cardData) {
|
||||
const c = document.createElement('div');
|
||||
@@ -1707,6 +1830,158 @@ function renderTable(id, rows) {
|
||||
}
|
||||
}
|
||||
|
||||
// ── Diagnostics ────────────────────────────────────────────────────────────────
|
||||
// Everything here is a cheap probe (POLL + one read) or a single write. None of
|
||||
// it walks the event chain. See docs/runbooks/wedged_unit_recovery.md.
|
||||
|
||||
function storageUsedLabel() {
|
||||
if (!storageInfo) return '—';
|
||||
if (storageInfo.is_empty) return 'empty';
|
||||
const f = storageInfo.first_key, l = storageInfo.last_key;
|
||||
return (f && l) ? `${f} → ${l}` : '—';
|
||||
}
|
||||
|
||||
function setDiagButtonsEnabled(on) {
|
||||
for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
|
||||
'diag-drip-btn','diag-blind-btn']) {
|
||||
const el = document.getElementById(id);
|
||||
if (el) el.disabled = !on;
|
||||
}
|
||||
diagCheckEraseConfirm();
|
||||
}
|
||||
|
||||
// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
|
||||
function diagCheckEraseConfirm() {
|
||||
const box = document.getElementById('diag-erase-confirm');
|
||||
const btn = document.getElementById('diag-erase-btn');
|
||||
if (!box || !btn) return;
|
||||
const serial = (unitInfo && unitInfo.serial) || '';
|
||||
btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
|
||||
}
|
||||
|
||||
function diagResult(id, text, cls) {
|
||||
const el = document.getElementById(id);
|
||||
if (!el) return;
|
||||
el.textContent = text;
|
||||
el.className = 'diag-result' + (cls ? ' ' + cls : '');
|
||||
}
|
||||
|
||||
async function refreshDiagnostics() {
|
||||
if (!devHost()) return;
|
||||
const st = document.getElementById('diag-status');
|
||||
if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
|
||||
|
||||
const [mon, store, idx] = await Promise.all([
|
||||
fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
|
||||
fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
|
||||
fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
|
||||
]);
|
||||
|
||||
if (!store._err) storageInfo = store;
|
||||
|
||||
const err = v => `<span style="color:var(--red)">${v}</span>`;
|
||||
const rows = [];
|
||||
|
||||
rows.push(['Monitoring', mon._err ? err(mon._err)
|
||||
: (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
|
||||
if (!mon._err) {
|
||||
rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
|
||||
if (mon.memory_total_bytes) {
|
||||
const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
|
||||
const pct = (used / mon.memory_total_bytes * 100).toFixed(1);
|
||||
rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
|
||||
}
|
||||
}
|
||||
|
||||
rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
|
||||
if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
|
||||
|
||||
// SUB 0x08. Known to report 0 on units with years of history — suspected
|
||||
// field-offset bug in the decode, so show it but do not trust it.
|
||||
rows.push(['Lifetime events', idx._err ? err(idx._err)
|
||||
: `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
|
||||
|
||||
renderTable('diag-table', rows);
|
||||
populateDeviceTab();
|
||||
if (st) { st.textContent = ''; st.className = ''; }
|
||||
}
|
||||
|
||||
async function diagStopMonitoring() {
|
||||
const btn = document.getElementById('diag-stop-btn');
|
||||
btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
|
||||
try {
|
||||
await fetchJson('/device/monitor/stop', { method: 'POST' });
|
||||
diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
|
||||
refreshDiagnostics();
|
||||
} catch (e) {
|
||||
diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagDisableAch() {
|
||||
const btn = document.getElementById('diag-ach-btn');
|
||||
btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
|
||||
try {
|
||||
const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
|
||||
const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
|
||||
diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
|
||||
} catch (e) {
|
||||
diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagEraseEvents() {
|
||||
const serial = (unitInfo && unitInfo.serial) || 'this unit';
|
||||
if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
|
||||
const btn = document.getElementById('diag-erase-btn');
|
||||
btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
|
||||
try {
|
||||
await fetchJson('/device/events/erase', { method: 'POST' });
|
||||
diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
|
||||
document.getElementById('diag-erase-confirm').value = '';
|
||||
eventList = []; eventsLoaded = false;
|
||||
setEventButtonsEnabled(); populateEventChips();
|
||||
refreshDiagnostics();
|
||||
} catch (e) {
|
||||
diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
diagCheckEraseConfirm();
|
||||
}
|
||||
|
||||
async function diagSlowDrip() {
|
||||
const btn = document.getElementById('diag-drip-btn');
|
||||
btn.disabled = true;
|
||||
diagResult('diag-drip-result', 'Holding a session for 120 s…');
|
||||
try {
|
||||
const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
|
||||
{ method: 'POST' });
|
||||
const good = (r.bytes_received || 0) > 0;
|
||||
diagResult('diag-drip-result',
|
||||
`drips ${r.drips_sent} · held ${r.duration_s}s · bytes back ${r.bytes_received}` +
|
||||
(r.send_error ? ` · ${r.send_error}` : '') +
|
||||
(good ? ' → device responded' : ' → no response; the modem may not be bridging'),
|
||||
good ? 'ok' : 'error');
|
||||
} catch (e) {
|
||||
diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
async function diagBlindStop() {
|
||||
const btn = document.getElementById('diag-blind-btn');
|
||||
btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
|
||||
try {
|
||||
const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
|
||||
diagResult('diag-blind-result',
|
||||
`Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
|
||||
} catch (e) {
|
||||
diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
|
||||
}
|
||||
btn.disabled = false;
|
||||
}
|
||||
|
||||
// ── Config form ────────────────────────────────────────────────────────────────
|
||||
function populateConfigFromDeviceInfo() {
|
||||
if (!unitInfo) return;
|
||||
|
||||
@@ -662,6 +662,11 @@ class WaveformStore:
|
||||
ev.raw_samples = idf_samples
|
||||
n_samples = max((len(idf_samples.get(ch, [])) for ch in ("Tran", "Vert", "Long", "MicL")), default=0)
|
||||
ev.total_samples = ev.total_samples or n_samples
|
||||
# Sensor self-check traces from the IDFW fixed header (waveform
|
||||
# events only; {} on histograms / when absent). Carried on the
|
||||
# bridged Event so the .h5 writer persists them like series-3.
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
ev.sensor_check = decode_idf_sensor_check(idf_bytes) or None
|
||||
|
||||
# For IDFH histograms there are no per-sample waveform arrays — the
|
||||
# device stores one peak ADC count per interval per channel. Synthesise
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,35 @@
|
||||
"""USBM/OSMRE compliance curve + scatter logic (sfm.compliance).
|
||||
Rendering is verified visually against Blastware reports."""
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
|
||||
from sfm.compliance import limit_at, channel_compliance_points
|
||||
|
||||
|
||||
def test_osmre_velocity_segments():
|
||||
assert abs(limit_at(6.0) - 0.75) < 1e-9 # 3.5–12 Hz flat
|
||||
assert abs(limit_at(50.0) - 2.00) < 1e-9 # 30–100 Hz flat
|
||||
|
||||
|
||||
def test_displacement_segments():
|
||||
assert abs(limit_at(2.0) - 2 * math.pi * 2.0 * 0.030) < 1e-9 # low-freq 0.030 in
|
||||
assert abs(limit_at(20.0) - 2 * math.pi * 20.0 * 0.008) < 1e-9 # rising diagonal 0.008 in
|
||||
|
||||
|
||||
def test_limit_clamps_below_1hz():
|
||||
assert limit_at(0.1) == limit_at(1.0)
|
||||
|
||||
|
||||
def test_scatter_ceiling_is_ppv_at_dominant_freq():
|
||||
# ~27 Hz blast-like trace whose energy peaks mid-record (inside full cycles,
|
||||
# as a real event does): the scatter cloud's ceiling is the trace PPV and the
|
||||
# top point sits near the dominant frequency.
|
||||
sps, n = 1024.0, 3328
|
||||
t = np.arange(n) / sps
|
||||
env = np.exp(-((t - 1.5) ** 2) / (2 * 0.3 ** 2))
|
||||
x = 0.9 * env * np.sin(2 * np.pi * 27.0 * t)
|
||||
f, v = channel_compliance_points(x, sps)
|
||||
assert len(f) > 20
|
||||
assert v.max() >= 0.99 * np.abs(x).max()
|
||||
assert 20.0 < f[int(np.argmax(v))] < 35.0
|
||||
@@ -0,0 +1,71 @@
|
||||
"""The event .h5 carries the sensor self-check traces (schema v2).
|
||||
|
||||
The sensor check is decoded by the per-series decoder and attached to the
|
||||
standardized Event, so the .h5 writer persists it device-agnostically and SFM
|
||||
reads it back without knowing which instrument produced it. Old v1 files (no
|
||||
sensor_check group) must still read cleanly.
|
||||
"""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from minimateplus.models import Event
|
||||
from minimateplus.event_file_io import read_blastware_file
|
||||
from sfm import event_hdf5
|
||||
|
||||
S3_FIX = Path(__file__).parent / "fixtures" / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W"
|
||||
|
||||
|
||||
def _write(ev, **kw):
|
||||
d = Path(tempfile.mkdtemp())
|
||||
p = d / "e.h5"
|
||||
event_hdf5.write_event_hdf5(p, ev, serial="BE12844", **kw)
|
||||
return p
|
||||
|
||||
|
||||
def test_sensor_check_roundtrips_through_hdf5():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, -3], "Vert": [0, 1], "Long": [2], "MicL": [5, -5]}
|
||||
ev.sample_rate = 1024
|
||||
sc = {"Tran": [0, -990, -500, -100], "Vert": [0, -980, -480],
|
||||
"Long": [0, -986, -470], "MicL": [0, -1800, 1800, -1800]}
|
||||
ev.sensor_check = sc
|
||||
|
||||
r = event_hdf5.read_event_hdf5(_write(ev))
|
||||
assert r["schema_version"] == 2
|
||||
assert set(r["sensor_check"]) == {"Tran", "Vert", "Long", "MicL"}
|
||||
for ch, vals in sc.items():
|
||||
assert r["sensor_check"][ch].tolist() == vals
|
||||
|
||||
|
||||
def test_plot_json_carries_sensor_check():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, 3]}
|
||||
ev.sample_rate = 1024
|
||||
ev.sensor_check = {"Tran": [0, -990, -500], "Vert": [0, -980],
|
||||
"Long": [0, -986]} # 3-channel: no MicL
|
||||
pj = event_hdf5.plot_json_from_hdf5(_write(ev))
|
||||
assert pj["sensor_check"] is not None
|
||||
assert "MicL" not in pj["sensor_check"]
|
||||
assert pj["sensor_check"]["Tran"] == [0, -990, -500]
|
||||
|
||||
|
||||
def test_event_without_sensor_check_still_reads_as_v2():
|
||||
ev = Event(index=0)
|
||||
ev.raw_samples = {"Tran": [1, 2, 3]}
|
||||
ev.sample_rate = 1024
|
||||
r = event_hdf5.read_event_hdf5(_write(ev))
|
||||
assert r["schema_version"] == 2
|
||||
assert r["sensor_check"] is None
|
||||
assert event_hdf5.plot_json_from_hdf5(_write(ev))["sensor_check"] is None
|
||||
|
||||
|
||||
def test_series3_decode_populates_event_sensor_check():
|
||||
# The real series-3 decoder attaches the traces to the Event, so the
|
||||
# ingest/backfill .h5 write picks them up with no extra plumbing.
|
||||
ev = read_blastware_file(S3_FIX)
|
||||
assert ev.sensor_check is not None
|
||||
assert set(ev.sensor_check) == {"Tran", "Vert", "Long", "MicL"}
|
||||
tran = np.asarray(ev.sensor_check["Tran"], dtype=float)
|
||||
assert tran.min() < -800 # the geophone ring-down deflection
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Blastware sensor self-check waveform decode (minimateplus.sensor_check).
|
||||
|
||||
Reverse-engineered 2026-09-15 against 7 BE12844 (MiniMate Plus) oracle events.
|
||||
After the main waveform record-chain and the trailing metadata / per-channel
|
||||
calibration records, a series-3 binary carries four length-prefixed records
|
||||
tagged 0x3c-0x3f: the sensor self-check traces the unit records when it pulses
|
||||
each sensor before monitoring (Blastware draws these as the little waveforms in
|
||||
the "Sensor Check" strip on the right of the Event Report).
|
||||
|
||||
* 0x3c / 0x3d / 0x3e = Tran / Vert / Long geophone ring-downs.
|
||||
* 0x3f = MicL, a pulse train at the mic self-test frequency.
|
||||
|
||||
The self-check injects a fixed pulse, so the response is near-identical across
|
||||
events — asserted here as an invariant shape (damped one-sided ring-down for
|
||||
the geophones, a multi-pulse train for the mic).
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from minimateplus.sensor_check import decode_sensor_check
|
||||
|
||||
FIXDIR = Path(__file__).parent / "fixtures" / "fft-oracle-2026-09-14"
|
||||
EVENTS = sorted(p.name for p in FIXDIR.iterdir()) # 7 BE12844 event binaries
|
||||
|
||||
|
||||
def _decode(name):
|
||||
return decode_sensor_check((FIXDIR / name).read_bytes())
|
||||
|
||||
|
||||
def test_all_four_channels_present():
|
||||
for name in EVENTS:
|
||||
sc = _decode(name)
|
||||
assert set(sc) == {"Tran", "Vert", "Long", "MicL"}, name
|
||||
|
||||
|
||||
def test_geo_channels_are_damped_ringdowns():
|
||||
# Each geophone self-check is a large one-sided deflection (~-990 raw) that
|
||||
# rings back and damps toward a settled value well above the trough.
|
||||
for name in EVENTS:
|
||||
sc = _decode(name)
|
||||
for ch in ("Tran", "Vert", "Long"):
|
||||
tr = np.asarray(sc[ch], dtype=float)
|
||||
assert 240 <= len(tr) <= 260, f"{name}:{ch} n={len(tr)}"
|
||||
assert abs(tr[:3].mean()) < 50, f"{name}:{ch} starts off-baseline"
|
||||
assert tr.min() < -800, f"{name}:{ch} min {tr.min()}"
|
||||
assert tr.max() < 60, f"{name}:{ch} unexpected positive swing {tr.max()}"
|
||||
# damped: settles between the trough and zero, well above the trough
|
||||
assert tr.min() < tr[-1] < 0, f"{name}:{ch} end {tr[-1]} not between trough and 0"
|
||||
assert abs(tr[-1]) < 0.6 * abs(tr.min()), f"{name}:{ch} not damped, end {tr[-1]}"
|
||||
|
||||
|
||||
def test_mic_channel_is_a_pulse_train():
|
||||
for name in EVENTS:
|
||||
tr = np.asarray(_decode(name)["MicL"], dtype=float)
|
||||
assert 235 <= len(tr) <= 255, f"{name} mic n={len(tr)}"
|
||||
# larger dynamic range than the geo ring-down, and swings both ways
|
||||
assert tr.min() < -1500, f"{name} mic min {tr.min()}"
|
||||
assert tr.max() > 100, f"{name} mic max {tr.max()}"
|
||||
# multiple pulses: several deep local minima
|
||||
deep = (tr[1:-1] < tr[:-2]) & (tr[1:-1] < tr[2:]) & (tr[1:-1] < -800)
|
||||
assert int(deep.sum()) >= 4, f"{name} mic pulses {int(deep.sum())}"
|
||||
|
||||
|
||||
def test_returns_empty_when_no_sensor_check_block():
|
||||
assert decode_sensor_check(b"not a blastware file") == {}
|
||||
assert decode_sensor_check(b"") == {}
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Series-4 (Thor / Micromate IDFW) sensor self-check waveform decode.
|
||||
|
||||
Reverse-engineered 2026-09-15 against 4 UM (Thor) oracle events. The IDFW
|
||||
binary carries the sensor self-check in its fixed-header region (before the
|
||||
waveform body) as up to four records tagged ``01 0e 3c/3d/3e/3f`` — the SAME
|
||||
channel ids as series-3 (Tran/Vert/Long/MicL). Unlike series-3's delta-coded
|
||||
trailing block, series-4 stores each trace as a raw int16-BE array after an
|
||||
18-byte record header whose sample count is a 2-byte field at offset +8.
|
||||
|
||||
Three-channel (mic-disabled) Thor units carry only 3c/3d/3e — no MicL record.
|
||||
|
||||
Validated by shape (geophone ring-down / mic pulse train) and cross-event
|
||||
consistency, since there's no Thor Event-Report strip to exact-match against.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from micromate.sensor_check import decode_idf_sensor_check
|
||||
|
||||
FIXDIR = Path(__file__).parent / "fixtures" / "thor-idf-sc"
|
||||
EVENTS = sorted(p.name for p in FIXDIR.glob("*.IDFW"))
|
||||
|
||||
|
||||
def _decode(name):
|
||||
return decode_idf_sensor_check((FIXDIR / name).read_bytes())
|
||||
|
||||
|
||||
def test_geo_channels_present_and_ringdown_shaped():
|
||||
# Every IDFW event has the three geophone self-checks; each is a large
|
||||
# one-sided deflection (~15000 raw counts) that rings back — the geophone's
|
||||
# damped impulse response.
|
||||
for name in EVENTS:
|
||||
sc = _decode(name)
|
||||
for ch in ("Tran", "Vert", "Long"):
|
||||
assert ch in sc, f"{name} missing {ch}"
|
||||
tr = np.asarray(sc[ch], dtype=float)
|
||||
tr = tr - tr[:4].mean() # reference to the pre-trigger baseline
|
||||
assert 40 <= len(tr) <= 300, f"{name}:{ch} n={len(tr)}"
|
||||
assert tr.min() < -8000, f"{name}:{ch} min {tr.min()}"
|
||||
# deflects one way and rings back toward / past the baseline
|
||||
assert tr.max() < abs(tr.min()), f"{name}:{ch} not one-sided"
|
||||
|
||||
|
||||
def test_mic_present_only_on_four_channel_units():
|
||||
# UM11719 / UM12947 record a mic; UM13981 / UM20147 are 3-channel
|
||||
# (mic-disabled) units and carry no MicL self-check.
|
||||
got = {name: ("MicL" in _decode(name)) for name in EVENTS}
|
||||
assert any(got.values()), "expected at least one 4-channel unit"
|
||||
assert not all(got.values()), "expected at least one 3-channel unit"
|
||||
for name, has_mic in got.items():
|
||||
if has_mic:
|
||||
tr = np.asarray(_decode(name)["MicL"], dtype=float)
|
||||
tr = tr - tr[:4].mean()
|
||||
# mic self-check is a bipolar pulse train — swings both ways, wide range
|
||||
assert tr.max() > 5000 and tr.min() < -5000, f"{name} mic not bipolar"
|
||||
|
||||
|
||||
def test_channel_ids_and_order():
|
||||
# ids decode to the canonical channel names, geo always in Tran/Vert/Long order
|
||||
sc = _decode(EVENTS[0])
|
||||
assert [c for c in ("Tran", "Vert", "Long") if c in sc] == ["Tran", "Vert", "Long"]
|
||||
|
||||
|
||||
def test_returns_empty_on_non_idf_input():
|
||||
assert decode_idf_sensor_check(b"not an IDF file") == {}
|
||||
assert decode_idf_sensor_check(b"") == {}
|
||||
Reference in New Issue
Block a user