Compare commits
5
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
154186a6cd | ||
|
|
1765b3300d | ||
|
|
1e76d08b37 | ||
|
|
a84a46e9d4 | ||
|
|
ada5bc2a82 |
@@ -4,6 +4,68 @@ All notable changes to seismo-relay are documented here.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Waveform event times were the monitoring-session start, not the trigger
|
||||||
|
(~hours off).** `read_blastware_file` stamped events with footer `ts1`, which
|
||||||
|
for a waveform is the session start a unit shares across every event that day
|
||||||
|
(a unit arming at 06:00 stamped 06:00 on all of them — the modal and PDF both
|
||||||
|
showed it, since it's the stored value). The event time is footer `ts2` (the
|
||||||
|
recording stop), and Blastware's trigger = `ts2 - record time`. The record
|
||||||
|
time is a big-endian float32 in the recording-setup config block (30 bytes
|
||||||
|
before the `Standard Recording Setup` marker), so the **exact trigger is now
|
||||||
|
recovered from the binary alone** — all 7 BE12844 oracle events decode to
|
||||||
|
their exact Blastware time (e.g. N844LQHB 10:33:29), no paired `.TXT` needed.
|
||||||
|
Histograms keep `ts1` (the ~24 h window start). A paired report's
|
||||||
|
`event_datetime` stays authoritative (unit-clock drift).
|
||||||
|
⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Diagnostics tab in the SFM standalone webapp.** Surfaces the device
|
||||||
|
endpoints that previously existed only as `curl`: `events/storage_range` and
|
||||||
|
`events/index` alongside `monitor/status`, then stop monitoring, disable ACH
|
||||||
|
(`rescue?erase=false`, so stored events survive), and erase. The wedged-unit
|
||||||
|
ladder — slow drip and blind stop — sits under its own heading pointing at
|
||||||
|
`docs/runbooks/wedged_unit_recovery.md`, with the reminder that `slow_drip`'s
|
||||||
|
success signal is `bytes_received > 0` and not a clean duration. Erase is
|
||||||
|
guarded by typing the unit's serial: auth answers *who*, not *did you mean
|
||||||
|
it*, and Swagger's try-it-out button on `/device/events/erase` is live on
|
||||||
|
`:8200/docs`.
|
||||||
|
|
||||||
|
- **`docs/sfm_tool_status.md`** — an honest per-capability maturity assessment:
|
||||||
|
what is production-grade (the codec library, the data side), what is
|
||||||
|
emergency-grade (the device side), what is a research artifact, the
|
||||||
|
known-issues table, and the gap to a real tool. Also records the **5A
|
||||||
|
page-boundary bug** as known: `parse_strt_end_offset()` discards the key's
|
||||||
|
page byte, so once a unit has recorded more than 64 KB since its last erase,
|
||||||
|
an event spanning the boundary reads an `end_offset` *behind* its own start —
|
||||||
|
the chunk loop fetches nothing and TERM packs a negative `offset_word`, which
|
||||||
|
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
|
||||||
|
files via the watcher path and never runs this walk.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Connecting to a unit no longer walks its event chain.** `/device/events`
|
||||||
|
reads every event header over the cellular link; on a unit with a large or
|
||||||
|
wrapped chain that takes minutes or fails outright, and it fired
|
||||||
|
automatically on every connect. Connect now uses only ~2 s probes —
|
||||||
|
`/device/info` (which already carried the compliance config the walk was
|
||||||
|
re-reading) plus `events/storage_range` — and the Device tab gains an Event
|
||||||
|
Chain card. The walk moved behind a **Load events** button in the Events
|
||||||
|
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
|
||||||
|
event it has stored.
|
||||||
|
|
||||||
|
### Migration
|
||||||
|
|
||||||
|
**None.** Frontend and documentation only — no codec, waveform-store or DB
|
||||||
|
change, no schema change, and no `TOOL_VERSION` bump. The webapp is served
|
||||||
|
from the image, so the change appears after the next `sfm` rebuild.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## v0.31.0 — 2026-09-18
|
## v0.31.0 — 2026-09-18
|
||||||
|
|
||||||
**Report parity, and a second way to rescue a runaway unit.** Two threads.
|
**Report parity, and a second way to rescue a runaway unit.** Two threads.
|
||||||
|
|||||||
@@ -296,6 +296,16 @@ def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
|
|||||||
event.sample_rate = report.sample_rate_sps
|
event.sample_rate = report.sample_rate_sps
|
||||||
if report.record_time_s is not None:
|
if report.record_time_s is not None:
|
||||||
event.rectime_seconds = report.record_time_s
|
event.rectime_seconds = report.record_time_s
|
||||||
|
# The report's event_datetime is Blastware's exact trigger time (parsed
|
||||||
|
# from Event Time + Event Date). Prefer it over the binary footer's stop
|
||||||
|
# time so a report-paired import matches BW to the second.
|
||||||
|
edt = report.event_datetime
|
||||||
|
if edt is not None:
|
||||||
|
event.timestamp = Timestamp(
|
||||||
|
raw=b"", flag=0x10,
|
||||||
|
year=edt.year, unknown_byte=0, month=edt.month, day=edt.day,
|
||||||
|
hour=edt.hour, minute=edt.minute, second=edt.second,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
|
def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
|
||||||
@@ -808,6 +818,30 @@ def derive_record_type_from_filename(filename, default: str = "Waveform") -> str
|
|||||||
return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
|
return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
|
||||||
|
|
||||||
|
|
||||||
|
# Marker for the recording-setup config block, and the offset of the record-time
|
||||||
|
# float32 within it. The configured post-trigger record time (seconds) is a
|
||||||
|
# big-endian float32 exactly 30 bytes before the "Standard Recording Setup"
|
||||||
|
# label. Verified across the corpus reading 1.0 / 2.0 / 3.0 s on different
|
||||||
|
# setups — and ts2 - record_time reproduces Blastware's trigger to the second
|
||||||
|
# (N844LQHB: stop 10:33:32 - 3.0 = 10:33:29).
|
||||||
|
_RECSETUP_MARKER = b"Standard Recording Setup"
|
||||||
|
_RECTIME_OFFSET_BEFORE_MARKER = 30
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_record_time_seconds(raw: bytes) -> Optional[float]:
|
||||||
|
"""The configured post-trigger record time in seconds, from the recording-
|
||||||
|
setup config block, or None when absent / implausible."""
|
||||||
|
a = raw.find(_RECSETUP_MARKER)
|
||||||
|
if a < _RECTIME_OFFSET_BEFORE_MARKER:
|
||||||
|
return None
|
||||||
|
off = a - _RECTIME_OFFSET_BEFORE_MARKER
|
||||||
|
try:
|
||||||
|
rt = struct.unpack(">f", raw[off:off + 4])[0]
|
||||||
|
except struct.error:
|
||||||
|
return None
|
||||||
|
return rt if 0.05 <= rt <= 600.0 else None
|
||||||
|
|
||||||
|
|
||||||
def read_blastware_file(path: Union[str, Path]) -> Event:
|
def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||||
"""
|
"""
|
||||||
Parse a Blastware waveform file into an Event.
|
Parse a Blastware waveform file into an Event.
|
||||||
@@ -917,6 +951,10 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
|||||||
# rest of the event (timestamp, waveform_key, project strings) is
|
# rest of the event (timestamp, waveform_key, project strings) is
|
||||||
# still recoverable and useful.
|
# still recoverable and useful.
|
||||||
decoded = decode_waveform_v2(body)
|
decoded = decode_waveform_v2(body)
|
||||||
|
# Discriminator for the timestamp logic below: a waveform (trigger) event
|
||||||
|
# vs a histogram window. Keyed on the codec, not the filename — the
|
||||||
|
# save_imported_bw path passes a tmp ".bw" name whose extension lies.
|
||||||
|
is_waveform_body = decoded is not None
|
||||||
if decoded is None:
|
if decoded is None:
|
||||||
decoded = decode_histogram_body(body)
|
decoded = decode_histogram_body(body)
|
||||||
if decoded is None:
|
if decoded is None:
|
||||||
@@ -948,7 +986,31 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
|||||||
ev.total_samples = strt_fields.get("total_samples")
|
ev.total_samples = strt_fields.get("total_samples")
|
||||||
ev.pretrig_samples = strt_fields.get("pretrig_samples")
|
ev.pretrig_samples = strt_fields.get("pretrig_samples")
|
||||||
|
|
||||||
if ts1 is not None:
|
# Event timestamp. The footer's two timestamps mean different things by
|
||||||
|
# record type:
|
||||||
|
# * Waveform: ts1 = the monitoring-SESSION start (shared across every
|
||||||
|
# event that day — a unit arming at 06:00 stamps 06:00 on all of them),
|
||||||
|
# ts2 = THIS event's recording STOP. Blastware's Date/Time is the
|
||||||
|
# TRIGGER = ts2 - record time, and the record time is a float32 in the
|
||||||
|
# recording-setup config block (see _parse_record_time_seconds), so the
|
||||||
|
# exact trigger is recoverable from the binary alone. Falls back to ts2
|
||||||
|
# (the stop, within the record duration) if the config block is absent.
|
||||||
|
# (Stamping ts1 showed the session start, hours off.)
|
||||||
|
# * Histogram / undecodable: ts1 = the window start, which IS the event
|
||||||
|
# time — keep it.
|
||||||
|
# Discriminate by ``is_waveform_body`` (the codec), not the filename.
|
||||||
|
if is_waveform_body and ts2 is not None:
|
||||||
|
_stop = datetime.datetime(ts2.year, ts2.month, ts2.day,
|
||||||
|
ts2.hour, ts2.minute, ts2.second)
|
||||||
|
_rt = _parse_record_time_seconds(raw)
|
||||||
|
_trig = _stop - datetime.timedelta(seconds=_rt) if _rt is not None else _stop
|
||||||
|
ev.timestamp = Timestamp(
|
||||||
|
raw=footer[10:18],
|
||||||
|
flag=0x10,
|
||||||
|
year=_trig.year, unknown_byte=0, month=_trig.month, day=_trig.day,
|
||||||
|
hour=_trig.hour, minute=_trig.minute, second=_trig.second,
|
||||||
|
)
|
||||||
|
elif ts1 is not None:
|
||||||
ev.timestamp = Timestamp(
|
ev.timestamp = Timestamp(
|
||||||
raw=footer[2:10],
|
raw=footer[2:10],
|
||||||
flag=0x10,
|
flag=0x10,
|
||||||
|
|||||||
Vendored
BIN
Binary file not shown.
@@ -0,0 +1,54 @@
|
|||||||
|
"""Event timestamp decode — waveform trigger/stop vs histogram window start.
|
||||||
|
|
||||||
|
The Blastware footer holds two timestamps: ts1 = footer[2:10], ts2 = footer[10:18].
|
||||||
|
Their meaning depends on record type:
|
||||||
|
|
||||||
|
* Waveform: ts1 is the monitoring-SESSION start (e.g. 06:00 for a unit that
|
||||||
|
arms at 06:00 daily — shared across every event that day), and ts2 is THIS
|
||||||
|
event's recording STOP. read_blastware_file used to stamp events with ts1 →
|
||||||
|
every waveform showed the session start (~4.5 h off). Binary-only, the best
|
||||||
|
estimate is ts2 (the stop); the exact trigger BW displays (= ts2 - record
|
||||||
|
duration) comes from the paired report's event_datetime, since the binary
|
||||||
|
STRT record-time byte is a misparsed record-type marker.
|
||||||
|
* Histogram: ts1/ts2 are the ~24 h window [start, stop]; the event time is the
|
||||||
|
window start = ts1 (unchanged).
|
||||||
|
"""
|
||||||
|
import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from minimateplus.event_file_io import read_blastware_file, apply_report_to_event
|
||||||
|
from minimateplus.bw_ascii_report import BwAsciiReport
|
||||||
|
from minimateplus.models import Event
|
||||||
|
|
||||||
|
FIX = Path(__file__).parent / "fixtures"
|
||||||
|
WAVEFORM = FIX / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W" # footer ts2 = 2026-08-25 10:33:32
|
||||||
|
HISTOGRAM = FIX / "ts-fix" / "K441LKZU.C30H" # window start 2026-05-10 19:04:50
|
||||||
|
|
||||||
|
|
||||||
|
def _tuple(ts):
|
||||||
|
return (ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second)
|
||||||
|
|
||||||
|
|
||||||
|
def test_waveform_timestamp_is_exact_trigger_from_binary():
|
||||||
|
ev = read_blastware_file(WAVEFORM)
|
||||||
|
# The EXACT Blastware trigger, from the binary alone: ts2 (stop 10:33:32)
|
||||||
|
# minus the config record time (3.0 s) = 10:33:29 — NOT the 06:00:13
|
||||||
|
# monitoring-session start the old decode used.
|
||||||
|
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29), _tuple(ev.timestamp)
|
||||||
|
|
||||||
|
|
||||||
|
def test_histogram_timestamp_is_window_start_unchanged():
|
||||||
|
ev = read_blastware_file(HISTOGRAM)
|
||||||
|
# Histogram event time = the window start (ts1); must NOT get the waveform
|
||||||
|
# ts2 treatment (that would land ~24 h off).
|
||||||
|
assert _tuple(ev.timestamp) == (2026, 5, 10, 19, 4, 50), _tuple(ev.timestamp)
|
||||||
|
|
||||||
|
|
||||||
|
def test_report_event_datetime_is_authoritative_over_binary():
|
||||||
|
# The binary already yields the exact trigger, but a paired report stays
|
||||||
|
# authoritative (e.g. if the unit clock had drifted) — applying it wins.
|
||||||
|
ev = read_blastware_file(WAVEFORM)
|
||||||
|
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29) # exact, from binary
|
||||||
|
apply_report_to_event(ev, BwAsciiReport(
|
||||||
|
event_datetime=datetime.datetime(2026, 8, 25, 10, 35, 0)))
|
||||||
|
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 35, 0) # report wins
|
||||||
Reference in New Issue
Block a user