Author SHA1 Message Date
serversdownandClaude Opus 5 0a96a34b02 docs(offset): CONFIRMED — BE12599 is a geophone-assembly fault, not the recorder
The first mechanism this investigation has established physically rather than
inferred. Brian pulled BE12599 and bench-tested it; a geophone swap settles it.

Same recorder, twenty minutes apart: Vert overswing 2026.0 with its own
geophone, 3.4 with BE9888's known-good one. Frequency 2.2 -> 7.4 Hz. All three
channels Passed, no offsets, no self-triggering, and a clean damped impulse with
broadband content. The recorder is functional and merely out of calibration.

A bench control -- trigger forced by slapping the microphone, geophone untouched
-- shows Vert producing 0.19 in/s of bipolar wander with no mechanical input at
all. The fault is continuous, not episodic. All three channels are affected:
Tran and Long sit on a small DC while their coils still pass the swing test,
Vert has lost damping entirely. That is leakage on several contacts plus one
gone open, not three elements failing in sequence.

The consequence that reaches past this unit: an autozero adjusts the RECORDER's
zero reference and cannot fix a geophone. If most offsets are geophone-assembly
faults, a re-zero succeeds only on the minority where the recorder really is at
fault -- which is exactly the ~10% rate recorded in section 5 and unexplained
since August. Procedure change: swap the geophone first.

Corrects 8e's title, which called it a connector rather than a geophone: the
swap narrows the fault to the assembly, and element / cable / connector remain
open within it. And reweights the cause -- the geophone is buried, not in the
enclosure with the nest, and the unit sat ~6 ft from I-80 WB, so road salt now
outranks the mice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
2026-09-23 02:57:34 +00:00
serversdownandClaude Opus 4.8 154186a6cd Merge feat/event-timestamp-fix: exact waveform trigger time from the binary
read_blastware_file stamped waveforms with footer ts1 (the monitoring-session
start, hours off — vomit-list #3).  The event time is ts2 (recording stop) and
the trigger = ts2 - record time, a float32 in the recording-setup config block,
so the exact Blastware trigger is recovered from the binary alone (no .TXT).
Histograms keep ts1; a paired report's event_datetime stays authoritative.

Needs a re-decode backfill to correct existing stored events' timestamps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 23:13:19 +00:00
serversdownandClaude Opus 4.8 1765b3300d docs(changelog): waveform event-time fix (exact trigger from binary)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 22:32:13 +00:00
serversdownandClaude Opus 4.8 1e76d08b37 fix(decode): recover the exact waveform trigger from the binary (no .TXT)
Follow-up to the ts1→ts2 fix: get the trigger to the second from the binary
alone, instead of falling back to the stop time (~record-duration late) for
no-report events.

The configured post-trigger record time is a big-endian float32 in the
recording-setup config block, exactly 30 bytes before the "Standard Recording
Setup" marker.  _parse_record_time_seconds reads it; the waveform branch now
stamps trigger = ts2 - record_time.  Verified: the field reads 1.0 / 2.0 / 3.0 s
across different setups in the corpus, and all 7 BE12844 oracle events now
decode to their exact Blastware trigger (N844LQHB 10:33:29) from the binary,
no paired .TXT needed.  Falls back to ts2 (the stop) if the config block is
absent.  A paired report's event_datetime stays authoritative (clock drift).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 22:31:22 +00:00
serversdownandClaude Opus 4.8 a84a46e9d4 fix(decode): stamp waveform events with the event time, not the session start
read_blastware_file built ev.timestamp from footer ts1, which for a WAVEFORM is
the monitoring-session start (a unit arming at 06:00 stamps 06:00 on every event
that day) — so every waveform's time was hours off (vomit-list #3, "~4.5 h off").
The event time is footer ts2 (the recording stop); BW's displayed Date/Time is
the trigger = ts2 - record duration.

Root cause proven against the BE12844 oracle set: 5 of 7 events decoded to the
identical 06:00:13 (the shared session start); ts2 gives distinct plausible
event times (N844LQHB ts2 = 10:33:32, BW trigger 10:33:29 = ts2 - 3.0 s rectime).

  * read_blastware_file now uses ts2 for waveforms (discriminated by which codec
    decoded the body, not the filename — save_imported_bw passes a tmp name).
    Histograms keep ts1 (the ~24 h window start, which IS the event time).
  * Binary-only decode can't get the exact trigger: the STRT record-time byte is
    a misparsed record-type marker (0x46=70), so ts2 (the stop, ~record duration
    after the trigger) is the best estimate. A paired BW report carries the exact
    trigger — apply_report_to_event now overlays event.timestamp from
    report.event_datetime, matching the existing build-path override (line ~441).

Tests: waveform → ts2, histogram → ts1 unchanged, report → exact trigger.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 18:21:07 +00:00
serversdownandClaude Opus 5 ada5bc2a82 docs(changelog): Unreleased — cheap connect, Diagnostics tab, tool status
Written on dev as part of finishing the merge, per the convention adopted
2026-09-18: feature branches do not touch CHANGELOG.md, and the entry describes
what actually landed rather than what a branch intended.

First time through the new way rather than discovering the conflict afterward —
the merge was clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-20 17:12:49 +00:00
serversdownandClaude Opus 5 f1ab5b1e9d docs: record the 5A page-boundary bug, and assess SFM as a tool
Two things Brian asked for after the BE12599 work.

The known bug: the 5A walk discards the key's page byte, so once a unit has
recorded more than 64 KB since its last erase, an event spanning the boundary
reads an end_offset behind its own start. The chunk loop then fetches nothing
and TERM packs a negative offset_word, which is the 500. Reproduced on BE12599.
It hid this long because every capture the walk was verified against came from
a freshly-erased BE11529 — all three confirmed TERM examples sit inside page
0x11. Prod is unaffected; it ingests complete files and never runs this walk.

The status doc exists because "is SFM reliable?" has three different answers
depending on which tier is meant. The codec library and the data side are
production — verified per-sample at scale, carrying Terra-View daily. The
device side is emergency-grade: it works, but it is synchronous,
unauthenticated, and thinly tested. The lab is research artifacts. Most
confusion comes from answering for the wrong tier.

It covers all three of what Brian asked for: maturity per capability, an
operator-facing "what to use when" (the cheap probes are cheap and the event
walk is not), the known-issues table, and the gap analysis. That gap is mostly
auth, async and guardrails — not protocol work. The protocol is the finished
part.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-20 01:30:14 +00:00
serversdownandClaude Opus 5 6589da445b feat(webapp): cheap connect, opt-in event walk, and a Diagnostics tab
Connecting to a unit fired /device/events automatically, which walks the whole
event chain — every event header over a cellular link. On BE12599 that took
minutes and then 500'd outright, because its buffer has wrapped past 0xFFFF and
the uint16 offset arithmetic goes negative. Wanting to know whether ACH was on
should not require reading every event the unit has stored.

Connect now uses only cheap probes: /device/info (which already carries the
compliance config the event walk was re-reading) plus /device/events/storage_
range. The chain walk moves behind a "Load events" button in the Events
toolbar, and the Device tab gains an Event Chain card showing the first/last
keys.

Adds a Diagnostics tab for the endpoints that previously existed only as curl:
storage_range and events/index alongside monitor/status, then stop monitoring,
disable ACH (rescue?erase=false, so events survive), and erase. The wedged-unit
ladder — slow drip and blind stop — sits under its own heading pointing at the
runbook, with the reminder that slow_drip's success signal is bytes_received>0
and not a clean duration.

Erase is guarded by typing the unit's serial. Auth answers who, not whether you
meant it, and Swagger's try-it-out button on /device/events/erase is live on
:8200/docs — the realistic risk here is an accident.

Lifetime events is displayed but labelled unreliable: SUB 0x08 reports 0 on
units with years of history, which is a decode bug we have not chased yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-19 22:05:55 +00:00
serversdownandClaude Opus 5 0b58415fe2 chore(release): v0.31.0 — report parity + the inverted rescue
Cuts Unreleased to v0.31.0 and writes the theme now that the whole release is
visible, per the convention adopted today.

Two threads landed. Blastware Event/FFT-Report parity — the FFT, the USBM
RI8507 compliance chart, and the sensor self-check decoded for both series and
standardized into the .h5 (schema v2, /sensor_check). And the ach_server rescue
flags out of the BE12599 field emergency, which invert the wedged-unit recovery:
answer the unit's call instead of racing a Stop into the gaps between its
dial-outs.

Version stamped in pyproject.toml, CLAUDE.md and README.md. TOOL_VERSION was
already at 0.31.0 — it came in with the sensor-check work, and it is what makes
the backfill pick up the new /sensor_check group without --force.

⚠ This release owes prod a backfill: .h5 schema v1 -> v2, ~2 h on the NAS.
Stated in the Migration block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 20:40:48 +00:00
10 changed files with 791 additions and 24 deletions
+85
View File
@@ -6,6 +6,91 @@ All notable changes to seismo-relay are documented here.
## Unreleased
### Fixed
- **Waveform event times were the monitoring-session start, not the trigger
(~hours off).** `read_blastware_file` stamped events with footer `ts1`, which
for a waveform is the session start a unit shares across every event that day
(a unit arming at 06:00 stamped 06:00 on all of them — the modal and PDF both
showed it, since it's the stored value). The event time is footer `ts2` (the
recording stop), and Blastware's trigger = `ts2 - record time`. The record
time is a big-endian float32 in the recording-setup config block (30 bytes
before the `Standard Recording Setup` marker), so the **exact trigger is now
recovered from the binary alone** — all 7 BE12844 oracle events decode to
their exact Blastware time (e.g. N844LQHB 10:33:29), no paired `.TXT` needed.
Histograms keep `ts1` (the ~24 h window start). A paired report's
`event_datetime` stays authoritative (unit-clock drift).
⚠ **Needs a re-decode backfill** to correct existing stored events' timestamps.
### Added
- **Diagnostics tab in the SFM standalone webapp.** Surfaces the device
endpoints that previously existed only as `curl`: `events/storage_range` and
`events/index` alongside `monitor/status`, then stop monitoring, disable ACH
(`rescue?erase=false`, so stored events survive), and erase. The wedged-unit
ladder — slow drip and blind stop — sits under its own heading pointing at
`docs/runbooks/wedged_unit_recovery.md`, with the reminder that `slow_drip`'s
success signal is `bytes_received > 0` and not a clean duration. Erase is
guarded by typing the unit's serial: auth answers *who*, not *did you mean
it*, and Swagger's try-it-out button on `/device/events/erase` is live on
`:8200/docs`.
- **`docs/sfm_tool_status.md`** — an honest per-capability maturity assessment:
what is production-grade (the codec library, the data side), what is
emergency-grade (the device side), what is a research artifact, the
known-issues table, and the gap to a real tool. Also records the **5A
page-boundary bug** as known: `parse_strt_end_offset()` discards the key's
page byte, so once a unit has recorded more than 64 KB since its last erase,
an event spanning the boundary reads an `end_offset` *behind* its own start —
the chunk loop fetches nothing and TERM packs a negative `offset_word`, which
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
files via the watcher path and never runs this walk.
### Changed
- **Connecting to a unit no longer walks its event chain.** `/device/events`
reads every event header over the cellular link; on a unit with a large or
wrapped chain that takes minutes or fails outright, and it fired
automatically on every connect. Connect now uses only ~2 s probes —
`/device/info` (which already carried the compliance config the walk was
re-reading) plus `events/storage_range` — and the Device tab gains an Event
Chain card. The walk moved behind a **Load events** button in the Events
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
event it has stored.
### Migration
**None.** Frontend and documentation only — no codec, waveform-store or DB
change, no schema change, and no `TOOL_VERSION` bump. The webapp is served
from the image, so the change appears after the next `sfm` rebuild.
---
## v0.31.0 — 2026-09-18
**Report parity, and a second way to rescue a runaway unit.** Two threads.
The first closes out Blastware Event/FFT-Report parity: the FFT, the USBM
RI8507 compliance chart and the sensor self-check now render on the event
report, reverse-engineered against BE12844 (MiniMate Plus) and UM (Thor)
events. The sensor check is decoded for **both** series and standardized into
the `.h5` (schema **v2**, a new `/sensor_check` group), so SFM serves it
device-agnostically rather than decoding at report time. The Inspector — an
annotated hex reader for series-3 binaries — is what made the trailing-block
structure findable, and it earned its keep by *ruling out* a stored FFT block
and proving Blastware computes it from the samples.
The second came out of a field emergency. BE12599's connector fault drove its
Tran channel to its trigger level, so the unit recorded back-to-back and dialed
the office ACH server every ~75 s, unreachable the whole time.
`bridges/ach_server.py` gained `--stop-monitoring` / `--disable-ach` /
`--rescue`, which **invert** the recovery: instead of racing a Stop into the
gaps between dial-outs, point the modem's Destination at our own ACH server and
answer the call. Proven in production the same night — the stop landed on the
first call-in and held. See `docs/runbooks/wedged_unit_recovery.md`.
⚠ **This release owes prod a backfill** — see Migration below.
### Added
- **Rescue-on-connect for `bridges/ach_server.py`** — `--stop-monitoring`
(SUB 0x97), `--disable-ach` (SUB 0x2C read → 0x7E write → 0x7F confirm) and
+19 -1
View File
@@ -2,7 +2,7 @@
Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
(Sierra Wireless RV50 / RV55). Current version: **v0.30.0**.
(Sierra Wireless RV50 / RV55). Current version: **v0.31.0**.
Stack-level context — which repo owns what, and how the three project versions
pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
@@ -61,6 +61,24 @@ Read this first when picking the project back up.
4th-decimal tick and are Thor's own rounding — no single linear LSB can
reproduce every printed value (the constraints are infeasible by 7e-5
relative), so do NOT retune `_GEO_LSB_IPS`.
- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
`parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
discarding the key's page byte. An event starting at `0x0111F2A2` and ending
at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
start. The chunk loop then exits before fetching anything and TERM computes
a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
`/device/events` walk 500s. Reproduced on BE12599 (2026-09-19), which had
78 KB stored and had rolled into page `0x12`.
**Why it hid so long:** every 5A capture the walk was verified against came
from a freshly-erased BE11529 — all three confirmed TERM examples in
`framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`. Prod is
unaffected: it ingests complete files via BW ACH, never this walk.
**Fixing it has two layers** — the arithmetic (`if end < start: end +=
0x10000`) stops the crash and bounds the loop correctly; carrying the page
byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
over) needs a BW capture of a spanning event first. Do not ship layer one
alone without a loud truncation warning — a silently short event is the
failure mode this codec has been bitten by repeatedly.
- **Open, not blocking:** 14 sensitive-range files show an exact 8x
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
inserts DB rows for store files that have none (one-time per store) and the
+6 -1
View File
@@ -1,4 +1,4 @@
# seismo-relay `v0.30.0`
# seismo-relay `v0.31.0`
A ground-up replacement for **Blastware** — Instantel's aging Windows-only
software for managing seismographs. Supports both the **MiniMate Plus
@@ -496,6 +496,11 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
## Roadmap (Future)
> **Where it stands *today*** — an honest per-capability maturity assessment,
> what to rely on, known issues, and the gap to a real tool:
> [`docs/sfm_tool_status.md`](docs/sfm_tool_status.md). This section covers
> where it is *going*.
### Strategic direction — where this is going
seismo-relay is being built as a **suite of cooperating components**
+118
View File
@@ -934,6 +934,120 @@ which looks like a different failure mode wearing the same signature.
---
### 8f. BE12599 — CONFIRMED on the bench: the geophone assembly, not the recorder (2026-09-23)
**The first mechanism in this investigation established physically rather than
inferred.** Brian pulled BE12599 from I-80 Abut 1 West on 2026-09-21 and bench-
tested it. A geophone swap settles it.
#### The swap test
Same recorder, twenty minutes apart, sensor self-check:
| geophone fitted | Tran | **Vert** | Long |
|---|---|---|---|
| BE12599's own (2026-09-22 21:10) | 4.2 | **2026.0** | 4.8 |
| BE9888's known-good (2026-09-22 22:48) | 3.6 | **3.4** | 3.6 |
Vert frequency likewise **2.2 Hz -> 7.4 Hz**. Fleet reference: overswing 3.3–5.1
(median 3.8, max 5.1 across 42,979 passing records), frequency 7.4–7.8 Hz.
With the good geophone the unit shows **no offsets, no self-triggering, all three
channels Passed**, and records a clean damped impulse with broadband 5–100 Hz
content. **BE12599 the recorder is functional** — it is only out of calibration
(due 2025-08-12).
**The fault is in the geophone assembly: element, cable, or its connector.**
The swap does not separate those three, because the cable is permanently glanded
into the geophone.
#### The bench evidence, with ground motion removed
17 events recorded on a workbench with nothing touching the sensor.
**The control** — `N599LRY0.4B0`, trigger forced by slapping the *microphone*,
geophone untouched:
| channel | peak in/s | frac < 0 | mean/peak |
|---|---|---|---|
| Tran | 0.070 | **0.00** | 0.87 |
| **Vert** | **0.190** | **0.35** | 0.13 |
| Long | 0.070 | **0.00** | 0.84 |
Vert produces 0.19 in/s of bipolar wander with **no mechanical input at all**.
The fault is *continuous*, not episodic — every field event was simply the
moment it crossed the trigger.
**All three channels are affected**, consistently across all 17 events:
Tran and Long parked at a small positive DC (`frac<0 = 0.00`, mp 0.64–0.95);
Vert wandering at 0.89–1.78 Hz, peaks 0.14–0.54. Yet the swing test calls Tran
and Long **Passed** (4.2 / 4.8). So their *coils* are healthy and merely sitting
on an offset, while Vert's circuit has lost damping entirely.
That reads as **leakage on several contacts plus one gone open** — which is what
progressive corrosion at a shared connection does. Three independent geophone
elements do not fail in sequence.
**It is still progressing.** Tran's DC offset was ~0.011 in/s on 2026-09-14 and
~0.064 on 2026-09-22 — 6x in eight days. Vert's overswing went 7.5 -> 2026.0 over
the same window.
#### ⚠ This explains the autozero's 10% success rate
The journal has framed this fault around the **recorder** throughout: Instantel's
autozero, the X1/X8 gains, the 2027–2069 window, `SUB 0x0E` as the best lead.
All of that adjusts the recorder's zero reference.
**An autozero cannot fix a geophone.** If most offsets are geophone-assembly
faults, a re-zero would succeed only on the minority where the recorder really is
at fault — which is exactly the ~10% Brian's father reported and which has sat
unexplained in §5 since 2026-08-28.
**Procedure change: swap the geophone first.** It is a five-minute field test
costing nothing. On this unit it would have answered the question in August.
⚠ Scoped to one unit. The other four confirmed units are the obvious check.
#### Correction to §8e
§8e is titled "a connector, not a geophone" and concluded the element was fine
because the fault hopped channels. The channel-hopping observation stands and the
swap test supports the general shape, but **that title overstated what was known**:
the swap narrows the fault to the *assembly*, and element / cable / connector are
still open within it. §8e's reasoning was sound; its confidence was not.
#### Cause — road salt now outranks the mice
The enclosure held a mouse nest on the battery ([[§8e]]), and that framed the
2026-09-22 analysis. But **the geophone is buried in the ground, not in the
enclosure** — they are two independent contamination paths, and the fault is in
the buried half.
The unit sat **~6 feet from I-80 westbound**. Winter road salt spray and highway
runoff put a buried geophone in chloride-rich water for months a year. Chloride
drives exactly this failure: corrosion at terminations, leakage across
insulation, eventually an open circuit.
The mice remain a real finding for the *enclosure* and the recorder end. They are
probably not the cause of this fault.
**Testable consequence:** road salt is a property of the **site**, not the unit.
Units deployed near salted highways should carry higher geophone-assembly failure
rates than units on quiet sites. Nothing in the archive records proximity to a
road, so this needs the deployment records rather than the event data.
#### Open — to be settled by opening the geophone
1. **Meter at the geophone plug** before disassembly: three coil pairs, and each
to shell. Prediction: **Vert open or very high** (that is the 2026 overswing);
Tran and Long matched and normal; **insulation to shell finite on all three**,
Vert worst (that is the leakage putting DC on healthy coils).
2. **Then open it.** Look for a water line, white/green corrosion at the coil
terminations, the state of the damping shunt, and the cable gland from the
inside. Measure the coils again **at the internal terminals** — if Vert reads
fine there, the fault is in the internal wiring or the gland, not the element.
---
### ⚠ Serial prefixes — four of these units are BlastMates, not MiniMates
Corrected 2026-09-06, after Brian queried "BA10895?" against a report that
@@ -985,3 +1099,7 @@ decoder lost the geo threshold along with the serial. See commit `9ceff65`.
| 2026-09-06 | The spread gate is **blind to onsets by construction** — it rejects a moving floor, which is what an onset is. It rejected the very record in which the ramp is visible. |
| 2026-09-07 | **The non-motion test** (Brian: "it doesn't cross zero"). `\|mean\|/peak` is bimodal with a 1.09% dead zone and returns exactly the 5 confirmed units from physics, not a threshold. Independent corroboration of the unit list. **BE11007 settled as NOT an offset** — a one-sided 9 in/s blast. |
| 2026-09-07 | **BE12599 is a connector fault, not a geophone fault.** Unipolar spikes with a 26→118 ms RC tail progressing to a flat pedestal, and the fault MOVES between Long and Tran while the sensor self-check passes on every event. An element cannot hop channels; a connector can. Inspect before reseating. |
| 2026-09-23 | **CONFIRMED: BE12599's fault is the geophone assembly, not the recorder.** Geophone swap on the same unit: Vert overswing **2026.0 -> 3.4**, frequency 2.2 -> 7.4 Hz, all three channels Passed, no self-triggering. First physically established mechanism in this investigation. |
| 2026-09-23 | **This explains the autozero's ~10% success rate** (§5): an autozero adjusts the *recorder's* zero reference and cannot fix a geophone. **Procedure change: swap the geophone first.** |
| 2026-09-23 | Bench control event (mic-slap trigger, geophone untouched) shows Vert wandering **0.19 in/s with no mechanical input** — the fault is continuous, not episodic. All three channels affected; Tran/Long parked on DC while their coils pass the swing test. |
| 2026-09-23 | **Cause: road salt now outranks the mice.** The geophone is buried, not in the enclosure — two independent contamination paths, and the fault is in the buried half. The unit sat ~6 ft from I-80 WB. Salt is a *site* property, so this predicts higher failure rates near salted highways. |
+150
View File
@@ -0,0 +1,150 @@
# SFM — where it actually stands as a tool
**Status as of 2026-09-20 (v0.31.0).** This is the honest assessment, not the
roadmap — `README.md § Roadmap` covers where it is *going*. Expect this file to
go stale; re-date it when you revise it.
---
## The framing
SFM is **three different things wearing one name**, at three very different
levels of maturity:
| | what it is | maturity |
|---|---|---|
| **The codec library** | `minimateplus/`, `micromate/` — bytes in, `Event` out | **Production.** Verified per-sample at scale. |
| **SDM — the data side** | the DB, waveform store, `/db/*`, ingest | **Production.** Terra-View depends on it daily. |
| **SFM — the device side** | `/device/*`, live connections to units | **Emergency-grade.** Works, but manual, unauthenticated, and thinly tested. |
| **The lab** | `seismo_lab.py`, `scratch/`, the Inspector | **Research artifacts.** Useful, not products. |
Brian's own description — *"right now it's an emergency tool and a research
project"* — is accurate, and it applies specifically to the **device side**.
The data side is not an emergency tool; it has been carrying production for
months.
Most confusion about "is SFM reliable?" comes from answering for the wrong
tier.
---
## 1. What you can rely on
### Production-grade — trust it
- **Series-3 decode.** 14,338 / 14,338 files decode per-sample exact against
preserved Blastware ASCII exports, 45 units, files back to 2018.
- **Series-4 (Thor) decode.** 1,057,536 / 1,057,536 geo samples exact against
Thor's own CSV exports; production IDFW 575/575 with zero truncations.
- **Histogram decode.** 1,211 / 1,211 production histograms exact, including
842,442 per-interval frequency comparisons with zero mismatches.
- **The ingest path.** `/db/import/blastware_file` and `/db/import/idf_file`
fed by the watchers — this is how prod actually gets its data, and it has
been running unattended for months.
- **`/db/*` read API.** Always-on, consumed by Terra-View for every fleet
listing, event detail and report.
- **The waveform store** — `.h5` + `.sfm.json` sidecars + retained raw
binaries, with operator review state preserved across regeneration.
- **`bridges/ach_server.py`** — speaks the full BW protocol to calling units.
Proven in the field, including as a rescue tool (see the runbook).
### Emergency-grade — works, but you are the error handling
- **`/device/*` live endpoints.** They do what they say. But they are
synchronous, unauthenticated, and a single cellular download can exceed the
60 s timeouts that sit in front of them.
- **The rescue ladder** (`rescue`, `stop_monitoring_*`, `events/erase`).
Each has worked in a real incident — but each has been used a handful of
times, by one person, with the runbook open.
- **The standalone webapp.** Perfectly usable, and as of v0.31.0 the cheap
probes and rescue actions are reachable without curl. No auth of any kind.
### Research artifacts — useful, not products
- **`seismo_lab.py`** — 2,789 lines of Tkinter (Bridge / Analyzer / Query DB /
Inspector). Desktop-only, single-user, no tests.
- **`scratch/`** — the verification harnesses (`verify_against_ascii.py`,
`verify_thor_against_csv.py`) and the offset detector (`offset_scan3.py`).
These produced the numbers the production claims rest on, so they matter —
but they are analysis scripts, not maintained code.
- **`docs/offset_investigation.md`** — an open investigation, not a feature.
---
## 2. What to use when
| you want to… | use | notes |
|---|---|---|
| Know if a unit is monitoring / its battery / memory | `GET /device/monitor/status?force=true` | ~2 s |
| Know whether ACH is on | `GET /device/call_home` | ~2 s. **Not** `/device/events`. |
| See how full a unit's buffer is | `GET /device/events/storage_range` | ~2 s, no chain walk |
| Stop a runaway unit | Diagnostics tab → Stop Monitoring | see the runbook first |
| Reach a unit that will not answer | **point its modem at an `ach_server` and answer its call** | runbook Method A — do not race it |
| List a unit's stored events | Events tab → Load events | **slow**, and broken past 64 KB (below) |
| Get event data into the DB | the watcher → `/db/import/*` path | not the live walk |
The single most useful habit: **the cheap probes are cheap and the event walk
is not.** Reaching for `/device/events` to answer a yes/no question about a
unit is the mistake that motivated the v0.31.0 webapp changes.
---
## 3. Known issues
| issue | impact | status |
|---|---|---|
| **5A walk dies once a unit's buffer crosses 64 KB** | `/device/events` 500s; event body never downloads | Known, documented in `CLAUDE.md`. Needs a BW capture of a spanning event to fix properly. |
| **No auth on SFM at all** | 21 `/device/*` endpoints, including destructive ones, open to anything that reaches the port | Design agreed (Terra-View as authenticated jump host); not built. |
| **Swagger try-it-out is live on destructive endpoints** | `POST /device/events/erase` is one click away at `:8200/docs` | Partially mitigated: the webapp's erase now requires typing the serial. `/docs` itself is unguarded. |
| **`SUB 0x08` lifetime counter reads 0** | `/device/events/index` returns a meaningless number | Suspected field-offset bug. Surfaced in the UI as "unreliable". |
| **Long device operations are synchronous** | 60 s timeouts in `routers/sfm.py` and the reverse proxy; a full download exceeds both | Known design constraint. Must be POST-starts-job / GET-polls before any remote lab. |
| **`backfill_sidecars.py --force` silently inserts DB rows** | store files with no DB row get one; the dry-run does not report the count | Known. Avoid `--force` — `TOOL_VERSION` gates regeneration anyway. |
| **14 sensitive-range files show an exact 8× discrepancy** | 10.0 / 1.25 — a units problem, not a decode problem | Open, not blocking. |
| **16 failing tests on `dev`** | 15 need gitignored fixture bundles; 1 is real (`sc["peak_values"]["transverse"]` returns `None` where `0.0` is expected) | The real one shipped in v0.31.0. |
---
## 4. What stands between this and a real tool
Roughly in dependency order — each unblocks the ones below it.
**1. Authentication.** Everything else is gated on this. SFM has none, and
the modem IP whitelist gives zero protection because SFM *is* the whitelisted
origin. The agreed design delegates rather than builds: Terra-View becomes the
authenticated jump host (`/api/sfm/*` already inherits deny-by-default operator
auth), and the `8200:8200` publish is dropped so Terra-View is the only door.
**2. Async long operations.** POST starts a job, GET polls. Retrofitting this
after building a remote lab on top of synchronous endpoints would be far worse
than designing for it now.
**3. Confirm-guards on the remaining destructive endpoints.** Auth answers
*who*, not *did you mean it*. The webapp's erase is guarded; the other seven
destructive POSTs and `/docs` are not.
**4. The 5A page-boundary fix.** Until this lands, live event download is
unreliable on exactly the units most likely to need attention — the ones that
have been recording heavily. Wants a Blastware capture of an event spanning a
page boundary before the chunk-addressing half is trustworthy.
**5. A live Thor / Micromate client.** The device side is MiniMate-only.
Series-4 units can only be read from forwarded files, so half the fleet has no
live path at all.
**6. Test coverage that runs from a clean checkout.** 15 of 16 current
failures are missing fixture bundles. A test suite that cannot go green on a
fresh clone cannot gate anything.
**7. The SDM rename.** Cosmetic relative to the above, but the longer `sfm/`
holds the data-side code the more the tiers blur. ~30–50 files here, ~10–15 in
Terra-View, plus a Docker volume migration. Do it when the codebase is quiet.
---
## The short version
The **data side is a real tool already**. The **device side is a set of sharp
instruments** that work in the hands of the person who wrote them, with the
runbook open. The gap between those two states is mostly **auth, async, and
guardrails** — not protocol work. The protocol is the part that is actually
finished.
+63 -1
View File
@@ -296,6 +296,16 @@ def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
event.sample_rate = report.sample_rate_sps
if report.record_time_s is not None:
event.rectime_seconds = report.record_time_s
# The report's event_datetime is Blastware's exact trigger time (parsed
# from Event Time + Event Date). Prefer it over the binary footer's stop
# time so a report-paired import matches BW to the second.
edt = report.event_datetime
if edt is not None:
event.timestamp = Timestamp(
raw=b"", flag=0x10,
year=edt.year, unknown_byte=0, month=edt.month, day=edt.day,
hour=edt.hour, minute=edt.minute, second=edt.second,
)
def apply_bw_report_dict_to_event(event: Event, bw_report: dict) -> None:
@@ -808,6 +818,30 @@ def derive_record_type_from_filename(filename, default: str = "Waveform") -> str
return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
# Marker for the recording-setup config block, and the offset of the record-time
# float32 within it. The configured post-trigger record time (seconds) is a
# big-endian float32 exactly 30 bytes before the "Standard Recording Setup"
# label. Verified across the corpus reading 1.0 / 2.0 / 3.0 s on different
# setups — and ts2 - record_time reproduces Blastware's trigger to the second
# (N844LQHB: stop 10:33:32 - 3.0 = 10:33:29).
_RECSETUP_MARKER = b"Standard Recording Setup"
_RECTIME_OFFSET_BEFORE_MARKER = 30
def _parse_record_time_seconds(raw: bytes) -> Optional[float]:
"""The configured post-trigger record time in seconds, from the recording-
setup config block, or None when absent / implausible."""
a = raw.find(_RECSETUP_MARKER)
if a < _RECTIME_OFFSET_BEFORE_MARKER:
return None
off = a - _RECTIME_OFFSET_BEFORE_MARKER
try:
rt = struct.unpack(">f", raw[off:off + 4])[0]
except struct.error:
return None
return rt if 0.05 <= rt <= 600.0 else None
def read_blastware_file(path: Union[str, Path]) -> Event:
"""
Parse a Blastware waveform file into an Event.
@@ -917,6 +951,10 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
# rest of the event (timestamp, waveform_key, project strings) is
# still recoverable and useful.
decoded = decode_waveform_v2(body)
# Discriminator for the timestamp logic below: a waveform (trigger) event
# vs a histogram window. Keyed on the codec, not the filename — the
# save_imported_bw path passes a tmp ".bw" name whose extension lies.
is_waveform_body = decoded is not None
if decoded is None:
decoded = decode_histogram_body(body)
if decoded is None:
@@ -948,7 +986,31 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
ev.total_samples = strt_fields.get("total_samples")
ev.pretrig_samples = strt_fields.get("pretrig_samples")
if ts1 is not None:
# Event timestamp. The footer's two timestamps mean different things by
# record type:
# * Waveform: ts1 = the monitoring-SESSION start (shared across every
# event that day — a unit arming at 06:00 stamps 06:00 on all of them),
# ts2 = THIS event's recording STOP. Blastware's Date/Time is the
# TRIGGER = ts2 - record time, and the record time is a float32 in the
# recording-setup config block (see _parse_record_time_seconds), so the
# exact trigger is recoverable from the binary alone. Falls back to ts2
# (the stop, within the record duration) if the config block is absent.
# (Stamping ts1 showed the session start, hours off.)
# * Histogram / undecodable: ts1 = the window start, which IS the event
# time — keep it.
# Discriminate by ``is_waveform_body`` (the codec), not the filename.
if is_waveform_body and ts2 is not None:
_stop = datetime.datetime(ts2.year, ts2.month, ts2.day,
ts2.hour, ts2.minute, ts2.second)
_rt = _parse_record_time_seconds(raw)
_trig = _stop - datetime.timedelta(seconds=_rt) if _rt is not None else _stop
ev.timestamp = Timestamp(
raw=footer[10:18],
flag=0x10,
year=_trig.year, unknown_byte=0, month=_trig.month, day=_trig.day,
hour=_trig.hour, minute=_trig.minute, second=_trig.second,
)
elif ts1 is not None:
ev.timestamp = Timestamp(
raw=footer[2:10],
flag=0x10,
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "seismo-relay"
version = "0.30.0"
version = "0.31.0"
description = "Python client and REST server for MiniMate Plus seismographs"
requires-python = ">=3.10"
dependencies = [
+295 -20
View File
@@ -108,6 +108,12 @@
color: var(--text);
}
.btn-ghost:hover { border-color: var(--blue-lt); color: var(--blue-lt); }
.btn-danger { background: var(--red); color: #fff; }
.btn-danger:hover:not(:disabled) { filter: brightness(1.15); }
.diag-result { display:block; margin-top:6px; font-size:12px; opacity:.85;
white-space:pre-wrap; word-break:break-word; }
.diag-result.ok { color: var(--green); }
.diag-result.error { color: var(--red); }
.btn:disabled { background: var(--surface2) !important; color: var(--text-mute) !important; cursor: not-allowed; border-color: var(--border2) !important; }
/* #connect-btn styles moved to #live-connect-bar block */
@@ -910,6 +916,7 @@
<button class="tab-btn" data-tab="events" onclick="switchTab('events')">Events</button>
<button class="tab-btn" data-tab="config" onclick="switchTab('config')">Config</button>
<button class="tab-btn" data-tab="call-home" onclick="switchTab('call-home')">Call Home</button>
<button class="tab-btn" data-tab="diagnostics" onclick="switchTab('diagnostics')">Diagnostics</button>
</div>
<!-- ════════════════════════════════════════════════════════════════
@@ -938,6 +945,10 @@
<div id="tab-events" class="tab-pane" style="display:flex; flex-direction:column; overflow:hidden;">
<div class="event-toolbar">
<button class="btn btn-ghost" id="load-events-btn" onclick="loadEventList()" disabled
title="Walk the device's event chain and list its stored events. This is the slow one — it reads every event header over the cellular link.">
⟳ Load events
</button>
<button class="btn btn-ghost" id="load-btn" onclick="loadWaveform()" disabled>Load Waveform</button>
<button class="btn btn-ghost" id="save-btn" onclick="saveEventToDb()" disabled
title="Download the full waveform from the device and save it to the SFM database + waveform store. Honors the Force refresh toggle.">
@@ -1205,6 +1216,77 @@
</div><!-- end #tab-call-home -->
<!-- ════════════════════════════════════════════════════════════════
TAB: Diagnostics
═══════════════════════════════════════════════════════════════════ -->
<div id="tab-diagnostics" class="tab-pane">
<div class="cfg-grid">
<div class="cfg-section">
<div class="cfg-section-title">Device State</div>
<div class="hint" style="margin-bottom:10px">
Fast probes — POLL plus one read each, about 2 s. None of these walk the event chain.
</div>
<div class="dev-table" id="diag-table"></div>
<div class="cfg-actions" style="margin-top:12px">
<button class="btn btn-ghost" id="diag-refresh-btn" onclick="refreshDiagnostics()" disabled>Refresh</button>
<span id="diag-status"></span>
</div>
</div>
<div class="cfg-section">
<div class="cfg-section-title">Actions</div>
<div class="cfg-field">
<label>Stop Monitoring</label>
<button class="btn btn-ghost" id="diag-stop-btn" onclick="diagStopMonitoring()" disabled>Send Stop (SUB 0x97)</button>
<div class="hint">Halts recording. On a unit triggering continuously, this is what breaks the call-home loop.</div>
<span class="diag-result" id="diag-stop-result"></span>
</div>
<div class="cfg-field">
<label>Disable Auto Call Home</label>
<button class="btn btn-ghost" id="diag-ach-btn" onclick="diagDisableAch()" disabled>Disable ACH</button>
<div class="hint">Stored events are left untouched (<code>rescue?erase=false</code>). The unit stops dialing out until ACH is re-enabled.</div>
<span class="diag-result" id="diag-ach-result"></span>
</div>
<div class="cfg-field">
<label>Erase All Events</label>
<input type="text" id="diag-erase-confirm" placeholder="Type the serial to enable"
oninput="diagCheckEraseConfirm()" autocomplete="off" />
<button class="btn btn-danger" id="diag-erase-btn" onclick="diagEraseEvents()" disabled>Erase Events</button>
<div class="hint">⚠ Permanent, and resets the event chain to key <code>0x01110000</code>. Download anything worth keeping first.</div>
<span class="diag-result" id="diag-erase-result"></span>
</div>
</div>
<div class="cfg-section">
<div class="cfg-section-title">Unresponsive Unit</div>
<div class="hint" style="margin-bottom:10px">
The escalation ladder from <code>docs/runbooks/wedged_unit_recovery.md</code>, for a unit too busy
to answer normal request/response. Prefer <b>Method A</b> — point the modem at an
<code>ach_server</code> and answer its call — before racing it with these.
</div>
<div class="cfg-field">
<label>Slow drip <span class="hint" style="display:inline">(one held session, a stop every 3 s)</span></label>
<button class="btn btn-ghost" id="diag-drip-btn" onclick="diagSlowDrip()" disabled>Run 120 s drip</button>
<div class="hint">Success is <code>bytes_received &gt; 0</code>. A full duration with <code>send_error: null</code> is <b>not</b> success on its own.</div>
<span class="diag-result" id="diag-drip-result"></span>
</div>
<div class="cfg-field">
<label>Blind stop <span class="hint" style="display:inline">(fire-and-forget, one attempt)</span></label>
<button class="btn btn-ghost" id="diag-blind-btn" onclick="diagBlindStop()" disabled>Send blind stop</button>
<span class="diag-result" id="diag-blind-result"></span>
</div>
</div>
</div>
</div><!-- end #tab-diagnostics -->
</div><!-- end #section-live -->
<!-- ════════════════════════════════════════════════════════════════
@@ -1361,6 +1443,8 @@
// ── State ──────────────────────────────────────────────────────────────────────
let unitInfo = null;
let eventList = [];
let storageInfo = null; // /device/events/storage_range — cheap, read on connect
let eventsLoaded = false; // the event chain walk is opt-in; see loadEventList()
let currentEvent = 0;
let charts = {};
let geoAdcScale = 6.206;
@@ -1458,6 +1542,7 @@ function switchTab(name) {
if (name === 'units') { if (!unitsLoaded) loadUnits(); }
if (name === 'monlog') { if (!monlogLoaded) loadMonitorLog(); }
if (name === 'sessions') { if (!sessLoaded) loadSessions(); }
if (name === 'diagnostics' && devHost() && unitInfo) refreshDiagnostics();
}
// ── Connect ────────────────────────────────────────────────────────────────────
@@ -1478,18 +1563,13 @@ async function connectUnit() {
btn.disabled = false; btn.textContent = 'Connect'; return;
}
setStatus('Fetching event list…', 'loading');
try {
const r = await fetch(`${api()}/device/events?${deviceParams()}`);
if (!r.ok) { const e = await r.json().catch(() => ({})); throw new Error(e.detail || r.statusText); }
const evData = await r.json();
eventList = evData.events || [];
// Merge compliance from /device/events response (it re-reads it)
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
} catch (e) {
setStatus(`Event fetch failed: ${e.message}`, 'error');
btn.disabled = false; btn.textContent = 'Reconnect'; return;
}
// Connecting deliberately does NOT walk the event chain. That walk reads
// every event header over the cellular link and can take minutes — or fail
// outright on a unit whose buffer has wrapped past 0xFFFF. Use the ~2 s
// probes instead; the event list is opt-in via loadEventList().
eventList = []; eventsLoaded = false;
setStatus('Reading device state…', 'loading');
storageInfo = await fetchJson(`/device/events/storage_range`).catch(() => null);
populateDeviceBar();
populateDeviceTab();
@@ -1498,11 +1578,9 @@ async function connectUnit() {
document.getElementById('device-bar').style.display = 'flex';
document.getElementById('monitor-panel').style.display = 'flex';
document.getElementById('load-btn').disabled = eventList.length === 0;
document.getElementById('save-btn').disabled = eventList.length === 0;
document.getElementById('download-btn').disabled = eventList.length === 0;
document.getElementById('prev-btn').disabled = true;
document.getElementById('next-btn').disabled = eventList.length <= 1;
setEventButtonsEnabled();
document.getElementById('load-events-btn').disabled = false;
setDiagButtonsEnabled(true);
document.getElementById('cfg-read-btn').disabled = false;
document.getElementById('cfg-write-btn').disabled = false;
document.getElementById('ch-read-btn').disabled = false;
@@ -1510,7 +1588,9 @@ async function connectUnit() {
btn.disabled = false; btn.textContent = 'Reconnect';
setStatus(`Connected — ${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
setStatus(storageInfo && storageInfo.is_empty
? 'Connected — no events stored.'
: 'Connected. Event list not loaded (Events → Load events).', 'ok');
// Fetch monitor status in background (non-blocking)
refreshMonitorStatus().catch(() => {});
@@ -1522,6 +1602,48 @@ async function connectUnit() {
}
}
// ── Shared fetch helper ────────────────────────────────────────────────────────
async function fetchJson(path, opts) {
const sep = path.includes('?') ? '&' : '?';
const r = await fetch(`${api()}${path}${sep}${deviceParams()}`, opts);
const body = await r.json().catch(() => ({}));
if (!r.ok) throw new Error(body.detail || r.statusText);
return body;
}
function setEventButtonsEnabled() {
const n = eventList.length;
document.getElementById('load-btn').disabled = n === 0;
document.getElementById('save-btn').disabled = n === 0;
document.getElementById('download-btn').disabled = n === 0;
document.getElementById('prev-btn').disabled = true;
document.getElementById('next-btn').disabled = n <= 1;
}
// ── Event list (opt-in — this is the slow chain walk) ──────────────────────────
async function loadEventList() {
if (!devHost()) { setStatus('Connect to a device first.', 'error'); return; }
const btn = document.getElementById('load-events-btn');
btn.disabled = true;
setStatus('Walking the event chain — this can take a while…', 'loading');
try {
const evData = await fetchJson('/device/events');
eventList = evData.events || [];
eventsLoaded = true;
// /device/events re-reads compliance; fold it in.
if (evData.device) unitInfo = { ...unitInfo, ...evData.device };
} catch (e) {
setStatus(`Event fetch failed: ${e.message}`, 'error');
btn.disabled = false; return;
}
populateDeviceBar();
populateDeviceTab();
populateEventChips();
setEventButtonsEnabled();
btn.disabled = false;
setStatus(`${eventList.length} event${eventList.length !== 1 ? 's' : ''} stored.`, 'ok');
}
// ── Device bar ─────────────────────────────────────────────────────────────────
function populateDeviceBar() {
qs('di-serial').textContent = unitInfo.serial || '—';
@@ -1530,7 +1652,7 @@ function populateDeviceBar() {
qs('di-sr').textContent = cc.sample_rate ? `${cc.sample_rate} sps` : '—';
qs('di-rt').textContent = cc.record_time != null ? `${cc.record_time.toFixed(1)} s` : '—';
qs('di-trig').textContent = cc.trigger_level_geo != null ? `${cc.trigger_level_geo.toFixed(3)} in/s` : '—';
qs('di-count').textContent = eventList.length;
qs('di-count').textContent = eventsLoaded ? eventList.length : '—';
qs('di-project').textContent = cc.project || '—';
qs('di-client').textContent = cc.client || '—';
qs('di-operator').textContent = cc.operator || '—';
@@ -1660,7 +1782,8 @@ function populateDeviceTab() {
{ label:'DSP', value: unitInfo.dsp_version || '—' },
{ label:'Model', value: unitInfo.model || '—' },
{ label:'Manufacturer', value: unitInfo.manufacturer || '—' },
{ label:'Stored Events', value: eventList.length },
{ label:'Stored Events', value: eventsLoaded ? eventList.length : 'not loaded' },
{ label:'Storage Used', value: storageUsedLabel() },
];
for (const {label, value} of cardData) {
const c = document.createElement('div');
@@ -1707,6 +1830,158 @@ function renderTable(id, rows) {
}
}
// ── Diagnostics ────────────────────────────────────────────────────────────────
// Everything here is a cheap probe (POLL + one read) or a single write. None of
// it walks the event chain. See docs/runbooks/wedged_unit_recovery.md.
function storageUsedLabel() {
if (!storageInfo) return '—';
if (storageInfo.is_empty) return 'empty';
const f = storageInfo.first_key, l = storageInfo.last_key;
return (f && l) ? `${f} → ${l}` : '—';
}
function setDiagButtonsEnabled(on) {
for (const id of ['diag-refresh-btn','diag-stop-btn','diag-ach-btn',
'diag-drip-btn','diag-blind-btn']) {
const el = document.getElementById(id);
if (el) el.disabled = !on;
}
diagCheckEraseConfirm();
}
// Erase is guarded by typing the serial — auth answers "who", not "did you mean it".
function diagCheckEraseConfirm() {
const box = document.getElementById('diag-erase-confirm');
const btn = document.getElementById('diag-erase-btn');
if (!box || !btn) return;
const serial = (unitInfo && unitInfo.serial) || '';
btn.disabled = !serial || box.value.trim().toUpperCase() !== serial.toUpperCase();
}
function diagResult(id, text, cls) {
const el = document.getElementById(id);
if (!el) return;
el.textContent = text;
el.className = 'diag-result' + (cls ? ' ' + cls : '');
}
async function refreshDiagnostics() {
if (!devHost()) return;
const st = document.getElementById('diag-status');
if (st) { st.textContent = 'Reading…'; st.className = 'loading'; }
const [mon, store, idx] = await Promise.all([
fetchJson('/device/monitor/status?force=true').catch(e => ({ _err: e.message })),
fetchJson('/device/events/storage_range').catch(e => ({ _err: e.message })),
fetchJson('/device/events/index').catch(e => ({ _err: e.message })),
]);
if (!store._err) storageInfo = store;
const err = v => `<span style="color:var(--red)">${v}</span>`;
const rows = [];
rows.push(['Monitoring', mon._err ? err(mon._err)
: (mon.is_monitoring ? '<b>MONITORING</b>' : 'idle')]);
if (!mon._err) {
rows.push(['Battery', mon.battery_v != null ? `${mon.battery_v.toFixed(2)} V` : '—']);
if (mon.memory_total_bytes) {
const used = mon.memory_total_bytes - (mon.memory_free_bytes ?? 0);
const pct = (used / mon.memory_total_bytes * 100).toFixed(1);
rows.push(['Memory used', `${used.toLocaleString()} / ${mon.memory_total_bytes.toLocaleString()} bytes (${pct}%)`]);
}
}
rows.push(['Event chain', store._err ? err(store._err) : storageUsedLabel()]);
if (!store._err) rows.push(['Chain empty', store.is_empty ? 'yes' : 'no']);
// SUB 0x08. Known to report 0 on units with years of history — suspected
// field-offset bug in the decode, so show it but do not trust it.
rows.push(['Lifetime events', idx._err ? err(idx._err)
: `${idx.lifetime_count} <span class="hint" style="display:inline">(unreliable — see CHANGELOG)</span>`]);
renderTable('diag-table', rows);
populateDeviceTab();
if (st) { st.textContent = ''; st.className = ''; }
}
async function diagStopMonitoring() {
const btn = document.getElementById('diag-stop-btn');
btn.disabled = true; diagResult('diag-stop-result', 'Sending…');
try {
await fetchJson('/device/monitor/stop', { method: 'POST' });
diagResult('diag-stop-result', 'Stop acknowledged — recording halted.', 'ok');
refreshDiagnostics();
} catch (e) {
diagResult('diag-stop-result', `Failed: ${e.message}`, 'error');
}
btn.disabled = false;
}
async function diagDisableAch() {
const btn = document.getElementById('diag-ach-btn');
btn.disabled = true; diagResult('diag-ach-result', 'Writing call-home config…');
try {
const r = await fetchJson('/device/rescue?erase=false', { method: 'POST' });
const steps = (r.steps || []).map(s => s.step).join(' → ') || 'done';
diagResult('diag-ach-result', `ACH disabled (${steps}). Events untouched.`, 'ok');
} catch (e) {
diagResult('diag-ach-result', `Failed: ${e.message}`, 'error');
}
btn.disabled = false;
}
async function diagEraseEvents() {
const serial = (unitInfo && unitInfo.serial) || 'this unit';
if (!confirm(`Permanently erase ALL events on ${serial}?\n\nThis cannot be undone.`)) return;
const btn = document.getElementById('diag-erase-btn');
btn.disabled = true; diagResult('diag-erase-result', 'Erasing…');
try {
await fetchJson('/device/events/erase', { method: 'POST' });
diagResult('diag-erase-result', 'Events erased — chain reset to 0x01110000.', 'ok');
document.getElementById('diag-erase-confirm').value = '';
eventList = []; eventsLoaded = false;
setEventButtonsEnabled(); populateEventChips();
refreshDiagnostics();
} catch (e) {
diagResult('diag-erase-result', `Failed: ${e.message}`, 'error');
}
diagCheckEraseConfirm();
}
async function diagSlowDrip() {
const btn = document.getElementById('diag-drip-btn');
btn.disabled = true;
diagResult('diag-drip-result', 'Holding a session for 120 s…');
try {
const r = await fetchJson('/device/stop_monitoring_slow_drip?duration_s=120&interval_s=3',
{ method: 'POST' });
const good = (r.bytes_received || 0) > 0;
diagResult('diag-drip-result',
`drips ${r.drips_sent} · held ${r.duration_s}s · bytes back ${r.bytes_received}` +
(r.send_error ? ` · ${r.send_error}` : '') +
(good ? ' → device responded' : ' → no response; the modem may not be bridging'),
good ? 'ok' : 'error');
} catch (e) {
diagResult('diag-drip-result', `Failed: ${e.message}`, 'error');
}
btn.disabled = false;
}
async function diagBlindStop() {
const btn = document.getElementById('diag-blind-btn');
btn.disabled = true; diagResult('diag-blind-result', 'Sending…');
try {
const r = await fetchJson('/device/stop_monitoring_blind', { method: 'POST' });
diagResult('diag-blind-result',
`Sent ${r.bytes_sent ?? '?'} bytes, no response read (fire-and-forget).`, 'ok');
} catch (e) {
diagResult('diag-blind-result', `Failed: ${e.message}`, 'error');
}
btn.disabled = false;
}
// ── Config form ────────────────────────────────────────────────────────────────
function populateConfigFromDeviceInfo() {
if (!unitInfo) return;
Binary file not shown.
+54
View File
@@ -0,0 +1,54 @@
"""Event timestamp decode — waveform trigger/stop vs histogram window start.
The Blastware footer holds two timestamps: ts1 = footer[2:10], ts2 = footer[10:18].
Their meaning depends on record type:
* Waveform: ts1 is the monitoring-SESSION start (e.g. 06:00 for a unit that
arms at 06:00 daily — shared across every event that day), and ts2 is THIS
event's recording STOP. read_blastware_file used to stamp events with ts1 →
every waveform showed the session start (~4.5 h off). Binary-only, the best
estimate is ts2 (the stop); the exact trigger BW displays (= ts2 - record
duration) comes from the paired report's event_datetime, since the binary
STRT record-time byte is a misparsed record-type marker.
* Histogram: ts1/ts2 are the ~24 h window [start, stop]; the event time is the
window start = ts1 (unchanged).
"""
import datetime
from pathlib import Path
from minimateplus.event_file_io import read_blastware_file, apply_report_to_event
from minimateplus.bw_ascii_report import BwAsciiReport
from minimateplus.models import Event
FIX = Path(__file__).parent / "fixtures"
WAVEFORM = FIX / "fft-oracle-2026-09-14" / "N844LQHB.ZT0W" # footer ts2 = 2026-08-25 10:33:32
HISTOGRAM = FIX / "ts-fix" / "K441LKZU.C30H" # window start 2026-05-10 19:04:50
def _tuple(ts):
return (ts.year, ts.month, ts.day, ts.hour, ts.minute, ts.second)
def test_waveform_timestamp_is_exact_trigger_from_binary():
ev = read_blastware_file(WAVEFORM)
# The EXACT Blastware trigger, from the binary alone: ts2 (stop 10:33:32)
# minus the config record time (3.0 s) = 10:33:29 — NOT the 06:00:13
# monitoring-session start the old decode used.
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29), _tuple(ev.timestamp)
def test_histogram_timestamp_is_window_start_unchanged():
ev = read_blastware_file(HISTOGRAM)
# Histogram event time = the window start (ts1); must NOT get the waveform
# ts2 treatment (that would land ~24 h off).
assert _tuple(ev.timestamp) == (2026, 5, 10, 19, 4, 50), _tuple(ev.timestamp)
def test_report_event_datetime_is_authoritative_over_binary():
# The binary already yields the exact trigger, but a paired report stays
# authoritative (e.g. if the unit clock had drifted) — applying it wins.
ev = read_blastware_file(WAVEFORM)
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 33, 29) # exact, from binary
apply_report_to_event(ev, BwAsciiReport(
event_datetime=datetime.datetime(2026, 8, 25, 10, 35, 0)))
assert _tuple(ev.timestamp) == (2026, 8, 25, 10, 35, 0) # report wins