feat: add thor/micromate compatibility v0.18.0

Reviewed-on: #21

## v0.17.0 — 2026-05-17

The "field rescue + DB management" release.  Hardened against units that are stuck in a runaway call-home loop, and added an operator-facing path for purging bogus events that those same units dump into the DB before recovery.  All work in this release was driven by the BE9558H incident (full incident log + recovery procedure at `docs/runbooks/wedged_unit_recovery.md`).

### Added — wedged-unit recovery toolkit

A toolkit for breaking the call-home loop on a misbehaving unit whose firmware is too busy to keep up with normal request/response handshakes.  Tested in production against BE9558H (16 May 2026) — a unit with a stuck-triggered Long-axis geophone that had been call-homing the office BW ACH server every 30 seconds for hours.  Endpoints layered from "single attempt" to "siege mode" to suit different contention levels:

- **`GET /device/events/storage_range`** — SUB 0x06 probe.  POLL + one read; ~2s.  Returns first/last event keys and an `is_empty` flag.  Use to triage whether a unit has stored events without invoking the slow `count_events()` 1E/1F chain (which choked on BE9558H's corrupted event chain).
- **`GET /device/events/index`** — SUB 0x08 probe.  POLL + one read; ~2s.  Returns the lifetime event counter (does NOT decrement on erase — use `storage_range` for "right now" state).
- **`POST /device/events/erase`** — full erase sequence `0xA3 → 0x1C → 0x06 → 0xA2` (confirmed 2026-04-11, see the protocol reference).  Resets event keys to `0x01110000`.  Caller's responsibility to disable ACH first if the underlying trigger condition will re-fill the buffer.
- **`POST /device/rescue`** — one TCP session, short connect+recv timeouts: POLL → disable ACH (compliance config write) → erase events → close.  Designed for race-loop usage when the device is busy in another session.  503 on connect-refused, 502 on protocol failure, 200 on full sequence success.
- **`POST /device/stop_monitoring_blind`** — fire-and-forget Stop Monitoring (SUB 0x97), TCP-only.  Dumps `SESSION_RESET + POLL_PROBE + SESSION_RESET + POLL_DATA + 0x97 × repeat` and closes without reading any S3 response.  The full POLL preamble is required — write commands without it are silently ignored by the device's protocol parser (false-positive surface area that bit the first version of this endpoint).  Use when the device's firmware can't keep up with full request/response but might process inbound bytes at its own pace.
- **`POST /device/stop_monitoring_spam`** — server-side hammer loop, duration-bounded.  Open TCP → write the same blind payload → close → repeat as fast as possible until `duration_s` elapses.  Configurable `connect_timeout` (default 500ms) and `repeat` (frames per session).  Reports `sent_ok`, `connect_failed`, `write_failed`, `rate_attempts_per_s`.  Clamped to 5min duration.
- **`POST /device/stop_monitoring_slow_drip`** — opposite of spam.  Open ONE TCP session, drip the wake handshake + stop frames at `interval_s` (default 3s) for `duration_s` (default 120s, max 10min).  Each drip is ~23 bytes — well under any UART FIFO size.  Opportunistically drains any inbound bytes the device sends back; `bytes_received > 0` in the response strongly suggests the device has started talking and the session is healthy.  **This is the endpoint that saved BE9558H.** Spam mode had been overrunning the device's UART FIFO; slow drip stayed under it.
- **Six rescue scripts** under `scripts/` — thin bash wrappers around the endpoints, default `SFM_BASE_URL=http://localhost:8200` (direct, not via Terra-View proxy whose 60s timeout would cut off the longer endpoints):
    - `rescue_device.sh` — race-loop wrapper for `/device/rescue`
    - `blind_stop.sh` — race-loop wrapper for `/device/stop_monitoring_blind`
    - `spam_stop.sh` — single-call burst hammer
    - `slow_drip.sh` — single-call held-session drip
    - `watch_unit.sh` — passive periodic reachability check (every N min, logs to file), useful for unattended overnight monitoring of a wedged unit
- **`docs/runbooks/wedged_unit_recovery.md`** — symptoms, quick-reference recovery procedure, the modem-layer mechanism (Sierra Wireless serial-port mode-flipping is the real failure mode — not the device firmware), and a table of "why simpler approaches don't work" so the next incident skips the dead ends.

### Added — operator event DB management

Endpoints powering Terra-View's new `/admin/events` page (v0.12.0).  Designed for purging bogus events from a unit that's been forwarding them in bulk (e.g. a stuck-triggered seismograph dumping hundreds of junk events before it's recovered).

- **`DELETE /db/events/{event_id}`** — hard-delete one event row.  Also unlinks the associated blastware binary (`.AB0*`), `.a5.pkl`, `.sfm.json` sidecar, and `.h5` clean-waveform files via the WaveformStore.  Returns the per-file removal status.  404 if the event doesn't exist.
- **`POST /db/events/delete_bulk`** — filter-based or id-list-based bulk delete with safety rails:
    - Filters (`serial`, `from_dt`, `to_dt`, `false_trigger`) combine with AND; same semantics as `GET /db/events`.  `ids` is an additional inclusion list.  Refuses to run with no filters (would wipe the whole table — raises 422).
    - `confirm` must be `true` to actually delete.  Otherwise returns a dry-run summary (`status: "dry_run"`, `matched: N`, `sample_serials: [...]`).
    - `max_rows` (default 10,000) caps how many rows can be deleted by-filter in one call.  If exceeded, returns `status: "too_many"` with a hint to narrow or raise the cap.  Bypassed when only `ids` is supplied.
- **`_cleanup_event_files(row)`** helper in `sfm/server.py` — best-effort `unlink()` of all four sidecar paths derived from the row's `blastware_filename`.  Logged at WARN if a path exists but unlink fails; the DB row deletion still proceeds.
- **`SeismoDb.delete_event(id)` and `SeismoDb.delete_events_bulk(...)`** in `sfm/database.py` — both return the deleted row dict(s) so callers can do file cleanup.  `delete_events_bulk` raises `ValueError` if no filters are supplied.

### Changed

- **Default protocol recv timeout dropped from 30s → 10s** in `_build_client()`.  The unit usually responds in well under a second over cellular; 10s leaves comfortable headroom for retransmits while failing reasonably fast when a unit is wedged.  The two endpoints that perform full 5A waveform downloads still pass `timeout=120.0` explicitly so multi-minute event transfers are unaffected.
- **`_build_client()` now accepts an optional `connect_timeout`** (TCP-only) so rescue / race-loop endpoints can fail fast on busy modems without affecting the protocol-level recv timeout.

### Fixed

- **`GET /device/monitor/status` returned HTTP 500 + uncaught traceback when the device was unresponsive**.  The retry-on-`Exception` inner block let the second `client.poll()`'s `ProtocolError` propagate out of the handler.  Now wrapped in proper try/except — returns 502 with `{"detail": "Protocol error: No S3 frame received within 10.0s ..."}` on timeout, 502 on connection errors, 500 only for genuinely unexpected exceptions.

### Migration

No schema changes.  No data migration required.

If you've been running a previous version against a wedged unit and accumulated bogus events, the new `/admin/events` page in Terra-View v0.12.0 (or direct `POST /db/events/delete_bulk` with `confirm: true`) is the cleanup tool.  Watcher state on the upstream DL2 PC does NOT need separate cleaning — the watcher's `sfm_forwarded.json` keys on file sha256 and won't re-forward the same files.

### Pairing

This release pairs with **Terra-View v0.12.0**, which adds the `/admin/events` UI that consumes the new bulk-delete endpoints, the bulk false-trigger flagging on `/unit/{id}`, and the field-deployment workflow that uses the same `series3-watcher` → SFM ingest path as before.

---

## v0.16.1 — 2026-05-14

### Fixed

- **`record_type` always "Waveform" for forwarded events.**  `read_blastware_file()` hardcoded `ev.record_type = "Waveform"` regardless of the file's actual type.  The watcher-forward pipeline (the main BW ACH ingest path) compounds this by parsing files from a tmp path with a `.bw` suffix, so even a filename-based fallback inside the parser still wouldn't see the original extension.  Now:

  1. New `derive_record_type_from_filename(filename)` helper in `minimateplus/event_file_io.py` derives the type from the LAST character of the filename's extension (V10.72+ AB0T scheme: `H`=Histogram, `W`=Waveform, `M`=Manual, `E`=Event, `C`=Combo).  Falls back to `"Waveform"` for old S338 firmware (3-char extensions ending in `0`) and any unrecognized suffix.
  2. `read_blastware_file()` now calls the helper with its `path.name` so direct callers (the `--dry-run` path in `scripts/import_bw.py`, tests, ad-hoc scripts) get the right value automatically.
  3. `WaveformStore.save_imported_bw()` overrides `ev.record_type` with the **original** filename's derived type after parsing (the tmp file inside the parser doesn't carry the original extension).  This is the path the live watcher-forwarder hits, so the DB column now reflects the actual event type going forward.

  Events ingested before this fix are stuck with `record_type="Waveform"` in the DB; a one-off backfill (`UPDATE events SET record_type = ... WHERE blastware_filename LIKE '%H'`) would fix them retroactively if desired.  Terra-view's event modal also derives client-side from the filename, so the UI already shows the correct type for old events even without the backfill.

---

.dockerignore

@@ -0,0 +1,28 @@
+.git
+.gitignore
+
+.venv
+venv
+env
+__pycache__
+*.pyc
+*.pyo
+*.pyd
+.pytest_cache
+.mypy_cache
+.ruff_cache
+
+*.db
+*.db-wal
+*.db-shm
+*.sqlite
+*.sqlite3
+
+sfm/data
+bridges/captures
+example-events
+captures
+logs
+
+.DS_Store
+Thumbs.db

CHANGELOG.md

@@ -4,6 +4,125 @@ All notable changes to seismo-relay are documented here.
 
 ---
 
+## v0.18.0 — 2026-05-19
+
+The "Thor / Series IV ingest adapter" release.  Seismo-relay can now accept event files from Instantel Micromate Series IV (Thor) units alongside the existing MiniMate Plus (Series III) Blastware pipeline.
+
+### Added — Thor (Series IV) IDF ingest
+
+- **`POST /db/import/idf_file`** (`sfm/server.py`) — multipart upload endpoint for `.IDFH` (histogram) and `.IDFW` (waveform) event files plus their `.IDFH.txt` / `.IDFW.txt` ASCII sidecars.  Mirrors the shape of `/db/import/blastware_file`: pairing by filename, optional `serial` query hint, per-file outcome reporting.
+- **`sfm/idf_ascii_report.py`** — parser for Thor's TXT sidecars (verified against 1,014 real-world samples).  Extracts device-authoritative PPV, ZC Freq, Peak Vector Sum, Mic PSPL, calibration date, firmware version, sensor self-check results, and project/client/operator strings.
+- **`WaveformStore.save_imported_idf()`** (`sfm/waveform_store.py`) — stores Thor binaries verbatim in `<root>/<serial>/<filename>`, writes a `.sfm.json` sidecar with `source.kind = "idf-import"` and the full parsed report under `extensions.idf_report`.  Reuses the existing `events` table — Thor events dedupe on (serial, timestamp) and surface in `/db/events` alongside BW events.
+- **`tests/test_idf_ascii_report.py`** — parser tests against the `thor-watcher/example-data/` corpus.
+
+### Changed
+
+- `event_to_sidecar_dict()` (`minimateplus/event_file_io.py`) allow-list for `source_kind` now includes `"idf-import"` so the existing sidecar machinery can carry Thor imports.
+- Bumped `pyproject.toml` version to `0.18.0`.
+
+### Companion release
+
+This release ships alongside **thor-watcher v0.3.0**, which adds the SFM forwarder that targets the new `/db/import/idf_file` endpoint.  Operators flip the switch in thor-watcher's new "SFM Forward" Settings tab; events POST to seismo-relay just like the series3-watcher BW forwarder does today.
+
+---
+
+## v0.17.0 — 2026-05-17
+
+The "field rescue + DB management" release.  Hardened against units that are stuck in a runaway call-home loop, and added an operator-facing path for purging bogus events that those same units dump into the DB before recovery.  All work in this release was driven by the BE9558H incident (full incident log + recovery procedure at `docs/runbooks/wedged_unit_recovery.md`).
+
+### Added — wedged-unit recovery toolkit
+
+A toolkit for breaking the call-home loop on a misbehaving unit whose firmware is too busy to keep up with normal request/response handshakes.  Tested in production against BE9558H (16 May 2026) — a unit with a stuck-triggered Long-axis geophone that had been call-homing the office BW ACH server every 30 seconds for hours.  Endpoints layered from "single attempt" to "siege mode" to suit different contention levels:
+
+- **`GET /device/events/storage_range`** — SUB 0x06 probe.  POLL + one read; ~2s.  Returns first/last event keys and an `is_empty` flag.  Use to triage whether a unit has stored events without invoking the slow `count_events()` 1E/1F chain (which choked on BE9558H's corrupted event chain).
+- **`GET /device/events/index`** — SUB 0x08 probe.  POLL + one read; ~2s.  Returns the lifetime event counter (does NOT decrement on erase — use `storage_range` for "right now" state).
+- **`POST /device/events/erase`** — full erase sequence `0xA3 → 0x1C → 0x06 → 0xA2` (confirmed 2026-04-11, see the protocol reference).  Resets event keys to `0x01110000`.  Caller's responsibility to disable ACH first if the underlying trigger condition will re-fill the buffer.
+- **`POST /device/rescue`** — one TCP session, short connect+recv timeouts: POLL → disable ACH (compliance config write) → erase events → close.  Designed for race-loop usage when the device is busy in another session.  503 on connect-refused, 502 on protocol failure, 200 on full sequence success.
+- **`POST /device/stop_monitoring_blind`** — fire-and-forget Stop Monitoring (SUB 0x97), TCP-only.  Dumps `SESSION_RESET + POLL_PROBE + SESSION_RESET + POLL_DATA + 0x97 × repeat` and closes without reading any S3 response.  The full POLL preamble is required — write commands without it are silently ignored by the device's protocol parser (false-positive surface area that bit the first version of this endpoint).  Use when the device's firmware can't keep up with full request/response but might process inbound bytes at its own pace.
+- **`POST /device/stop_monitoring_spam`** — server-side hammer loop, duration-bounded.  Open TCP → write the same blind payload → close → repeat as fast as possible until `duration_s` elapses.  Configurable `connect_timeout` (default 500ms) and `repeat` (frames per session).  Reports `sent_ok`, `connect_failed`, `write_failed`, `rate_attempts_per_s`.  Clamped to 5min duration.
+- **`POST /device/stop_monitoring_slow_drip`** — opposite of spam.  Open ONE TCP session, drip the wake handshake + stop frames at `interval_s` (default 3s) for `duration_s` (default 120s, max 10min).  Each drip is ~23 bytes — well under any UART FIFO size.  Opportunistically drains any inbound bytes the device sends back; `bytes_received > 0` in the response strongly suggests the device has started talking and the session is healthy.  **This is the endpoint that saved BE9558H.** Spam mode had been overrunning the device's UART FIFO; slow drip stayed under it.
+- **Six rescue scripts** under `scripts/` — thin bash wrappers around the endpoints, default `SFM_BASE_URL=http://localhost:8200` (direct, not via Terra-View proxy whose 60s timeout would cut off the longer endpoints):
+    - `rescue_device.sh` — race-loop wrapper for `/device/rescue`
+    - `blind_stop.sh` — race-loop wrapper for `/device/stop_monitoring_blind`
+    - `spam_stop.sh` — single-call burst hammer
+    - `slow_drip.sh` — single-call held-session drip
+    - `watch_unit.sh` — passive periodic reachability check (every N min, logs to file), useful for unattended overnight monitoring of a wedged unit
+- **`docs/runbooks/wedged_unit_recovery.md`** — symptoms, quick-reference recovery procedure, the modem-layer mechanism (Sierra Wireless serial-port mode-flipping is the real failure mode — not the device firmware), and a table of "why simpler approaches don't work" so the next incident skips the dead ends.
+
+### Added — operator event DB management
+
+Endpoints powering Terra-View's new `/admin/events` page (v0.12.0).  Designed for purging bogus events from a unit that's been forwarding them in bulk (e.g. a stuck-triggered seismograph dumping hundreds of junk events before it's recovered).
+
+- **`DELETE /db/events/{event_id}`** — hard-delete one event row.  Also unlinks the associated blastware binary (`.AB0*`), `.a5.pkl`, `.sfm.json` sidecar, and `.h5` clean-waveform files via the WaveformStore.  Returns the per-file removal status.  404 if the event doesn't exist.
+- **`POST /db/events/delete_bulk`** — filter-based or id-list-based bulk delete with safety rails:
+    - Filters (`serial`, `from_dt`, `to_dt`, `false_trigger`) combine with AND; same semantics as `GET /db/events`.  `ids` is an additional inclusion list.  Refuses to run with no filters (would wipe the whole table — raises 422).
+    - `confirm` must be `true` to actually delete.  Otherwise returns a dry-run summary (`status: "dry_run"`, `matched: N`, `sample_serials: [...]`).
+    - `max_rows` (default 10,000) caps how many rows can be deleted by-filter in one call.  If exceeded, returns `status: "too_many"` with a hint to narrow or raise the cap.  Bypassed when only `ids` is supplied.
+- **`_cleanup_event_files(row)`** helper in `sfm/server.py` — best-effort `unlink()` of all four sidecar paths derived from the row's `blastware_filename`.  Logged at WARN if a path exists but unlink fails; the DB row deletion still proceeds.
+- **`SeismoDb.delete_event(id)` and `SeismoDb.delete_events_bulk(...)`** in `sfm/database.py` — both return the deleted row dict(s) so callers can do file cleanup.  `delete_events_bulk` raises `ValueError` if no filters are supplied.
+
+### Changed
+
+- **Default protocol recv timeout dropped from 30s → 10s** in `_build_client()`.  The unit usually responds in well under a second over cellular; 10s leaves comfortable headroom for retransmits while failing reasonably fast when a unit is wedged.  The two endpoints that perform full 5A waveform downloads still pass `timeout=120.0` explicitly so multi-minute event transfers are unaffected.
+- **`_build_client()` now accepts an optional `connect_timeout`** (TCP-only) so rescue / race-loop endpoints can fail fast on busy modems without affecting the protocol-level recv timeout.
+
+### Fixed
+
+- **`GET /device/monitor/status` returned HTTP 500 + uncaught traceback when the device was unresponsive**.  The retry-on-`Exception` inner block let the second `client.poll()`'s `ProtocolError` propagate out of the handler.  Now wrapped in proper try/except — returns 502 with `{"detail": "Protocol error: No S3 frame received within 10.0s ..."}` on timeout, 502 on connection errors, 500 only for genuinely unexpected exceptions.
+
+### Migration
+
+No schema changes.  No data migration required.
+
+If you've been running a previous version against a wedged unit and accumulated bogus events, the new `/admin/events` page in Terra-View v0.12.0 (or direct `POST /db/events/delete_bulk` with `confirm: true`) is the cleanup tool.  Watcher state on the upstream DL2 PC does NOT need separate cleaning — the watcher's `sfm_forwarded.json` keys on file sha256 and won't re-forward the same files.
+
+### Pairing
+
+This release pairs with **Terra-View v0.12.0**, which adds the `/admin/events` UI that consumes the new bulk-delete endpoints, the bulk false-trigger flagging on `/unit/{id}`, and the field-deployment workflow that uses the same `series3-watcher` → SFM ingest path as before.
+
+---
+
+## v0.16.1 — 2026-05-14
+
+### Fixed
+
+- **`record_type` always "Waveform" for forwarded events.**  `read_blastware_file()` hardcoded `ev.record_type = "Waveform"` regardless of the file's actual type.  The watcher-forward pipeline (the main BW ACH ingest path) compounds this by parsing files from a tmp path with a `.bw` suffix, so even a filename-based fallback inside the parser still wouldn't see the original extension.  Now:
+
+  1. New `derive_record_type_from_filename(filename)` helper in `minimateplus/event_file_io.py` derives the type from the LAST character of the filename's extension (V10.72+ AB0T scheme: `H`=Histogram, `W`=Waveform, `M`=Manual, `E`=Event, `C`=Combo).  Falls back to `"Waveform"` for old S338 firmware (3-char extensions ending in `0`) and any unrecognized suffix.
+  2. `read_blastware_file()` now calls the helper with its `path.name` so direct callers (the `--dry-run` path in `scripts/import_bw.py`, tests, ad-hoc scripts) get the right value automatically.
+  3. `WaveformStore.save_imported_bw()` overrides `ev.record_type` with the **original** filename's derived type after parsing (the tmp file inside the parser doesn't carry the original extension).  This is the path the live watcher-forwarder hits, so the DB column now reflects the actual event type going forward.
+
+  Events ingested before this fix are stuck with `record_type="Waveform"` in the DB; a one-off backfill (`UPDATE events SET record_type = ... WHERE blastware_filename LIKE '%H'`) would fix them retroactively if desired.  Terra-view's event modal also derives client-side from the filename, so the UI already shows the correct type for old events even without the backfill.
+
+---
+
+## v0.16.0 — 2026-05-11
+
+The "BW ACH ingestion" release.  When paired with **series3-watcher v1.5.0**, every Blastware ACH event (binary + `_ASCII.TXT` report) lands in SeismoDb with device-authoritative peaks, project metadata, sensor self-check, and ZC/Time-of-Peak data — without depending on the still-undecoded waveform body codec.  This is the end-to-end product win discussed in v0.15.0's "out of scope" notes: sortable / filterable monthly-summary review of historical events, populated from the BW ASCII export rather than re-decoded samples.
+
+### Added — `/db/import/blastware_file` rich-metadata ingestion
+
+- **Paired BW ASCII reports.**  The endpoint now accepts the `<binary>_<ext>_ASCII.TXT` partner BW writes alongside each event.  Pairing handles both filename conventions: ACH (`M529LK44_AB0_ASCII.TXT`) and manual-export (`M529LK44.AB0.TXT`).  When both present, ACH wins.
+- **`minimateplus/bw_ascii_report.py`** (new) — parser + `BwAsciiReport` dataclass for BW's per-event ASCII export.  Handles every field BW writes: identity, trigger config, per-channel PPV / ZC Freq / Time of Peak / Peak Acceleration / Peak Displacement, Peak Vector Sum + time, MicL PSPL / Time of Peak / ZC Freq, sensor self-check (Test Freq / Test Ratio / Test Amplitude / Pass-Fail per channel), monitor log, PC SW version.
+- **Position-based user-notes parsing.**  BW's Compliance Setup → Notes tab labels (Project / Client / User Name / Seis Loc) are *operator-editable* — an operator can rename them to "Building:", "Site Address:", etc.  Rather than maintain a label-spelling map, the parser uses positional matching between the `Units :` and `Geo Range :` anchors in the ASCII output.  The four canonical slots (project / client / operator / sensor_location) populate by position regardless of label; the original labels BW wrote are preserved in `report.user_note_labels` for downstream UIs (terra-view) to display verbatim.
+- **`bw_report` sidecar block.**  New top-level block in `.sfm.json` carrying the parsed BW report (trigger config, peaks with per-channel stats, mic block, sensor_check, monitor_log, PC SW version, operator-label labels).
+- **`apply_report_to_event(event, report)` helper.**  Overlays the report's device-authoritative fields onto an in-memory `Event` so `SeismoDb.insert_events()` writes correct DB columns instead of the broken-codec values from `_peaks_from_samples()`.
+
+### Fixed — three compounding bugs that left forwarded events with garbage data
+
+- **Import endpoint inserted under `serial="UNKNOWN"`.**  `_serial_from_event(ev)` was a stub that always returned `None`; the BW-filename-decoded serial that `WaveformStore` had already resolved was never surfaced to `db.insert_events`.  Now uses `rec["serial"]` as the authoritative source.  `scripts/repair_unknown_serials.py` repairs existing DB rows.
+- **`/db/units` ignored events from non-ACH ingest paths.**  `query_units()` only aggregated from `ach_sessions` — events that arrived via `save_imported_bw()` were never visible in the fleet overview even though they populated `events` correctly.  Now unions both tables.
+- **Re-imports left stale DB rows.**  The `IntegrityError` handler in `insert_events()` only refreshed filename / sidecar columns when a duplicate `(serial, timestamp)` arrived.  Peak values, project info, sample_rate, record_type stayed locked at whatever the first (often broken-codec) insert wrote.  Now the upsert path refreshes every device-authoritative column from the new data while preserving `false_trigger` and immutable fields (`id`, `created_at`).
+- **Server-side TXT pairing only knew the legacy convention.**  The endpoint stripped `.TXT` and looked up `<binary>` — which works for manual exports (`<binary>.TXT`) but not BW ACH (`<stem>_<ext>_ASCII.TXT`).  Reports were arriving in the multipart but silently dropped.  Now recognises both conventions and registers each report under all matching binary names.
+
+### Migration
+
+For existing deployments where events were forwarded by an older watcher (broken pairing) or imported during the UNKNOWN-bucketing window:
+
+1. `python -m scripts.repair_unknown_serials --db <path> --apply` to re-attribute `serial="UNKNOWN"` rows.
+2. Delete the watcher's `sfm_forwarded.json` state file and let it re-forward.  The server's upsert path will refresh the existing DB rows with the report's authoritative values.
+3. Operator review state (`false_trigger`, sidecar `review` block) is preserved across the re-import.
+
 ## v0.15.0 — 2026-05-07
 
 ### Added

CLAUDE.md

@@ -2,7 +2,7 @@
 
 Ground-up Python replacement for **Blastware**, Instantel's Windows-only software for
 managing MiniMate Plus seismographs. Connects over direct RS-232 or cellular modem
-(Sierra Wireless RV50 / RV55). Current version: **v0.14.3**.
+(Sierra Wireless RV50 / RV55). Current version: **v0.17.0**.
 
 When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
 
@@ -1353,6 +1353,8 @@ body) because writing a dial string may require DLE escaping for embedded contro
 
 ## What's next
 
+**See [README.md → Roadmap (Future)](README.md#roadmap-future) for the canonical deferred-work list.** This section is kept as a status log of in-progress / recently-shipped technical details (encoding schemes, byte layouts, etc.) that are too low-level for the README's roadmap.
+
 - **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
 - **Histograms** — decode histogram-mode A5 data (noise floor tracking)
 - **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed BYTE-PERFECT against BW reference (v0.14.3, 2026-05-05):** when fed the BW 5-1-26 3-sec capture's A5 frames, the SFM-built file matches BW's saved `M529LKIQ.G10` byte-for-byte (8708 bytes, 0 differences).  Live SFM downloads of event 0 (3-sec) and event 1 (3-sec continuation) both open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots.  Body assembly is just contiguous concatenation of frame contributions in stream order (probe → meta@0x1002 → meta@0x1004 → samples → TERM); no stripping, no overlay, no special handling.  Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that may need different handling — untested under v0.14.x). Extension mapping: extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`

Dockerfile

@@ -0,0 +1,18 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends curl && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY pyproject.toml requirements.txt ./
+COPY minimateplus ./minimateplus
+COPY sfm ./sfm
+COPY bridges ./bridges
+
+RUN pip install --no-cache-dir -e .
+
+EXPOSE 8200
+
+CMD ["python", "-m", "uvicorn", "sfm.server:app", "--host", "0.0.0.0", "--port", "8200"]

README.md

@@ -1,4 +1,4 @@
-# seismo-relay  `v0.15.0`
+# seismo-relay  `v0.17.0`
 
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing MiniMate Plus seismographs.
@@ -14,11 +14,12 @@ over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).
 > byte-perfect against Blastware captures across 2-sec, 3-sec, and 10-sec
 > events.** Generated `.G10` / `.AB0` files open cleanly in Blastware with
 > full Event Reports, frequency analysis, and waveform plots.
-> **v0.15.0 (2026-05-07)** adds layered per-event storage (BW binary +
-> raw 5A pickle + HDF5 + `.sfm.json` sidecar), a plot-ready
-> `sfm.plot.v1` JSON shape with server-side ADC-to-physical-units
-> conversion, and a BW-file importer for ingesting externally-produced
-> events.  See [CHANGELOG.md](CHANGELOG.md) for full version history.
+> **v0.16.0 (2026-05-11)** adds BW ASCII report ingestion to
+> `/db/import/blastware_file` — paired with **series3-watcher v1.5.0**,
+> every Blastware ACH event lands in SeismoDb with device-authoritative
+> peaks, project metadata, sensor self-check, and ZC/Time-of-Peak data,
+> without depending on the still-undecoded waveform body codec.
+> See [CHANGELOG.md](CHANGELOG.md) for full version history.
 
 ---
 
@@ -356,10 +357,40 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
 
 ## Roadmap (Future)
 
-- [ ] Verify 30-sec event download — body may exceed `0xFFFF` and force the device into a different `end_key` encoding (none of 2/3/10-sec test cases hit this boundary)
-- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
-- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
-- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
-- [ ] Modem manager — push RV50/RV55 configs via Sierra Wireless API
-- [ ] Histogram mode recording support (5A stream analysis for mode 0x03)
-- [ ] Call Home dial_string write support (requires DLE escaping for embedded control characters)
+### High-impact (unblocks product features)
+
+- [ ] **Waveform body codec reverse-engineering.**  The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`).  Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open.  Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise.  Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec.
+- [ ] **In-app waveform viewer accuracy.**  Depends on codec decode.  Plot.v1 JSON pipeline + viewer skeleton already exist; will start showing real waveforms automatically once `_decode_a5_waveform` produces correct samples.
+- [ ] **Terra-view integration** — seismo-relay router, unit detail page, VISON-style event listing.
+- [ ] **Vibration summary reports** — highest legit PPV per project → Word doc (false-trigger filtering first).
+
+### BW ASCII report parser enhancements (built in v0.16.0)
+
+- [ ] **Histogram-specific structural fields.**  Current parser handles the shared fields (PPV, ZC Freq, sensor self-check, project) but silently drops histogram-only fields: `Histogram Start/Stop Time`, `Histogram Start/Stop Date`, `Number of Intervals`, `Interval Size`, per-channel `Peak Time` + `Peak Date` (absolute timestamps rather than the waveform's `Time of Peak` relative seconds).
+- [ ] **Histogram interval bin-table parsing.**  Trailing 792-row table (per-interval Peak/Freq per channel + MicL) in histogram TXTs is unparsed.  Probably too big for the sidecar JSON; may want a separate `.histogram.h5` companion file.
+- [ ] **`>100 Hz` value parsing.**  Histogram TXTs use `>100 Hz` for out-of-range ZC freq; current `_parse_number()` returns `None` for these (loses information).
+
+### Ingestion gaps
+
+- [ ] **MLG forwarding.**  `series3-watcher` forwards event binaries + their `_ASCII.TXT` reports, but skips `.MLG` per-unit monitor log files entirely.  Adding an `POST /db/import/mlg_file` endpoint + watcher scan path would populate `monitor_log` for non-ACH-routed units (coverage queries, "was this unit monitoring on date X" lookups).
+- [ ] **0C-record raw bytes persistence in the sidecar.**  Currently on branch `claude/codec-re-cBGNe` as commit `a187124`; cherry-pick if useful as a standalone fix.  Preserves the 210-byte 0C record under `extensions.raw_records.waveform_record_b64` so future field-offset analysis (Peak Acceleration / Time of Peak / etc. — the fields BW computes client-side from samples) can run offline.
+
+### Operational
+
+- [ ] **`series3-watcher` file archive manager** — 90-day-old events moved to `<watch_folder>_archive/<year>/<month>/` subfolders.  Plan drafted in `claude/codec-re-cBGNe`'s plan-mode session; awaiting a 5-minute test on whether Blastware UI walks subfolders before any code lands (determines layout: in-place subfolders vs sibling archive).
+- [ ] **Compliance config encoder** — build raw write payloads from a `ComplianceConfig` object.
+- [ ] **Modem manager** — push RV50/RV55 configs via Sierra Wireless API.
+- [ ] **Call Home dial_string write support** (requires DLE escaping for embedded control characters).
+- [ ] **Histogram mode recording support** (5A stream analysis for mode 0x03 — separate from histogram ASCII parsing above).
+
+### Test coverage
+
+- [ ] Verify 30-sec event download — body may exceed `0xFFFF` and force the device into a different `end_key` encoding (none of the 2/3/10-sec test cases hit this boundary).
+- [ ] Histogram mode (0x03) write via SFM — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state.
+
+### Lower-priority cleanups
+
+- [ ] Compliance write anchor-9 cleanup — when changing recording_mode via SFM, a spurious `0x10` may persist after Histogram→other mode transitions.  Doesn't affect device operation but differs from BW's byte-perfect output.
+- [ ] Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring).
+- [ ] Call Home — map time slots 3/4 offsets; confirm `modem_power_relay_enabled`.
+- [ ] RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred).

docs/runbooks/wedged_unit_recovery.md

@@ -0,0 +1,255 @@
+# Runbook — Recovering a wedged unit stuck in a call-home loop
+
+**Original incident:** BE9558H at `166.246.130.1:9034`, recovered 2026-05-17.
+
+A field unit with a stuck-triggered geophone (or any hardware fault causing
+constant event triggering) will record events back-to-back, and if Auto Call
+Home is set to "After Event Recorded" the device will dial the office BW
+ACH server in a tight loop. Combined with a Sierra Wireless modem in
+bidirectional serial-TCP mode, this makes the unit effectively unreachable
+from SFM — every TCP connection we open gets killed when the modem flips
+from server-mode to client-mode to honor the device's next AT dial command.
+
+This runbook describes how to break the loop and recover control.
+
+---
+
+## Symptoms
+
+- Terra-View / SFM `/device/info` either hangs or fails on `count_events()`.
+- `/device/monitor/status` and `/device/rescue` return 502 (protocol timeout
+  waiting for POLL response) or 503 (TCP connect refused).
+- ACEmanager serial log shows repeating
+  `Connect to IP: <BW_IP> Port: <BW_PORT>` → `Shutdown TCP socket` cycles
+  every 30-60 seconds.
+- Spam-mode endpoints (`/device/stop_monitoring_spam`) report many
+  `sent_ok` but the device's monitoring state never changes.
+- `slow_drip` reports `[Errno 32] Broken pipe` after sending the preamble
+  but before completing the drip loop.
+
+If you see *all* of these, the unit is in this exact failure mode.
+
+---
+
+## Quick reference — how to recover
+
+You need **ACEmanager access** to the unit's modem.
+
+### Step 1: stop the modem's mode-flipping
+
+In ACEmanager → **Serial → Port Configuration**:
+
+| Field | Set to |
+|---|---|
+| **Destination Address** | clear (blank) |
+| **Destination Port** | `0` |
+
+Click **Apply**. This removes the modem's auto-dial-out target. The device's
+AT dial commands now error back at the modem instead of triggering a
+mode-flip, so the modem stays in TCP-server mode permanently and our inbound
+TCP sessions stay alive.
+
+*(Optional belt-and-suspenders: also add the BW server's port to
+**Security → Port Filtering - Outbound** as a blocked port, with
+Outbound Port Filtering Mode = Blocked Ports.)*
+
+### Step 2: stop monitoring on the device (slow drip)
+
+From the SFM host:
+
+```bash
+/home/serversdown/seismo-relay/scripts/slow_drip.sh <DEVICE_IP> <PORT>
+```
+
+Defaults are 120s duration with a drip every 3s. Watch the response:
+
+- `duration_s ≈ 120` and `drips_sent ≈ 40` → session held the full duration ✓
+- `bytes_received > 0` → device is responding ✓ (this is the success signal)
+
+If `duration_s` is small or `send_error: "Broken pipe"`, Step 1 didn't take
+hold — re-check ACEmanager, may need to reboot the modem after Apply.
+
+### Step 3: confirm monitoring stopped
+
+```bash
+curl 'http://localhost:8200/device/monitor/status?host=<DEVICE_IP>&tcp_port=<PORT>&force=true'
+# expect: {"is_monitoring": false, ...}
+```
+
+### Step 4: disable ACH at the device level + erase corrupted events
+
+Either fire the rescue endpoint:
+
+```bash
+/home/serversdown/seismo-relay/scripts/rescue_device.sh <DEVICE_IP> <PORT>
+```
+
+Or do the two steps manually:
+
+```bash
+# Disable ACH in the device's compliance config
+curl -X POST 'http://localhost:8200/device/call_home?host=<DEVICE_IP>&tcp_port=<PORT>' \
+  -H 'Content-Type: application/json' \
+  -d '{"auto_call_home_enabled": false}'
+
+# Erase corrupted event chain
+curl -X POST 'http://localhost:8200/device/events/erase?host=<DEVICE_IP>&tcp_port=<PORT>'
+```
+
+You can also do this via the SFM standalone UI → **Call Home** tab → set
+`Enable Auto Call Home` to `Disabled` → **Write to Device**.
+
+### Step 5: restore modem config (housekeeping)
+
+Once the device-side ACH is disabled, restore the modem's Destination
+Address and Port to the original values (e.g. `50.197.32.92` / `12345`) in
+ACEmanager. The modem will resume normal bidirectional behavior, but the
+unit won't issue any dial commands until ACH is explicitly re-enabled on
+the device.
+
+### Step 6: do NOT re-enable ACH on this unit until the underlying hardware
+fault is repaired. If you do, the call-home loop starts again immediately
+and you'll be running this runbook a second time.
+
+---
+
+## Why this works — the failure mode explained
+
+The Sierra Wireless RV50/RV55 serial port operates in one of two TCP modes
+at any moment:
+
+- **Server mode** — listens on `Device Port` (e.g. 9034), bridges inbound
+  TCP to the device's serial port. This is what we need to interact with
+  the device.
+- **Client mode** — when the device sends an AT dial command on its serial
+  TX line, the modem opens an outbound TCP to `Destination Address:Port`
+  and bridges that to serial.
+
+A serial port in this configuration is **bidirectional**: the modem flips
+between server and client modes on demand. When the device's firmware is
+healthy and only dials occasionally, this works fine.
+
+When the unit is constantly triggering events and ACH is set to "After
+Event Recorded", the device sends an AT dial command every few seconds.
+Each one causes the modem to:
+
+1. Drop any active inbound TCP session
+2. Flip to client mode
+3. Attempt outbound TCP to `Destination Address:Port`
+4. Hang for up to a minute waiting for it to succeed/fail
+5. Drop back to server mode
+
+**During the entire hang, no inbound TCP can establish.** Even between
+hangs, the modem closes any existing inbound session before flipping. So
+any tool that needs more than a few seconds of held TCP (e.g. POLL +
+config read + write) gets repeatedly kicked off.
+
+Clearing `Destination Address` removes step 3-4 from the cycle: the modem
+has nowhere to dial, so it doesn't flip modes when it receives an AT dial
+command. The serial port effectively becomes server-only, and inbound TCP
+sessions can stay open as long as needed.
+
+**This is a modem-layer issue, not a device firmware issue.** The device
+is alive and responsive the whole time — confirmed in the BE9558H
+recovery by 990 bytes of S3 responses received over a 120s slow-drip
+session once the modem was no longer mode-flipping.
+
+---
+
+## Why simpler approaches don't work
+
+| Approach | Why it fails |
+|---|---|
+| Standard `/device/info` | Triggers `count_events()` 1E/1F walk, takes 90s+ and hits corrupted event chain in this scenario |
+| `/device/rescue` race loop | Gets 502 (protocol timeout) because the modem closes the TCP before the POLL handshake can complete |
+| `/device/stop_monitoring_blind` (single frame) | Even if the bytes leave the wire, the device's protocol parser ignores write commands without a preceding POLL handshake (early-version bug, now fixed by including POLL preamble in blind sends) |
+| `/device/stop_monitoring_spam` (sub-second cadence) | Each session is killed by the modem's mode-flip before the device can drain its UART RX buffer; high-rate spam also risks UART FIFO overrun on the device side |
+| Outbound port firewall block alone | Stops the outbound TCP from succeeding, but doesn't stop the modem from *trying* and mode-flipping. Reduces but doesn't eliminate the contention. |
+| Modem reboot | Temporary — as soon as the device starts triggering again, the loop resumes within seconds |
+
+The combination of `slow_drip` + cleared `Destination Address` works because:
+
+1. The modem stops mode-flipping → TCP session stays open for the full
+   drip duration
+2. Slow drip rate → device's UART RX FIFO never overflows even if
+   firmware is busy with event recording
+3. The drip is `SESSION_RESET + STOP_MONITORING` every 3s → many
+   independent chances for the parser to land one valid frame
+4. Once one Stop Monitoring is parsed, event recording halts → firmware
+   has CPU to spare → subsequent operations are trivially easy
+
+---
+
+## Tooling reference
+
+All endpoints live in `seismo-relay/sfm/server.py`. All scripts live in
+`seismo-relay/scripts/` and default to SFM direct (`http://localhost:8200`),
+overridable via `SFM_BASE_URL`.
+
+### Endpoints added during BE9558H recovery
+
+| Endpoint | Purpose |
+|---|---|
+| `GET /device/events/storage_range` | SUB 0x06 — first/last event keys, `is_empty` flag. ~2s, no event walk. |
+| `GET /device/events/index` | SUB 0x08 — lifetime event counter (does NOT decrement on erase). ~2s. |
+| `POST /device/events/erase` | Full erase sequence 0xA3 → 0x1C → 0x06 → 0xA2. |
+| `POST /device/rescue` | Disable ACH + erase in one TCP session. Short timeouts for race-loop usage. |
+| `POST /device/stop_monitoring_blind` | Fire-and-forget Stop with full POLL preamble (single attempt). |
+| `POST /device/stop_monitoring_spam` | Server-side tight retry loop, sub-second cadence, duration-bounded. |
+| `POST /device/stop_monitoring_slow_drip` | One held TCP session, slow trickle of stop frames. **The endpoint that saved BE9558H.** |
+
+Also changed: default protocol recv timeout dropped from 30s → 10s in
+`_build_client`. Added `connect_timeout` knob to same. Cleaned up
+unhandled-exception path in `/device/monitor/status` so it returns 502
+instead of 500 on protocol timeouts.
+
+### Scripts
+
+| Script | Purpose |
+|---|---|
+| `scripts/rescue_device.sh` | Race-loop wrapper around `/device/rescue` |
+| `scripts/blind_stop.sh` | Race-loop wrapper around `/device/stop_monitoring_blind` |
+| `scripts/spam_stop.sh` | Single-call burst hammer (`/device/stop_monitoring_spam`) |
+| `scripts/slow_drip.sh` | Single-call held-session drip (`/device/stop_monitoring_slow_drip`) |
+| `scripts/watch_unit.sh` | Passive periodic reachability check, logs to file |
+
+---
+
+## Incident log — BE9558H, 2026-05-16/17
+
+What was wrong: Long-axis geophone developed an offset, constantly above
+trigger threshold → constant event recording → after-event ACH set →
+modem dialing office BW server (`50.197.32.92:12345`) every 30-60s.
+Local event chain corrupted (`next_boundary 0x100EE exceeds uint16`).
+
+Diagnostic path:
+
+1. `/device/info` slow, choked on event walk
+2. Built lightweight probe endpoints (`storage_range`, `index`) — useful
+   but didn't reach the wedged unit
+3. Built `/device/rescue` with short timeouts — got 502 (POLL no response)
+4. Built `/device/stop_monitoring_blind` — first version was a false
+   positive (no POLL preamble); fixed by including
+   `SESSION_RESET+POLL_PROBE+SESSION_RESET+POLL_DATA` in the dump
+5. Verified blind stop works on bench unit
+6. Built `/device/stop_monitoring_spam` — 420 successful sends over
+   5 min, zero behavior change on field unit
+7. Inspected ACEmanager logs → saw outbound dial-out attempts every ~30s,
+   confirmed device was not fully locked up
+8. Added outbound port-12345 firewall block → outbound attempts now fail
+   instantly but contention persisted
+9. Built `/device/stop_monitoring_slow_drip` — session died at 3s with
+   broken pipe (modem closing on us)
+10. Looked at full ACEmanager Port Configuration → **found
+    `Destination Address: 50.197.32.92` configured**, realized every AT
+    dial command was triggering a modem mode-flip that killed our inbound
+11. Cleared Destination Address + Port → slow_drip held 120s, device
+    responded with 990 bytes, 39 stop commands acked
+12. Disabled ACH at device level via `/device/call_home`, erased events
+
+Final state: device IDLE, memory 958.1 / 960 KB free, ACH disabled at
+device level, modem destination cleared (to be restored after physical
+service).
+
+Total time from "i was wondering if its possible to" first attempt to
+recovery: ~7 hours of intermittent debugging across one evening.

minimateplus/bw_ascii_report.py

@@ -0,0 +1,522 @@
+"""
+minimateplus/bw_ascii_report.py — parser for Blastware's per-event ASCII
+report (the .TXT file BW writes alongside each saved event binary).
+
+The ASCII export is the authoritative source for every "rich" per-event
+field that BW computes from the waveform but never persists in the BW
+binary itself:
+
+  - Per-channel PPV (Tran / Vert / Long / MicL)
+  - Peak Vector Sum + Peak Vector Sum Time
+  - Per-channel ZC Freq, Time of Peak, Peak Acceleration, Peak Displacement
+  - MicL PSPL, MicL Time of Peak, MicL ZC Freq
+  - Per-channel Sensor Self-Check (Test Freq / Test Ratio / Test Results)
+  - MicL Test Amplitude (mV)
+  - Battery, calibration date, monitor-log timestamps
+
+Persisting these values into the SFM database lets the monthly-summary
+review workflow ("show me events at Location X with PVS > 0.5") work
+without depending on the (still-undecoded) waveform body codec.
+
+Format (verified against decode-re/5-8-26 4-event bundle):
+
+  - One field per line, wrapped in double quotes:   `"Field Name : Value"`
+  - Field/value separator: literal ` : ` (space-colon-space).
+  - Some field names contain an internal `:` already (e.g. `"Project:"`),
+    so we split on the FIRST ` : ` only.
+  - Some fields have unit suffixes:  `"0.500 in/s"` / `"7.5 Hz"` / `"533 mv"`.
+  - A `"Monitor Log(s)"` marker line is followed by tab-separated rows
+    of `start_time<TAB>stop_time<TAB>description`.
+  - Final `"PC SW Version : ..."` line ends the metadata block.
+  - A blank line separates metadata from the sample table.
+  - Sample table starts with `   Tran   <TAB>   Vert   <TAB>...`, then
+    one row per sample (tab-separated, right-padded numeric values).
+  - Geo channel values are in in/s; MicL in dB(L) (or 0.000 below threshold).
+
+Because some metadata fields have whitespace quirks ("MicL  Time of
+Peak" has two spaces; the leading "Project:" value has its own colon),
+we normalise whitespace in the key before lookup.
+"""
+
+from __future__ import annotations
+
+import datetime
+import re
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple, Union
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Output dataclasses
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+@dataclass
+class ChannelStats:
+    """Per-channel derived stats, populated from an event report."""
+    ppv_ips:           Optional[float] = None      # in/s            (geo channels only)
+    zc_freq_hz:        Optional[float] = None      # Hz
+    time_of_peak_s:    Optional[float] = None      # seconds (relative to trigger; can be negative)
+    peak_accel_g:      Optional[float] = None      # g               (geo channels only)
+    peak_disp_in:      Optional[float] = None      # in              (geo channels only)
+
+
+@dataclass
+class MicStats:
+    """MicL-specific stats."""
+    weighting:         Optional[str]   = None      # e.g. "Linear Weighting"
+    pspl_dbl:          Optional[float] = None      # dB(L)
+    zc_freq_hz:        Optional[float] = None
+    time_of_peak_s:    Optional[float] = None
+
+
+@dataclass
+class SensorCheck:
+    """Per-channel sensor self-check result.
+
+    Geo channels report a frequency + ratio; MicL reports a frequency +
+    amplitude (mV).  All channels also have a Pass/Fail string.
+    """
+    test_freq_hz:      Optional[float] = None
+    test_ratio:        Optional[float] = None      # geo channels only
+    test_amplitude_mv: Optional[float] = None      # MicL only
+    test_results:      Optional[str]   = None      # "Passed" / "Failed"
+
+
+@dataclass
+class MonitorLogEntry:
+    """One row of the trailing Monitor Log(s) block."""
+    start_time:  Optional[datetime.datetime] = None
+    stop_time:   Optional[datetime.datetime] = None
+    description: Optional[str] = None
+
+
+@dataclass
+class BwAsciiReport:
+    """Structured representation of one BW per-event ASCII export."""
+    # ── Identity ─────────────────────────────────────────────────────────────
+    event_type:        Optional[str] = None         # e.g. "Full Waveform"
+    serial:            Optional[str] = None         # e.g. "BE11529"
+    version:           Optional[str] = None         # firmware version line
+    file_name:         Optional[str] = None         # e.g. "M529LK44.AB0"
+    event_datetime:    Optional[datetime.datetime] = None  # parsed from Event Time + Event Date
+
+    # ── Trigger / recording config ──────────────────────────────────────────
+    trigger_channel:        Optional[str]   = None  # e.g. "Vert" or "From Unit"
+    geo_trigger_level_ips:  Optional[float] = None
+    pretrig_s:              Optional[float] = None  # negative seconds
+    record_time_s:          Optional[float] = None
+    record_stop_mode:       Optional[str]   = None
+    sample_rate_sps:        Optional[int]   = None
+    battery_volts:          Optional[float] = None
+    calibration_date:       Optional[datetime.date] = None
+    calibration_by:         Optional[str]   = None  # e.g. "Instantel"
+    units:                  Optional[str]   = None  # e.g. "in/s and dB(L)"
+
+    # ── Operator-supplied metadata ──────────────────────────────────────────
+    # Parsed by POSITION from the 4-line "User Notes" block BW writes
+    # between the `Units :` and `Geo Range :` lines.  Position-based so
+    # the values populate correctly even when an operator renames the
+    # labels in Blastware's Compliance Setup → Notes tab (the 4 labels
+    # are user-editable, e.g. "Seis Loc:" → "Building:" → "Site Address:").
+    # The original labels BW wrote are preserved in `user_note_labels`
+    # so terra-view can render them as the operator named them.
+    project:           Optional[str] = None     # position 1 (BW default label "Project:")
+    client:            Optional[str] = None     # position 2 (BW default label "Client:")
+    operator:          Optional[str] = None     # position 3 (BW default label "User Name:")
+    sensor_location:   Optional[str] = None     # position 4 (BW default label "Seis Loc:")
+
+    # Maps canonical slot name → the literal label BW wrote in the ASCII
+    # export.  Empty if the User Notes block wasn't present.  Example
+    # when the operator renamed slot 4 to "Building:":
+    #     {"project": "Project:", "client": "Client:",
+    #      "operator": "User Name:", "sensor_location": "Building:"}
+    user_note_labels:  Dict[str, str] = field(default_factory=dict)
+
+    # ── Geo channel scaling ─────────────────────────────────────────────────
+    geo_range_ips:     Optional[float] = None       # 10.000 / 1.250
+
+    # ── Per-channel derived stats (geo + mic) ───────────────────────────────
+    channels:          Dict[str, ChannelStats] = field(default_factory=dict)
+    mic:               MicStats = field(default_factory=MicStats)
+
+    # ── Vector sum ──────────────────────────────────────────────────────────
+    peak_vector_sum_ips:    Optional[float] = None
+    peak_vector_sum_time_s: Optional[float] = None
+
+    # ── Sensor self-check (per channel) ─────────────────────────────────────
+    sensor_check:      Dict[str, SensorCheck] = field(default_factory=dict)
+
+    # ── Monitor log + tooling version ───────────────────────────────────────
+    monitor_log:       List[MonitorLogEntry] = field(default_factory=list)
+    pc_sw_version:     Optional[str] = None
+
+    # ── Sample table (optional; only parsed if requested) ───────────────────
+    # Each entry: (Tran, Vert, Long, MicL) in the report's units (geo
+    # channels in in/s, MicL in dB(L)).  None when parse_samples=False.
+    samples:           Optional[List[Tuple[float, float, float, float]]] = None
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Helpers
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+_KEY_NORMALISE_RE = re.compile(r"\s+")
+_NUMERIC_RE       = re.compile(r"^-?\d+(?:\.\d+)?")
+
+
+def _normalise_key(k: str) -> str:
+    """Collapse whitespace runs (incl. tabs) and strip — handles BW's
+    "MicL  Time of Peak" double-space and leading-colon quirks."""
+    return _KEY_NORMALISE_RE.sub(" ", k).strip()
+
+
+def _strip_quotes(line: str) -> str:
+    line = line.rstrip("\r\n")
+    if len(line) >= 2 and line.startswith('"') and line.endswith('"'):
+        return line[1:-1]
+    return line
+
+
+def _parse_number(value: str) -> Optional[float]:
+    """Pull the leading numeric portion out of a value like "0.500 in/s"."""
+    m = _NUMERIC_RE.match(value.strip())
+    if not m:
+        return None
+    try:
+        return float(m.group(0))
+    except ValueError:
+        return None
+
+
+def _parse_int(value: str) -> Optional[int]:
+    n = _parse_number(value)
+    return None if n is None else int(round(n))
+
+
+# Months exactly as BW writes them.
+_MONTHS = {
+    "January": 1, "February": 2, "March": 3, "April": 4,
+    "May": 5, "June": 6, "July": 7, "August": 8,
+    "September": 9, "October": 10, "November": 11, "December": 12,
+    # Short forms used in monitor-log rows ("Apr 23 /26").
+    "Jan": 1, "Feb": 2, "Mar": 3, "Apr": 4, "Jun": 6, "Jul": 7,
+    "Aug": 8, "Sep": 9, "Oct": 10, "Nov": 11, "Dec": 12,
+}
+
+
+def _parse_event_date(s: str) -> Optional[datetime.date]:
+    """Parse "April 23, 2026" or "May 8, 2026" → date."""
+    s = s.strip()
+    parts = s.replace(",", " ").split()
+    if len(parts) < 3:
+        return None
+    month_name, day_str, year_str = parts[0], parts[1], parts[2]
+    month = _MONTHS.get(month_name)
+    if month is None:
+        return None
+    try:
+        return datetime.date(int(year_str), month, int(day_str))
+    except ValueError:
+        return None
+
+
+def _parse_event_time(s: str) -> Optional[datetime.time]:
+    """Parse "15:56:35" → time."""
+    s = s.strip()
+    try:
+        h, m, sec = s.split(":")
+        return datetime.time(int(h), int(m), int(sec))
+    except (ValueError, IndexError):
+        return None
+
+
+def _parse_calibration(value: str) -> Tuple[Optional[datetime.date], Optional[str]]:
+    """Parse "April 29, 2025 by Instantel" → (date, "Instantel")."""
+    parts = value.split(" by ", 1)
+    date = _parse_event_date(parts[0])
+    by = parts[1].strip() if len(parts) > 1 else None
+    return date, by
+
+
+def _parse_monitor_row(line: str) -> Optional[MonitorLogEntry]:
+    """Parse a tab-separated monitor log row.
+
+    Format: `<start>\t<stop>\t<desc>` where each timestamp is BW's
+    short form "Mon DD /YY HH:MM:SS" (e.g. "Apr 23 /26 15:46:16").
+    Year is encoded as a 2-digit suffix; we expand "/26" → 2026.
+    """
+    parts = line.split("\t")
+    if len(parts) < 2:
+        return None
+    start = _parse_monitor_ts(parts[0])
+    stop  = _parse_monitor_ts(parts[1])
+    desc  = parts[2].strip() if len(parts) > 2 else None
+    if start is None and stop is None and not desc:
+        return None
+    return MonitorLogEntry(start_time=start, stop_time=stop, description=desc)
+
+
+def _parse_monitor_ts(s: str) -> Optional[datetime.datetime]:
+    """Parse "Apr 23 /26 15:46:16" → datetime."""
+    s = s.strip()
+    parts = s.split()
+    if len(parts) < 4:
+        return None
+    month = _MONTHS.get(parts[0])
+    if month is None:
+        return None
+    try:
+        day = int(parts[1])
+        # parts[2] looks like "/26" → century-flip to 2026
+        yy = int(parts[2].lstrip("/"))
+        year = 2000 + yy if yy < 80 else 1900 + yy
+        h, m, sec = (int(x) for x in parts[3].split(":"))
+        return datetime.datetime(year, month, day, h, m, sec)
+    except (ValueError, IndexError):
+        return None
+
+
+# ── User-notes positional slot map ──────────────────────────────────────────
+#
+# Blastware's Compliance Setup → Notes tab shows four operator-supplied
+# fields whose LABELS the operator can rename (see screenshot in
+# project archive).  Defaults are "Project:" / "Client:" /
+# "User Name:" / "Seis Loc:", but an operator using a different
+# convention can rename them to anything ("Building:", "Site:",
+# "Address:", etc.).  The ASCII export reflects whatever the operator
+# typed, so label-based matching is fragile.
+#
+# What IS reliable: BW always writes the 4 user-notes lines in the
+# same order, contiguously between the `Units :` line and the
+# `Geo Range :` line.  We parse them by POSITION and preserve the
+# operator's labels in `report.user_note_labels` so terra-view can
+# render them as the operator intended.
+
+_USER_NOTE_SLOTS = ("project", "client", "operator", "sensor_location")
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Top-level parser
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+def parse_report(text: Union[str, bytes], *, parse_samples: bool = False) -> BwAsciiReport:
+    """Parse a BW per-event ASCII export into a structured BwAsciiReport.
+
+    Set ``parse_samples=True`` to also populate ``report.samples`` with
+    the trailing sample table.  Default False because the table is
+    huge and most callers only want metadata for indexing.
+    """
+    if isinstance(text, bytes):
+        text = text.decode("ascii", errors="replace")
+
+    report = BwAsciiReport()
+    # Pre-create channel stat slots so callers can rely on them existing.
+    for ch in ("Tran", "Vert", "Long", "MicL"):
+        report.channels.setdefault(ch, ChannelStats())
+        report.sensor_check.setdefault(ch, SensorCheck())
+
+    lines = text.splitlines()
+    i = 0
+    n = len(lines)
+
+    in_monitor_log_section = False
+    event_time_str: Optional[str] = None
+    event_date: Optional[datetime.date] = None
+
+    # User-notes block detection.  We enter the block after parsing
+    # the "Units :" line and exit on the "Geo Range :" line.  Inside,
+    # the first 4 unmatched `<label> : <value>` lines are assigned to
+    # the 4 canonical operator-supplied slots by POSITION (project,
+    # client, operator, sensor_location) regardless of what the
+    # operator named the labels in BW's Compliance Setup → Notes tab.
+    in_user_notes_block = False
+    user_note_position = 0
+
+    while i < n:
+        raw_line = lines[i]
+        i += 1
+        # Blank line marks the start of the sample table.
+        if raw_line.strip() == "":
+            break
+
+        line = _strip_quotes(raw_line)
+
+        # Monitor log section: "Monitor Log(s)" header followed by N rows
+        # (still inside double-quoted lines), terminated by a non-row line
+        # like "PC SW Version : ..." or a blank line.
+        if not in_monitor_log_section and line.strip() == "Monitor Log(s)":
+            in_monitor_log_section = True
+            continue
+        if in_monitor_log_section:
+            # Heuristic: monitor rows contain a tab; the next "Field : Value"
+            # line ends the section.
+            if "\t" in line:
+                entry = _parse_monitor_row(line)
+                if entry:
+                    report.monitor_log.append(entry)
+                continue
+            # Falls through to the field parser below; clear the flag.
+            in_monitor_log_section = False
+
+        # "Field : Value" — split on FIRST occurrence of " : "
+        idx = line.find(" : ")
+        if idx < 0:
+            continue
+        key = _normalise_key(line[:idx])
+        value = line[idx + 3 :].strip()
+
+        # ── Identity / config ────────────────────────────────────────────────
+        if   key == "Event Type":           report.event_type = value
+        elif key == "Serial Number":        report.serial = value
+        elif key == "Version":              report.version = value
+        elif key == "File Name":            report.file_name = value
+        elif key == "Event Time":           event_time_str = value
+        elif key == "Event Date":           event_date = _parse_event_date(value)
+
+        elif key == "Trigger":              report.trigger_channel = value
+        elif key == "Geo Trigger Level":    report.geo_trigger_level_ips = _parse_number(value)
+        elif key == "Pre-trigger Length":   report.pretrig_s = _parse_number(value)
+        elif key == "Record Time":          report.record_time_s = _parse_number(value)
+        elif key == "Record Stop Mode":     report.record_stop_mode = value
+        elif key == "Sample Rate":          report.sample_rate_sps = _parse_int(value)
+        elif key == "Battery Level":        report.battery_volts = _parse_number(value)
+        elif key == "Calibration":
+            report.calibration_date, report.calibration_by = _parse_calibration(value)
+        elif key == "Units":
+            report.units = value
+            # Entering the user-notes block.  Next ~4 lines until
+            # "Geo Range :" are the operator-supplied notes.
+            in_user_notes_block = True
+            user_note_position = 0
+
+        elif key == "Geo Range":
+            # Exiting the user-notes block.
+            in_user_notes_block = False
+            report.geo_range_ips = _parse_number(value)
+
+        # User-notes block: assign by position (operator may have
+        # renamed the labels, so we don't trust them).  Preserve the
+        # original labels in `user_note_labels` for downstream UIs
+        # (terra-view) that want to display them as the operator
+        # named them.
+        elif in_user_notes_block and user_note_position < len(_USER_NOTE_SLOTS):
+            slot = _USER_NOTE_SLOTS[user_note_position]
+            setattr(report, slot, value)
+            report.user_note_labels[slot] = key
+            user_note_position += 1
+
+        # ── Per-channel stats ────────────────────────────────────────────────
+        # All match the pattern "{Channel} <stat-name>"
+        elif key in (
+            "Tran PPV", "Vert PPV", "Long PPV",
+            "Tran ZC Freq", "Vert ZC Freq", "Long ZC Freq",
+            "Tran Time of Peak", "Vert Time of Peak", "Long Time of Peak",
+            "Tran Peak Acceleration", "Vert Peak Acceleration", "Long Peak Acceleration",
+            "Tran Peak Displacement", "Vert Peak Displacement", "Long Peak Displacement",
+        ):
+            ch_name, stat = key.split(" ", 1)
+            cs = report.channels.setdefault(ch_name, ChannelStats())
+            num = _parse_number(value)
+            if   stat == "PPV":                 cs.ppv_ips        = num
+            elif stat == "ZC Freq":             cs.zc_freq_hz     = num
+            elif stat == "Time of Peak":        cs.time_of_peak_s = num
+            elif stat == "Peak Acceleration":   cs.peak_accel_g   = num
+            elif stat == "Peak Displacement":   cs.peak_disp_in   = num
+
+        # ── Vector Sum ───────────────────────────────────────────────────────
+        elif key == "Peak Vector Sum":
+            report.peak_vector_sum_ips = _parse_number(value)
+        elif key == "Peak Vector Sum Time":
+            report.peak_vector_sum_time_s = _parse_number(value)
+
+        # ── Microphone block ────────────────────────────────────────────────
+        elif key == "Microphone":
+            report.mic.weighting = value
+        elif key == "MicL PSPL":
+            report.mic.pspl_dbl = _parse_number(value)
+            # Mirror onto the "MicL" entry in channels so callers querying
+            # `channels["MicL"].ppv_ips` see something — but it's dB(L), not
+            # in/s, so we store as-is in the MicStats and mark the channel.
+        elif key == "MicL Time of Peak":
+            report.mic.time_of_peak_s = _parse_number(value)
+            cs = report.channels.setdefault("MicL", ChannelStats())
+            cs.time_of_peak_s = report.mic.time_of_peak_s
+        elif key == "MicL ZC Freq":
+            report.mic.zc_freq_hz = _parse_number(value)
+            cs = report.channels.setdefault("MicL", ChannelStats())
+            cs.zc_freq_hz = report.mic.zc_freq_hz
+
+        # ── Sensor self-check ────────────────────────────────────────────────
+        elif key in (
+            "Tran Test Freq", "Vert Test Freq", "Long Test Freq", "MicL Test Freq",
+            "Tran Test Ratio", "Vert Test Ratio", "Long Test Ratio",
+            "MicL Test Amplitude",
+            "Tran Test Results", "Vert Test Results", "Long Test Results", "MicL Test Results",
+        ):
+            ch_name, stat = key.split(" ", 1)
+            sc = report.sensor_check.setdefault(ch_name, SensorCheck())
+            if   stat == "Test Freq":      sc.test_freq_hz      = _parse_number(value)
+            elif stat == "Test Ratio":     sc.test_ratio        = _parse_number(value)
+            elif stat == "Test Amplitude": sc.test_amplitude_mv = _parse_number(value)
+            elif stat == "Test Results":   sc.test_results      = value
+
+        # ── Trailer ─────────────────────────────────────────────────────────
+        elif key == "PC SW Version":
+            report.pc_sw_version = value
+
+        # Unknown keys are silently dropped — forward-compat for future
+        # BW versions that may add fields.
+
+    # Combine event date + time into a datetime
+    if event_date is not None and event_time_str is not None:
+        t = _parse_event_time(event_time_str)
+        if t is not None:
+            report.event_datetime = datetime.datetime.combine(event_date, t)
+
+    if parse_samples:
+        report.samples = _parse_sample_table(lines, i)
+
+    return report
+
+
+def _parse_sample_table(
+    lines: List[str], start: int,
+) -> List[Tuple[float, float, float, float]]:
+    """Parse the trailing sample table.
+
+    The table starts with a header row ("   Tran   <TAB>...") and continues
+    until EOF.  Each data row is a tab-separated quartet of numeric values.
+    """
+    samples: List[Tuple[float, float, float, float]] = []
+    seen_header = False
+    for line in lines[start:]:
+        line = line.rstrip("\r\n")
+        if not line.strip():
+            continue
+        cols = [c.strip() for c in line.split("\t") if c.strip()]
+        if not seen_header:
+            # Header row contains channel names; numeric rows don't.
+            if any(c in ("Tran", "Vert", "Long", "MicL") for c in cols):
+                seen_header = True
+            continue
+        if len(cols) < 4:
+            continue
+        try:
+            samples.append((
+                float(cols[0]), float(cols[1]),
+                float(cols[2]), float(cols[3]),
+            ))
+        except ValueError:
+            continue
+    return samples
+
+
+def parse_report_file(
+    path: Union[str, Path], *, parse_samples: bool = False,
+) -> BwAsciiReport:
+    """Convenience: read a .TXT file from disk and parse it."""
+    return parse_report(Path(path).read_bytes(), parse_samples=parse_samples)

minimateplus/client.py

@@ -1362,20 +1362,6 @@ def _decode_waveform_record_into(data: bytes, event: Event) -> None:
 
     Modifies event in-place.
     """
-    # ── Always preserve the raw 210 bytes ─────────────────────────────────────
-    # The 0C record carries far more than just peaks + project strings:
-    # ZC Freq, Time of Peak, Peak Acceleration, Peak Displacement, Vector
-    # Sum Time, MicL Time of Peak, and the per-channel sensor self-check
-    # results (Test Freq / Ratio / Pass-Fail) all live somewhere in this
-    # 210-byte block.  Their byte offsets are not yet mapped — keeping the
-    # raw bytes lets us decode those fields offline once we have a paired
-    # (raw 0C, BW-report) sample to fit against.  Cheap to keep around
-    # (210 bytes per event).
-    try:
-        event._raw_record = bytes(data[:210])
-    except Exception:
-        pass
-
     # ── Record type + format detection ────────────────────────────────────────
     # `record_type` is the user-facing label ("Waveform" for any triggered
     # event regardless of timestamp-header layout).  `fmt` is the internal

minimateplus/event_file_io.py

@@ -15,7 +15,6 @@ declared in `event_to_sidecar_dict()`.
 
 from __future__ import annotations
 
-import base64
 import datetime
 import hashlib
 import json
@@ -27,6 +26,12 @@ from typing import Optional, Union
 
 from .models import Event, PeakValues, ProjectInfo, Timestamp
 from . import blastware_file as _bw  # avoid circular reference at module load
+from .bw_ascii_report import BwAsciiReport
+
+# Reference pressure for dB(L) → psi conversion (20 µPa expressed in psi).
+# Same constant as sfm/sfm_webapp.html so server-side and browser-side
+# conversions agree.
+_DBL_REF_PSI = 2.9e-9
 
 log = logging.getLogger(__name__)
 
@@ -42,7 +47,7 @@ SIDECAR_KIND   = "sfm.event"
 # bumped without a `pip install` re-run — leading to confusing stale
 # version stamps in sidecars.  Bump this constant and CHANGELOG.md
 # together at release time.
-TOOL_VERSION = "0.15.0"
+TOOL_VERSION = "0.16.1"
 
 try:
     # Best-effort: prefer the installed metadata when it's NEWER than the
@@ -95,6 +100,158 @@ def _peak_values_to_dict(pv: Optional[PeakValues]) -> dict:
     }
 
 
+def _bw_report_to_dict(report: BwAsciiReport) -> dict:
+    """Project a parsed BW ASCII report into the sidecar's `bw_report` block.
+
+    All fields are rendered as plain JSON-compatible types (no datetime
+    objects).  Channels are uniformly lowercased for stable JSON keys.
+    """
+    def _ch(ch_name: str) -> dict:
+        cs = report.channels.get(ch_name)
+        if cs is None:
+            return {}
+        out = {
+            "ppv_ips":         cs.ppv_ips,
+            "zc_freq_hz":      cs.zc_freq_hz,
+            "time_of_peak_s":  cs.time_of_peak_s,
+            "peak_accel_g":    cs.peak_accel_g,
+            "peak_disp_in":    cs.peak_disp_in,
+        }
+        # Drop all-None entries — keeps the JSON tidy for partial reports.
+        return {k: v for k, v in out.items() if v is not None}
+
+    def _sc(ch_name: str) -> dict:
+        sc = report.sensor_check.get(ch_name)
+        if sc is None:
+            return {}
+        out = {
+            "freq_hz":      sc.test_freq_hz,
+            "ratio":        sc.test_ratio,
+            "amplitude_mv": sc.test_amplitude_mv,
+            "result":       sc.test_results,
+        }
+        return {k: v for k, v in out.items() if v is not None}
+
+    monitor_log = []
+    for entry in report.monitor_log:
+        e = {
+            "start":       entry.start_time.isoformat() if entry.start_time else None,
+            "stop":        entry.stop_time.isoformat()  if entry.stop_time  else None,
+            "description": entry.description,
+        }
+        monitor_log.append({k: v for k, v in e.items() if v is not None})
+
+    return {
+        "available":   True,
+        "event_type":  report.event_type,
+        "version":     report.version,
+        "trigger": {
+            "channel":       report.trigger_channel,
+            "geo_level_ips": report.geo_trigger_level_ips,
+        },
+        "recording": {
+            "sample_rate_sps":  report.sample_rate_sps,
+            "record_time_s":    report.record_time_s,
+            "pretrig_s":        report.pretrig_s,
+            "stop_mode":        report.record_stop_mode,
+            "geo_range_ips":    report.geo_range_ips,
+            "units":            report.units,
+        },
+        "device": {
+            "battery_volts":    report.battery_volts,
+            "calibration_date": report.calibration_date.isoformat() if report.calibration_date else None,
+            "calibration_by":   report.calibration_by,
+        },
+        "peaks": {
+            "tran":         _ch("Tran"),
+            "vert":         _ch("Vert"),
+            "long":         _ch("Long"),
+            "vector_sum": {
+                "ips":     report.peak_vector_sum_ips,
+                "time_s":  report.peak_vector_sum_time_s,
+            },
+        },
+        "mic": {
+            "weighting":        report.mic.weighting,
+            "pspl_dbl":         report.mic.pspl_dbl,
+            "zc_freq_hz":       report.mic.zc_freq_hz,
+            "time_of_peak_s":   report.mic.time_of_peak_s,
+        },
+        "sensor_check": {
+            "tran": _sc("Tran"),
+            "vert": _sc("Vert"),
+            "long": _sc("Long"),
+            "mic":  _sc("MicL"),
+        },
+        "monitor_log":   monitor_log,
+        "pc_sw_version": report.pc_sw_version,
+    }
+
+
+def _dbl_to_psi(pspl_dbl: float) -> float:
+    """Convert dB(L) sound pressure level back to psi.  Uses the same
+    20 µPa reference (= 2.9e-9 psi) as the webapp so server-side and
+    browser-side conversions agree."""
+    return _DBL_REF_PSI * (10.0 ** (pspl_dbl / 20.0))
+
+
+def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
+    """Overlay device-authoritative fields from a parsed BW ASCII report
+    onto an in-memory Event, IN-PLACE.
+
+    Why this exists
+    ───────────────
+    `read_blastware_file()` parses the BW binary and fills `Event.peak_values`
+    via `_peaks_from_samples()` — which runs the (still-undecoded) BW body
+    codec assuming raw int16 LE and produces ±32K-shaped noise on every
+    channel.  Result: peak values land in the SeismoDb event row as
+    ~10 in/s on every event regardless of the actual signal.
+
+    When a paired BW ASCII report is available, the report carries the
+    device's own authoritative peak / project / sample-rate / record-time
+    values.  This helper folds those onto the Event before it flows to
+    `SeismoDb.insert_events()`, so the DB columns reflect the report
+    rather than the broken-codec output.
+
+    Fields overlaid (only when the report supplies a non-None value):
+      - peak_values.tran / .vert / .long              (from report.channels)
+      - peak_values.peak_vector_sum                   (from report.peak_vector_sum_ips)
+      - peak_values.micl  (psi)                       (from report.mic.pspl_dbl → psi)
+      - project_info.project / .client / .operator / .sensor_location
+      - sample_rate                                   (from report.sample_rate_sps)
+      - rectime_seconds                               (from report.record_time_s)
+
+    Fields NOT touched (operator-edit / parser-output preserved):
+      - timestamp, raw_samples, record_type, total_samples,
+        pretrig_samples, _waveform_key, _a5_frames, _raw_record
+      - false_trigger and review state (those live on the sidecar, not on Event)
+    """
+    if event.peak_values is None:
+        event.peak_values = PeakValues()
+    pv = event.peak_values
+    ch = report.channels
+    if (t := ch.get("Tran")) and t.ppv_ips is not None: pv.tran = t.ppv_ips
+    if (v := ch.get("Vert")) and v.ppv_ips is not None: pv.vert = v.ppv_ips
+    if (l := ch.get("Long")) and l.ppv_ips is not None: pv.long = l.ppv_ips
+    if report.peak_vector_sum_ips is not None:
+        pv.peak_vector_sum = report.peak_vector_sum_ips
+    if report.mic.pspl_dbl is not None and report.mic.pspl_dbl > 0:
+        pv.micl = _dbl_to_psi(report.mic.pspl_dbl)
+
+    if event.project_info is None:
+        event.project_info = ProjectInfo()
+    pi = event.project_info
+    if report.project:         pi.project         = report.project
+    if report.client:          pi.client          = report.client
+    if report.operator:        pi.operator        = report.operator
+    if report.sensor_location: pi.sensor_location = report.sensor_location
+
+    if report.sample_rate_sps:
+        event.sample_rate = report.sample_rate_sps
+    if report.record_time_s is not None:
+        event.rectime_seconds = report.record_time_s
+
+
 def _project_info_to_dict(pi: Optional[ProjectInfo]) -> dict:
     if pi is None:
         return {
@@ -124,49 +281,104 @@ def event_to_sidecar_dict(
     captured_at: Optional[datetime.datetime] = None,
     review: Optional[dict] = None,
     extensions: Optional[dict] = None,
+    bw_report: Optional[BwAsciiReport] = None,
 ) -> dict:
     """
     Build a v1 sidecar dict from an Event + the surrounding metadata.
 
     Pure helper — no file I/O.  Callers stitch the result into a sidecar
     via `write_sidecar()` (or POST it back via the PATCH endpoint).
+
+    When *bw_report* is supplied (e.g. by the ACH-forwarded import path
+    where Blastware writes a per-event ASCII report alongside the binary),
+    its decoded fields are folded into the sidecar:
+
+      - A new top-level ``bw_report`` block carries the rich derived
+        per-channel stats (Peak Acceleration, Peak Displacement, ZC Freq,
+        Time of Peak), the Peak Vector Sum + time, the per-channel sensor
+        self-check results, and monitor-log timestamps.
+      - ``peak_values`` is overlaid from the report (the report's PPV/PVS
+        values are computed by the device firmware and are authoritative;
+        anything ``read_blastware_file()`` derived from samples is
+        approximate at best until the body codec is decoded).
+      - ``project_info`` is overlaid from the report when the report
+        supplies a non-empty value (the report mirrors the device's
+        compliance config, which is what BW shows in its event report).
+      - ``event.timestamp`` is overlaid from the report's Event Date +
+        Event Time (BW's report timestamps are second-resolution and
+        match the binary's footer; we prefer the report value because
+        the BW-binary footer timestamp can drift on some firmware).
     """
-    if source_kind not in {"sfm-live", "sfm-ach", "bw-import"}:
+    if source_kind not in {"sfm-live", "sfm-ach", "bw-import", "idf-import"}:
         raise ValueError(f"unknown source_kind: {source_kind!r}")
 
     captured_at = captured_at or datetime.datetime.utcnow()
 
-    # Stash raw 0C record bytes in `extensions.raw_records` so future
-    # field-decoding work (Peak Acceleration, ZC Freq, Time of Peak,
-    # sensor self-check results, etc.) can run offline against committed
-    # sidecars without a live device.  Cheap (~280 bytes base64) and
-    # forward-compatible (older readers ignore unknown extensions keys).
-    ext_dict: dict = dict(extensions) if extensions else {}
-    raw_0c = getattr(event, "_raw_record", None)
-    if raw_0c:
-        rr = ext_dict.setdefault("raw_records", {})
-        # Don't clobber a raw_0c that callers explicitly passed in via
-        # `extensions=...` (e.g. round-trip preservation in patch_sidecar).
-        rr.setdefault("waveform_record_b64", base64.b64encode(raw_0c).decode("ascii"))
-        rr.setdefault("waveform_record_len", len(raw_0c))
+    # ── Overlay event fields from the report when present ───────────────────
+    timestamp_iso = _ts_iso(event.timestamp)
+    if bw_report and bw_report.event_datetime:
+        timestamp_iso = bw_report.event_datetime.isoformat()
 
-    return {
+    # Build peak_values, optionally overlaid from the report.  The report
+    # stores Mic peak as PSPL (dB(L)); we convert to psi to match the
+    # existing peak_values.mic_psi field.
+    peak_dict = _peak_values_to_dict(event.peak_values)
+    if bw_report:
+        ch = bw_report.channels
+        if (t := ch.get("Tran")) and t.ppv_ips is not None: peak_dict["transverse"]   = t.ppv_ips
+        if (v := ch.get("Vert")) and v.ppv_ips is not None: peak_dict["vertical"]     = v.ppv_ips
+        if (l := ch.get("Long")) and l.ppv_ips is not None: peak_dict["longitudinal"] = l.ppv_ips
+        if bw_report.peak_vector_sum_ips is not None:
+            peak_dict["vector_sum"] = bw_report.peak_vector_sum_ips
+        if bw_report.mic.pspl_dbl is not None and bw_report.mic.pspl_dbl > 0:
+            peak_dict["mic_psi"] = _dbl_to_psi(bw_report.mic.pspl_dbl)
+
+    # Project info: overlay from report (the report mirrors the
+    # session-start compliance config that BW renders in event reports).
+    proj_dict = _project_info_to_dict(event.project_info)
+    if bw_report:
+        if bw_report.project:         proj_dict["project"]         = bw_report.project
+        if bw_report.client:          proj_dict["client"]          = bw_report.client
+        if bw_report.operator:        proj_dict["operator"]        = bw_report.operator
+        if bw_report.sensor_location: proj_dict["sensor_location"] = bw_report.sensor_location
+
+    # Event-block fields: overlay from report where available.
+    event_block = {
+        "serial":           serial,
+        "timestamp":        timestamp_iso,
+        "waveform_key":     event._waveform_key.hex() if event._waveform_key else None,
+        "record_type":      event.record_type,
+        "sample_rate":      event.sample_rate,
+        "rectime_seconds":  event.rectime_seconds,
+        "total_samples":    event.total_samples,
+        "pretrig_samples":  event.pretrig_samples,
+    }
+    if bw_report:
+        # Report values are authoritative — they're the user-configured
+        # values BW reads back, not STRT-derived guesses.  In particular
+        # `event.rectime_seconds` from `read_blastware_file()` reads
+        # STRT[18] which is actually the `0x46` record-type marker (= 70)
+        # rather than the user's Record Time setting.  Always overwrite.
+        if bw_report.sample_rate_sps:
+            event_block["sample_rate"] = bw_report.sample_rate_sps
+        if bw_report.record_time_s is not None:
+            event_block["rectime_seconds"] = bw_report.record_time_s
+        # Derive total_samples + pretrig_samples per channel from the
+        # report's sample_rate × times.  These match the row count of
+        # the report's sample table (verified: event-c reports 1024 sps
+        # × (1.0 + 0.25) = 1280 rows).
+        if (sr := bw_report.sample_rate_sps) and bw_report.record_time_s is not None:
+            pretrig_s = abs(bw_report.pretrig_s) if bw_report.pretrig_s is not None else 0.0
+            event_block["total_samples"]   = int(round(sr * (bw_report.record_time_s + pretrig_s)))
+            event_block["pretrig_samples"] = int(round(sr * pretrig_s))
+
+    out = {
         "schema_version": SCHEMA_VERSION,
         "kind":           SIDECAR_KIND,
 
-        "event": {
-            "serial":           serial,
-            "timestamp":        _ts_iso(event.timestamp),
-            "waveform_key":     event._waveform_key.hex() if event._waveform_key else None,
-            "record_type":      event.record_type,
-            "sample_rate":      event.sample_rate,
-            "rectime_seconds":  event.rectime_seconds,
-            "total_samples":    event.total_samples,
-            "pretrig_samples":  event.pretrig_samples,
-        },
-
-        "peak_values":  _peak_values_to_dict(event.peak_values),
-        "project_info": _project_info_to_dict(event.project_info),
+        "event":        event_block,
+        "peak_values":  peak_dict,
+        "project_info": proj_dict,
 
         "blastware": {
             "filename":  blastware_filename,
@@ -189,9 +401,14 @@ def event_to_sidecar_dict(
             "notes":         "",
         },
 
-        "extensions": ext_dict,
+        "extensions": extensions or {},
     }
 
+    if bw_report:
+        out["bw_report"] = _bw_report_to_dict(bw_report)
+
+    return out
+
 
 # ── Sidecar IO ────────────────────────────────────────────────────────────────
 
@@ -429,6 +646,50 @@ def _peaks_from_samples(samples: dict[str, list[int]]) -> PeakValues:
     )
 
 
+_RECORD_TYPE_BY_EXT_SUFFIX = {
+    'H': 'Histogram',
+    'W': 'Waveform',
+    'M': 'Manual',
+    'E': 'Event',
+    'C': 'Combo',
+}
+
+
+def derive_record_type_from_filename(filename, default: str = "Waveform") -> str:
+    """Derive a BW Event's record_type from its filename's extension suffix.
+
+    V10.72+ MiniMate Plus firmware encodes the event type as the LAST
+    character of the extension (the `T` in BW's `AB0T` scheme):
+
+        ``M529LKIQ.G10H``  →  H  →  ``"Histogram"``
+        ``T350L385.VY0W``  →  W  →  ``"Waveform"``
+        ``...M``           →  M  →  ``"Manual"``
+        ``...E``           →  E  →  ``"Event"``
+        ``...C``           →  C  →  ``"Combo"``
+
+    Old S338 firmware uses 3-char extensions ending in ``0`` whose
+    encoding is not yet known — those fall through to ``default``.
+    Micromate Series 4 uses a different scheme entirely (observed:
+    ``IDFH``, ``IDFW``) but the LAST-char convention (H / W) still holds
+    for the type code, so it works for both families.
+
+    Returns ``default`` if filename is empty, has no extension, or the
+    suffix char isn't a recognized type code.
+    """
+    if not filename:
+        return default
+    try:
+        name = Path(filename).name
+    except (TypeError, ValueError):
+        return default
+    if '.' not in name:
+        return default
+    ext = name.rsplit('.', 1)[1]
+    if not ext:
+        return default
+    return _RECORD_TYPE_BY_EXT_SUFFIX.get(ext[-1].upper(), default)
+
+
 def read_blastware_file(path: Union[str, Path]) -> Event:
     """
     Parse a Blastware waveform file into an Event.
@@ -510,7 +771,12 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
     ev = Event(index=-1)
     if strt_fields.get("waveform_key"):
         ev._waveform_key = bytes.fromhex(strt_fields["waveform_key"])
-    ev.record_type     = "Waveform"
+    # Derive record_type from the filename's extension suffix (H/W/M/E/C).
+    # When called from save_imported_bw the path here is a tmp file with a
+    # ".bw" suffix, so the derivation falls back to "Waveform" and the
+    # caller overrides ev.record_type using the original filename — see
+    # waveform_store.save_imported_bw.
+    ev.record_type     = derive_record_type_from_filename(path.name)
     ev.rectime_seconds = strt_fields.get("rectime_seconds")
     ev.total_samples   = strt_fields.get("total_samples")
     ev.pretrig_samples = strt_fields.get("pretrig_samples")

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "seismo-relay"
-version = "0.15.0"
+version = "0.18.0"
 description = "Python client and REST server for MiniMate Plus seismographs"
 requires-python = ">=3.10"
 dependencies = [

scripts/backfill_record_type.py

@@ -0,0 +1,150 @@
+"""
+scripts/backfill_record_type.py — fix `record_type` on legacy event
+rows whose value was hardcoded to "Waveform" regardless of actual type.
+
+Why this is needed
+──────────────────
+Pre-v0.16.1 the BW file importer (`event_file_io.read_blastware_file`)
+hardcoded `ev.record_type = "Waveform"` for every imported event.  Fixed
+in commit aac1c8e — new ingests now derive the type from the Blastware
+filename's extension last character (H=Histogram, W=Waveform, M=Manual,
+E=Event, C=Combo) per the V10.72+ MiniMate Plus AB0T filename scheme.
+
+Effect on a server that imported events under the old code: every
+events row has `record_type = "Waveform"`, even for histograms,
+manuals, etc.  Visible in terra-view's event-detail modal under the
+"Record Type" field.  Terra-view also has a client-side workaround
+that derives the type from the filename for display purposes, so
+operators see the correct type in the UI even before this backfill.
+This script makes the DB column match what the UI is already showing,
+which matters for reporting and any downstream consumer that reads
+events.record_type directly.
+
+This script
+───────────
+Walks the `events` table and updates each row's `record_type` to the
+derived value from its `blastware_filename`.  Old S338 firmware files
+(3-char extensions ending in `0`) and any unrecognized suffix get
+left at the existing value (defaults to "Waveform").
+
+Idempotent: re-running after a successful backfill finds zero rows
+needing updates and exits cleanly (it always re-derives but only
+writes when the value would change).
+
+Usage
+─────
+  # Dry-run (default): print what would change, don't touch the DB
+  python -m scripts.backfill_record_type --db bridges/captures/seismo_relay.db
+
+  # Apply the backfill
+  python -m scripts.backfill_record_type --db bridges/captures/seismo_relay.db --apply
+"""
+
+from __future__ import annotations
+
+import argparse
+import sqlite3
+import sys
+from collections import Counter
+from pathlib import Path
+
+
+# Must stay in sync with minimateplus.event_file_io._RECORD_TYPE_BY_EXT_SUFFIX.
+_TYPE_FROM_SUFFIX = {
+    "H": "Histogram",
+    "W": "Waveform",
+    "M": "Manual",
+    "E": "Event",
+    "C": "Combo",
+}
+
+
+def derive_record_type(filename: str | None, default: str = "Waveform") -> str:
+    """Mirror of minimateplus.event_file_io.derive_record_type_from_filename.
+
+    Vendored here so this script runs without needing the seismo-relay
+    package on the Python path (useful on prod where you might be
+    running it via `docker exec` against a container's DB volume).
+    """
+    if not filename:
+        return default
+    name = Path(filename).name
+    if "." not in name:
+        return default
+    ext = name.rsplit(".", 1)[1]
+    if not ext:
+        return default
+    return _TYPE_FROM_SUFFIX.get(ext[-1].upper(), default)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument("--db", required=True, help="Path to seismo_relay.db")
+    ap.add_argument("--apply", action="store_true",
+                    help="Actually write changes (default is dry-run).")
+    ap.add_argument("--default", default="Waveform",
+                    help="Fallback record_type when filename doesn't encode one. "
+                         "Default: Waveform (matches the pre-fix bug's behavior).")
+    args = ap.parse_args()
+
+    db_path = Path(args.db)
+    if not db_path.exists():
+        print(f"ERROR: database not found at {db_path}", file=sys.stderr)
+        return 1
+
+    conn = sqlite3.connect(str(db_path))
+    conn.row_factory = sqlite3.Row
+    cur = conn.cursor()
+
+    cur.execute("""
+        SELECT id, blastware_filename, record_type
+        FROM events
+        WHERE blastware_filename IS NOT NULL
+          AND blastware_filename != ''
+    """)
+    rows = cur.fetchall()
+    total = len(rows)
+    print(f"Scanning {total:,} event rows…")
+    print()
+
+    # Tally proposed changes.
+    transitions: Counter[tuple[str, str]] = Counter()
+    update_ids: list[tuple[str, str]] = []
+    unrecognized = 0
+
+    for row in rows:
+        derived = derive_record_type(row["blastware_filename"], default=args.default)
+        current = row["record_type"] or ""
+        if derived == current:
+            continue
+        transitions[(current, derived)] += 1
+        update_ids.append((row["id"], derived))
+
+    if not update_ids:
+        print("Nothing to update — all rows already match.")
+        conn.close()
+        return 0
+
+    print(f"{len(update_ids):,} row(s) need updating:")
+    for (old, new), count in sorted(transitions.items(), key=lambda x: -x[1]):
+        print(f"  {count:>6,}  {old!r:14s} → {new!r}")
+    print()
+
+    if not args.apply:
+        print("(dry-run — re-run with --apply to write changes)")
+        conn.close()
+        return 0
+
+    print("Applying changes…")
+    cur.executemany(
+        "UPDATE events SET record_type = ? WHERE id = ?",
+        [(new, eid) for eid, new in update_ids],
+    )
+    conn.commit()
+    print(f"Done. Updated {cur.rowcount:,} row(s).")
+    conn.close()
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/blind_stop.sh

@@ -0,0 +1,100 @@
+#!/usr/bin/env bash
+# Fire-and-forget Stop Monitoring loop — for wedged or constantly-triggering units.
+#
+# Hammers POST /device/stop_monitoring_blind in a tight loop.  The endpoint
+# opens TCP, dumps SESSION_RESET + a few copies of the SUB 0x97 frame, and
+# closes — without ever reading an S3 response.  Each TCP-won attempt is
+# ~50ms of wire activity instead of the multi-frame handshake the regular
+# rescue endpoint does, so windows that are too small for the full rescue
+# can still land a stop-monitoring command.
+#
+# Usage:
+#   ./blind_stop.sh <host> [tcp_port]
+#
+# Env:
+#   SFM_BASE_URL    Default: http://localhost:8200 (SFM direct).
+#                   Set to http://localhost:8001/api/sfm to route through
+#                   Terra-View's proxy.
+#   MAX_ATTEMPTS    Default: 600
+#   SLEEP_S         Default: 0  (no backoff — hammer it)
+#   MAX_TIME_S      Default: 15
+#   CONNECT_TIMEOUT Default: 5
+#   REPEAT          Frames per TCP session (default 3 — increases hit rate
+#                   if the device is busy reading its own buffer).
+#   STOP_ON_OK      Default: 1.  Set to 0 to keep hammering indefinitely
+#                   even after successful sends (every 503 means the device
+#                   is in *another* session, every 200 means our bytes got
+#                   through — but the device may not have processed them).
+
+set -u
+
+host="${1:-}"
+tcp_port="${2:-9034}"
+if [[ -z "$host" ]]; then
+  echo "usage: $0 <host> [tcp_port]" >&2
+  exit 2
+fi
+
+base="${SFM_BASE_URL:-http://localhost:8200}"
+max_attempts="${MAX_ATTEMPTS:-600}"
+sleep_s="${SLEEP_S:-0}"
+max_time_s="${MAX_TIME_S:-15}"
+connect_timeout="${CONNECT_TIMEOUT:-5}"
+repeat="${REPEAT:-3}"
+stop_on_ok="${STOP_ON_OK:-1}"
+
+url="${base}/device/stop_monitoring_blind?host=${host}&tcp_port=${tcp_port}&connect_timeout=${connect_timeout}&repeat=${repeat}"
+
+echo "blind_stop: target ${host}:${tcp_port}  connect_timeout=${connect_timeout}s  repeat=${repeat}"
+echo "blind_stop: POST ${url}"
+echo "blind_stop: up to ${max_attempts} attempts, ${sleep_s}s between, ${max_time_s}s per request"
+echo "blind_stop: stop_on_ok=${stop_on_ok}"
+echo
+
+ok_count=0
+busy_count=0
+err_count=0
+started=$(date +%s)
+
+for ((i=1; i<=max_attempts; i++)); do
+  printf "[%4d] %s  " "$i" "$(date +%H:%M:%S)"
+  http_code=$(curl -sS -o /tmp/blind_resp.$$ -w "%{http_code}" \
+    --max-time "$max_time_s" \
+    -X POST "$url" || echo "000")
+  body=$(cat /tmp/blind_resp.$$ 2>/dev/null || true)
+  rm -f /tmp/blind_resp.$$
+
+  case "$http_code" in
+    200|201)
+      ok_count=$((ok_count + 1))
+      echo "SENT  $body"
+      if [[ "$stop_on_ok" == "1" ]]; then
+        elapsed=$(( $(date +%s) - started ))
+        echo
+        echo "blind_stop: success after ${i} attempts (${elapsed}s).  ok=${ok_count} busy=${busy_count} err=${err_count}"
+        echo "blind_stop: NEXT — wait ~10s, then try the full rescue:"
+        echo "  /home/serversdown/seismo-relay/scripts/rescue_device.sh ${host} ${tcp_port}"
+        exit 0
+      fi
+      ;;
+    503)
+      busy_count=$((busy_count + 1))
+      echo "busy (503)"
+      ;;
+    000)
+      err_count=$((err_count + 1))
+      echo "curl error"
+      ;;
+    *)
+      err_count=$((err_count + 1))
+      echo "HTTP $http_code  $body" | head -c 400
+      echo
+      ;;
+  esac
+  [[ "$sleep_s" != "0" ]] && sleep "$sleep_s"
+done
+
+elapsed=$(( $(date +%s) - started ))
+echo
+echo "blind_stop: gave up after ${max_attempts} attempts (${elapsed}s).  ok=${ok_count} busy=${busy_count} err=${err_count}" >&2
+exit 1

scripts/repair_unknown_serials.py

@@ -0,0 +1,151 @@
+"""
+scripts/repair_unknown_serials.py — re-attribute events stuck under
+`serial = 'UNKNOWN'` to their correct serial by decoding the BW filename.
+
+Why this is needed
+──────────────────
+The /db/import/blastware_file endpoint had a bug (fixed in commit a032fa5+1
+on the ach-report-ingestion branch) where every forwarded event was inserted
+with serial='UNKNOWN' because the endpoint's `_serial_from_event(ev)` stub
+returned None and never consulted the BW-filename serial that
+`WaveformStore.save_imported_bw()` had already decoded.
+
+Effect on a server that ran a buggy version: every forwarded event's
+SeismoDb row has `serial='UNKNOWN'`, even though the on-disk waveform
+store has correctly bucketed the files into `BE<NNNN>/` folders.  So
+the BW binaries / sidecars / HDF5s are fine, but `/db/units` and
+`/db/events?serial=...` queries don't surface the events.
+
+This script
+───────────
+Walks the events table looking for rows with `serial='UNKNOWN'` and
+re-attributes each one to the serial decoded from its
+`blastware_filename` column.  If the row's serial would collide with
+an existing row (already-correct duplicate from a later re-forward),
+the UNKNOWN row is deleted.  Otherwise the row's `serial` column is
+updated in-place.
+
+Idempotent: re-running after a successful repair finds zero matching
+rows and exits cleanly.
+
+Usage
+─────
+  # Dry-run (default): print what would change, don't touch the DB
+  python -m scripts.repair_unknown_serials --db bridges/captures/seismo_relay.db
+
+  # Apply the repair
+  python -m scripts.repair_unknown_serials --db bridges/captures/seismo_relay.db --apply
+"""
+
+from __future__ import annotations
+
+import argparse
+import sqlite3
+import sys
+from pathlib import Path
+
+# Reach into sfm.waveform_store for the serial decoder.  This script
+# is run from the repo root via `python -m scripts.repair_unknown_serials`.
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+from sfm.waveform_store import _serial_from_bw_filename
+
+
+def main(argv: list[str] | None = None) -> int:
+    p = argparse.ArgumentParser(
+        description="Re-attribute events stuck under serial='UNKNOWN'.",
+    )
+    p.add_argument(
+        "--db", required=True, type=Path,
+        help="Path to seismo_relay.db (e.g. bridges/captures/seismo_relay.db)",
+    )
+    p.add_argument(
+        "--apply", action="store_true",
+        help="Apply the repair.  Without this flag the script runs in "
+             "dry-run mode and only reports what would change.",
+    )
+    args = p.parse_args(argv)
+
+    if not args.db.exists():
+        print(f"DB not found: {args.db}", file=sys.stderr)
+        return 2
+
+    conn = sqlite3.connect(str(args.db))
+    conn.row_factory = sqlite3.Row
+
+    rows = list(conn.execute(
+        "SELECT id, serial, timestamp, blastware_filename "
+        "  FROM events "
+        " WHERE serial = 'UNKNOWN' "
+        " ORDER BY timestamp",
+    ))
+    print(f"Found {len(rows)} UNKNOWN-serial rows in events table.")
+    if not rows:
+        return 0
+
+    updated   = 0
+    deleted   = 0
+    unresolved = 0
+    by_serial: dict[str, int] = {}
+
+    for row in rows:
+        rid       = row["id"]
+        ts        = row["timestamp"]
+        bw_name   = row["blastware_filename"]
+        new_serial = _serial_from_bw_filename(bw_name) if bw_name else None
+        if not new_serial:
+            print(f"  ⚠ id={rid[:8]} ts={ts} filename={bw_name!r} — "
+                  f"cannot decode serial from filename; skipping")
+            unresolved += 1
+            continue
+
+        # Check for an existing row at the target (serial, timestamp).
+        existing = conn.execute(
+            "SELECT id FROM events WHERE serial = ? AND timestamp = ?",
+            (new_serial, ts),
+        ).fetchone()
+        action: str
+        if existing is None:
+            # Safe to UPDATE in place.
+            if args.apply:
+                conn.execute(
+                    "UPDATE events SET serial = ? WHERE id = ?",
+                    (new_serial, rid),
+                )
+            action = "UPDATE"
+            updated += 1
+        else:
+            # A correctly-attributed row already exists.  Drop the
+            # UNKNOWN duplicate.
+            if args.apply:
+                conn.execute("DELETE FROM events WHERE id = ?", (rid,))
+            action = "DELETE (dup)"
+            deleted += 1
+
+        by_serial[new_serial] = by_serial.get(new_serial, 0) + 1
+        print(f"  {action:14s}  id={rid[:8]}  ts={ts}  "
+              f"filename={bw_name}  →  {new_serial}")
+
+    if args.apply:
+        conn.commit()
+    conn.close()
+
+    print()
+    print(f"Summary:")
+    print(f"  UNKNOWN rows scanned:       {len(rows)}")
+    print(f"  Updated to real serial:     {updated}")
+    print(f"  Deleted (duplicate of an    ")
+    print(f"   already-correct row):      {deleted}")
+    print(f"  Unresolved (bad filename):  {unresolved}")
+    print()
+    if by_serial:
+        print(f"Per-serial breakdown of repaired rows:")
+        for serial, count in sorted(by_serial.items()):
+            print(f"  {serial:12s}  {count}")
+    if not args.apply:
+        print()
+        print("(dry-run — re-run with --apply to commit)")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/rescue_device.sh

@@ -0,0 +1,99 @@
+#!/usr/bin/env bash
+# Rescue an uncooperative MiniMate that's busy with another ACH session.
+#
+# Hammers POST /device/rescue in a tight loop with a short timeout.  When the
+# device is in an ACH session our SYN either gets refused or silently dropped
+# (5s connect timeout inside the endpoint) and we retry immediately.  When the
+# device is between sessions, our TCP wins, the endpoint disables Auto Call
+# Home and erases events inside the same session, then returns success.
+#
+# Usage:
+#   ./rescue_device.sh <host> [tcp_port] [--no-erase] [--no-disable-ach]
+#
+# Examples:
+#   ./rescue_device.sh 166.246.130.1 9034
+#   ./rescue_device.sh 166.246.130.1 9034 --no-erase     # just silence it
+#
+# Environment:
+#   SFM_BASE_URL    Defaults to http://localhost:8200 (SFM direct).
+#                   Set to http://localhost:8001/api/sfm to route through
+#                   Terra-View's proxy.  Direct mode avoids the proxy's
+#                   60s timeout, which matters for long-running endpoints.
+#   MAX_ATTEMPTS    Cap on retries (default 600 ≈ 30+ min).
+#   SLEEP_S         Backoff between attempts (default 1).
+#   MAX_TIME_S      Per-request timeout (default 60).
+#   CONNECT_TIMEOUT TCP connect timeout (default 5).
+#   RECV_TIMEOUT    Per-frame S3 recv timeout (default 5).  If POLL or any
+#                   subsequent frame doesn't respond within this window, the
+#                   rescue endpoint bails and this script retries.
+
+set -u
+
+host="${1:-}"
+tcp_port="${2:-9034}"
+shift 2 2>/dev/null || shift $# 2>/dev/null
+
+if [[ -z "$host" ]]; then
+  echo "usage: $0 <host> [tcp_port] [--no-erase] [--no-disable-ach]" >&2
+  exit 2
+fi
+
+disable_ach="true"
+erase="true"
+for arg in "$@"; do
+  case "$arg" in
+    --no-erase)        erase="false" ;;
+    --no-disable-ach)  disable_ach="false" ;;
+    *) echo "unknown flag: $arg" >&2; exit 2 ;;
+  esac
+done
+
+base="${SFM_BASE_URL:-http://localhost:8200}"
+max_attempts="${MAX_ATTEMPTS:-600}"
+sleep_s="${SLEEP_S:-1}"
+max_time_s="${MAX_TIME_S:-60}"
+connect_timeout="${CONNECT_TIMEOUT:-5}"
+recv_timeout="${RECV_TIMEOUT:-5}"
+
+url="${base}/device/rescue?host=${host}&tcp_port=${tcp_port}&disable_ach=${disable_ach}&erase=${erase}&connect_timeout=${connect_timeout}&recv_timeout=${recv_timeout}"
+
+echo "rescue: target ${host}:${tcp_port}  disable_ach=${disable_ach}  erase=${erase}"
+echo "rescue: connect_timeout=${connect_timeout}s  recv_timeout=${recv_timeout}s"
+echo "rescue: POST ${url}"
+echo "rescue: up to ${max_attempts} attempts, ${sleep_s}s between, ${max_time_s}s per request"
+echo
+
+started=$(date +%s)
+for ((i=1; i<=max_attempts; i++)); do
+  printf "[%3d] %s  " "$i" "$(date +%H:%M:%S)"
+  http_code=$(curl -sS -o /tmp/rescue_resp.$$ -w "%{http_code}" \
+    --max-time "$max_time_s" \
+    -X POST "$url" || echo "000")
+  body=$(cat /tmp/rescue_resp.$$ 2>/dev/null || true)
+  rm -f /tmp/rescue_resp.$$
+
+  case "$http_code" in
+    200|201)
+      elapsed=$(( $(date +%s) - started ))
+      echo "OK  (${elapsed}s total)"
+      echo "$body"
+      exit 0
+      ;;
+    503)
+      # Connection refused / timeout — device busy in another session.  Retry fast.
+      echo "busy (503)"
+      ;;
+    000)
+      echo "curl error (network)"
+      ;;
+    *)
+      echo "HTTP $http_code"
+      echo "  $body" | head -c 400
+      echo
+      ;;
+  esac
+  sleep "$sleep_s"
+done
+
+echo "rescue: gave up after ${max_attempts} attempts" >&2
+exit 1

scripts/slow_drip.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+# Hold a single TCP session open and drip stop-monitoring frames at a slow
+# rate, so the device's UART RX FIFO has time to drain between sends.
+#
+# Use when high-rate spam isn't landing — typically because the device's
+# firmware is too busy to drain its serial buffer fast enough and bytes
+# are being lost to UART overrun.
+#
+# Usage:
+#   ./slow_drip.sh <host> [tcp_port] [duration_s]
+#
+# Env:
+#   DURATION         Default: 120 (seconds; arg 3 overrides). Clamped 1..600.
+#   INTERVAL         Seconds between drip sends (default 3).  Lower = more
+#                    aggressive, more risk of FIFO overrun.  Higher = safer
+#                    but fewer total drips per duration.
+#   CONNECT_TIMEOUT  Default: 5
+#   SFM_BASE_URL     Default: http://localhost:8200 (SFM direct).
+
+set -u
+
+host="${1:-}"
+tcp_port="${2:-9034}"
+duration="${3:-${DURATION:-120}}"
+if [[ -z "$host" ]]; then
+  echo "usage: $0 <host> [tcp_port] [duration_s]" >&2
+  exit 2
+fi
+
+base="${SFM_BASE_URL:-http://localhost:8200}"
+interval="${INTERVAL:-3}"
+connect_timeout="${CONNECT_TIMEOUT:-5}"
+
+url="${base}/device/stop_monitoring_slow_drip?host=${host}&tcp_port=${tcp_port}&duration_s=${duration}&interval_s=${interval}&connect_timeout=${connect_timeout}"
+
+echo "slow_drip: target ${host}:${tcp_port}  duration=${duration}s  interval=${interval}s  connect_timeout=${connect_timeout}s"
+echo "slow_drip: POST ${url}"
+echo
+
+# Give curl enough slack to wait out the duration plus a buffer
+max_time=$(awk -v d="$duration" 'BEGIN { printf "%d", d + 30 }')
+
+curl -sS --max-time "$max_time" -X POST "$url"
+echo

scripts/spam_stop.sh

@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+# Hammer a device with blind stop-monitoring sessions as fast as possible.
+# Single HTTP call kicks off the burst inside SFM (no per-attempt HTTP
+# overhead).  Default: 10 seconds, ~500 ms per attempt = ~20 attempts/sec.
+#
+# Usage:
+#   ./spam_stop.sh <host> [tcp_port] [duration_s]
+#
+# Examples:
+#   ./spam_stop.sh 166.246.130.1                  # 10s burst
+#   ./spam_stop.sh 166.246.130.1 9034 30          # 30s burst
+#   DURATION=60 CONNECT_TIMEOUT=0.2 ./spam_stop.sh 166.246.130.1
+#
+# Env:
+#   SFM_BASE_URL     Default: http://localhost:8200 (SFM direct).
+#                    Set to http://localhost:8001/api/sfm to route through
+#                    Terra-View's proxy — but note the proxy has a 60s
+#                    timeout, so long bursts need direct mode.
+#   DURATION         Default: 10 (seconds; arg 3 overrides)
+#   CONNECT_TIMEOUT  Default: 0.5 (seconds)
+#   REPEAT           Default: 3   (stop frames per TCP session)
+
+set -u
+
+host="${1:-}"
+tcp_port="${2:-9034}"
+duration="${3:-${DURATION:-10}}"
+
+if [[ -z "$host" ]]; then
+  echo "usage: $0 <host> [tcp_port] [duration_s]" >&2
+  exit 2
+fi
+
+base="${SFM_BASE_URL:-http://localhost:8200}"
+connect_timeout="${CONNECT_TIMEOUT:-0.5}"
+repeat="${REPEAT:-3}"
+
+url="${base}/device/stop_monitoring_spam?host=${host}&tcp_port=${tcp_port}&duration_s=${duration}&connect_timeout=${connect_timeout}&repeat=${repeat}"
+
+echo "spam_stop: target ${host}:${tcp_port}  duration=${duration}s  connect_timeout=${connect_timeout}s  repeat=${repeat}"
+echo "spam_stop: POST ${url}"
+echo
+
+# Give curl enough slack to wait out the duration plus a buffer
+max_time=$(awk -v d="$duration" 'BEGIN { printf "%d", d + 10 }')
+
+curl -sS --max-time "$max_time" -X POST "$url"
+echo

scripts/watch_unit.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Passive monitor for a misbehaving unit.  Every INTERVAL seconds, attempts
+# a single short TCP probe + storage_range read and logs the result.  Designed
+# to run unattended for hours/days and tell you when the unit comes back.
+#
+# Usage:
+#   ./watch_unit.sh <host> [tcp_port]
+#
+# Env:
+#   INTERVAL    Seconds between checks (default 300 = 5 min)
+#   LOG_FILE    Append results here (default /tmp/watch_<host>.log)
+#   SFM_BASE_URL  Default: http://localhost:8200
+
+set -u
+
+host="${1:-}"
+tcp_port="${2:-9034}"
+if [[ -z "$host" ]]; then
+  echo "usage: $0 <host> [tcp_port]" >&2
+  exit 2
+fi
+
+interval="${INTERVAL:-300}"
+log_file="${LOG_FILE:-/tmp/watch_${host}.log}"
+base="${SFM_BASE_URL:-http://localhost:8200}"
+
+url="${base}/device/events/storage_range?host=${host}&tcp_port=${tcp_port}"
+
+echo "watch_unit: target ${host}:${tcp_port}  interval=${interval}s  log=${log_file}"
+echo "watch_unit: Ctrl-C to stop"
+
+while true; do
+  ts=$(date '+%Y-%m-%d %H:%M:%S')
+  http_code=$(curl -sS -o /tmp/watch_resp.$$ -w "%{http_code}" \
+    --max-time 20 "$url" || echo "000")
+  body=$(cat /tmp/watch_resp.$$ 2>/dev/null || true)
+  rm -f /tmp/watch_resp.$$
+
+  case "$http_code" in
+    200|201)
+      # Strip the raw_hex for readability
+      summary=$(echo "$body" | sed 's/"raw_hex":"[^"]*",*//; s/,*$//' | head -c 200)
+      echo "$ts  REACHABLE  $summary" | tee -a "$log_file"
+      ;;
+    502|503)
+      err=$(echo "$body" | head -c 150)
+      echo "$ts  ERROR_$http_code  $err" | tee -a "$log_file"
+      ;;
+    000)
+      echo "$ts  CURL_FAIL  (network/timeout)" | tee -a "$log_file"
+      ;;
+    *)
+      echo "$ts  HTTP_$http_code  $(echo "$body" | head -c 150)" | tee -a "$log_file"
+      ;;
+  esac
+
+  sleep "$interval"
+done

sfm/database.py

@@ -374,28 +374,64 @@ class SeismoDb:
                     inserted += 1
                 except sqlite3.IntegrityError:
                     skipped += 1
-                    # Upsert waveform fields onto the existing dedup row so a
-                    # re-download via the live endpoint refreshes filename /
-                    # size / sidecar without churning the rest of the row.
-                    if rec and ts:
-                        conn.execute(
-                            """
-                            UPDATE events
-                               SET blastware_filename = ?,
-                                   blastware_filesize = ?,
-                                   a5_pickle_filename = ?,
-                                   sidecar_filename   = ?
-                             WHERE serial = ? AND timestamp = ?
-                            """,
-                            (
-                                rec.get("filename"),
-                                rec.get("filesize"),
-                                rec.get("a5_pickle_filename"),
-                                rec.get("sidecar_filename"),
-                                serial,
-                                ts,
-                            ),
-                        )
+                    # UPSERT path: a row for this (serial, timestamp) already
+                    # exists.  Refresh every device-authoritative field from
+                    # the new data so that a re-import with better data (e.g.
+                    # a watcher re-forward where the previous attempt missed
+                    # the paired BW ASCII report) replaces stale peaks /
+                    # project info / sample_rate.
+                    #
+                    # Preserved (not in this UPDATE):
+                    #   id, waveform_key, session_id, created_at  — immutable / FK
+                    #   false_trigger                              — operator review state
+                    #
+                    # Behaviour change vs prior versions: this UPDATE used
+                    # to only refresh filename / filesize / a5_pickle /
+                    # sidecar fields.  As a result, the first insert's
+                    # broken-codec peak values were locked in forever even
+                    # if subsequent re-forwards arrived with correct
+                    # report-derived values.  Now every re-import lifts the
+                    # DB row up to whatever the latest Event carries.
+                    conn.execute(
+                        """
+                        UPDATE events
+                           SET tran_ppv           = ?,
+                               vert_ppv           = ?,
+                               long_ppv           = ?,
+                               peak_vector_sum    = ?,
+                               mic_ppv            = ?,
+                               project            = ?,
+                               client             = ?,
+                               operator           = ?,
+                               sensor_location    = ?,
+                               sample_rate        = ?,
+                               record_type        = ?,
+                               blastware_filename = ?,
+                               blastware_filesize = ?,
+                               a5_pickle_filename = ?,
+                               sidecar_filename   = ?
+                         WHERE serial = ? AND timestamp = ?
+                        """,
+                        (
+                            pv.tran            if pv else None,
+                            pv.vert            if pv else None,
+                            pv.long            if pv else None,
+                            pv.peak_vector_sum if pv else None,
+                            pv.micl            if pv else None,
+                            pi.project         if pi else None,
+                            pi.client          if pi else None,
+                            pi.operator        if pi else None,
+                            pi.sensor_location if pi else None,
+                            ev.sample_rate,
+                            ev.record_type,
+                            rec.get("filename")           if rec else None,
+                            rec.get("filesize")           if rec else None,
+                            rec.get("a5_pickle_filename") if rec else None,
+                            rec.get("sidecar_filename")   if rec else None,
+                            serial,
+                            ts,
+                        ),
+                    )
 
         log.debug("insert_events serial=%s inserted=%d skipped=%d",
                   serial, inserted, skipped)
@@ -455,6 +491,75 @@ class SeismoDb:
             )
         return cur.rowcount > 0
 
+    def delete_event(self, event_id: str) -> Optional[dict]:
+        """
+        Hard-delete one event row by id.  Returns the deleted row (so the
+        caller can clean up any on-disk files referenced by it) or None
+        if no row matched.
+        """
+        with self._connect() as conn:
+            row = conn.execute(
+                "SELECT * FROM events WHERE id = ?", (event_id,),
+            ).fetchone()
+            if row is None:
+                return None
+            conn.execute("DELETE FROM events WHERE id = ?", (event_id,))
+        return dict(row)
+
+    def delete_events_bulk(
+        self,
+        serial: Optional[str] = None,
+        from_dt: Optional[datetime.datetime] = None,
+        to_dt: Optional[datetime.datetime] = None,
+        false_trigger: Optional[bool] = None,
+        ids: Optional[list[str]] = None,
+    ) -> list[dict]:
+        """
+        Hard-delete events matching the given filters.  Returns the list
+        of deleted row dicts.  Refuses to delete with no filters at all
+        (would wipe the whole table) — raises ValueError.
+
+        Filter semantics match query_events: serial / from_dt / to_dt /
+        false_trigger combine with AND.  `ids` is an additional inclusion
+        list (event_id IN (...)); if supplied alongside other filters,
+        only rows matching all conditions are deleted.
+        """
+        clauses: list[str] = []
+        params:  list      = []
+
+        if serial:
+            clauses.append("serial = ?")
+            params.append(serial)
+        if from_dt:
+            clauses.append("timestamp >= ?")
+            params.append(from_dt.isoformat())
+        if to_dt:
+            clauses.append("timestamp <= ?")
+            params.append(to_dt.isoformat())
+        if false_trigger is not None:
+            clauses.append("false_trigger = ?")
+            params.append(1 if false_trigger else 0)
+        if ids:
+            placeholders = ",".join("?" * len(ids))
+            clauses.append(f"id IN ({placeholders})")
+            params.extend(ids)
+
+        if not clauses:
+            raise ValueError(
+                "delete_events_bulk refuses to delete with no filters "
+                "(would wipe the entire events table)"
+            )
+
+        where = "WHERE " + " AND ".join(clauses)
+
+        with self._connect() as conn:
+            rows = conn.execute(
+                f"SELECT * FROM events {where}", params,
+            ).fetchall()
+            if rows:
+                conn.execute(f"DELETE FROM events {where}", params)
+        return [dict(r) for r in rows]
+
     def update_event_review(self, event_id: str, review: dict) -> bool:
         """
         Sync derived index columns from a sidecar's `review` block.
@@ -564,21 +669,79 @@ class SeismoDb:
 
     def query_units(self) -> list[dict]:
         """
-        Return one row per known serial with summary stats:
-        last_seen, total_events, total_monitor_entries.
+        Return one row per known serial with summary stats.
+
+        Aggregates from BOTH source tables:
+          - `events`        — populated by every ingest path
+                              (live ACH, /db/import/blastware_file
+                              from the series3-watcher forwarder, etc.)
+          - `ach_sessions`  — only populated by the live ACH server;
+                              empty for events that came in via the
+                              BW-importer route.
+
+        Earlier this method only joined on `ach_sessions`, which made
+        watcher-forwarded units invisible to the SFM webapp's fleet
+        overview even though their events were correctly populated in
+        `events`.  Now we union the two and surface every serial that
+        has activity in either table.
+
+        Fields:
+          serial                — unit serial number (e.g. "BE11529")
+          last_seen             — most recent of MAX(events.timestamp)
+                                  and MAX(ach_sessions.session_time)
+          total_events          — COUNT(*) from `events` (the
+                                  authoritative count regardless of
+                                  ingest path)
+          total_monitor_entries — from `ach_sessions`, 0 when absent
+          total_sessions        — COUNT(*) from `ach_sessions`, 0 when absent
         """
         with self._connect() as conn:
-            rows = conn.execute(
-                """
-                SELECT
-                    s.serial,
-                    MAX(s.session_time)  AS last_seen,
-                    SUM(s.events_downloaded)  AS total_events,
-                    SUM(s.monitor_entries)    AS total_monitor_entries,
-                    COUNT(*)                  AS total_sessions
-                FROM ach_sessions s
-                GROUP BY s.serial
-                ORDER BY last_seen DESC
-                """
-            ).fetchall()
-        return [dict(r) for r in rows]
+            event_stats = {
+                row["serial"]: row
+                for row in conn.execute(
+                    """
+                    SELECT serial,
+                           MAX(timestamp) AS last_event_at,
+                           COUNT(*)        AS total_events
+                      FROM events
+                     GROUP BY serial
+                    """,
+                ).fetchall()
+            }
+            session_stats = {
+                row["serial"]: row
+                for row in conn.execute(
+                    """
+                    SELECT serial,
+                           MAX(session_time)         AS last_session_at,
+                           SUM(monitor_entries)      AS total_monitor_entries,
+                           COUNT(*)                  AS total_sessions
+                      FROM ach_sessions
+                     GROUP BY serial
+                    """,
+                ).fetchall()
+            }
+
+        all_serials = set(event_stats) | set(session_stats)
+        units = []
+        for serial in all_serials:
+            e = event_stats.get(serial)
+            s = session_stats.get(serial)
+            last_event_at   = e["last_event_at"]   if e else None
+            last_session_at = s["last_session_at"] if s else None
+            # Prefer whichever timestamp is more recent
+            last_seen = max(
+                (t for t in (last_event_at, last_session_at) if t),
+                default=None,
+            )
+            units.append({
+                "serial":                serial,
+                "last_seen":             last_seen,
+                "total_events":          e["total_events"]          if e else 0,
+                "total_monitor_entries": s["total_monitor_entries"] if s else 0,
+                "total_sessions":        s["total_sessions"]        if s else 0,
+            })
+
+        # Sort by last_seen desc; serials with no timestamp at all sink to the bottom.
+        units.sort(key=lambda u: u.get("last_seen") or "", reverse=True)
+        return units

sfm/dump_0c.py

@@ -1,216 +0,0 @@
-"""
-sfm.dump_0c — inspect the raw 210-byte SUB 0C waveform record stored in a
-sidecar JSON's `extensions.raw_records.waveform_record_b64`.
-
-Usage:
-
-    python -m sfm.dump_0c <sidecar.sfm.json> [<sidecar.sfm.json> ...]
-
-Prints, for each input:
-  - A header summarising the sidecar's metadata-block claims (peaks,
-    project, timestamp) — the "what BW says this event measured" view.
-  - A 16-byte-wide hex dump of the raw 0C record, annotated with known
-    field anchors (STRT, channel labels, project strings).
-  - A "candidate float regions" scan that brute-forces every byte
-    position as a float32 BE and prints any that yield a value in a
-    plausible range (1e-7 to 1e3) — useful for hunting where Peak
-    Acceleration / Peak Displacement / ZC Freq / Time of Peak live.
-
-Pairing the printed candidates with the BW Event Report values lets
-us nail down byte offsets for the missing fields without a live
-device.
-"""
-
-from __future__ import annotations
-
-import argparse
-import base64
-import json
-import struct
-import sys
-from pathlib import Path
-
-
-# ── Annotations for known anchors in a 210-byte 0C record ──────────────────
-
-# Anchors we look for and label inline in the hex dump.  Each is a needle
-# (bytes to find) and a short label.  Found via .find() — the first
-# occurrence wins.
-_ANCHORS = [
-    (b"Tran",            "Tran label  (PPV @ +6, PVS @ -12)"),
-    (b"Vert",            "Vert label  (PPV @ +6)"),
-    (b"Long",            "Long label  (PPV @ +6)"),
-    (b"MicL",            "MicL label  (peak psi @ +6)"),
-    (b"Project:",        "Project: label"),
-    (b"Client:",         "Client: label"),
-    (b"User Name:",      "User Name: label"),
-    (b"Seis Loc:",       "Seis Loc: label"),
-    (b"Extended Notes",  "Extended Notes label"),
-]
-
-
-def _hex_dump(data: bytes, anchors: dict[int, str]) -> str:
-    """Return a 16-byte-wide hex+ASCII dump, with anchor labels printed
-    on the line that contains the anchor's start byte."""
-    lines = []
-    for off in range(0, len(data), 16):
-        chunk = data[off : off + 16]
-        hex_part   = " ".join(f"{b:02x}" for b in chunk)
-        ascii_part = "".join(chr(b) if 32 <= b < 127 else "." for b in chunk)
-        line = f"  {off:04x}  {hex_part:<47}  |{ascii_part}|"
-
-        # If any anchor lands on a byte in this row, append a tag
-        tags = [
-            f"[{a:#04x}: {label}]"
-            for a, label in anchors.items()
-            if off <= a < off + 16
-        ]
-        if tags:
-            line += "  " + "  ".join(tags)
-        lines.append(line)
-    return "\n".join(lines)
-
-
-def _scan_float32_be(data: bytes, lo: float, hi: float) -> list[tuple[int, float]]:
-    """Brute-force every offset where data[off:off+4] is a float32 BE in
-    (lo, hi).  Includes negatives in the symmetric range."""
-    hits = []
-    for i in range(len(data) - 3):
-        try:
-            v = struct.unpack_from(">f", data, i)[0]
-        except struct.error:
-            continue
-        if v != v:                     # NaN
-            continue
-        if abs(v) < 1e-30 or abs(v) > 1e10:   # crap range
-            continue
-        a = abs(v)
-        if lo <= a <= hi:
-            hits.append((i, v))
-    return hits
-
-
-def _scan_uint16_be(data: bytes, lo: int, hi: int) -> list[tuple[int, int]]:
-    """Find every offset where uint16 BE is in [lo, hi]."""
-    hits = []
-    for i in range(len(data) - 1):
-        v = (data[i] << 8) | data[i + 1]
-        if lo <= v <= hi:
-            hits.append((i, v))
-    return hits
-
-
-def _summarize_sidecar(side: dict) -> str:
-    ev   = side.get("event", {})
-    pv   = side.get("peak_values", {})
-    pi   = side.get("project_info", {})
-    bw   = side.get("blastware", {})
-    return (
-        f"  serial:     {ev.get('serial')}\n"
-        f"  timestamp:  {ev.get('timestamp')}\n"
-        f"  waveform:   {ev.get('waveform_key')}  ({ev.get('record_type')})\n"
-        f"  sample_rate:{ev.get('sample_rate')} sps  rectime:{ev.get('rectime_seconds')}s\n"
-        f"  bw file:    {bw.get('filename')}  ({bw.get('filesize')} B)\n"
-        f"  peaks:      "
-        f"Tran={pv.get('transverse'):.5f}  "
-        f"Vert={pv.get('vertical'):.5f}  "
-        f"Long={pv.get('longitudinal'):.5f}  "
-        f"PVS={pv.get('vector_sum'):.5f} in/s  "
-        f"Mic={pv.get('mic_psi'):.6e} psi"
-        if all(pv.get(k) is not None for k in
-               ("transverse", "vertical", "longitudinal", "vector_sum", "mic_psi"))
-        else f"  peaks:      {pv}\n  project:    {pi}"
-    ) + (
-        f"\n  project:    {pi.get('project')!r}  / {pi.get('client')!r}  / "
-        f"operator={pi.get('operator')!r}  loc={pi.get('sensor_location')!r}"
-    )
-
-
-def dump_one(path: Path) -> int:
-    side = json.loads(path.read_text(encoding="utf-8"))
-
-    raw_b64 = (
-        side.get("extensions", {})
-            .get("raw_records", {})
-            .get("waveform_record_b64")
-    )
-    if not raw_b64:
-        print(f"\n=== {path} ===")
-        print("  ! no extensions.raw_records.waveform_record_b64 — sidecar")
-        print("    pre-dates raw-0C persistence (added in v0.15.x).  Re-save")
-        print("    the event from the device to capture the bytes.")
-        return 1
-
-    raw = base64.b64decode(raw_b64)
-
-    # Build anchor map
-    anchors: dict[int, str] = {}
-    for needle, label in _ANCHORS:
-        i = raw.find(needle)
-        if i >= 0:
-            anchors[i] = label
-
-    print(f"\n=== {path} ===")
-    print("metadata claimed by sidecar:")
-    print(_summarize_sidecar(side))
-
-    print(f"\nraw 0C record  ({len(raw)} bytes):")
-    print(_hex_dump(raw, anchors))
-
-    # Float32 BE candidates in geo-relevant ranges
-    geo_hits = _scan_float32_be(raw, 1e-5, 50.0)
-    # Filter: only show hits that are NOT trivially the per-channel labels'
-    # +6 PPV floats already documented (those will land in any sweep too).
-    print("\nfloat32 BE candidates (1e-5 .. 50.0):")
-    for off, v in geo_hits:
-        annotation = ""
-        for needle, _ in _ANCHORS[:4]:   # geo + mic labels
-            i = raw.find(needle)
-            if i >= 0 and off == i + 6:
-                annotation = f"  ← {needle.decode()} PPV (label+6)"
-                break
-        print(f"    {off:#04x}  ({off:3d})  {v:>+15.6f}{annotation}")
-
-    print("\nuint16 BE candidates ZC-Freq-ish (1..200):")
-    for off, v in _scan_uint16_be(raw, 1, 200):
-        if v < 5:    # too noisy at very low end
-            continue
-        print(f"    {off:#04x}  ({off:3d})  = {v}")
-
-    print("\nuint16 BE candidates Time-of-Peak-ish if stored as ms (1..30000):")
-    for off, v in _scan_uint16_be(raw, 1, 30000):
-        if v < 100:  # noise filter
-            continue
-        # Only the first ~80 are worth showing — too many hits otherwise
-        if off > 80:
-            break
-        print(f"    {off:#04x}  ({off:3d})  = {v} ms ?")
-
-    print()
-    return 0
-
-
-def main(argv: list[str] | None = None) -> int:
-    p = argparse.ArgumentParser(
-        description="Inspect a saved 0C waveform record from a sidecar JSON.",
-    )
-    p.add_argument(
-        "sidecars",
-        nargs="+",
-        type=Path,
-        help="Path(s) to <event>.sfm.json sidecar file(s).",
-    )
-    args = p.parse_args(argv)
-
-    rc = 0
-    for path in args.sidecars:
-        try:
-            rc |= dump_one(path)
-        except Exception as exc:
-            print(f"\n=== {path} ===\n  ERROR: {exc}", file=sys.stderr)
-            rc |= 2
-    return rc
-
-
-if __name__ == "__main__":
-    sys.exit(main())

sfm/idf_ascii_report.py

@@ -0,0 +1,291 @@
+"""
+sfm/idf_ascii_report.py — parse Thor (Micromate Series IV) IDF ASCII reports.
+
+Thor exports a `.IDFW.txt` or `.IDFH.txt` sidecar next to each `.IDFW`
+(waveform) or `.IDFH` (histogram) event binary.  Each sidecar is a
+plain-text file with `"Key : Value"` lines covering the full device-
+authoritative event metadata — PPV per channel, ZC Freq, Time of Peak,
+Peak Acceleration / Displacement, sensor self-check results, project
+strings, calibration date, battery level, etc. — followed by a raw
+waveform-samples block headed by the literal line "Waveform Data Channels".
+
+This is the Thor analogue of `minimateplus/bw_ascii_report.py` for the
+Blastware (Series III) report format.  The parser is intentionally
+permissive: we extract everything we recognise into a flat dict and
+silently ignore anything we don't.  Downstream callers parse units
+(`"0.2119 in/s"` → 0.2119) only on the fields they need.
+
+Example input (truncated):
+
+    "EventType : Full Waveform"
+    "SampleRate : 1024 sps"
+    "EventTime : 16:27:23"
+    "EventDate : 2023-12-19"
+    "TranPPV : 0.0251 in/s"
+    "VertPPV : 0.2119 in/s"
+    "LongPPV : 0.0282 in/s"
+    "PeakVectorSum : 0.2131 in/s"
+    "MicPSPL : 99.4 dB(L)"
+    "TranZCFreq : 6.5 Hz"
+    "SerialNumber : UM11719"
+    "Version : Micromate ISEE 11.0AK"
+    "FileName : UM11719_20231219162723.IDFW"
+    "BatteryLevel : 3.8 volts"
+    "Calibration : November 22, 2023 by Instantel"
+    "TranTestResults : Passed"
+    "TitleString1 : UPMC Presby-Loc 3-Level1-1R Elevator Rm"
+    Waveform Data Channels
+        Tran    Vert    Long    MicL
+        0.0003  -0.0003  0.0003  0.00013
+        ...
+"""
+
+from __future__ import annotations
+
+import datetime
+import re
+from typing import Any, Dict, Optional, Tuple, Union
+
+
+# Lines look like:  "Key : Value"   (quotes literal, single ":" separator)
+_LINE_RE = re.compile(r'^\s*"?([^":]+?)"?\s*:\s*"?(.*?)"?\s*$')
+
+# Marker that ends the metadata block — everything after is raw sample data.
+_WAVEFORM_BLOCK_MARKER = "waveform data channels"
+
+
+def _normalize_key(raw: str) -> str:
+    """Convert "TranPPV" / "PreTriggerLength" → snake_case."""
+    s = raw.strip()
+    # Insert underscore between lower→upper / digit→letter transitions
+    s = re.sub(r"(?<=[a-z0-9])(?=[A-Z])", "_", s)
+    s = re.sub(r"(?<=[A-Z])(?=[A-Z][a-z])", "_", s)
+    s = s.replace("-", "_").replace(" ", "_")
+    return s.lower()
+
+
+def _strip_unit_suffix(value: str) -> str:
+    """Return the numeric part of values like "0.2119 in/s" → "0.2119"."""
+    parts = value.strip().split()
+    return parts[0] if parts else value.strip()
+
+
+def _parse_float(value: str) -> Optional[float]:
+    try:
+        return float(_strip_unit_suffix(value))
+    except (ValueError, TypeError):
+        return None
+
+
+def _parse_int(value: str) -> Optional[int]:
+    try:
+        return int(float(_strip_unit_suffix(value)))
+    except (ValueError, TypeError):
+        return None
+
+
+def parse_idf_report(text: Union[str, bytes]) -> Dict[str, Any]:
+    """
+    Parse a Thor IDFW.txt / IDFH.txt sidecar.
+
+    Returns a flat dict with two kinds of entries:
+
+      - **Raw fields** — every `Key : Value` line, keyed by snake_case
+        of the original key, value as a string (unit suffix preserved).
+        Lets callers grab any field we haven't explicitly normalised.
+
+      - **Derived fields** — a curated set with parsed types:
+          * `serial_number`     str
+          * `event_type`        str  ("Full Waveform" / "Full Histogram")
+          * `event_datetime`    ISO-8601 string ("YYYY-MM-DDTHH:MM:SS") when
+                                 both EventDate and EventTime are present
+          * `sample_rate`       int  (samples/sec)
+          * `tran_ppv`,`vert_ppv`,`long_ppv` float (in/s)
+          * `mic_ppv`           float (dB or psi — same units as MicPSPL)
+          * `peak_vector_sum`   float (in/s)
+          * `tran_zc_freq`,`vert_zc_freq`,`long_zc_freq` float (Hz)
+          * `record_time_sec`   float (seconds)
+          * `pre_trigger_sec`   float (seconds)
+          * `project`           str  (from TitleString1 — Thor's location)
+          * `client`            str  (TitleString2)
+          * `operator`          str  (TitleString3 — company/operator)
+          * `notes`             str  (TitleString4)
+          * `setup`             str
+          * `version`           str  (firmware)
+          * `battery_volts`     float
+          * `calibration_text`  str  (e.g. "November 22, 2023 by Instantel")
+          * `tran_test_passed`, `vert_test_passed`, `long_test_passed`,
+            `mic_test_passed`  bool  ("Passed" → True; anything else → False)
+          * `filename`          str  (FileName line — useful sanity check)
+
+    Stops parsing at the literal "Waveform Data Channels" line; the
+    raw-samples block is left to whoever wants to decode the binary.
+
+    Input may be `str` or `bytes` (`utf-8`/`latin-1` tolerant).
+    """
+    if isinstance(text, bytes):
+        try:
+            text = text.decode("utf-8")
+        except UnicodeDecodeError:
+            text = text.decode("latin-1", errors="replace")
+
+    raw: Dict[str, str] = {}
+
+    for line in text.splitlines():
+        stripped = line.strip()
+        if not stripped:
+            continue
+        if stripped.lower().startswith(_WAVEFORM_BLOCK_MARKER):
+            break
+        m = _LINE_RE.match(stripped)
+        if not m:
+            continue
+        key = _normalize_key(m.group(1))
+        value = m.group(2).strip()
+        # Multi-value lines (Channel, Units, etc.) — coalesce by appending.
+        if key in raw:
+            raw[key] = raw[key] + "; " + value
+        else:
+            raw[key] = value
+
+    out: Dict[str, Any] = dict(raw)  # keep all raw fields
+
+    # ── Derived fields ───────────────────────────────────────────────────────
+
+    def _take(*candidates: str) -> Optional[str]:
+        for c in candidates:
+            if c in raw:
+                return raw[c]
+        return None
+
+    # Event identity
+    if "serial_number" in raw:
+        out["serial_number"] = raw["serial_number"]
+    if "event_type" in raw:
+        out["event_type"] = raw["event_type"]
+    if "file_name" in raw:
+        out["filename"] = raw["file_name"]
+
+    # Combined date+time.  Waveform sidecars use "EventDate" / "EventTime";
+    # histogram sidecars use "HistogramStartDate" / "HistogramStartTime".
+    # Prefer the event_* names when both are present.
+    ed = raw.get("event_date") or raw.get("histogram_start_date")
+    et = raw.get("event_time") or raw.get("histogram_start_time")
+    if ed and et:
+        try:
+            dt = datetime.datetime.strptime(f"{ed} {et}", "%Y-%m-%d %H:%M:%S")
+            out["event_datetime"] = dt.isoformat()
+        except ValueError:
+            pass
+
+    # Numeric scalars
+    for key in ("sample_rate",):
+        v = raw.get(key)
+        if v is not None:
+            iv = _parse_int(v)
+            if iv is not None:
+                out[key] = iv
+
+    for key in ("tran_ppv", "vert_ppv", "long_ppv", "peak_vector_sum",
+                "tran_zc_freq", "vert_zc_freq", "long_zc_freq",
+                "tran_peak_acceleration", "vert_peak_acceleration",
+                "long_peak_acceleration",
+                "tran_peak_displacement", "vert_peak_displacement",
+                "long_peak_displacement",
+                "tran_time_of_peak", "vert_time_of_peak", "long_time_of_peak",
+                "mic_time_of_peak", "mic_zc_freq"):
+        v = raw.get(key)
+        if v is not None:
+            fv = _parse_float(v)
+            if fv is not None:
+                out[key] = fv
+
+    # Microphone — Thor reports MicPSPL (dB(L)) which is the closest
+    # analogue to BW's mic_ppv.  Stored as a float; units are in the
+    # original raw field (`mic_pspl` raw entry preserves "99.4 dB(L)").
+    mic = raw.get("mic_pspl")
+    if mic is not None:
+        fv = _parse_float(mic)
+        if fv is not None:
+            out["mic_ppv"] = fv
+
+    # Record / pre-trigger duration
+    rt = raw.get("record_time")
+    if rt is not None:
+        fv = _parse_float(rt)
+        if fv is not None:
+            out["record_time_sec"] = fv
+    pt = raw.get("pre_trigger_length")
+    if pt is not None:
+        fv = _parse_float(pt)
+        if fv is not None:
+            out["pre_trigger_sec"] = fv
+
+    # Project / client / operator / location strings.  Thor's title
+    # strings are operator-defined; conventional mapping (per Thor's
+    # default TitleNote labels in the example data):
+    #   TitleString1 = Location  → project (sensor location identifier)
+    #   TitleString2 = Client    → client
+    #   TitleString3 = Company   → operator (the monitoring company)
+    #   TitleString4 = Notes     → notes
+    out["project"]  = _take("title_string1")
+    out["client"]   = _take("title_string2")
+    out["operator"] = _take("title_string3", "operator")
+    out["notes"]    = _take("title_string4", "post_event_note")
+
+    if "setup" in raw:
+        out["setup"] = raw["setup"]
+    if "version" in raw:
+        out["version"] = raw["version"]
+
+    # Battery (e.g. "3.8 volts" → 3.8)
+    bl = raw.get("battery_level")
+    if bl is not None:
+        fv = _parse_float(bl)
+        if fv is not None:
+            out["battery_volts"] = fv
+
+    # Calibration line is free-form (e.g. "November 22, 2023 by Instantel").
+    if "calibration" in raw:
+        out["calibration_text"] = raw["calibration"]
+
+    # Sensor self-check results — bool flags
+    for key, out_key in (
+        ("tran_test_results", "tran_test_passed"),
+        ("vert_test_results", "vert_test_passed"),
+        ("long_test_results", "long_test_passed"),
+        ("mic_test_results",  "mic_test_passed"),
+    ):
+        v = raw.get(key)
+        if v is not None:
+            out[out_key] = v.strip().lower() == "passed"
+
+    return out
+
+
+def serial_from_filename(name: str) -> Optional[str]:
+    """Convenience: pull the serial prefix from a Thor event filename.
+
+    Thor uses the literal serial as the filename prefix:
+      UM11719_20231219163444.IDFW  →  "UM11719"
+      BE9439_20200713124251.IDFH   →  "BE9439"
+    """
+    m = re.match(r"^([A-Z]{2}\d+)_\d{14}\.(IDFH|IDFW)(?:\.txt)?$",
+                 name, re.IGNORECASE)
+    return m.group(1).upper() if m else None
+
+
+def parse_event_filename(name: str) -> Optional[Tuple[str, datetime.datetime, str]]:
+    """Parse `<SERIAL>_<YYYYMMDDHHMMSS>.<KIND>` → (serial, datetime, kind).
+
+    `kind` is "IDFH" or "IDFW" (upper-case).  Returns None on no match.
+    """
+    m = re.match(r"^([A-Z]{2}\d+)_(\d{14})\.(IDFH|IDFW)$",
+                 name, re.IGNORECASE)
+    if not m:
+        return None
+    try:
+        ts = datetime.datetime.strptime(m.group(2), "%Y%m%d%H%M%S")
+    except ValueError:
+        return None
+    return m.group(1).upper(), ts, m.group(3).upper()

sfm/waveform_store.py

@@ -34,7 +34,7 @@ import logging
 import pickle
 import shutil
 from pathlib import Path
-from typing import Optional
+from typing import Optional, Union
 
 from minimateplus import event_file_io
 from minimateplus.blastware_file import blastware_filename, write_blastware_file
@@ -258,6 +258,7 @@ class WaveformStore:
         source_path: Path,
         *,
         serial_hint: Optional[str] = None,
+        bw_report_text: Optional[Union[str, bytes]] = None,
     ) -> tuple[Event, dict]:
         """
         Ingest a Blastware event file produced by an external tool
@@ -267,10 +268,17 @@ class WaveformStore:
         Workflow:
           1. Parse the bytes via event_file_io.read_blastware_file (writes
              a temp file to do that, since the parser takes a path).
-          2. Resolve serial from BW filename (`<P><serial3>...`) or use
+          2. Optionally parse a paired BW ASCII event report (the .TXT
+             file BW writes alongside the binary).  When supplied, its
+             decoded fields land in the sidecar's `bw_report` block AND
+             overlay the device-authoritative peak values into the
+             top-level `peak_values` block.  This is the right path for
+             the ACH-forwarder daemon use case where Blastware's own
+             ACH writes both files into the watch folder.
+          3. Resolve serial from BW filename (`<P><serial3>...`) or use
              serial_hint.  Falls back to "UNKNOWN".
-          3. Copy the BW bytes verbatim into <root>/<serial>/<filename>.
-          4. Write the .sfm.json sidecar with source.kind = "bw-import"
+          4. Copy the BW bytes verbatim into <root>/<serial>/<filename>.
+          5. Write the .sfm.json sidecar with source.kind = "bw-import"
              and a5_pickle_filename = None.  Does NOT write a .a5.pkl
              (no A5 source available; byte-for-byte regeneration not
              possible — the on-disk BW file IS the byte-for-byte source).
@@ -292,6 +300,47 @@ class WaveformStore:
             except FileNotFoundError:
                 pass
 
+        # read_blastware_file derives record_type from its path arg, but
+        # that arg is the tmp file (suffix ".bw") — so override with the
+        # original filename's encoded type (H/W/M/E/C in the BW AB0T
+        # scheme).  Without this override every BW-imported event lands
+        # in the DB with record_type="Waveform" regardless of the actual
+        # type (Histogram, Manual, etc.).
+        ev.record_type = event_file_io.derive_record_type_from_filename(
+            source_path.name
+        )
+
+        # Parse the BW ASCII report if one was supplied.  Failures here
+        # are non-fatal: we still write the binary + sidecar without the
+        # rich derived fields.
+        bw_report = None
+        if bw_report_text is not None:
+            try:
+                from minimateplus.bw_ascii_report import parse_report
+                bw_report = parse_report(bw_report_text)
+            except Exception as exc:
+                log.warning(
+                    "save_imported_bw: BW report parse failed: %s — continuing without it",
+                    exc,
+                )
+
+        # If we have a report, overlay its device-authoritative fields
+        # (peaks, project, sample_rate, record_time) onto the Event
+        # BEFORE handing it to db.insert_events().  Without this overlay
+        # the DB row gets `peak_values` from _peaks_from_samples(), which
+        # runs the still-undecoded waveform codec on the BW body and
+        # produces ±10 in/s saturation values on every channel for every
+        # event.  The sidecar JSON had the correct values via
+        # event_to_sidecar_dict(bw_report=...) but the DB columns didn't.
+        if bw_report is not None:
+            try:
+                event_file_io.apply_report_to_event(ev, bw_report)
+            except Exception as exc:
+                log.warning(
+                    "save_imported_bw: failed to overlay report onto event: %s",
+                    exc,
+                )
+
         # Resolve serial.  blastware_filename derives a 4-char prefix from
         # the numeric serial (e.g. BE11529 → M529); we go the other way
         # via the source filename if a hint wasn't given.
@@ -345,6 +394,7 @@ class WaveformStore:
             source_kind="bw-import",
             a5_pickle_filename=None,
             review=existing_review,
+            bw_report=bw_report,
         )
         event_file_io.write_sidecar(sidecar_path, sidecar)
 
@@ -360,6 +410,180 @@ class WaveformStore:
             "a5_pickle_filename": None,
             "hdf5_filename":      hdf5_filename,
             "sidecar_filename":   sidecar_path.name,
+            "serial":             serial,
+        }
+
+    def save_imported_idf(
+        self,
+        idf_bytes: bytes,
+        source_path: Path,
+        *,
+        serial_hint: Optional[str] = None,
+        idf_report_text: Optional[Union[str, bytes]] = None,
+    ) -> tuple[Optional["Event"], dict]:
+        """
+        Ingest a Thor (Micromate Series IV) IDF event file (`.IDFW` or
+        `.IDFH`) produced by Thor's TXT exporter.
+
+        Thor binaries are stored as opaque bytes — seismo-relay doesn't
+        decode the proprietary IDF binary format.  Device-authoritative
+        metadata comes from the paired `.IDFW.txt` / `.IDFH.txt` sidecar
+        when supplied; we parse that text and surface its fields onto
+        the returned Event so the SFM database row has real PPV/project
+        values instead of NULLs.
+
+        Workflow:
+          1. Parse the paired TXT report (when supplied) via
+             `sfm.idf_ascii_report.parse_idf_report`.
+          2. Build a minimal `Event` populated from the report fields
+             (timestamp, peaks, project info, sample_rate, record_type).
+          3. Resolve serial from filename prefix or `serial_hint`.
+          4. Copy bytes verbatim into <root>/<serial>/<filename>.
+          5. Write the `.sfm.json` sidecar with source.kind = "idf-import".
+
+        Returns (event, record_dict) so the endpoint can both insert
+        into SeismoDb and surface the parsed event.
+        """
+        from sfm.idf_ascii_report import (
+            parse_idf_report,
+            parse_event_filename,
+            serial_from_filename as _idf_serial_from_filename,
+        )
+        from minimateplus.models import (
+            Event, PeakValues, ProjectInfo, Timestamp,
+        )
+
+        # Parse the .txt sidecar (best-effort; non-fatal on failure).
+        report: dict = {}
+        if idf_report_text is not None:
+            try:
+                report = parse_idf_report(idf_report_text)
+            except Exception as exc:
+                log.warning(
+                    "save_imported_idf: report parse failed: %s — continuing without it",
+                    exc,
+                )
+
+        # Resolve serial: prefer the explicit hint, fall back to filename prefix.
+        serial = (
+            serial_hint
+            or report.get("serial_number")
+            or _idf_serial_from_filename(source_path.name)
+            or "UNKNOWN"
+        )
+
+        # Resolve event timestamp + kind from the filename (always present).
+        parsed_name = parse_event_filename(source_path.name)
+        kind = "Waveform"
+        ts_dt: Optional[datetime.datetime] = None
+        if parsed_name is not None:
+            _, ts_dt, kind_token = parsed_name
+            kind = "Histogram" if kind_token == "IDFH" else "Waveform"
+        # Report's event_datetime is the device-authoritative value; prefer it.
+        if "event_datetime" in report:
+            try:
+                ts_dt = datetime.datetime.fromisoformat(report["event_datetime"])
+            except (TypeError, ValueError):
+                pass
+
+        ts_obj: Optional[Timestamp] = None
+        if ts_dt is not None:
+            ts_obj = Timestamp(
+                raw=bytes(9),
+                flag=0,
+                year=ts_dt.year,
+                unknown_byte=0,
+                month=ts_dt.month,
+                day=ts_dt.day,
+                hour=ts_dt.hour,
+                minute=ts_dt.minute,
+                second=ts_dt.second,
+            )
+
+        # Build PeakValues from the report (fields are None when absent).
+        pv = PeakValues(
+            tran=report.get("tran_ppv"),
+            vert=report.get("vert_ppv"),
+            long=report.get("long_ppv"),
+            micl=report.get("mic_ppv"),
+            peak_vector_sum=report.get("peak_vector_sum"),
+        )
+
+        # Build ProjectInfo.  See idf_ascii_report — Thor's title strings
+        # carry project / client / company / notes in TitleString1..4.
+        pi = ProjectInfo(
+            setup_name=report.get("setup"),
+            project=report.get("project"),
+            client=report.get("client"),
+            operator=report.get("operator"),
+            sensor_location=None,  # Thor folds location into TitleString1 = project
+            notes=report.get("notes"),
+        )
+
+        # Filesystem write.
+        filename = source_path.name
+        bw_path = self._serial_dir(serial) / filename
+        bw_path.write_bytes(idf_bytes)
+        filesize = bw_path.stat().st_size
+        sha256   = event_file_io.file_sha256(bw_path)
+
+        # _waveform_key dedups (serial, timestamp) rows in the events
+        # table.  Use the binary's sha256 (first 16 bytes) as a stable
+        # surrogate — every distinct binary maps to a distinct row.
+        waveform_key = bytes.fromhex(sha256)[:16]
+
+        ev = Event(
+            index=0,
+            timestamp=ts_obj,
+            sample_rate=report.get("sample_rate"),
+            peak_values=pv,
+            project_info=pi,
+            record_type=kind,
+            rectime_seconds=report.get("record_time_sec"),
+        )
+        ev._waveform_key = waveform_key
+
+        # Write the sidecar.  Source kind "idf-import" was added to the
+        # allow-list in event_file_io.event_to_sidecar_dict for this.
+        sidecar_path = self.sidecar_path_for(serial, filename)
+        existing_review = None
+        if sidecar_path.exists():
+            try:
+                existing_review = event_file_io.read_sidecar(sidecar_path).get("review")
+            except Exception:
+                pass
+
+        sidecar = event_file_io.event_to_sidecar_dict(
+            ev,
+            serial=serial,
+            blastware_filename=filename,
+            blastware_filesize=filesize,
+            blastware_sha256=sha256,
+            source_kind="idf-import",
+            a5_pickle_filename=None,
+            review=existing_review,
+        )
+        # Stash the full parsed IDF report under extensions so downstream
+        # consumers can recover the rich derived fields that don't fit
+        # the BW-shaped event model (Peak Acceleration / Displacement,
+        # Time of Peak, sensor self-check, calibration, firmware).
+        if report:
+            sidecar["extensions"]["idf_report"] = report
+        event_file_io.write_sidecar(sidecar_path, sidecar)
+
+        log.info(
+            "WaveformStore.save_imported_idf serial=%s filename=%s filesize=%d "
+            "report_attached=%s",
+            serial, filename, filesize, bool(report),
+        )
+        return ev, {
+            "filename":           filename,
+            "filesize":           filesize,
+            "sha256":             sha256,
+            "a5_pickle_filename": None,
+            "hdf5_filename":      None,
+            "sidecar_filename":   sidecar_path.name,
+            "serial":             serial,
         }
 
     def load_a5(self, serial: str, filename: str) -> Optional[list[S3Frame]]:

tests/test_bw_ascii_report.py

@@ -0,0 +1,407 @@
+"""
+test_bw_ascii_report.py — parser for Blastware's per-event ASCII export.
+
+Run:
+    python -m pytest tests/test_bw_ascii_report.py -q
+"""
+
+from __future__ import annotations
+
+import datetime
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from minimateplus.bw_ascii_report import (
+    BwAsciiReport,
+    parse_report,
+    parse_report_file,
+)
+
+
+FIXTURES = Path(__file__).parent.parent / "decode-re" / "5-8-26"
+
+
+def _fixture(event_name: str) -> Path:
+    """Find the .TXT file inside a fixture event folder."""
+    for p in (FIXTURES / event_name).iterdir():
+        if p.suffix.lower() == ".txt":
+            return p
+    raise FileNotFoundError(f"no .TXT in {FIXTURES / event_name}")
+
+
+# ── Identity / config ───────────────────────────────────────────────────────
+
+
+def test_event_c_identity_and_config():
+    r = parse_report_file(_fixture("event-c"))
+
+    assert r.event_type == "Full Waveform"
+    assert r.serial == "BE11529"
+    assert r.file_name == "M529LK44.AB0"
+    assert r.event_datetime == datetime.datetime(2026, 4, 23, 15, 56, 35)
+
+    assert r.trigger_channel == "Vert"
+    assert r.geo_trigger_level_ips == pytest.approx(0.5)
+    assert r.pretrig_s == pytest.approx(-0.25)
+    assert r.record_time_s == pytest.approx(1.0)
+    assert r.record_stop_mode == "Fixed"
+    assert r.sample_rate_sps == 1024
+    assert r.battery_volts == pytest.approx(6.8)
+    assert r.calibration_date == datetime.date(2025, 4, 29)
+    assert r.calibration_by == "Instantel"
+    assert r.units == "in/s and dB(L)"
+
+
+def test_event_c_operator_metadata():
+    r = parse_report_file(_fixture("event-c"))
+
+    # The "Project: : value" pattern (key has its own trailing colon)
+    # is handled by stripping the colon at lookup time.
+    assert r.project == "Test4-21-26"
+    assert r.client == "Test-Client1"
+    assert r.operator == "Brian and claude"
+    assert r.sensor_location == "catbed"
+
+
+def test_event_c_geo_range():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.geo_range_ips == pytest.approx(10.0)
+
+
+# ── Per-channel derived stats ───────────────────────────────────────────────
+
+
+def test_event_c_per_channel_stats():
+    r = parse_report_file(_fixture("event-c"))
+
+    tran = r.channels["Tran"]
+    assert tran.ppv_ips         == pytest.approx(0.065)
+    assert tran.zc_freq_hz      == pytest.approx(47.0)
+    assert tran.time_of_peak_s  == pytest.approx(0.007)
+    assert tran.peak_accel_g    == pytest.approx(0.066)
+    assert tran.peak_disp_in    == pytest.approx(0.001)
+
+    vert = r.channels["Vert"]
+    assert vert.ppv_ips         == pytest.approx(0.610)
+    assert vert.zc_freq_hz      == pytest.approx(16.0)
+    assert vert.time_of_peak_s  == pytest.approx(0.024)
+    assert vert.peak_accel_g    == pytest.approx(0.437)
+    assert vert.peak_disp_in    == pytest.approx(0.006)
+
+    long_ = r.channels["Long"]
+    assert long_.ppv_ips        == pytest.approx(0.070)
+    assert long_.zc_freq_hz     == pytest.approx(22.0)
+    assert long_.time_of_peak_s == pytest.approx(0.019)
+    assert long_.peak_accel_g   == pytest.approx(0.040)
+    assert long_.peak_disp_in   == pytest.approx(0.001)
+
+
+def test_event_c_micl_stats():
+    r = parse_report_file(_fixture("event-c"))
+
+    # MicL specific block
+    assert r.mic.weighting       == "Linear Weighting"
+    assert r.mic.pspl_dbl        == pytest.approx(88.0)
+    assert r.mic.zc_freq_hz      == pytest.approx(57.0)
+    assert r.mic.time_of_peak_s  == pytest.approx(-0.004)
+
+    # Mirrored onto channels["MicL"] for uniform per-channel access
+    micl_ch = r.channels["MicL"]
+    assert micl_ch.zc_freq_hz     == pytest.approx(57.0)
+    assert micl_ch.time_of_peak_s == pytest.approx(-0.004)
+
+
+def test_event_c_vector_sum():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.peak_vector_sum_ips    == pytest.approx(0.612)
+    assert r.peak_vector_sum_time_s == pytest.approx(0.024)
+
+
+# ── Sensor self-check ───────────────────────────────────────────────────────
+
+
+def test_event_c_sensor_check_geo_channels():
+    r = parse_report_file(_fixture("event-c"))
+
+    for ch_name, expected_freq, expected_ratio in [
+        ("Tran", 7.4, 3.7),
+        ("Vert", 7.6, 3.5),
+        ("Long", 7.5, 3.8),
+    ]:
+        sc = r.sensor_check[ch_name]
+        assert sc.test_freq_hz   == pytest.approx(expected_freq), ch_name
+        assert sc.test_ratio     == pytest.approx(expected_ratio), ch_name
+        assert sc.test_results   == "Passed", ch_name
+        # Geo channels don't have an Test Amplitude
+        assert sc.test_amplitude_mv is None
+
+
+def test_event_c_sensor_check_micl():
+    r = parse_report_file(_fixture("event-c"))
+
+    sc = r.sensor_check["MicL"]
+    assert sc.test_freq_hz       == pytest.approx(20.1)
+    assert sc.test_amplitude_mv  == pytest.approx(533.0)
+    assert sc.test_results       == "Passed"
+    # MicL doesn't have a ratio — it has amplitude instead
+    assert sc.test_ratio is None
+
+
+# ── Monitor log + tooling ───────────────────────────────────────────────────
+
+
+def test_event_c_monitor_log_and_pc_version():
+    r = parse_report_file(_fixture("event-c"))
+
+    assert len(r.monitor_log) == 1
+    e = r.monitor_log[0]
+    assert e.start_time == datetime.datetime(2026, 4, 23, 15, 46, 16)
+    assert e.stop_time  == datetime.datetime(2026, 4, 23, 15, 56, 36)
+    assert e.description == "Event recorded."
+
+    assert r.pc_sw_version == "V 10.74"
+
+
+# ── Sample table ─────────────────────────────────────────────────────────────
+
+
+def test_event_c_sample_table_parsed_when_requested():
+    r = parse_report_file(_fixture("event-c"), parse_samples=True)
+
+    # 1 sec event @ 1024 sps + 0.25 sec pretrig = 1280 samples
+    assert r.samples is not None
+    assert len(r.samples) == 1280, f"expected 1280 samples, got {len(r.samples)}"
+
+    # First row: "0.000     \t0.005     \t0.005     \t-81.94"
+    t, v, l, m = r.samples[0]
+    assert t == pytest.approx(0.000)
+    assert v == pytest.approx(0.005)
+    assert l == pytest.approx(0.005)
+    assert m == pytest.approx(-81.94)
+
+
+def test_event_c_sample_table_skipped_by_default():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.samples is None
+
+
+# ── Cross-event smoke ───────────────────────────────────────────────────────
+
+
+@pytest.mark.parametrize("event_name", ["event-a", "event-b", "event-c", "event-d"])
+def test_all_fixtures_parse_without_error(event_name):
+    """Every fixture in the bundle must parse cleanly with the same parser."""
+    r = parse_report_file(_fixture(event_name))
+
+    # Common invariants: serial, event_datetime, sample rate, all four
+    # channels surfaced.
+    assert r.serial == "BE11529"
+    assert r.event_datetime is not None
+    assert r.sample_rate_sps in (1024, 2048, 4096)
+    for ch in ("Tran", "Vert", "Long", "MicL"):
+        assert ch in r.channels
+        assert ch in r.sensor_check
+
+    # PVS should be present and positive on triggered events
+    if r.peak_vector_sum_ips is not None:
+        assert r.peak_vector_sum_ips >= 0
+
+
+# ── Edge cases / defensive parsing ──────────────────────────────────────────
+
+
+def test_parse_empty_input():
+    r = parse_report("")
+    assert r.serial is None
+    assert r.event_datetime is None
+    assert all(cs.ppv_ips is None for cs in r.channels.values())
+
+
+def test_parse_unknown_keys_ignored():
+    """Forward-compat: future BW versions may add fields we don't recognise.
+    Those should be silently dropped, not raise."""
+    text = (
+        '"Serial Number : BE99999"\n'
+        '"Future Field That Does Not Exist : 42 widgets"\n'
+        '"Tran PPV : 0.123 in/s"\n'
+    )
+    r = parse_report(text)
+    assert r.serial == "BE99999"
+    assert r.channels["Tran"].ppv_ips == pytest.approx(0.123)
+
+
+def test_parse_numeric_with_units_strips_unit():
+    text = (
+        '"Vert PPV : 1.275 in/s"\n'
+        '"Vert ZC Freq : 23 Hz"\n'
+        '"MicL  Test Amplitude : 569 mv"\n'
+    )
+    r = parse_report(text)
+    assert r.channels["Vert"].ppv_ips    == pytest.approx(1.275)
+    assert r.channels["Vert"].zc_freq_hz == pytest.approx(23.0)
+    assert r.sensor_check["MicL"].test_amplitude_mv == pytest.approx(569.0)
+
+
+def test_parse_handles_micl_double_space_in_key():
+    """BW writes "MicL  Time of Peak" with TWO spaces; the parser must
+    normalise whitespace before key lookup."""
+    text = (
+        '"MicL  Time of Peak : 0.012 sec"\n'
+        '"MicL  ZC Freq : 51 Hz"\n'
+    )
+    r = parse_report(text)
+    assert r.mic.time_of_peak_s == pytest.approx(0.012)
+    assert r.mic.zc_freq_hz     == pytest.approx(51.0)
+
+
+# ── Position-based user-notes parsing ───────────────────────────────────────
+#
+# The 4 user-supplied note slots (Project / Client / User Name / Seis Loc
+# by default) have OPERATOR-EDITABLE labels in BW's Compliance Setup →
+# Notes tab.  An operator could rename them to "Building:", "Site:",
+# "Address:", etc. and the ASCII export would write those labels
+# verbatim.  We parse by POSITION between the `Units :` and `Geo Range :`
+# anchors, NOT by matching the label text.
+
+
+def _wrap_user_notes(*lines: str) -> str:
+    """Helper: wrap N user-note lines in the minimal context the parser
+    needs (`Units :` opens the block, `Geo Range :` closes it)."""
+    body = ['"Units : in/s and dB(L)"']
+    body.extend('"' + l + '"' for l in lines)
+    body.append('"Geo Range : 10.000 in/s"')
+    return "\n".join(body) + "\n"
+
+
+def test_user_notes_default_labels_populate_by_position():
+    """The BW-default labels (Project / Client / User Name / Seis Loc)
+    populate the four canonical slots in order."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Test4-21-26",
+        "Client: : Acme Inc",
+        "User Name: : Brian",
+        "Seis Loc: : Catbed",
+    ))
+    assert r.project          == "Test4-21-26"
+    assert r.client           == "Acme Inc"
+    assert r.operator         == "Brian"
+    assert r.sensor_location  == "Catbed"
+    assert r.user_note_labels == {
+        "project":         "Project:",
+        "client":          "Client:",
+        "operator":        "User Name:",
+        "sensor_location": "Seis Loc:",
+    }
+
+
+def test_user_notes_operator_renamed_labels_still_populate():
+    """If the operator renames the labels in BW's UI (e.g. "Seis Loc:"
+    → "Building:"), the values STILL populate the canonical slots by
+    position — and the operator's labels are preserved in
+    `user_note_labels` for terra-view to display."""
+    r = parse_report(_wrap_user_notes(
+        "Building : Main Office",
+        "Project Manager : Brian",
+        "Inspector : Claude",
+        "Site Address : 123 Main St",
+    ))
+    assert r.project         == "Main Office"
+    assert r.client          == "Brian"
+    assert r.operator        == "Claude"
+    assert r.sensor_location == "123 Main St"
+    assert r.user_note_labels == {
+        "project":         "Building",
+        "client":          "Project Manager",
+        "operator":        "Inspector",
+        "sensor_location": "Site Address",
+    }
+
+
+def test_user_notes_with_histogram_label_spelling():
+    """Histogram exports use 'Seis. Location:' (with period and colon)
+    instead of 'Seis Loc:'.  Position-based parsing handles both."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Plum Cont.- Rainbow Run",
+        "Client: : Plum Contracting In.c",
+        "User Name: : Terra-Mechanics Inc.",
+        "Seis. Location: : Loc #1 - 2652 Hepner",
+    ))
+    assert r.project == "Plum Cont.- Rainbow Run"
+    assert r.client == "Plum Contracting In.c"
+    assert r.operator == "Terra-Mechanics Inc."
+    assert r.sensor_location == "Loc #1 - 2652 Hepner"
+    # And the histogram's specific label spelling is preserved
+    assert r.user_note_labels["sensor_location"] == "Seis. Location:"
+
+
+def test_user_notes_outside_block_are_ignored():
+    """Lines that look like user-notes but appear OUTSIDE the
+    Units→Geo Range range don't get assigned to user-note slots."""
+    # No Units anchor — these lines shouldn't populate user-note slots
+    text = (
+        '"Serial Number : BE11529"\n'
+        '"Project: : SHOULD NOT POPULATE"\n'
+    )
+    r = parse_report(text)
+    assert r.serial == "BE11529"
+    assert r.project is None
+
+
+def test_user_notes_partial_block_only_fills_present_slots():
+    """If BW writes fewer than 4 user-notes (e.g. operator disabled
+    Extended Notes mid-block), only the present positions populate;
+    later slots stay None."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Just-a-project",
+        "Client: : Just-a-client",
+    ))
+    assert r.project == "Just-a-project"
+    assert r.client  == "Just-a-client"
+    assert r.operator is None
+    assert r.sensor_location is None
+
+
+def test_user_notes_extra_lines_beyond_four_are_dropped():
+    """If somehow more than 4 lines appear in the user-notes block
+    (e.g. BW adds an Extended Notes line), only the first 4 are
+    captured — slots 5+ have nowhere to go."""
+    r = parse_report(_wrap_user_notes(
+        "L1 : v1",
+        "L2 : v2",
+        "L3 : v3",
+        "L4 : v4",
+        "L5 : v5",   # ignored — no fifth slot
+    ))
+    assert r.project         == "v1"
+    assert r.client          == "v2"
+    assert r.operator        == "v3"
+    assert r.sensor_location == "v4"
+    # 5th label not captured
+    assert "L5" not in r.user_note_labels.values()
+
+
+def test_real_histogram_fixture_populates_sensor_location():
+    """End-to-end: the histogram fixture uses 'Seis. Location:' — must
+    successfully populate sensor_location via position-based parsing."""
+    fixture_dir = (
+        Path(__file__).parent.parent / "example-events" / "histogram"
+    )
+    if not fixture_dir.exists():
+        pytest.skip("histogram fixtures not present")
+    txt = next(fixture_dir.glob("*_ASCII.TXT"), None)
+    if txt is None:
+        pytest.skip("no histogram TXT in fixture dir")
+
+    r = parse_report_file(txt)
+    assert r.sensor_location is not None
+    assert len(r.sensor_location) > 0
+    assert r.user_note_labels.get("sensor_location") is not None
+    # Sanity: other shared fields still parse correctly
+    assert r.serial is not None
+    assert r.serial.startswith("BE")
+    assert r.geo_range_ips is not None

tests/test_event_file_io.py

@@ -127,59 +127,6 @@ def test_sidecar_write_and_read_round_trip(tmp_path: Path):
     assert loaded["source"]["kind"] == "sfm-ach"
 
 
-def test_sidecar_persists_raw_0c_record_in_extensions(tmp_path: Path):
-    """An Event with _raw_record populated should land its 210 bytes
-    base64-encoded in extensions.raw_records.waveform_record_b64, so
-    later analysis (e.g. mapping Peak Acceleration / Time of Peak / ZC
-    Freq byte offsets) can run offline against the saved sidecar."""
-    import base64
-
-    ev, _ = _make_synthetic_event()
-    # Synthesize a 210-byte 0C record with embedded label needles so
-    # the dump tool's anchor scan has something to find.
-    raw = bytearray(210)
-    raw[10:14]   = b"Tran"
-    raw[60:64]   = b"Vert"
-    raw[110:114] = b"Long"
-    raw[160:164] = b"MicL"
-    ev._raw_record = bytes(raw)
-
-    d = event_file_io.event_to_sidecar_dict(
-        ev, serial="BE11529",
-        blastware_filename="M529LKIQ.7M0W", blastware_filesize=1024,
-        blastware_sha256="x" * 64, source_kind="sfm-live",
-    )
-
-    rr = d["extensions"]["raw_records"]
-    assert rr["waveform_record_len"] == 210
-    decoded = base64.b64decode(rr["waveform_record_b64"])
-    assert decoded == ev._raw_record
-
-    # Round-trip through write/read
-    path = tmp_path / "raw0c.sfm.json"
-    event_file_io.write_sidecar(path, d)
-    loaded = event_file_io.read_sidecar(path)
-    assert (
-        base64.b64decode(loaded["extensions"]["raw_records"]["waveform_record_b64"])
-        == ev._raw_record
-    )
-
-
-def test_sidecar_omits_raw_records_when_event_has_no_0c(tmp_path: Path):
-    """Events without a _raw_record (e.g. constructed by importers that
-    never see 0C) should NOT add an empty raw_records block — keep the
-    sidecar clean for those flows."""
-    ev, _ = _make_synthetic_event()
-    assert ev._raw_record is None
-
-    d = event_file_io.event_to_sidecar_dict(
-        ev, serial="BE11529",
-        blastware_filename="M529LKIQ.7M0W", blastware_filesize=1024,
-        blastware_sha256="x" * 64, source_kind="bw-import",
-    )
-    assert d["extensions"] == {}
-
-
 def test_sidecar_rejects_unsupported_schema_version(tmp_path: Path):
     path = tmp_path / "future.sfm.json"
     path.write_text(json.dumps({
@@ -347,6 +294,114 @@ def test_read_blastware_file_round_trip(tmp_path: Path):
     assert parsed.peak_values.peak_vector_sum == 0.0
 
 
+def test_save_imported_bw_with_paired_report(tmp_path: Path):
+    """save_imported_bw + a paired BW ASCII report fold the report's
+    rich derived fields into the sidecar.  This is the daemon-forwarded
+    ACH workflow: BW writes <event>.AB0 and <event>.AB0.TXT side by side;
+    the daemon ships both; we overlay the report-decoded values onto the
+    sidecar (peaks, project, plus the rich `bw_report` block)."""
+    from minimateplus.blastware_file import write_blastware_file, blastware_filename
+    from sfm.waveform_store import WaveformStore
+
+    ev, frames = _make_synthetic_event()
+    fname = blastware_filename(ev, "BE11529")
+    src = tmp_path / fname
+    write_blastware_file(ev, frames, src)
+
+    # Use one of the real BW ASCII exports as the paired report.
+    report_path = (
+        Path(__file__).parent.parent
+        / "decode-re" / "5-8-26" / "event-c" / "M529LK44.AB0.TXT"
+    )
+    if not report_path.exists():
+        import pytest as _pt
+        _pt.skip("decode-re fixtures not present")
+    report_bytes = report_path.read_bytes()
+
+    store = WaveformStore(tmp_path / "waveforms")
+    parsed_ev, rec = store.save_imported_bw(
+        src.read_bytes(),
+        source_path=src,
+        bw_report_text=report_bytes,
+    )
+
+    sc = store.load_sidecar("BE11529", fname)
+    assert sc is not None
+
+    # ── bw_report block populated with the rich fields ──────────────────
+    assert "bw_report" in sc
+    br = sc["bw_report"]
+    assert br["available"] is True
+    assert br["event_type"] == "Full Waveform"
+    assert br["recording"]["sample_rate_sps"] == 1024
+    assert br["recording"]["geo_range_ips"]   == 10.0
+
+    # Per-channel derived stats
+    assert br["peaks"]["tran"]["ppv_ips"]        == 0.065
+    assert br["peaks"]["vert"]["ppv_ips"]        == 0.610
+    assert br["peaks"]["long"]["ppv_ips"]        == 0.070
+    assert br["peaks"]["vert"]["peak_accel_g"]   == 0.437
+    assert br["peaks"]["vert"]["peak_disp_in"]   == 0.006
+    assert br["peaks"]["tran"]["zc_freq_hz"]     == 47.0
+    assert br["peaks"]["vector_sum"]["ips"]      == 0.612
+    assert br["peaks"]["vector_sum"]["time_s"]   == 0.024
+
+    # Sensor self-check per channel
+    assert br["sensor_check"]["tran"]["freq_hz"]  == 7.4
+    assert br["sensor_check"]["tran"]["ratio"]    == 3.7
+    assert br["sensor_check"]["tran"]["result"]   == "Passed"
+    assert br["sensor_check"]["mic"]["amplitude_mv"] == 533.0
+
+    # Mic block
+    assert br["mic"]["weighting"]    == "Linear Weighting"
+    assert br["mic"]["pspl_dbl"]     == 88.0
+
+    # Monitor log roundtripped
+    assert len(br["monitor_log"]) == 1
+    assert "2026-04-23T15:46:16" in br["monitor_log"][0]["start"]
+    assert br["pc_sw_version"] == "V 10.74"
+
+    # ── Overlay onto canonical peak_values ──────────────────────────────
+    # Report values win over the broken-codec samples-derived peaks.
+    assert sc["peak_values"]["transverse"]   == 0.065
+    assert sc["peak_values"]["vertical"]     == 0.610
+    assert sc["peak_values"]["longitudinal"] == 0.070
+    assert sc["peak_values"]["vector_sum"]   == 0.612
+    # Mic PSPL converted to psi (dbl=88 → 10^(88/20) * 2.9e-9)
+    assert sc["peak_values"]["mic_psi"] is not None
+    assert 1e-5 < sc["peak_values"]["mic_psi"] < 1e-3
+
+    # ── Overlay onto project_info ───────────────────────────────────────
+    assert sc["project_info"]["project"]         == "Test4-21-26"
+    assert sc["project_info"]["client"]          == "Test-Client1"
+    assert sc["project_info"]["operator"]        == "Brian and claude"
+    assert sc["project_info"]["sensor_location"] == "catbed"
+
+    # ── Event timestamp overlaid from report ───────────────────────────
+    assert sc["event"]["timestamp"] == "2026-04-23T15:56:35"
+
+
+def test_save_imported_bw_without_report_works_unchanged(tmp_path: Path):
+    """Calling save_imported_bw with no bw_report_text behaves exactly
+    as before — no `bw_report` block, peak_values come from samples."""
+    from minimateplus.blastware_file import write_blastware_file, blastware_filename
+    from sfm.waveform_store import WaveformStore
+
+    ev, frames = _make_synthetic_event()
+    fname = blastware_filename(ev, "BE11529")
+    src = tmp_path / fname
+    write_blastware_file(ev, frames, src)
+
+    store = WaveformStore(tmp_path / "waveforms")
+    store.save_imported_bw(src.read_bytes(), source_path=src)
+
+    sc = store.load_sidecar("BE11529", fname)
+    assert sc is not None
+    assert "bw_report" not in sc          # block is absent without a report
+    # Synthetic event has zero samples → peaks all zero (was true before this change)
+    assert sc["peak_values"]["transverse"] == 0.0
+
+
 def test_save_imported_bw_round_trip(tmp_path: Path):
     """save_imported_bw stores a copy + sidecar with source.kind = bw-import."""
     from minimateplus.blastware_file import write_blastware_file, blastware_filename
@@ -363,6 +418,10 @@ def test_save_imported_bw_round_trip(tmp_path: Path):
 
     assert rec["filename"] == fname
     assert rec["a5_pickle_filename"] is None  # no A5 source for BW imports
+    # The serial decoded from the BW filename surfaces on the record so
+    # the import endpoint can use it when calling SeismoDb.insert_events()
+    # (otherwise forwarded events would all bucket into serial="UNKNOWN").
+    assert rec["serial"] == "BE11529"
     sc = store.load_sidecar("BE11529", fname)
     assert sc is not None
     assert sc["source"]["kind"] == "bw-import"

tests/test_idf_ascii_report.py

@@ -0,0 +1,234 @@
+"""
+test_idf_ascii_report.py — parser for Thor's per-event IDF ASCII export.
+
+Run:
+    python -m pytest tests/test_idf_ascii_report.py -q
+
+Tests use real Thor sample data shipped under
+`thor-watcher/example-data/THORDATA_example/`.  When that path is not
+available (e.g. running from a checkout where the watcher repo isn't
+sibling), tests gracefully skip.
+"""
+
+from __future__ import annotations
+
+import datetime
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from sfm.idf_ascii_report import (
+    parse_event_filename,
+    parse_idf_report,
+    serial_from_filename,
+)
+
+
+# ── Sample data ──────────────────────────────────────────────────────────────
+
+
+SAMPLE_REPO = Path("/home/serversdown/thor-watcher/example-data/"
+                   "THORDATA_example/THORDATA_example")
+
+
+def _sample_path(rel: str) -> Path:
+    return SAMPLE_REPO / rel
+
+
+@pytest.fixture
+def upmc_waveform_txt() -> str:
+    p = _sample_path("UPMC Presby/UM11719/TXT/UM11719_20231219162723.IDFW.txt")
+    if not p.exists():
+        pytest.skip(f"sample missing: {p}")
+    return p.read_text()
+
+
+@pytest.fixture
+def upmc_histogram_txt() -> str:
+    p = _sample_path("UPMC Presby/UM11719/TXT/UM11719_20231219163444.IDFH.txt")
+    if not p.exists():
+        pytest.skip(f"sample missing: {p}")
+    return p.read_text()
+
+
+# ── Filename parsing ─────────────────────────────────────────────────────────
+
+
+def test_parse_event_filename_waveform():
+    parsed = parse_event_filename("UM11719_20231219163444.IDFW")
+    assert parsed is not None
+    serial, ts, kind = parsed
+    assert serial == "UM11719"
+    assert ts == datetime.datetime(2023, 12, 19, 16, 34, 44)
+    assert kind == "IDFW"
+
+
+def test_parse_event_filename_histogram():
+    parsed = parse_event_filename("BE9439_20200713124251.IDFH")
+    assert parsed is not None
+    serial, ts, kind = parsed
+    assert serial == "BE9439"
+    assert kind == "IDFH"
+
+
+def test_parse_event_filename_case_insensitive():
+    parsed = parse_event_filename("um11719_20231219163444.idfw")
+    assert parsed is not None
+    assert parsed[0] == "UM11719"
+    assert parsed[2] == "IDFW"
+
+
+def test_parse_event_filename_rejects_invalid():
+    for name in [
+        "UM11719_20231219163444.MLG",
+        "UM11719.IDFW",
+        "UM11719_20231219163444.IDFW.txt",  # report sidecar — not a binary
+        "UM11719_2023121916344X.IDFW",
+        "garbage",
+        "",
+    ]:
+        assert parse_event_filename(name) is None, name
+
+
+def test_serial_from_filename():
+    assert serial_from_filename("UM11719_20231219163444.IDFW") == "UM11719"
+    assert serial_from_filename("BE9439_20200713124251.IDFH") == "BE9439"
+    # Works on the .txt sidecar name too — handy in pairing code paths
+    assert serial_from_filename("UM11719_20231219163444.IDFW.txt") == "UM11719"
+    assert serial_from_filename("not_a_thor_file.bin") is None
+
+
+# ── Report parsing — derived fields against real Thor sample ─────────────────
+
+
+def test_waveform_report_derives_serial_event_type_and_datetime(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["serial_number"]  == "UM11719"
+    assert r["event_type"]     == "Full Waveform"
+    assert r["event_datetime"] == "2023-12-19T16:27:23"
+    assert r["filename"]       == "UM11719_20231219162723.IDFW"
+
+
+def test_waveform_report_parses_peak_velocities(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["tran_ppv"]        == pytest.approx(0.0251)
+    assert r["vert_ppv"]        == pytest.approx(0.2119)
+    assert r["long_ppv"]        == pytest.approx(0.0282)
+    assert r["peak_vector_sum"] == pytest.approx(0.2131)
+
+
+def test_waveform_report_parses_zc_freq_and_mic(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["tran_zc_freq"] == pytest.approx(6.5)
+    assert r["vert_zc_freq"] == pytest.approx(73.1)
+    assert r["long_zc_freq"] == pytest.approx(85.3)
+    assert r["mic_ppv"]      == pytest.approx(99.4)
+
+
+def test_waveform_report_parses_record_and_pretrigger_durations(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["record_time_sec"] == pytest.approx(2.0)
+    assert r["pre_trigger_sec"] == pytest.approx(0.25)
+
+
+def test_waveform_report_parses_sample_rate(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["sample_rate"] == 1024
+
+
+def test_waveform_report_extracts_title_strings(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    # TitleString1 (location) → project
+    assert r["project"] == "UPMC Presby-Loc 3-Level1-1R Elevator Rm"
+    # TitleString2 → client
+    assert r["client"]  == "Whiting-Turner - PJ Dick - Joint Venture"
+    # TitleString3 → operator (company)
+    assert r["operator"] == "Terra-Mechanics, Inc. - D. Harrsion"
+
+
+def test_waveform_report_extracts_setup_version_and_calibration(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["setup"]            == "UPMC Loc 3.mmb"
+    assert r["version"]          == "Micromate ISEE 11.0AK"
+    assert r["calibration_text"] == "November 22, 2023 by Instantel"
+    assert r["battery_volts"]    == pytest.approx(3.8)
+
+
+def test_waveform_report_decodes_sensor_self_check(upmc_waveform_txt):
+    r = parse_idf_report(upmc_waveform_txt)
+    assert r["tran_test_passed"] is True
+    assert r["vert_test_passed"] is True
+    assert r["long_test_passed"] is True
+    assert r["mic_test_passed"]  is True
+
+
+def test_histogram_report_parses(upmc_histogram_txt):
+    """Histogram sidecars have the same shape as waveform — both
+    decode through the same parser without errors."""
+    r = parse_idf_report(upmc_histogram_txt)
+    assert r["serial_number"] == "UM11719"
+    # IDFH timestamp in the sample
+    assert r["event_datetime"] == "2023-12-19T16:34:44"
+    assert r["event_type"]    .lower().startswith("full histogram") or \
+           r["event_type"]    .lower().startswith("histogram")
+    # Sample rate present
+    assert "sample_rate" in r
+
+
+# ── Edge cases ───────────────────────────────────────────────────────────────
+
+
+def test_parses_bytes_input():
+    text = (
+        '"SerialNumber : UM11719"\n'
+        '"TranPPV : 0.0251 in/s"\n'
+    )
+    r = parse_idf_report(text.encode("utf-8"))
+    assert r["serial_number"] == "UM11719"
+    assert r["tran_ppv"] == pytest.approx(0.0251)
+
+
+def test_parses_latin1_fallback():
+    """Garbled non-UTF8 bytes fall back to latin-1 instead of crashing."""
+    text = b'"SerialNumber : UM11719"\n"Operator : Caf\xe9"\n'
+    r = parse_idf_report(text)
+    assert r["serial_number"] == "UM11719"
+    assert r["operator"] == "Café"
+
+
+def test_stops_at_waveform_data_marker():
+    """Lines after the 'Waveform Data Channels' marker are not parsed
+    as key/value pairs — they're tabular sample data."""
+    text = (
+        '"SerialNumber : UM11719"\n'
+        '"TranPPV : 0.0251 in/s"\n'
+        'Waveform Data Channels\n'
+        '    Tran    Vert    Long    MicL\n'
+        '    0.0003  -0.0003  0.0003  0.00013\n'
+    )
+    r = parse_idf_report(text)
+    assert r["serial_number"] == "UM11719"
+    assert r["tran_ppv"] == pytest.approx(0.0251)
+    # No spurious entries from the table body
+    assert "tran" not in r
+    assert "0.0003" not in r
+
+
+def test_missing_event_time_omits_datetime():
+    r = parse_idf_report('"SerialNumber : UM11719"\n')
+    assert r["serial_number"] == "UM11719"
+    assert "event_datetime" not in r
+
+
+def test_handles_empty_input():
+    r = parse_idf_report("")
+    assert r == {
+        "project":  None,
+        "client":   None,
+        "operator": None,
+        "notes":    None,
+    }