Merge branch 'main' into codec-re

CHANGELOG.md

@@ -4,6 +4,33 @@ All notable changes to seismo-relay are documented here.
 
 ---
 
+## v0.16.0 — 2026-05-11
+
+The "BW ACH ingestion" release.  When paired with **series3-watcher v1.5.0**, every Blastware ACH event (binary + `_ASCII.TXT` report) lands in SeismoDb with device-authoritative peaks, project metadata, sensor self-check, and ZC/Time-of-Peak data — without depending on the still-undecoded waveform body codec.  This is the end-to-end product win discussed in v0.15.0's "out of scope" notes: sortable / filterable monthly-summary review of historical events, populated from the BW ASCII export rather than re-decoded samples.
+
+### Added — `/db/import/blastware_file` rich-metadata ingestion
+
+- **Paired BW ASCII reports.**  The endpoint now accepts the `<binary>_<ext>_ASCII.TXT` partner BW writes alongside each event.  Pairing handles both filename conventions: ACH (`M529LK44_AB0_ASCII.TXT`) and manual-export (`M529LK44.AB0.TXT`).  When both present, ACH wins.
+- **`minimateplus/bw_ascii_report.py`** (new) — parser + `BwAsciiReport` dataclass for BW's per-event ASCII export.  Handles every field BW writes: identity, trigger config, per-channel PPV / ZC Freq / Time of Peak / Peak Acceleration / Peak Displacement, Peak Vector Sum + time, MicL PSPL / Time of Peak / ZC Freq, sensor self-check (Test Freq / Test Ratio / Test Amplitude / Pass-Fail per channel), monitor log, PC SW version.
+- **Position-based user-notes parsing.**  BW's Compliance Setup → Notes tab labels (Project / Client / User Name / Seis Loc) are *operator-editable* — an operator can rename them to "Building:", "Site Address:", etc.  Rather than maintain a label-spelling map, the parser uses positional matching between the `Units :` and `Geo Range :` anchors in the ASCII output.  The four canonical slots (project / client / operator / sensor_location) populate by position regardless of label; the original labels BW wrote are preserved in `report.user_note_labels` for downstream UIs (terra-view) to display verbatim.
+- **`bw_report` sidecar block.**  New top-level block in `.sfm.json` carrying the parsed BW report (trigger config, peaks with per-channel stats, mic block, sensor_check, monitor_log, PC SW version, operator-label labels).
+- **`apply_report_to_event(event, report)` helper.**  Overlays the report's device-authoritative fields onto an in-memory `Event` so `SeismoDb.insert_events()` writes correct DB columns instead of the broken-codec values from `_peaks_from_samples()`.
+
+### Fixed — three compounding bugs that left forwarded events with garbage data
+
+- **Import endpoint inserted under `serial="UNKNOWN"`.**  `_serial_from_event(ev)` was a stub that always returned `None`; the BW-filename-decoded serial that `WaveformStore` had already resolved was never surfaced to `db.insert_events`.  Now uses `rec["serial"]` as the authoritative source.  `scripts/repair_unknown_serials.py` repairs existing DB rows.
+- **`/db/units` ignored events from non-ACH ingest paths.**  `query_units()` only aggregated from `ach_sessions` — events that arrived via `save_imported_bw()` were never visible in the fleet overview even though they populated `events` correctly.  Now unions both tables.
+- **Re-imports left stale DB rows.**  The `IntegrityError` handler in `insert_events()` only refreshed filename / sidecar columns when a duplicate `(serial, timestamp)` arrived.  Peak values, project info, sample_rate, record_type stayed locked at whatever the first (often broken-codec) insert wrote.  Now the upsert path refreshes every device-authoritative column from the new data while preserving `false_trigger` and immutable fields (`id`, `created_at`).
+- **Server-side TXT pairing only knew the legacy convention.**  The endpoint stripped `.TXT` and looked up `<binary>` — which works for manual exports (`<binary>.TXT`) but not BW ACH (`<stem>_<ext>_ASCII.TXT`).  Reports were arriving in the multipart but silently dropped.  Now recognises both conventions and registers each report under all matching binary names.
+
+### Migration
+
+For existing deployments where events were forwarded by an older watcher (broken pairing) or imported during the UNKNOWN-bucketing window:
+
+1. `python -m scripts.repair_unknown_serials --db <path> --apply` to re-attribute `serial="UNKNOWN"` rows.
+2. Delete the watcher's `sfm_forwarded.json` state file and let it re-forward.  The server's upsert path will refresh the existing DB rows with the report's authoritative values.
+3. Operator review state (`false_trigger`, sidecar `review` block) is preserved across the re-import.
+
 ## v0.15.0 — 2026-05-07
 
 ### Added

CLAUDE.md

@@ -1353,6 +1353,8 @@ body) because writing a dial string may require DLE escaping for embedded contro
 
 ## What's next
 
+**See [README.md → Roadmap (Future)](README.md#roadmap-future) for the canonical deferred-work list.** This section is kept as a status log of in-progress / recently-shipped technical details (encoding schemes, byte layouts, etc.) that are too low-level for the README's roadmap.
+
 - **Database** — SQLite store for events + monitor log entries; dedup by key; queryable
 - **Histograms** — decode histogram-mode A5 data (noise floor tracking)
 - **Blastware-compatible file output** — `write_blastware_file()` and `write_mlg()` implemented. `blastware_filename()` generates correct Blastware filenames (AB0 for direct, AB0W/AB0H for ACH). **Confirmed BYTE-PERFECT against BW reference (v0.14.3, 2026-05-05):** when fed the BW 5-1-26 3-sec capture's A5 frames, the SFM-built file matches BW's saved `M529LKIQ.G10` byte-for-byte (8708 bytes, 0 differences).  Live SFM downloads of event 0 (3-sec) and event 1 (3-sec continuation) both open cleanly in Blastware with full Event Reports, frequency analysis, and waveform plots.  Body assembly is just contiguous concatenation of frame contributions in stream order (probe → meta@0x1002 → meta@0x1004 → samples → TERM); no stripping, no overlay, no special handling.  Histogram+Continuous mode deferred (5A stream for those events embeds histogram interval records that may need different handling — untested under v0.14.x). Extension mapping: extensions encode timestamp (AB0T for ACH, AB0 for direct), NOT recording mode. Filename format: `<prefix_letter><serial3><4-char-base36-stem><ext>`

README.md

@@ -1,4 +1,4 @@
-# seismo-relay  `v0.15.0`
+# seismo-relay  `v0.16.0`
 
 A ground-up replacement for **Blastware** — Instantel's aging Windows-only
 software for managing MiniMate Plus seismographs.
@@ -14,11 +14,12 @@ over direct RS-232 or cellular modem (Sierra Wireless RV50 / RV55).
 > byte-perfect against Blastware captures across 2-sec, 3-sec, and 10-sec
 > events.** Generated `.G10` / `.AB0` files open cleanly in Blastware with
 > full Event Reports, frequency analysis, and waveform plots.
-> **v0.15.0 (2026-05-07)** adds layered per-event storage (BW binary +
+> **v0.16.0 (2026-05-11)** adds BW ASCII report ingestion to
-> raw 5A pickle + HDF5 + `.sfm.json` sidecar), a plot-ready
+> `/db/import/blastware_file` — paired with **series3-watcher v1.5.0**,
-> `sfm.plot.v1` JSON shape with server-side ADC-to-physical-units
+> every Blastware ACH event lands in SeismoDb with device-authoritative
-> conversion, and a BW-file importer for ingesting externally-produced
+> peaks, project metadata, sensor self-check, and ZC/Time-of-Peak data,
-> events.  See [CHANGELOG.md](CHANGELOG.md) for full version history.
+> without depending on the still-undecoded waveform body codec.
+> See [CHANGELOG.md](CHANGELOG.md) for full version history.
 
 ---
 
@@ -356,10 +357,40 @@ Use **com0com** or **VSPD** to create the virtual COM pair on Windows.
 
 ## Roadmap (Future)
 
-- [ ] Verify 30-sec event download — body may exceed `0xFFFF` and force the device into a different `end_key` encoding (none of 2/3/10-sec test cases hit this boundary)
+### High-impact (unblocks product features)
-- [ ] Terra-view integration — seismo-relay router, unit detail page, VISON-style event listing
+
-- [ ] Vibration summary reports — highest legit PPV per project → Word doc (false trigger filtering first)
+- [ ] **Waveform body codec reverse-engineering.**  The 5A bulk-stream body is some kind of compressed/encoded format (not raw int16 LE as previously assumed — see §7.6.1 retraction in `docs/instantel_protocol_reference.md`).  Structural framing is ~50% decoded on branch `claude/codec-re-cBGNe` (tagged-block walker, segment counters); per-byte sample mapping is still open.  Until this lands, the in-app waveform viewer renders garbage and BW-import peak values fall back to `_peaks_from_samples()` saturation noise.  Workaround: pair every BW-imported event with its `_ASCII.TXT` so the device-authoritative peaks land in the DB regardless of codec.
-- [ ] Compliance config encoder — build raw write payloads from a `ComplianceConfig` object
+- [ ] **In-app waveform viewer accuracy.**  Depends on codec decode.  Plot.v1 JSON pipeline + viewer skeleton already exist; will start showing real waveforms automatically once `_decode_a5_waveform` produces correct samples.
-- [ ] Modem manager — push RV50/RV55 configs via Sierra Wireless API
+- [ ] **Terra-view integration** — seismo-relay router, unit detail page, VISON-style event listing.
-- [ ] Histogram mode recording support (5A stream analysis for mode 0x03)
+- [ ] **Vibration summary reports** — highest legit PPV per project → Word doc (false-trigger filtering first).
-- [ ] Call Home dial_string write support (requires DLE escaping for embedded control characters)
+
+### BW ASCII report parser enhancements (built in v0.16.0)
+
+- [ ] **Histogram-specific structural fields.**  Current parser handles the shared fields (PPV, ZC Freq, sensor self-check, project) but silently drops histogram-only fields: `Histogram Start/Stop Time`, `Histogram Start/Stop Date`, `Number of Intervals`, `Interval Size`, per-channel `Peak Time` + `Peak Date` (absolute timestamps rather than the waveform's `Time of Peak` relative seconds).
+- [ ] **Histogram interval bin-table parsing.**  Trailing 792-row table (per-interval Peak/Freq per channel + MicL) in histogram TXTs is unparsed.  Probably too big for the sidecar JSON; may want a separate `.histogram.h5` companion file.
+- [ ] **`>100 Hz` value parsing.**  Histogram TXTs use `>100 Hz` for out-of-range ZC freq; current `_parse_number()` returns `None` for these (loses information).
+
+### Ingestion gaps
+
+- [ ] **MLG forwarding.**  `series3-watcher` forwards event binaries + their `_ASCII.TXT` reports, but skips `.MLG` per-unit monitor log files entirely.  Adding an `POST /db/import/mlg_file` endpoint + watcher scan path would populate `monitor_log` for non-ACH-routed units (coverage queries, "was this unit monitoring on date X" lookups).
+- [ ] **0C-record raw bytes persistence in the sidecar.**  Currently on branch `claude/codec-re-cBGNe` as commit `a187124`; cherry-pick if useful as a standalone fix.  Preserves the 210-byte 0C record under `extensions.raw_records.waveform_record_b64` so future field-offset analysis (Peak Acceleration / Time of Peak / etc. — the fields BW computes client-side from samples) can run offline.
+
+### Operational
+
+- [ ] **`series3-watcher` file archive manager** — 90-day-old events moved to `<watch_folder>_archive/<year>/<month>/` subfolders.  Plan drafted in `claude/codec-re-cBGNe`'s plan-mode session; awaiting a 5-minute test on whether Blastware UI walks subfolders before any code lands (determines layout: in-place subfolders vs sibling archive).
+- [ ] **Compliance config encoder** — build raw write payloads from a `ComplianceConfig` object.
+- [ ] **Modem manager** — push RV50/RV55 configs via Sierra Wireless API.
+- [ ] **Call Home dial_string write support** (requires DLE escaping for embedded control characters).
+- [ ] **Histogram mode recording support** (5A stream analysis for mode 0x03 — separate from histogram ASCII parsing above).
+
+### Test coverage
+
+- [ ] Verify 30-sec event download — body may exceed `0xFFFF` and force the device into a different `end_key` encoding (none of the 2/3/10-sec test cases hit this boundary).
+- [ ] Histogram mode (0x03) write via SFM — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state.
+
+### Lower-priority cleanups
+
+- [ ] Compliance write anchor-9 cleanup — when changing recording_mode via SFM, a spurious `0x10` may persist after Histogram→other mode transitions.  Doesn't affect device operation but differs from BW's byte-perfect output.
+- [ ] Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring).
+- [ ] Call Home — map time slots 3/4 offsets; confirm `modem_power_relay_enabled`.
+- [ ] RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred).

minimateplus/bw_ascii_report.py

@@ -0,0 +1,522 @@
+"""
+minimateplus/bw_ascii_report.py — parser for Blastware's per-event ASCII
+report (the .TXT file BW writes alongside each saved event binary).
+
+The ASCII export is the authoritative source for every "rich" per-event
+field that BW computes from the waveform but never persists in the BW
+binary itself:
+
+  - Per-channel PPV (Tran / Vert / Long / MicL)
+  - Peak Vector Sum + Peak Vector Sum Time
+  - Per-channel ZC Freq, Time of Peak, Peak Acceleration, Peak Displacement
+  - MicL PSPL, MicL Time of Peak, MicL ZC Freq
+  - Per-channel Sensor Self-Check (Test Freq / Test Ratio / Test Results)
+  - MicL Test Amplitude (mV)
+  - Battery, calibration date, monitor-log timestamps
+
+Persisting these values into the SFM database lets the monthly-summary
+review workflow ("show me events at Location X with PVS > 0.5") work
+without depending on the (still-undecoded) waveform body codec.
+
+Format (verified against decode-re/5-8-26 4-event bundle):
+
+  - One field per line, wrapped in double quotes:   `"Field Name : Value"`
+  - Field/value separator: literal ` : ` (space-colon-space).
+  - Some field names contain an internal `:` already (e.g. `"Project:"`),
+    so we split on the FIRST ` : ` only.
+  - Some fields have unit suffixes:  `"0.500 in/s"` / `"7.5 Hz"` / `"533 mv"`.
+  - A `"Monitor Log(s)"` marker line is followed by tab-separated rows
+    of `start_time<TAB>stop_time<TAB>description`.
+  - Final `"PC SW Version : ..."` line ends the metadata block.
+  - A blank line separates metadata from the sample table.
+  - Sample table starts with `   Tran   <TAB>   Vert   <TAB>...`, then
+    one row per sample (tab-separated, right-padded numeric values).
+  - Geo channel values are in in/s; MicL in dB(L) (or 0.000 below threshold).
+
+Because some metadata fields have whitespace quirks ("MicL  Time of
+Peak" has two spaces; the leading "Project:" value has its own colon),
+we normalise whitespace in the key before lookup.
+"""
+
+from __future__ import annotations
+
+import datetime
+import re
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Dict, List, Optional, Tuple, Union
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Output dataclasses
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+@dataclass
+class ChannelStats:
+    """Per-channel derived stats, populated from an event report."""
+    ppv_ips:           Optional[float] = None      # in/s            (geo channels only)
+    zc_freq_hz:        Optional[float] = None      # Hz
+    time_of_peak_s:    Optional[float] = None      # seconds (relative to trigger; can be negative)
+    peak_accel_g:      Optional[float] = None      # g               (geo channels only)
+    peak_disp_in:      Optional[float] = None      # in              (geo channels only)
+
+
+@dataclass
+class MicStats:
+    """MicL-specific stats."""
+    weighting:         Optional[str]   = None      # e.g. "Linear Weighting"
+    pspl_dbl:          Optional[float] = None      # dB(L)
+    zc_freq_hz:        Optional[float] = None
+    time_of_peak_s:    Optional[float] = None
+
+
+@dataclass
+class SensorCheck:
+    """Per-channel sensor self-check result.
+
+    Geo channels report a frequency + ratio; MicL reports a frequency +
+    amplitude (mV).  All channels also have a Pass/Fail string.
+    """
+    test_freq_hz:      Optional[float] = None
+    test_ratio:        Optional[float] = None      # geo channels only
+    test_amplitude_mv: Optional[float] = None      # MicL only
+    test_results:      Optional[str]   = None      # "Passed" / "Failed"
+
+
+@dataclass
+class MonitorLogEntry:
+    """One row of the trailing Monitor Log(s) block."""
+    start_time:  Optional[datetime.datetime] = None
+    stop_time:   Optional[datetime.datetime] = None
+    description: Optional[str] = None
+
+
+@dataclass
+class BwAsciiReport:
+    """Structured representation of one BW per-event ASCII export."""
+    # ── Identity ─────────────────────────────────────────────────────────────
+    event_type:        Optional[str] = None         # e.g. "Full Waveform"
+    serial:            Optional[str] = None         # e.g. "BE11529"
+    version:           Optional[str] = None         # firmware version line
+    file_name:         Optional[str] = None         # e.g. "M529LK44.AB0"
+    event_datetime:    Optional[datetime.datetime] = None  # parsed from Event Time + Event Date
+
+    # ── Trigger / recording config ──────────────────────────────────────────
+    trigger_channel:        Optional[str]   = None  # e.g. "Vert" or "From Unit"
+    geo_trigger_level_ips:  Optional[float] = None
+    pretrig_s:              Optional[float] = None  # negative seconds
+    record_time_s:          Optional[float] = None
+    record_stop_mode:       Optional[str]   = None
+    sample_rate_sps:        Optional[int]   = None
+    battery_volts:          Optional[float] = None
+    calibration_date:       Optional[datetime.date] = None
+    calibration_by:         Optional[str]   = None  # e.g. "Instantel"
+    units:                  Optional[str]   = None  # e.g. "in/s and dB(L)"
+
+    # ── Operator-supplied metadata ──────────────────────────────────────────
+    # Parsed by POSITION from the 4-line "User Notes" block BW writes
+    # between the `Units :` and `Geo Range :` lines.  Position-based so
+    # the values populate correctly even when an operator renames the
+    # labels in Blastware's Compliance Setup → Notes tab (the 4 labels
+    # are user-editable, e.g. "Seis Loc:" → "Building:" → "Site Address:").
+    # The original labels BW wrote are preserved in `user_note_labels`
+    # so terra-view can render them as the operator named them.
+    project:           Optional[str] = None     # position 1 (BW default label "Project:")
+    client:            Optional[str] = None     # position 2 (BW default label "Client:")
+    operator:          Optional[str] = None     # position 3 (BW default label "User Name:")
+    sensor_location:   Optional[str] = None     # position 4 (BW default label "Seis Loc:")
+
+    # Maps canonical slot name → the literal label BW wrote in the ASCII
+    # export.  Empty if the User Notes block wasn't present.  Example
+    # when the operator renamed slot 4 to "Building:":
+    #     {"project": "Project:", "client": "Client:",
+    #      "operator": "User Name:", "sensor_location": "Building:"}
+    user_note_labels:  Dict[str, str] = field(default_factory=dict)
+
+    # ── Geo channel scaling ─────────────────────────────────────────────────
+    geo_range_ips:     Optional[float] = None       # 10.000 / 1.250
+
+    # ── Per-channel derived stats (geo + mic) ───────────────────────────────
+    channels:          Dict[str, ChannelStats] = field(default_factory=dict)
+    mic:               MicStats = field(default_factory=MicStats)
+
+    # ── Vector sum ──────────────────────────────────────────────────────────
+    peak_vector_sum_ips:    Optional[float] = None
+    peak_vector_sum_time_s: Optional[float] = None
+
+    # ── Sensor self-check (per channel) ─────────────────────────────────────
+    sensor_check:      Dict[str, SensorCheck] = field(default_factory=dict)
+
+    # ── Monitor log + tooling version ───────────────────────────────────────
+    monitor_log:       List[MonitorLogEntry] = field(default_factory=list)
+    pc_sw_version:     Optional[str] = None
+
+    # ── Sample table (optional; only parsed if requested) ───────────────────
+    # Each entry: (Tran, Vert, Long, MicL) in the report's units (geo
+    # channels in in/s, MicL in dB(L)).  None when parse_samples=False.
+    samples:           Optional[List[Tuple[float, float, float, float]]] = None
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Helpers
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+_KEY_NORMALISE_RE = re.compile(r"\s+")
+_NUMERIC_RE       = re.compile(r"^-?\d+(?:\.\d+)?")
+
+
+def _normalise_key(k: str) -> str:
+    """Collapse whitespace runs (incl. tabs) and strip — handles BW's
+    "MicL  Time of Peak" double-space and leading-colon quirks."""
+    return _KEY_NORMALISE_RE.sub(" ", k).strip()
+
+
+def _strip_quotes(line: str) -> str:
+    line = line.rstrip("\r\n")
+    if len(line) >= 2 and line.startswith('"') and line.endswith('"'):
+        return line[1:-1]
+    return line
+
+
+def _parse_number(value: str) -> Optional[float]:
+    """Pull the leading numeric portion out of a value like "0.500 in/s"."""
+    m = _NUMERIC_RE.match(value.strip())
+    if not m:
+        return None
+    try:
+        return float(m.group(0))
+    except ValueError:
+        return None
+
+
+def _parse_int(value: str) -> Optional[int]:
+    n = _parse_number(value)
+    return None if n is None else int(round(n))
+
+
+# Months exactly as BW writes them.
+_MONTHS = {
+    "January": 1, "February": 2, "March": 3, "April": 4,
+    "May": 5, "June": 6, "July": 7, "August": 8,
+    "September": 9, "October": 10, "November": 11, "December": 12,
+    # Short forms used in monitor-log rows ("Apr 23 /26").
+    "Jan": 1, "Feb": 2, "Mar": 3, "Apr": 4, "Jun": 6, "Jul": 7,
+    "Aug": 8, "Sep": 9, "Oct": 10, "Nov": 11, "Dec": 12,
+}
+
+
+def _parse_event_date(s: str) -> Optional[datetime.date]:
+    """Parse "April 23, 2026" or "May 8, 2026" → date."""
+    s = s.strip()
+    parts = s.replace(",", " ").split()
+    if len(parts) < 3:
+        return None
+    month_name, day_str, year_str = parts[0], parts[1], parts[2]
+    month = _MONTHS.get(month_name)
+    if month is None:
+        return None
+    try:
+        return datetime.date(int(year_str), month, int(day_str))
+    except ValueError:
+        return None
+
+
+def _parse_event_time(s: str) -> Optional[datetime.time]:
+    """Parse "15:56:35" → time."""
+    s = s.strip()
+    try:
+        h, m, sec = s.split(":")
+        return datetime.time(int(h), int(m), int(sec))
+    except (ValueError, IndexError):
+        return None
+
+
+def _parse_calibration(value: str) -> Tuple[Optional[datetime.date], Optional[str]]:
+    """Parse "April 29, 2025 by Instantel" → (date, "Instantel")."""
+    parts = value.split(" by ", 1)
+    date = _parse_event_date(parts[0])
+    by = parts[1].strip() if len(parts) > 1 else None
+    return date, by
+
+
+def _parse_monitor_row(line: str) -> Optional[MonitorLogEntry]:
+    """Parse a tab-separated monitor log row.
+
+    Format: `<start>\t<stop>\t<desc>` where each timestamp is BW's
+    short form "Mon DD /YY HH:MM:SS" (e.g. "Apr 23 /26 15:46:16").
+    Year is encoded as a 2-digit suffix; we expand "/26" → 2026.
+    """
+    parts = line.split("\t")
+    if len(parts) < 2:
+        return None
+    start = _parse_monitor_ts(parts[0])
+    stop  = _parse_monitor_ts(parts[1])
+    desc  = parts[2].strip() if len(parts) > 2 else None
+    if start is None and stop is None and not desc:
+        return None
+    return MonitorLogEntry(start_time=start, stop_time=stop, description=desc)
+
+
+def _parse_monitor_ts(s: str) -> Optional[datetime.datetime]:
+    """Parse "Apr 23 /26 15:46:16" → datetime."""
+    s = s.strip()
+    parts = s.split()
+    if len(parts) < 4:
+        return None
+    month = _MONTHS.get(parts[0])
+    if month is None:
+        return None
+    try:
+        day = int(parts[1])
+        # parts[2] looks like "/26" → century-flip to 2026
+        yy = int(parts[2].lstrip("/"))
+        year = 2000 + yy if yy < 80 else 1900 + yy
+        h, m, sec = (int(x) for x in parts[3].split(":"))
+        return datetime.datetime(year, month, day, h, m, sec)
+    except (ValueError, IndexError):
+        return None
+
+
+# ── User-notes positional slot map ──────────────────────────────────────────
+#
+# Blastware's Compliance Setup → Notes tab shows four operator-supplied
+# fields whose LABELS the operator can rename (see screenshot in
+# project archive).  Defaults are "Project:" / "Client:" /
+# "User Name:" / "Seis Loc:", but an operator using a different
+# convention can rename them to anything ("Building:", "Site:",
+# "Address:", etc.).  The ASCII export reflects whatever the operator
+# typed, so label-based matching is fragile.
+#
+# What IS reliable: BW always writes the 4 user-notes lines in the
+# same order, contiguously between the `Units :` line and the
+# `Geo Range :` line.  We parse them by POSITION and preserve the
+# operator's labels in `report.user_note_labels` so terra-view can
+# render them as the operator intended.
+
+_USER_NOTE_SLOTS = ("project", "client", "operator", "sensor_location")
+
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Top-level parser
+# ─────────────────────────────────────────────────────────────────────────────
+
+
+def parse_report(text: Union[str, bytes], *, parse_samples: bool = False) -> BwAsciiReport:
+    """Parse a BW per-event ASCII export into a structured BwAsciiReport.
+
+    Set ``parse_samples=True`` to also populate ``report.samples`` with
+    the trailing sample table.  Default False because the table is
+    huge and most callers only want metadata for indexing.
+    """
+    if isinstance(text, bytes):
+        text = text.decode("ascii", errors="replace")
+
+    report = BwAsciiReport()
+    # Pre-create channel stat slots so callers can rely on them existing.
+    for ch in ("Tran", "Vert", "Long", "MicL"):
+        report.channels.setdefault(ch, ChannelStats())
+        report.sensor_check.setdefault(ch, SensorCheck())
+
+    lines = text.splitlines()
+    i = 0
+    n = len(lines)
+
+    in_monitor_log_section = False
+    event_time_str: Optional[str] = None
+    event_date: Optional[datetime.date] = None
+
+    # User-notes block detection.  We enter the block after parsing
+    # the "Units :" line and exit on the "Geo Range :" line.  Inside,
+    # the first 4 unmatched `<label> : <value>` lines are assigned to
+    # the 4 canonical operator-supplied slots by POSITION (project,
+    # client, operator, sensor_location) regardless of what the
+    # operator named the labels in BW's Compliance Setup → Notes tab.
+    in_user_notes_block = False
+    user_note_position = 0
+
+    while i < n:
+        raw_line = lines[i]
+        i += 1
+        # Blank line marks the start of the sample table.
+        if raw_line.strip() == "":
+            break
+
+        line = _strip_quotes(raw_line)
+
+        # Monitor log section: "Monitor Log(s)" header followed by N rows
+        # (still inside double-quoted lines), terminated by a non-row line
+        # like "PC SW Version : ..." or a blank line.
+        if not in_monitor_log_section and line.strip() == "Monitor Log(s)":
+            in_monitor_log_section = True
+            continue
+        if in_monitor_log_section:
+            # Heuristic: monitor rows contain a tab; the next "Field : Value"
+            # line ends the section.
+            if "\t" in line:
+                entry = _parse_monitor_row(line)
+                if entry:
+                    report.monitor_log.append(entry)
+                continue
+            # Falls through to the field parser below; clear the flag.
+            in_monitor_log_section = False
+
+        # "Field : Value" — split on FIRST occurrence of " : "
+        idx = line.find(" : ")
+        if idx < 0:
+            continue
+        key = _normalise_key(line[:idx])
+        value = line[idx + 3 :].strip()
+
+        # ── Identity / config ────────────────────────────────────────────────
+        if   key == "Event Type":           report.event_type = value
+        elif key == "Serial Number":        report.serial = value
+        elif key == "Version":              report.version = value
+        elif key == "File Name":            report.file_name = value
+        elif key == "Event Time":           event_time_str = value
+        elif key == "Event Date":           event_date = _parse_event_date(value)
+
+        elif key == "Trigger":              report.trigger_channel = value
+        elif key == "Geo Trigger Level":    report.geo_trigger_level_ips = _parse_number(value)
+        elif key == "Pre-trigger Length":   report.pretrig_s = _parse_number(value)
+        elif key == "Record Time":          report.record_time_s = _parse_number(value)
+        elif key == "Record Stop Mode":     report.record_stop_mode = value
+        elif key == "Sample Rate":          report.sample_rate_sps = _parse_int(value)
+        elif key == "Battery Level":        report.battery_volts = _parse_number(value)
+        elif key == "Calibration":
+            report.calibration_date, report.calibration_by = _parse_calibration(value)
+        elif key == "Units":
+            report.units = value
+            # Entering the user-notes block.  Next ~4 lines until
+            # "Geo Range :" are the operator-supplied notes.
+            in_user_notes_block = True
+            user_note_position = 0
+
+        elif key == "Geo Range":
+            # Exiting the user-notes block.
+            in_user_notes_block = False
+            report.geo_range_ips = _parse_number(value)
+
+        # User-notes block: assign by position (operator may have
+        # renamed the labels, so we don't trust them).  Preserve the
+        # original labels in `user_note_labels` for downstream UIs
+        # (terra-view) that want to display them as the operator
+        # named them.
+        elif in_user_notes_block and user_note_position < len(_USER_NOTE_SLOTS):
+            slot = _USER_NOTE_SLOTS[user_note_position]
+            setattr(report, slot, value)
+            report.user_note_labels[slot] = key
+            user_note_position += 1
+
+        # ── Per-channel stats ────────────────────────────────────────────────
+        # All match the pattern "{Channel} <stat-name>"
+        elif key in (
+            "Tran PPV", "Vert PPV", "Long PPV",
+            "Tran ZC Freq", "Vert ZC Freq", "Long ZC Freq",
+            "Tran Time of Peak", "Vert Time of Peak", "Long Time of Peak",
+            "Tran Peak Acceleration", "Vert Peak Acceleration", "Long Peak Acceleration",
+            "Tran Peak Displacement", "Vert Peak Displacement", "Long Peak Displacement",
+        ):
+            ch_name, stat = key.split(" ", 1)
+            cs = report.channels.setdefault(ch_name, ChannelStats())
+            num = _parse_number(value)
+            if   stat == "PPV":                 cs.ppv_ips        = num
+            elif stat == "ZC Freq":             cs.zc_freq_hz     = num
+            elif stat == "Time of Peak":        cs.time_of_peak_s = num
+            elif stat == "Peak Acceleration":   cs.peak_accel_g   = num
+            elif stat == "Peak Displacement":   cs.peak_disp_in   = num
+
+        # ── Vector Sum ───────────────────────────────────────────────────────
+        elif key == "Peak Vector Sum":
+            report.peak_vector_sum_ips = _parse_number(value)
+        elif key == "Peak Vector Sum Time":
+            report.peak_vector_sum_time_s = _parse_number(value)
+
+        # ── Microphone block ────────────────────────────────────────────────
+        elif key == "Microphone":
+            report.mic.weighting = value
+        elif key == "MicL PSPL":
+            report.mic.pspl_dbl = _parse_number(value)
+            # Mirror onto the "MicL" entry in channels so callers querying
+            # `channels["MicL"].ppv_ips` see something — but it's dB(L), not
+            # in/s, so we store as-is in the MicStats and mark the channel.
+        elif key == "MicL Time of Peak":
+            report.mic.time_of_peak_s = _parse_number(value)
+            cs = report.channels.setdefault("MicL", ChannelStats())
+            cs.time_of_peak_s = report.mic.time_of_peak_s
+        elif key == "MicL ZC Freq":
+            report.mic.zc_freq_hz = _parse_number(value)
+            cs = report.channels.setdefault("MicL", ChannelStats())
+            cs.zc_freq_hz = report.mic.zc_freq_hz
+
+        # ── Sensor self-check ────────────────────────────────────────────────
+        elif key in (
+            "Tran Test Freq", "Vert Test Freq", "Long Test Freq", "MicL Test Freq",
+            "Tran Test Ratio", "Vert Test Ratio", "Long Test Ratio",
+            "MicL Test Amplitude",
+            "Tran Test Results", "Vert Test Results", "Long Test Results", "MicL Test Results",
+        ):
+            ch_name, stat = key.split(" ", 1)
+            sc = report.sensor_check.setdefault(ch_name, SensorCheck())
+            if   stat == "Test Freq":      sc.test_freq_hz      = _parse_number(value)
+            elif stat == "Test Ratio":     sc.test_ratio        = _parse_number(value)
+            elif stat == "Test Amplitude": sc.test_amplitude_mv = _parse_number(value)
+            elif stat == "Test Results":   sc.test_results      = value
+
+        # ── Trailer ─────────────────────────────────────────────────────────
+        elif key == "PC SW Version":
+            report.pc_sw_version = value
+
+        # Unknown keys are silently dropped — forward-compat for future
+        # BW versions that may add fields.
+
+    # Combine event date + time into a datetime
+    if event_date is not None and event_time_str is not None:
+        t = _parse_event_time(event_time_str)
+        if t is not None:
+            report.event_datetime = datetime.datetime.combine(event_date, t)
+
+    if parse_samples:
+        report.samples = _parse_sample_table(lines, i)
+
+    return report
+
+
+def _parse_sample_table(
+    lines: List[str], start: int,
+) -> List[Tuple[float, float, float, float]]:
+    """Parse the trailing sample table.
+
+    The table starts with a header row ("   Tran   <TAB>...") and continues
+    until EOF.  Each data row is a tab-separated quartet of numeric values.
+    """
+    samples: List[Tuple[float, float, float, float]] = []
+    seen_header = False
+    for line in lines[start:]:
+        line = line.rstrip("\r\n")
+        if not line.strip():
+            continue
+        cols = [c.strip() for c in line.split("\t") if c.strip()]
+        if not seen_header:
+            # Header row contains channel names; numeric rows don't.
+            if any(c in ("Tran", "Vert", "Long", "MicL") for c in cols):
+                seen_header = True
+            continue
+        if len(cols) < 4:
+            continue
+        try:
+            samples.append((
+                float(cols[0]), float(cols[1]),
+                float(cols[2]), float(cols[3]),
+            ))
+        except ValueError:
+            continue
+    return samples
+
+
+def parse_report_file(
+    path: Union[str, Path], *, parse_samples: bool = False,
+) -> BwAsciiReport:
+    """Convenience: read a .TXT file from disk and parse it."""
+    return parse_report(Path(path).read_bytes(), parse_samples=parse_samples)

minimateplus/event_file_io.py

@@ -26,6 +26,12 @@ from typing import Optional, Union
 
 from .models import Event, PeakValues, ProjectInfo, Timestamp
 from . import blastware_file as _bw  # avoid circular reference at module load
+from .bw_ascii_report import BwAsciiReport
+
+# Reference pressure for dB(L) → psi conversion (20 µPa expressed in psi).
+# Same constant as sfm/sfm_webapp.html so server-side and browser-side
+# conversions agree.
+_DBL_REF_PSI = 2.9e-9
 
 log = logging.getLogger(__name__)
 
@@ -41,7 +47,7 @@ SIDECAR_KIND   = "sfm.event"
 # bumped without a `pip install` re-run — leading to confusing stale
 # version stamps in sidecars.  Bump this constant and CHANGELOG.md
 # together at release time.
-TOOL_VERSION = "0.15.0"
+TOOL_VERSION = "0.16.0"
 
 try:
     # Best-effort: prefer the installed metadata when it's NEWER than the
@@ -94,6 +100,158 @@ def _peak_values_to_dict(pv: Optional[PeakValues]) -> dict:
     }
 
 
+def _bw_report_to_dict(report: BwAsciiReport) -> dict:
+    """Project a parsed BW ASCII report into the sidecar's `bw_report` block.
+
+    All fields are rendered as plain JSON-compatible types (no datetime
+    objects).  Channels are uniformly lowercased for stable JSON keys.
+    """
+    def _ch(ch_name: str) -> dict:
+        cs = report.channels.get(ch_name)
+        if cs is None:
+            return {}
+        out = {
+            "ppv_ips":         cs.ppv_ips,
+            "zc_freq_hz":      cs.zc_freq_hz,
+            "time_of_peak_s":  cs.time_of_peak_s,
+            "peak_accel_g":    cs.peak_accel_g,
+            "peak_disp_in":    cs.peak_disp_in,
+        }
+        # Drop all-None entries — keeps the JSON tidy for partial reports.
+        return {k: v for k, v in out.items() if v is not None}
+
+    def _sc(ch_name: str) -> dict:
+        sc = report.sensor_check.get(ch_name)
+        if sc is None:
+            return {}
+        out = {
+            "freq_hz":      sc.test_freq_hz,
+            "ratio":        sc.test_ratio,
+            "amplitude_mv": sc.test_amplitude_mv,
+            "result":       sc.test_results,
+        }
+        return {k: v for k, v in out.items() if v is not None}
+
+    monitor_log = []
+    for entry in report.monitor_log:
+        e = {
+            "start":       entry.start_time.isoformat() if entry.start_time else None,
+            "stop":        entry.stop_time.isoformat()  if entry.stop_time  else None,
+            "description": entry.description,
+        }
+        monitor_log.append({k: v for k, v in e.items() if v is not None})
+
+    return {
+        "available":   True,
+        "event_type":  report.event_type,
+        "version":     report.version,
+        "trigger": {
+            "channel":       report.trigger_channel,
+            "geo_level_ips": report.geo_trigger_level_ips,
+        },
+        "recording": {
+            "sample_rate_sps":  report.sample_rate_sps,
+            "record_time_s":    report.record_time_s,
+            "pretrig_s":        report.pretrig_s,
+            "stop_mode":        report.record_stop_mode,
+            "geo_range_ips":    report.geo_range_ips,
+            "units":            report.units,
+        },
+        "device": {
+            "battery_volts":    report.battery_volts,
+            "calibration_date": report.calibration_date.isoformat() if report.calibration_date else None,
+            "calibration_by":   report.calibration_by,
+        },
+        "peaks": {
+            "tran":         _ch("Tran"),
+            "vert":         _ch("Vert"),
+            "long":         _ch("Long"),
+            "vector_sum": {
+                "ips":     report.peak_vector_sum_ips,
+                "time_s":  report.peak_vector_sum_time_s,
+            },
+        },
+        "mic": {
+            "weighting":        report.mic.weighting,
+            "pspl_dbl":         report.mic.pspl_dbl,
+            "zc_freq_hz":       report.mic.zc_freq_hz,
+            "time_of_peak_s":   report.mic.time_of_peak_s,
+        },
+        "sensor_check": {
+            "tran": _sc("Tran"),
+            "vert": _sc("Vert"),
+            "long": _sc("Long"),
+            "mic":  _sc("MicL"),
+        },
+        "monitor_log":   monitor_log,
+        "pc_sw_version": report.pc_sw_version,
+    }
+
+
+def _dbl_to_psi(pspl_dbl: float) -> float:
+    """Convert dB(L) sound pressure level back to psi.  Uses the same
+    20 µPa reference (= 2.9e-9 psi) as the webapp so server-side and
+    browser-side conversions agree."""
+    return _DBL_REF_PSI * (10.0 ** (pspl_dbl / 20.0))
+
+
+def apply_report_to_event(event: Event, report: BwAsciiReport) -> None:
+    """Overlay device-authoritative fields from a parsed BW ASCII report
+    onto an in-memory Event, IN-PLACE.
+
+    Why this exists
+    ───────────────
+    `read_blastware_file()` parses the BW binary and fills `Event.peak_values`
+    via `_peaks_from_samples()` — which runs the (still-undecoded) BW body
+    codec assuming raw int16 LE and produces ±32K-shaped noise on every
+    channel.  Result: peak values land in the SeismoDb event row as
+    ~10 in/s on every event regardless of the actual signal.
+
+    When a paired BW ASCII report is available, the report carries the
+    device's own authoritative peak / project / sample-rate / record-time
+    values.  This helper folds those onto the Event before it flows to
+    `SeismoDb.insert_events()`, so the DB columns reflect the report
+    rather than the broken-codec output.
+
+    Fields overlaid (only when the report supplies a non-None value):
+      - peak_values.tran / .vert / .long              (from report.channels)
+      - peak_values.peak_vector_sum                   (from report.peak_vector_sum_ips)
+      - peak_values.micl  (psi)                       (from report.mic.pspl_dbl → psi)
+      - project_info.project / .client / .operator / .sensor_location
+      - sample_rate                                   (from report.sample_rate_sps)
+      - rectime_seconds                               (from report.record_time_s)
+
+    Fields NOT touched (operator-edit / parser-output preserved):
+      - timestamp, raw_samples, record_type, total_samples,
+        pretrig_samples, _waveform_key, _a5_frames, _raw_record
+      - false_trigger and review state (those live on the sidecar, not on Event)
+    """
+    if event.peak_values is None:
+        event.peak_values = PeakValues()
+    pv = event.peak_values
+    ch = report.channels
+    if (t := ch.get("Tran")) and t.ppv_ips is not None: pv.tran = t.ppv_ips
+    if (v := ch.get("Vert")) and v.ppv_ips is not None: pv.vert = v.ppv_ips
+    if (l := ch.get("Long")) and l.ppv_ips is not None: pv.long = l.ppv_ips
+    if report.peak_vector_sum_ips is not None:
+        pv.peak_vector_sum = report.peak_vector_sum_ips
+    if report.mic.pspl_dbl is not None and report.mic.pspl_dbl > 0:
+        pv.micl = _dbl_to_psi(report.mic.pspl_dbl)
+
+    if event.project_info is None:
+        event.project_info = ProjectInfo()
+    pi = event.project_info
+    if report.project:         pi.project         = report.project
+    if report.client:          pi.client          = report.client
+    if report.operator:        pi.operator        = report.operator
+    if report.sensor_location: pi.sensor_location = report.sensor_location
+
+    if report.sample_rate_sps:
+        event.sample_rate = report.sample_rate_sps
+    if report.record_time_s is not None:
+        event.rectime_seconds = report.record_time_s
+
+
 def _project_info_to_dict(pi: Optional[ProjectInfo]) -> dict:
     if pi is None:
         return {
@@ -123,35 +281,104 @@ def event_to_sidecar_dict(
     captured_at: Optional[datetime.datetime] = None,
     review: Optional[dict] = None,
     extensions: Optional[dict] = None,
+    bw_report: Optional[BwAsciiReport] = None,
 ) -> dict:
     """
     Build a v1 sidecar dict from an Event + the surrounding metadata.
 
     Pure helper — no file I/O.  Callers stitch the result into a sidecar
     via `write_sidecar()` (or POST it back via the PATCH endpoint).
+
+    When *bw_report* is supplied (e.g. by the ACH-forwarded import path
+    where Blastware writes a per-event ASCII report alongside the binary),
+    its decoded fields are folded into the sidecar:
+
+      - A new top-level ``bw_report`` block carries the rich derived
+        per-channel stats (Peak Acceleration, Peak Displacement, ZC Freq,
+        Time of Peak), the Peak Vector Sum + time, the per-channel sensor
+        self-check results, and monitor-log timestamps.
+      - ``peak_values`` is overlaid from the report (the report's PPV/PVS
+        values are computed by the device firmware and are authoritative;
+        anything ``read_blastware_file()`` derived from samples is
+        approximate at best until the body codec is decoded).
+      - ``project_info`` is overlaid from the report when the report
+        supplies a non-empty value (the report mirrors the device's
+        compliance config, which is what BW shows in its event report).
+      - ``event.timestamp`` is overlaid from the report's Event Date +
+        Event Time (BW's report timestamps are second-resolution and
+        match the binary's footer; we prefer the report value because
+        the BW-binary footer timestamp can drift on some firmware).
     """
     if source_kind not in {"sfm-live", "sfm-ach", "bw-import"}:
         raise ValueError(f"unknown source_kind: {source_kind!r}")
 
     captured_at = captured_at or datetime.datetime.utcnow()
 
-    return {
+    # ── Overlay event fields from the report when present ───────────────────
+    timestamp_iso = _ts_iso(event.timestamp)
+    if bw_report and bw_report.event_datetime:
+        timestamp_iso = bw_report.event_datetime.isoformat()
+
+    # Build peak_values, optionally overlaid from the report.  The report
+    # stores Mic peak as PSPL (dB(L)); we convert to psi to match the
+    # existing peak_values.mic_psi field.
+    peak_dict = _peak_values_to_dict(event.peak_values)
+    if bw_report:
+        ch = bw_report.channels
+        if (t := ch.get("Tran")) and t.ppv_ips is not None: peak_dict["transverse"]   = t.ppv_ips
+        if (v := ch.get("Vert")) and v.ppv_ips is not None: peak_dict["vertical"]     = v.ppv_ips
+        if (l := ch.get("Long")) and l.ppv_ips is not None: peak_dict["longitudinal"] = l.ppv_ips
+        if bw_report.peak_vector_sum_ips is not None:
+            peak_dict["vector_sum"] = bw_report.peak_vector_sum_ips
+        if bw_report.mic.pspl_dbl is not None and bw_report.mic.pspl_dbl > 0:
+            peak_dict["mic_psi"] = _dbl_to_psi(bw_report.mic.pspl_dbl)
+
+    # Project info: overlay from report (the report mirrors the
+    # session-start compliance config that BW renders in event reports).
+    proj_dict = _project_info_to_dict(event.project_info)
+    if bw_report:
+        if bw_report.project:         proj_dict["project"]         = bw_report.project
+        if bw_report.client:          proj_dict["client"]          = bw_report.client
+        if bw_report.operator:        proj_dict["operator"]        = bw_report.operator
+        if bw_report.sensor_location: proj_dict["sensor_location"] = bw_report.sensor_location
+
+    # Event-block fields: overlay from report where available.
+    event_block = {
+        "serial":           serial,
+        "timestamp":        timestamp_iso,
+        "waveform_key":     event._waveform_key.hex() if event._waveform_key else None,
+        "record_type":      event.record_type,
+        "sample_rate":      event.sample_rate,
+        "rectime_seconds":  event.rectime_seconds,
+        "total_samples":    event.total_samples,
+        "pretrig_samples":  event.pretrig_samples,
+    }
+    if bw_report:
+        # Report values are authoritative — they're the user-configured
+        # values BW reads back, not STRT-derived guesses.  In particular
+        # `event.rectime_seconds` from `read_blastware_file()` reads
+        # STRT[18] which is actually the `0x46` record-type marker (= 70)
+        # rather than the user's Record Time setting.  Always overwrite.
+        if bw_report.sample_rate_sps:
+            event_block["sample_rate"] = bw_report.sample_rate_sps
+        if bw_report.record_time_s is not None:
+            event_block["rectime_seconds"] = bw_report.record_time_s
+        # Derive total_samples + pretrig_samples per channel from the
+        # report's sample_rate × times.  These match the row count of
+        # the report's sample table (verified: event-c reports 1024 sps
+        # × (1.0 + 0.25) = 1280 rows).
+        if (sr := bw_report.sample_rate_sps) and bw_report.record_time_s is not None:
+            pretrig_s = abs(bw_report.pretrig_s) if bw_report.pretrig_s is not None else 0.0
+            event_block["total_samples"]   = int(round(sr * (bw_report.record_time_s + pretrig_s)))
+            event_block["pretrig_samples"] = int(round(sr * pretrig_s))
+
+    out = {
         "schema_version": SCHEMA_VERSION,
         "kind":           SIDECAR_KIND,
 
-        "event": {
+        "event":        event_block,
-            "serial":           serial,
+        "peak_values":  peak_dict,
-            "timestamp":        _ts_iso(event.timestamp),
+        "project_info": proj_dict,
-            "waveform_key":     event._waveform_key.hex() if event._waveform_key else None,
-            "record_type":      event.record_type,
-            "sample_rate":      event.sample_rate,
-            "rectime_seconds":  event.rectime_seconds,
-            "total_samples":    event.total_samples,
-            "pretrig_samples":  event.pretrig_samples,
-        },
-
-        "peak_values":  _peak_values_to_dict(event.peak_values),
-        "project_info": _project_info_to_dict(event.project_info),
 
         "blastware": {
             "filename":  blastware_filename,
@@ -177,6 +404,11 @@ def event_to_sidecar_dict(
         "extensions": extensions or {},
     }
 
+    if bw_report:
+        out["bw_report"] = _bw_report_to_dict(bw_report)
+
+    return out
+
 
 # ── Sidecar IO ────────────────────────────────────────────────────────────────
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "seismo-relay"
-version = "0.15.0"
+version = "0.16.0"
 description = "Python client and REST server for MiniMate Plus seismographs"
 requires-python = ">=3.10"
 dependencies = [

scripts/repair_unknown_serials.py

@@ -0,0 +1,151 @@
+"""
+scripts/repair_unknown_serials.py — re-attribute events stuck under
+`serial = 'UNKNOWN'` to their correct serial by decoding the BW filename.
+
+Why this is needed
+──────────────────
+The /db/import/blastware_file endpoint had a bug (fixed in commit a032fa5+1
+on the ach-report-ingestion branch) where every forwarded event was inserted
+with serial='UNKNOWN' because the endpoint's `_serial_from_event(ev)` stub
+returned None and never consulted the BW-filename serial that
+`WaveformStore.save_imported_bw()` had already decoded.
+
+Effect on a server that ran a buggy version: every forwarded event's
+SeismoDb row has `serial='UNKNOWN'`, even though the on-disk waveform
+store has correctly bucketed the files into `BE<NNNN>/` folders.  So
+the BW binaries / sidecars / HDF5s are fine, but `/db/units` and
+`/db/events?serial=...` queries don't surface the events.
+
+This script
+───────────
+Walks the events table looking for rows with `serial='UNKNOWN'` and
+re-attributes each one to the serial decoded from its
+`blastware_filename` column.  If the row's serial would collide with
+an existing row (already-correct duplicate from a later re-forward),
+the UNKNOWN row is deleted.  Otherwise the row's `serial` column is
+updated in-place.
+
+Idempotent: re-running after a successful repair finds zero matching
+rows and exits cleanly.
+
+Usage
+─────
+  # Dry-run (default): print what would change, don't touch the DB
+  python -m scripts.repair_unknown_serials --db bridges/captures/seismo_relay.db
+
+  # Apply the repair
+  python -m scripts.repair_unknown_serials --db bridges/captures/seismo_relay.db --apply
+"""
+
+from __future__ import annotations
+
+import argparse
+import sqlite3
+import sys
+from pathlib import Path
+
+# Reach into sfm.waveform_store for the serial decoder.  This script
+# is run from the repo root via `python -m scripts.repair_unknown_serials`.
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+from sfm.waveform_store import _serial_from_bw_filename
+
+
+def main(argv: list[str] | None = None) -> int:
+    p = argparse.ArgumentParser(
+        description="Re-attribute events stuck under serial='UNKNOWN'.",
+    )
+    p.add_argument(
+        "--db", required=True, type=Path,
+        help="Path to seismo_relay.db (e.g. bridges/captures/seismo_relay.db)",
+    )
+    p.add_argument(
+        "--apply", action="store_true",
+        help="Apply the repair.  Without this flag the script runs in "
+             "dry-run mode and only reports what would change.",
+    )
+    args = p.parse_args(argv)
+
+    if not args.db.exists():
+        print(f"DB not found: {args.db}", file=sys.stderr)
+        return 2
+
+    conn = sqlite3.connect(str(args.db))
+    conn.row_factory = sqlite3.Row
+
+    rows = list(conn.execute(
+        "SELECT id, serial, timestamp, blastware_filename "
+        "  FROM events "
+        " WHERE serial = 'UNKNOWN' "
+        " ORDER BY timestamp",
+    ))
+    print(f"Found {len(rows)} UNKNOWN-serial rows in events table.")
+    if not rows:
+        return 0
+
+    updated   = 0
+    deleted   = 0
+    unresolved = 0
+    by_serial: dict[str, int] = {}
+
+    for row in rows:
+        rid       = row["id"]
+        ts        = row["timestamp"]
+        bw_name   = row["blastware_filename"]
+        new_serial = _serial_from_bw_filename(bw_name) if bw_name else None
+        if not new_serial:
+            print(f"  ⚠ id={rid[:8]} ts={ts} filename={bw_name!r} — "
+                  f"cannot decode serial from filename; skipping")
+            unresolved += 1
+            continue
+
+        # Check for an existing row at the target (serial, timestamp).
+        existing = conn.execute(
+            "SELECT id FROM events WHERE serial = ? AND timestamp = ?",
+            (new_serial, ts),
+        ).fetchone()
+        action: str
+        if existing is None:
+            # Safe to UPDATE in place.
+            if args.apply:
+                conn.execute(
+                    "UPDATE events SET serial = ? WHERE id = ?",
+                    (new_serial, rid),
+                )
+            action = "UPDATE"
+            updated += 1
+        else:
+            # A correctly-attributed row already exists.  Drop the
+            # UNKNOWN duplicate.
+            if args.apply:
+                conn.execute("DELETE FROM events WHERE id = ?", (rid,))
+            action = "DELETE (dup)"
+            deleted += 1
+
+        by_serial[new_serial] = by_serial.get(new_serial, 0) + 1
+        print(f"  {action:14s}  id={rid[:8]}  ts={ts}  "
+              f"filename={bw_name}  →  {new_serial}")
+
+    if args.apply:
+        conn.commit()
+    conn.close()
+
+    print()
+    print(f"Summary:")
+    print(f"  UNKNOWN rows scanned:       {len(rows)}")
+    print(f"  Updated to real serial:     {updated}")
+    print(f"  Deleted (duplicate of an    ")
+    print(f"   already-correct row):      {deleted}")
+    print(f"  Unresolved (bad filename):  {unresolved}")
+    print()
+    if by_serial:
+        print(f"Per-serial breakdown of repaired rows:")
+        for serial, count in sorted(by_serial.items()):
+            print(f"  {serial:12s}  {count}")
+    if not args.apply:
+        print()
+        print("(dry-run — re-run with --apply to commit)")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

sfm/database.py

@@ -374,28 +374,64 @@ class SeismoDb:
                     inserted += 1
                 except sqlite3.IntegrityError:
                     skipped += 1
-                    # Upsert waveform fields onto the existing dedup row so a
+                    # UPSERT path: a row for this (serial, timestamp) already
-                    # re-download via the live endpoint refreshes filename /
+                    # exists.  Refresh every device-authoritative field from
-                    # size / sidecar without churning the rest of the row.
+                    # the new data so that a re-import with better data (e.g.
-                    if rec and ts:
+                    # a watcher re-forward where the previous attempt missed
-                        conn.execute(
+                    # the paired BW ASCII report) replaces stale peaks /
-                            """
+                    # project info / sample_rate.
-                            UPDATE events
+                    #
-                               SET blastware_filename = ?,
+                    # Preserved (not in this UPDATE):
-                                   blastware_filesize = ?,
+                    #   id, waveform_key, session_id, created_at  — immutable / FK
-                                   a5_pickle_filename = ?,
+                    #   false_trigger                              — operator review state
-                                   sidecar_filename   = ?
+                    #
-                             WHERE serial = ? AND timestamp = ?
+                    # Behaviour change vs prior versions: this UPDATE used
-                            """,
+                    # to only refresh filename / filesize / a5_pickle /
-                            (
+                    # sidecar fields.  As a result, the first insert's
-                                rec.get("filename"),
+                    # broken-codec peak values were locked in forever even
-                                rec.get("filesize"),
+                    # if subsequent re-forwards arrived with correct
-                                rec.get("a5_pickle_filename"),
+                    # report-derived values.  Now every re-import lifts the
-                                rec.get("sidecar_filename"),
+                    # DB row up to whatever the latest Event carries.
-                                serial,
+                    conn.execute(
-                                ts,
+                        """
-                            ),
+                        UPDATE events
-                        )
+                           SET tran_ppv           = ?,
+                               vert_ppv           = ?,
+                               long_ppv           = ?,
+                               peak_vector_sum    = ?,
+                               mic_ppv            = ?,
+                               project            = ?,
+                               client             = ?,
+                               operator           = ?,
+                               sensor_location    = ?,
+                               sample_rate        = ?,
+                               record_type        = ?,
+                               blastware_filename = ?,
+                               blastware_filesize = ?,
+                               a5_pickle_filename = ?,
+                               sidecar_filename   = ?
+                         WHERE serial = ? AND timestamp = ?
+                        """,
+                        (
+                            pv.tran            if pv else None,
+                            pv.vert            if pv else None,
+                            pv.long            if pv else None,
+                            pv.peak_vector_sum if pv else None,
+                            pv.micl            if pv else None,
+                            pi.project         if pi else None,
+                            pi.client          if pi else None,
+                            pi.operator        if pi else None,
+                            pi.sensor_location if pi else None,
+                            ev.sample_rate,
+                            ev.record_type,
+                            rec.get("filename")           if rec else None,
+                            rec.get("filesize")           if rec else None,
+                            rec.get("a5_pickle_filename") if rec else None,
+                            rec.get("sidecar_filename")   if rec else None,
+                            serial,
+                            ts,
+                        ),
+                    )
 
         log.debug("insert_events serial=%s inserted=%d skipped=%d",
                   serial, inserted, skipped)
@@ -564,21 +600,79 @@ class SeismoDb:
 
     def query_units(self) -> list[dict]:
         """
-        Return one row per known serial with summary stats:
+        Return one row per known serial with summary stats.
-        last_seen, total_events, total_monitor_entries.
+
+        Aggregates from BOTH source tables:
+          - `events`        — populated by every ingest path
+                              (live ACH, /db/import/blastware_file
+                              from the series3-watcher forwarder, etc.)
+          - `ach_sessions`  — only populated by the live ACH server;
+                              empty for events that came in via the
+                              BW-importer route.
+
+        Earlier this method only joined on `ach_sessions`, which made
+        watcher-forwarded units invisible to the SFM webapp's fleet
+        overview even though their events were correctly populated in
+        `events`.  Now we union the two and surface every serial that
+        has activity in either table.
+
+        Fields:
+          serial                — unit serial number (e.g. "BE11529")
+          last_seen             — most recent of MAX(events.timestamp)
+                                  and MAX(ach_sessions.session_time)
+          total_events          — COUNT(*) from `events` (the
+                                  authoritative count regardless of
+                                  ingest path)
+          total_monitor_entries — from `ach_sessions`, 0 when absent
+          total_sessions        — COUNT(*) from `ach_sessions`, 0 when absent
         """
         with self._connect() as conn:
-            rows = conn.execute(
+            event_stats = {
-                """
+                row["serial"]: row
-                SELECT
+                for row in conn.execute(
-                    s.serial,
+                    """
-                    MAX(s.session_time)  AS last_seen,
+                    SELECT serial,
-                    SUM(s.events_downloaded)  AS total_events,
+                           MAX(timestamp) AS last_event_at,
-                    SUM(s.monitor_entries)    AS total_monitor_entries,
+                           COUNT(*)        AS total_events
-                    COUNT(*)                  AS total_sessions
+                      FROM events
-                FROM ach_sessions s
+                     GROUP BY serial
-                GROUP BY s.serial
+                    """,
-                ORDER BY last_seen DESC
+                ).fetchall()
-                """
+            }
-            ).fetchall()
+            session_stats = {
-        return [dict(r) for r in rows]
+                row["serial"]: row
+                for row in conn.execute(
+                    """
+                    SELECT serial,
+                           MAX(session_time)         AS last_session_at,
+                           SUM(monitor_entries)      AS total_monitor_entries,
+                           COUNT(*)                  AS total_sessions
+                      FROM ach_sessions
+                     GROUP BY serial
+                    """,
+                ).fetchall()
+            }
+
+        all_serials = set(event_stats) | set(session_stats)
+        units = []
+        for serial in all_serials:
+            e = event_stats.get(serial)
+            s = session_stats.get(serial)
+            last_event_at   = e["last_event_at"]   if e else None
+            last_session_at = s["last_session_at"] if s else None
+            # Prefer whichever timestamp is more recent
+            last_seen = max(
+                (t for t in (last_event_at, last_session_at) if t),
+                default=None,
+            )
+            units.append({
+                "serial":                serial,
+                "last_seen":             last_seen,
+                "total_events":          e["total_events"]          if e else 0,
+                "total_monitor_entries": s["total_monitor_entries"] if s else 0,
+                "total_sessions":        s["total_sessions"]        if s else 0,
+            })
+
+        # Sort by last_seen desc; serials with no timestamp at all sink to the bottom.
+        units.sort(key=lambda u: u.get("last_seen") or "", reverse=True)
+        return units

sfm/server.py

@@ -1619,6 +1619,21 @@ async def db_import_blastware_file(
          writes a .sfm.json sidecar with source.kind = "bw-import".
       2. Upsert a row into `events` (dedup'd on serial+timestamp).
 
+    **Paired BW ASCII reports.**  When Blastware's ACH writes events,
+    it also emits a per-event report alongside each binary as
+    ``<binary>.TXT`` (e.g. ``M529LK44.AB0`` + ``M529LK44.AB0.TXT``).
+    If a request includes ``.TXT`` files matching a binary's filename,
+    the report is parsed and its decoded fields land in the sidecar's
+    ``bw_report`` block — including device-authoritative peaks, ZC
+    Freq, Peak Acceleration, Peak Displacement, Time of Peak, sensor
+    self-check results, and monitor-log timestamps.  The daemon-
+    forwarded ACH workflow should always send both files together
+    so the SFM database has the rich metadata for sort/filter/report.
+
+    Pairing is by exact filename match (case-insensitive on the
+    extension): a binary named ``foo.AB0`` is paired with a report
+    named ``foo.AB0.TXT`` or ``foo.AB0.txt``.
+
     Response includes per-file outcomes so the caller can see which
     landed cleanly and which failed (e.g. malformed file, unknown
     serial, etc.).
@@ -1627,46 +1642,108 @@ async def db_import_blastware_file(
     db    = _get_db()
     results: list[dict] = []
 
+    # Read every upload up front (UploadFile.read() is one-shot under
+    # FastAPI's spooled-tempfile backing) and split into binaries vs
+    # paired ASCII reports.
+    binaries: list[tuple[str, bytes]] = []
+    reports:  dict[str, bytes]        = {}   # keyed by lower-cased stem (without .txt)
     for upload in files:
+        name = upload.filename or ""
         try:
             content = await upload.read()
         except Exception as exc:
             results.append({
-                "filename": upload.filename, "status": "error",
+                "filename": name or "<unnamed>", "status": "error",
-                "detail": f"read failed: {exc}",
+                "detail":   f"read failed: {exc}",
             })
             continue
 
+        if name.lower().endswith(".txt"):
+            # Pair the report back to its binary.  BW writes ASCII
+            # reports under two conventions:
+            #
+            #   1. ACH convention (Blastware's official Auto Call Home):
+            #        binary:  M529LK44.AB0
+            #        report:  M529LK44_AB0_ASCII.TXT
+            #      (replaces the "." with "_", appends "_ASCII.TXT")
+            #
+            #   2. Manual-export convention (operator clicks Save As Text
+            #      in BW's UI):
+            #        binary:  M529LK44.AB0
+            #        report:  M529LK44.AB0.TXT
+            #      (literal binary filename + ".TXT" suffix)
+            #
+            # We register BOTH possible binary names as keys so the
+            # subsequent lookup matches whichever convention was used.
+            stripped = name[:-4]            # remove ".TXT"
+            # ACH convention: strip "_ASCII" and convert the last "_"
+            # back to "." to recover the binary's filename.
+            if stripped.lower().endswith("_ascii"):
+                inner = stripped[:-6]       # remove "_ASCII"
+                under = inner.rfind("_")
+                if under >= 0:
+                    ach_binary = inner[:under] + "." + inner[under + 1 :]
+                    reports[ach_binary.lower()] = content
+            # Legacy convention: the stripped name IS the binary's name.
+            reports[stripped.lower()] = content
+        else:
+            binaries.append((name, content))
+
+    for filename, content in binaries:
+        report_bytes = reports.get(filename.lower())
         try:
             ev, rec = store.save_imported_bw(
                 content,
-                source_path=Path(upload.filename or "imported.bw"),
+                source_path=Path(filename or "imported.bw"),
                 serial_hint=serial,
+                bw_report_text=report_bytes,
+            )
+            # WaveformStore decoded the serial from the BW filename
+            # (e.g. T104… → BE18104) and surfaces it on `rec`.  Use that
+            # rather than the placeholder `_serial_from_event(ev)` stub,
+            # which always returned None and was silently bucketing every
+            # forwarded event into serial="UNKNOWN" in the DB.
+            resolved_serial = (
+                serial
+                or rec.get("serial")
+                or _serial_from_event(ev)
+                or "UNKNOWN"
             )
             inserted, skipped = db.insert_events(
                 [ev],
-                serial=(serial or _serial_from_event(ev) or "UNKNOWN"),
+                serial=resolved_serial,
                 waveform_records={
                     ev._waveform_key.hex(): rec
                     if ev._waveform_key else None
                 } if ev._waveform_key else None,
             )
             results.append({
-                "filename":         upload.filename,
+                "filename":         filename,
                 "status":           "ok",
                 "stored_filename":  rec["filename"],
                 "filesize":         rec["filesize"],
                 "sha256":           rec["sha256"],
+                "serial":           resolved_serial,
+                "report_attached":  report_bytes is not None,
                 "inserted":         inserted,
                 "skipped":          skipped,
             })
         except Exception as exc:
-            log.error("import failed for %s: %s", upload.filename, exc, exc_info=True)
+            log.error("import failed for %s: %s", filename, exc, exc_info=True)
             results.append({
-                "filename": upload.filename, "status": "error",
+                "filename": filename, "status": "error",
                 "detail":   str(exc),
             })
 
+    # Surface unmatched .txt uploads so the daemon can detect mis-pairings.
+    used_report_keys = {fn.lower() for fn, _ in binaries}
+    for stem in reports.keys() - used_report_keys:
+        results.append({
+            "filename":  stem + ".txt",
+            "status":    "warning",
+            "detail":    "BW ASCII report supplied but no matching binary in this upload",
+        })
+
     return {"count": len(results), "results": results}
 
 

sfm/waveform_store.py

@@ -34,7 +34,7 @@ import logging
 import pickle
 import shutil
 from pathlib import Path
-from typing import Optional
+from typing import Optional, Union
 
 from minimateplus import event_file_io
 from minimateplus.blastware_file import blastware_filename, write_blastware_file
@@ -258,6 +258,7 @@ class WaveformStore:
         source_path: Path,
         *,
         serial_hint: Optional[str] = None,
+        bw_report_text: Optional[Union[str, bytes]] = None,
     ) -> tuple[Event, dict]:
         """
         Ingest a Blastware event file produced by an external tool
@@ -267,10 +268,17 @@ class WaveformStore:
         Workflow:
           1. Parse the bytes via event_file_io.read_blastware_file (writes
              a temp file to do that, since the parser takes a path).
-          2. Resolve serial from BW filename (`<P><serial3>...`) or use
+          2. Optionally parse a paired BW ASCII event report (the .TXT
+             file BW writes alongside the binary).  When supplied, its
+             decoded fields land in the sidecar's `bw_report` block AND
+             overlay the device-authoritative peak values into the
+             top-level `peak_values` block.  This is the right path for
+             the ACH-forwarder daemon use case where Blastware's own
+             ACH writes both files into the watch folder.
+          3. Resolve serial from BW filename (`<P><serial3>...`) or use
              serial_hint.  Falls back to "UNKNOWN".
-          3. Copy the BW bytes verbatim into <root>/<serial>/<filename>.
+          4. Copy the BW bytes verbatim into <root>/<serial>/<filename>.
-          4. Write the .sfm.json sidecar with source.kind = "bw-import"
+          5. Write the .sfm.json sidecar with source.kind = "bw-import"
              and a5_pickle_filename = None.  Does NOT write a .a5.pkl
              (no A5 source available; byte-for-byte regeneration not
              possible — the on-disk BW file IS the byte-for-byte source).
@@ -292,6 +300,37 @@ class WaveformStore:
             except FileNotFoundError:
                 pass
 
+        # Parse the BW ASCII report if one was supplied.  Failures here
+        # are non-fatal: we still write the binary + sidecar without the
+        # rich derived fields.
+        bw_report = None
+        if bw_report_text is not None:
+            try:
+                from minimateplus.bw_ascii_report import parse_report
+                bw_report = parse_report(bw_report_text)
+            except Exception as exc:
+                log.warning(
+                    "save_imported_bw: BW report parse failed: %s — continuing without it",
+                    exc,
+                )
+
+        # If we have a report, overlay its device-authoritative fields
+        # (peaks, project, sample_rate, record_time) onto the Event
+        # BEFORE handing it to db.insert_events().  Without this overlay
+        # the DB row gets `peak_values` from _peaks_from_samples(), which
+        # runs the still-undecoded waveform codec on the BW body and
+        # produces ±10 in/s saturation values on every channel for every
+        # event.  The sidecar JSON had the correct values via
+        # event_to_sidecar_dict(bw_report=...) but the DB columns didn't.
+        if bw_report is not None:
+            try:
+                event_file_io.apply_report_to_event(ev, bw_report)
+            except Exception as exc:
+                log.warning(
+                    "save_imported_bw: failed to overlay report onto event: %s",
+                    exc,
+                )
+
         # Resolve serial.  blastware_filename derives a 4-char prefix from
         # the numeric serial (e.g. BE11529 → M529); we go the other way
         # via the source filename if a hint wasn't given.
@@ -345,6 +384,7 @@ class WaveformStore:
             source_kind="bw-import",
             a5_pickle_filename=None,
             review=existing_review,
+            bw_report=bw_report,
         )
         event_file_io.write_sidecar(sidecar_path, sidecar)
 
@@ -360,6 +400,7 @@ class WaveformStore:
             "a5_pickle_filename": None,
             "hdf5_filename":      hdf5_filename,
             "sidecar_filename":   sidecar_path.name,
+            "serial":             serial,
         }
 
     def load_a5(self, serial: str, filename: str) -> Optional[list[S3Frame]]:

tests/test_bw_ascii_report.py

@@ -0,0 +1,407 @@
+"""
+test_bw_ascii_report.py — parser for Blastware's per-event ASCII export.
+
+Run:
+    python -m pytest tests/test_bw_ascii_report.py -q
+"""
+
+from __future__ import annotations
+
+import datetime
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from minimateplus.bw_ascii_report import (
+    BwAsciiReport,
+    parse_report,
+    parse_report_file,
+)
+
+
+FIXTURES = Path(__file__).parent.parent / "decode-re" / "5-8-26"
+
+
+def _fixture(event_name: str) -> Path:
+    """Find the .TXT file inside a fixture event folder."""
+    for p in (FIXTURES / event_name).iterdir():
+        if p.suffix.lower() == ".txt":
+            return p
+    raise FileNotFoundError(f"no .TXT in {FIXTURES / event_name}")
+
+
+# ── Identity / config ───────────────────────────────────────────────────────
+
+
+def test_event_c_identity_and_config():
+    r = parse_report_file(_fixture("event-c"))
+
+    assert r.event_type == "Full Waveform"
+    assert r.serial == "BE11529"
+    assert r.file_name == "M529LK44.AB0"
+    assert r.event_datetime == datetime.datetime(2026, 4, 23, 15, 56, 35)
+
+    assert r.trigger_channel == "Vert"
+    assert r.geo_trigger_level_ips == pytest.approx(0.5)
+    assert r.pretrig_s == pytest.approx(-0.25)
+    assert r.record_time_s == pytest.approx(1.0)
+    assert r.record_stop_mode == "Fixed"
+    assert r.sample_rate_sps == 1024
+    assert r.battery_volts == pytest.approx(6.8)
+    assert r.calibration_date == datetime.date(2025, 4, 29)
+    assert r.calibration_by == "Instantel"
+    assert r.units == "in/s and dB(L)"
+
+
+def test_event_c_operator_metadata():
+    r = parse_report_file(_fixture("event-c"))
+
+    # The "Project: : value" pattern (key has its own trailing colon)
+    # is handled by stripping the colon at lookup time.
+    assert r.project == "Test4-21-26"
+    assert r.client == "Test-Client1"
+    assert r.operator == "Brian and claude"
+    assert r.sensor_location == "catbed"
+
+
+def test_event_c_geo_range():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.geo_range_ips == pytest.approx(10.0)
+
+
+# ── Per-channel derived stats ───────────────────────────────────────────────
+
+
+def test_event_c_per_channel_stats():
+    r = parse_report_file(_fixture("event-c"))
+
+    tran = r.channels["Tran"]
+    assert tran.ppv_ips         == pytest.approx(0.065)
+    assert tran.zc_freq_hz      == pytest.approx(47.0)
+    assert tran.time_of_peak_s  == pytest.approx(0.007)
+    assert tran.peak_accel_g    == pytest.approx(0.066)
+    assert tran.peak_disp_in    == pytest.approx(0.001)
+
+    vert = r.channels["Vert"]
+    assert vert.ppv_ips         == pytest.approx(0.610)
+    assert vert.zc_freq_hz      == pytest.approx(16.0)
+    assert vert.time_of_peak_s  == pytest.approx(0.024)
+    assert vert.peak_accel_g    == pytest.approx(0.437)
+    assert vert.peak_disp_in    == pytest.approx(0.006)
+
+    long_ = r.channels["Long"]
+    assert long_.ppv_ips        == pytest.approx(0.070)
+    assert long_.zc_freq_hz     == pytest.approx(22.0)
+    assert long_.time_of_peak_s == pytest.approx(0.019)
+    assert long_.peak_accel_g   == pytest.approx(0.040)
+    assert long_.peak_disp_in   == pytest.approx(0.001)
+
+
+def test_event_c_micl_stats():
+    r = parse_report_file(_fixture("event-c"))
+
+    # MicL specific block
+    assert r.mic.weighting       == "Linear Weighting"
+    assert r.mic.pspl_dbl        == pytest.approx(88.0)
+    assert r.mic.zc_freq_hz      == pytest.approx(57.0)
+    assert r.mic.time_of_peak_s  == pytest.approx(-0.004)
+
+    # Mirrored onto channels["MicL"] for uniform per-channel access
+    micl_ch = r.channels["MicL"]
+    assert micl_ch.zc_freq_hz     == pytest.approx(57.0)
+    assert micl_ch.time_of_peak_s == pytest.approx(-0.004)
+
+
+def test_event_c_vector_sum():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.peak_vector_sum_ips    == pytest.approx(0.612)
+    assert r.peak_vector_sum_time_s == pytest.approx(0.024)
+
+
+# ── Sensor self-check ───────────────────────────────────────────────────────
+
+
+def test_event_c_sensor_check_geo_channels():
+    r = parse_report_file(_fixture("event-c"))
+
+    for ch_name, expected_freq, expected_ratio in [
+        ("Tran", 7.4, 3.7),
+        ("Vert", 7.6, 3.5),
+        ("Long", 7.5, 3.8),
+    ]:
+        sc = r.sensor_check[ch_name]
+        assert sc.test_freq_hz   == pytest.approx(expected_freq), ch_name
+        assert sc.test_ratio     == pytest.approx(expected_ratio), ch_name
+        assert sc.test_results   == "Passed", ch_name
+        # Geo channels don't have an Test Amplitude
+        assert sc.test_amplitude_mv is None
+
+
+def test_event_c_sensor_check_micl():
+    r = parse_report_file(_fixture("event-c"))
+
+    sc = r.sensor_check["MicL"]
+    assert sc.test_freq_hz       == pytest.approx(20.1)
+    assert sc.test_amplitude_mv  == pytest.approx(533.0)
+    assert sc.test_results       == "Passed"
+    # MicL doesn't have a ratio — it has amplitude instead
+    assert sc.test_ratio is None
+
+
+# ── Monitor log + tooling ───────────────────────────────────────────────────
+
+
+def test_event_c_monitor_log_and_pc_version():
+    r = parse_report_file(_fixture("event-c"))
+
+    assert len(r.monitor_log) == 1
+    e = r.monitor_log[0]
+    assert e.start_time == datetime.datetime(2026, 4, 23, 15, 46, 16)
+    assert e.stop_time  == datetime.datetime(2026, 4, 23, 15, 56, 36)
+    assert e.description == "Event recorded."
+
+    assert r.pc_sw_version == "V 10.74"
+
+
+# ── Sample table ─────────────────────────────────────────────────────────────
+
+
+def test_event_c_sample_table_parsed_when_requested():
+    r = parse_report_file(_fixture("event-c"), parse_samples=True)
+
+    # 1 sec event @ 1024 sps + 0.25 sec pretrig = 1280 samples
+    assert r.samples is not None
+    assert len(r.samples) == 1280, f"expected 1280 samples, got {len(r.samples)}"
+
+    # First row: "0.000     \t0.005     \t0.005     \t-81.94"
+    t, v, l, m = r.samples[0]
+    assert t == pytest.approx(0.000)
+    assert v == pytest.approx(0.005)
+    assert l == pytest.approx(0.005)
+    assert m == pytest.approx(-81.94)
+
+
+def test_event_c_sample_table_skipped_by_default():
+    r = parse_report_file(_fixture("event-c"))
+    assert r.samples is None
+
+
+# ── Cross-event smoke ───────────────────────────────────────────────────────
+
+
+@pytest.mark.parametrize("event_name", ["event-a", "event-b", "event-c", "event-d"])
+def test_all_fixtures_parse_without_error(event_name):
+    """Every fixture in the bundle must parse cleanly with the same parser."""
+    r = parse_report_file(_fixture(event_name))
+
+    # Common invariants: serial, event_datetime, sample rate, all four
+    # channels surfaced.
+    assert r.serial == "BE11529"
+    assert r.event_datetime is not None
+    assert r.sample_rate_sps in (1024, 2048, 4096)
+    for ch in ("Tran", "Vert", "Long", "MicL"):
+        assert ch in r.channels
+        assert ch in r.sensor_check
+
+    # PVS should be present and positive on triggered events
+    if r.peak_vector_sum_ips is not None:
+        assert r.peak_vector_sum_ips >= 0
+
+
+# ── Edge cases / defensive parsing ──────────────────────────────────────────
+
+
+def test_parse_empty_input():
+    r = parse_report("")
+    assert r.serial is None
+    assert r.event_datetime is None
+    assert all(cs.ppv_ips is None for cs in r.channels.values())
+
+
+def test_parse_unknown_keys_ignored():
+    """Forward-compat: future BW versions may add fields we don't recognise.
+    Those should be silently dropped, not raise."""
+    text = (
+        '"Serial Number : BE99999"\n'
+        '"Future Field That Does Not Exist : 42 widgets"\n'
+        '"Tran PPV : 0.123 in/s"\n'
+    )
+    r = parse_report(text)
+    assert r.serial == "BE99999"
+    assert r.channels["Tran"].ppv_ips == pytest.approx(0.123)
+
+
+def test_parse_numeric_with_units_strips_unit():
+    text = (
+        '"Vert PPV : 1.275 in/s"\n'
+        '"Vert ZC Freq : 23 Hz"\n'
+        '"MicL  Test Amplitude : 569 mv"\n'
+    )
+    r = parse_report(text)
+    assert r.channels["Vert"].ppv_ips    == pytest.approx(1.275)
+    assert r.channels["Vert"].zc_freq_hz == pytest.approx(23.0)
+    assert r.sensor_check["MicL"].test_amplitude_mv == pytest.approx(569.0)
+
+
+def test_parse_handles_micl_double_space_in_key():
+    """BW writes "MicL  Time of Peak" with TWO spaces; the parser must
+    normalise whitespace before key lookup."""
+    text = (
+        '"MicL  Time of Peak : 0.012 sec"\n'
+        '"MicL  ZC Freq : 51 Hz"\n'
+    )
+    r = parse_report(text)
+    assert r.mic.time_of_peak_s == pytest.approx(0.012)
+    assert r.mic.zc_freq_hz     == pytest.approx(51.0)
+
+
+# ── Position-based user-notes parsing ───────────────────────────────────────
+#
+# The 4 user-supplied note slots (Project / Client / User Name / Seis Loc
+# by default) have OPERATOR-EDITABLE labels in BW's Compliance Setup →
+# Notes tab.  An operator could rename them to "Building:", "Site:",
+# "Address:", etc. and the ASCII export would write those labels
+# verbatim.  We parse by POSITION between the `Units :` and `Geo Range :`
+# anchors, NOT by matching the label text.
+
+
+def _wrap_user_notes(*lines: str) -> str:
+    """Helper: wrap N user-note lines in the minimal context the parser
+    needs (`Units :` opens the block, `Geo Range :` closes it)."""
+    body = ['"Units : in/s and dB(L)"']
+    body.extend('"' + l + '"' for l in lines)
+    body.append('"Geo Range : 10.000 in/s"')
+    return "\n".join(body) + "\n"
+
+
+def test_user_notes_default_labels_populate_by_position():
+    """The BW-default labels (Project / Client / User Name / Seis Loc)
+    populate the four canonical slots in order."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Test4-21-26",
+        "Client: : Acme Inc",
+        "User Name: : Brian",
+        "Seis Loc: : Catbed",
+    ))
+    assert r.project          == "Test4-21-26"
+    assert r.client           == "Acme Inc"
+    assert r.operator         == "Brian"
+    assert r.sensor_location  == "Catbed"
+    assert r.user_note_labels == {
+        "project":         "Project:",
+        "client":          "Client:",
+        "operator":        "User Name:",
+        "sensor_location": "Seis Loc:",
+    }
+
+
+def test_user_notes_operator_renamed_labels_still_populate():
+    """If the operator renames the labels in BW's UI (e.g. "Seis Loc:"
+    → "Building:"), the values STILL populate the canonical slots by
+    position — and the operator's labels are preserved in
+    `user_note_labels` for terra-view to display."""
+    r = parse_report(_wrap_user_notes(
+        "Building : Main Office",
+        "Project Manager : Brian",
+        "Inspector : Claude",
+        "Site Address : 123 Main St",
+    ))
+    assert r.project         == "Main Office"
+    assert r.client          == "Brian"
+    assert r.operator        == "Claude"
+    assert r.sensor_location == "123 Main St"
+    assert r.user_note_labels == {
+        "project":         "Building",
+        "client":          "Project Manager",
+        "operator":        "Inspector",
+        "sensor_location": "Site Address",
+    }
+
+
+def test_user_notes_with_histogram_label_spelling():
+    """Histogram exports use 'Seis. Location:' (with period and colon)
+    instead of 'Seis Loc:'.  Position-based parsing handles both."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Plum Cont.- Rainbow Run",
+        "Client: : Plum Contracting In.c",
+        "User Name: : Terra-Mechanics Inc.",
+        "Seis. Location: : Loc #1 - 2652 Hepner",
+    ))
+    assert r.project == "Plum Cont.- Rainbow Run"
+    assert r.client == "Plum Contracting In.c"
+    assert r.operator == "Terra-Mechanics Inc."
+    assert r.sensor_location == "Loc #1 - 2652 Hepner"
+    # And the histogram's specific label spelling is preserved
+    assert r.user_note_labels["sensor_location"] == "Seis. Location:"
+
+
+def test_user_notes_outside_block_are_ignored():
+    """Lines that look like user-notes but appear OUTSIDE the
+    Units→Geo Range range don't get assigned to user-note slots."""
+    # No Units anchor — these lines shouldn't populate user-note slots
+    text = (
+        '"Serial Number : BE11529"\n'
+        '"Project: : SHOULD NOT POPULATE"\n'
+    )
+    r = parse_report(text)
+    assert r.serial == "BE11529"
+    assert r.project is None
+
+
+def test_user_notes_partial_block_only_fills_present_slots():
+    """If BW writes fewer than 4 user-notes (e.g. operator disabled
+    Extended Notes mid-block), only the present positions populate;
+    later slots stay None."""
+    r = parse_report(_wrap_user_notes(
+        "Project: : Just-a-project",
+        "Client: : Just-a-client",
+    ))
+    assert r.project == "Just-a-project"
+    assert r.client  == "Just-a-client"
+    assert r.operator is None
+    assert r.sensor_location is None
+
+
+def test_user_notes_extra_lines_beyond_four_are_dropped():
+    """If somehow more than 4 lines appear in the user-notes block
+    (e.g. BW adds an Extended Notes line), only the first 4 are
+    captured — slots 5+ have nowhere to go."""
+    r = parse_report(_wrap_user_notes(
+        "L1 : v1",
+        "L2 : v2",
+        "L3 : v3",
+        "L4 : v4",
+        "L5 : v5",   # ignored — no fifth slot
+    ))
+    assert r.project         == "v1"
+    assert r.client          == "v2"
+    assert r.operator        == "v3"
+    assert r.sensor_location == "v4"
+    # 5th label not captured
+    assert "L5" not in r.user_note_labels.values()
+
+
+def test_real_histogram_fixture_populates_sensor_location():
+    """End-to-end: the histogram fixture uses 'Seis. Location:' — must
+    successfully populate sensor_location via position-based parsing."""
+    fixture_dir = (
+        Path(__file__).parent.parent / "example-events" / "histogram"
+    )
+    if not fixture_dir.exists():
+        pytest.skip("histogram fixtures not present")
+    txt = next(fixture_dir.glob("*_ASCII.TXT"), None)
+    if txt is None:
+        pytest.skip("no histogram TXT in fixture dir")
+
+    r = parse_report_file(txt)
+    assert r.sensor_location is not None
+    assert len(r.sensor_location) > 0
+    assert r.user_note_labels.get("sensor_location") is not None
+    # Sanity: other shared fields still parse correctly
+    assert r.serial is not None
+    assert r.serial.startswith("BE")
+    assert r.geo_range_ips is not None

tests/test_event_file_io.py

@@ -294,6 +294,114 @@ def test_read_blastware_file_round_trip(tmp_path: Path):
     assert parsed.peak_values.peak_vector_sum == 0.0
 
 
+def test_save_imported_bw_with_paired_report(tmp_path: Path):
+    """save_imported_bw + a paired BW ASCII report fold the report's
+    rich derived fields into the sidecar.  This is the daemon-forwarded
+    ACH workflow: BW writes <event>.AB0 and <event>.AB0.TXT side by side;
+    the daemon ships both; we overlay the report-decoded values onto the
+    sidecar (peaks, project, plus the rich `bw_report` block)."""
+    from minimateplus.blastware_file import write_blastware_file, blastware_filename
+    from sfm.waveform_store import WaveformStore
+
+    ev, frames = _make_synthetic_event()
+    fname = blastware_filename(ev, "BE11529")
+    src = tmp_path / fname
+    write_blastware_file(ev, frames, src)
+
+    # Use one of the real BW ASCII exports as the paired report.
+    report_path = (
+        Path(__file__).parent.parent
+        / "decode-re" / "5-8-26" / "event-c" / "M529LK44.AB0.TXT"
+    )
+    if not report_path.exists():
+        import pytest as _pt
+        _pt.skip("decode-re fixtures not present")
+    report_bytes = report_path.read_bytes()
+
+    store = WaveformStore(tmp_path / "waveforms")
+    parsed_ev, rec = store.save_imported_bw(
+        src.read_bytes(),
+        source_path=src,
+        bw_report_text=report_bytes,
+    )
+
+    sc = store.load_sidecar("BE11529", fname)
+    assert sc is not None
+
+    # ── bw_report block populated with the rich fields ──────────────────
+    assert "bw_report" in sc
+    br = sc["bw_report"]
+    assert br["available"] is True
+    assert br["event_type"] == "Full Waveform"
+    assert br["recording"]["sample_rate_sps"] == 1024
+    assert br["recording"]["geo_range_ips"]   == 10.0
+
+    # Per-channel derived stats
+    assert br["peaks"]["tran"]["ppv_ips"]        == 0.065
+    assert br["peaks"]["vert"]["ppv_ips"]        == 0.610
+    assert br["peaks"]["long"]["ppv_ips"]        == 0.070
+    assert br["peaks"]["vert"]["peak_accel_g"]   == 0.437
+    assert br["peaks"]["vert"]["peak_disp_in"]   == 0.006
+    assert br["peaks"]["tran"]["zc_freq_hz"]     == 47.0
+    assert br["peaks"]["vector_sum"]["ips"]      == 0.612
+    assert br["peaks"]["vector_sum"]["time_s"]   == 0.024
+
+    # Sensor self-check per channel
+    assert br["sensor_check"]["tran"]["freq_hz"]  == 7.4
+    assert br["sensor_check"]["tran"]["ratio"]    == 3.7
+    assert br["sensor_check"]["tran"]["result"]   == "Passed"
+    assert br["sensor_check"]["mic"]["amplitude_mv"] == 533.0
+
+    # Mic block
+    assert br["mic"]["weighting"]    == "Linear Weighting"
+    assert br["mic"]["pspl_dbl"]     == 88.0
+
+    # Monitor log roundtripped
+    assert len(br["monitor_log"]) == 1
+    assert "2026-04-23T15:46:16" in br["monitor_log"][0]["start"]
+    assert br["pc_sw_version"] == "V 10.74"
+
+    # ── Overlay onto canonical peak_values ──────────────────────────────
+    # Report values win over the broken-codec samples-derived peaks.
+    assert sc["peak_values"]["transverse"]   == 0.065
+    assert sc["peak_values"]["vertical"]     == 0.610
+    assert sc["peak_values"]["longitudinal"] == 0.070
+    assert sc["peak_values"]["vector_sum"]   == 0.612
+    # Mic PSPL converted to psi (dbl=88 → 10^(88/20) * 2.9e-9)
+    assert sc["peak_values"]["mic_psi"] is not None
+    assert 1e-5 < sc["peak_values"]["mic_psi"] < 1e-3
+
+    # ── Overlay onto project_info ───────────────────────────────────────
+    assert sc["project_info"]["project"]         == "Test4-21-26"
+    assert sc["project_info"]["client"]          == "Test-Client1"
+    assert sc["project_info"]["operator"]        == "Brian and claude"
+    assert sc["project_info"]["sensor_location"] == "catbed"
+
+    # ── Event timestamp overlaid from report ───────────────────────────
+    assert sc["event"]["timestamp"] == "2026-04-23T15:56:35"
+
+
+def test_save_imported_bw_without_report_works_unchanged(tmp_path: Path):
+    """Calling save_imported_bw with no bw_report_text behaves exactly
+    as before — no `bw_report` block, peak_values come from samples."""
+    from minimateplus.blastware_file import write_blastware_file, blastware_filename
+    from sfm.waveform_store import WaveformStore
+
+    ev, frames = _make_synthetic_event()
+    fname = blastware_filename(ev, "BE11529")
+    src = tmp_path / fname
+    write_blastware_file(ev, frames, src)
+
+    store = WaveformStore(tmp_path / "waveforms")
+    store.save_imported_bw(src.read_bytes(), source_path=src)
+
+    sc = store.load_sidecar("BE11529", fname)
+    assert sc is not None
+    assert "bw_report" not in sc          # block is absent without a report
+    # Synthetic event has zero samples → peaks all zero (was true before this change)
+    assert sc["peak_values"]["transverse"] == 0.0
+
+
 def test_save_imported_bw_round_trip(tmp_path: Path):
     """save_imported_bw stores a copy + sidecar with source.kind = bw-import."""
     from minimateplus.blastware_file import write_blastware_file, blastware_filename
@@ -310,6 +418,10 @@ def test_save_imported_bw_round_trip(tmp_path: Path):
 
     assert rec["filename"] == fname
     assert rec["a5_pickle_filename"] is None  # no A5 source for BW imports
+    # The serial decoded from the BW filename surfaces on the record so
+    # the import endpoint can use it when calling SeismoDb.insert_events()
+    # (otherwise forwarded events would all bucket into serial="UNKNOWN").
+    assert rec["serial"] == "BE11529"
     sc = store.load_sidecar("BE11529", fname)
     assert sc is not None
     assert sc["source"]["kind"] == "bw-import"