bw_ascii_report: parse OORANGE saturation marker + TimeSum typo

BW writes "OORANGE" (truncation of "Out Of Range") when a channel
exceeds its full-scale, and uses a typo'd label "Peak Vector Sum
TimeSum" for the PVS time field.  Both confirmed against real ASCII
files pulled from a Windows watcher PC 2026-05-27:

  T190LD5Q.LK0W  Vert PPV = OORANGE  (Normal range, 10 in/s exceeded)
  T438L713.RY0W  All three PPVs OORANGE  (Sensitive range, 1.25 in/s)
  K557L3YM.OE0W  Tran+Vert PPV OORANGE + MicL PSPL OORANGE

Previously our _parse_number() returned None for OORANGE → DB columns
ended up NULL → events vanished from filters / sorts / dashboards
despite being legitimate high-amplitude events.

New behavior — substitute a conservative bound + set a saturation flag:
  - Channel PPV       → geo_range_ips + ChannelStats.ppv_saturated
  - Peak Vector Sum   → sqrt(3) * geo_range_ips + peak_vector_sum_saturated
  - MicL PSPL         → 140 dB(L) + MicStats.pspl_saturated

Flags propagate to the sidecar's bw_report block so the SFM UI can
render "> 10 in/s" / "> 140 dBL" rather than treating the substituted
value as exact.

Same commit also accepts "Peak Vector Sum TimeSum" as an alias for
"Peak Vector Sum Time" (BW always writes the typo on OORANGE PVS
lines — every example file confirms it).

Tests: new test_oorange_marker_treated_as_saturation (synthetic) +
test_real_oorange_event_t190_parses (skips if real fixture absent).
177/177 tests pass; 16 pre-existing missing-fixture skips unchanged.

Five events on prod (T190, T438, K557, plus 2 others matching the
same fault pattern) will pick up correct peaks + saturation flags
once watchers re-forward.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-27 20:32:56 +00:00
parent d21e3b5298
commit 3457ed0072
4 changed files with 151 additions and 13 deletions
+10
View File
@@ -6,6 +6,16 @@ All notable changes to seismo-relay are documented here.
## [Unreleased]
### Fixed
- **bw_ascii_report parser now handles `OORANGE` saturation marker.** BW writes `"OORANGE"` (truncation of "Out Of Range") in PPV / PVS / MicL PSPL fields when the underlying measurement exceeded the channel's full-scale. Previously our `_parse_number()` returned None → DB ended up with NULL peaks for legitimate high-amplitude events. Confirmed on real ASCII files pulled 2026-05-27 from the Windows watcher PC: T190LD5Q.LK0W (Vert saturated at Normal range 10 in/s), T438L713.RY0W (all three channels saturated at Sensitive range 1.25 in/s), K557L3YM.OE0W (Tran+Vert saturated + Mic PSPL OORANGE). New behavior:
- Per-channel PPV: substitute `geo_range_ips` as a conservative lower bound + set `ppv_saturated` flag
- Peak Vector Sum: substitute `sqrt(3) * geo_range_ips` (the theoretical max when all 3 channels are simultaneously at full-scale) + `peak_vector_sum_saturated` flag
- MicL PSPL: substitute 140 dB(L) (conservative NL-43 max) + `pspl_saturated` flag
- Saturation flags are propagated into the sidecar's `bw_report` block for downstream UI rendering (`> 10 in/s` or similar)
- Five events on prod (T190 / T438 / K557 + 2 others matching the same fault pattern) will pick up correct DB peaks + saturation flags once re-forwarded
- **bw_ascii_report parser handles `Peak Vector Sum TimeSum` typo'd label.** Real BW output uses this misspelled label (Sum appended twice instead of "Peak Vector Sum Time"). Now accepted as an alias. Confirmed against all three OORANGE example files — every one has the typo.
### Added
- **Histogram per-interval aggregation in `waveform.json`.** Histogram events now render with one bar per BW-reported interval (matching the Blastware printout) instead of ~200 bars per event (the raw codec output). When the sidecar's `bw_report.histogram.n_intervals` is populated (events ingested with the new parser, see next bullet), the `/db/events/{id}/waveform.json` endpoint groups the codec samples into N intervals via max-per-group and returns the aggregated array. `time_axis` gains `histogram_aggregated: true`, `n_intervals`, `interval_size_s`, and `interval_times` (HH:MM:SS strings). Both the modal chart and the standalone event browser use those interval timestamps as x-axis labels when present. Defensive: no-op for events ingested before the parser extension landed (their sidecars lack `histogram.n_intervals`) — those continue to render with raw codec output.
+63 -3
View File
@@ -60,6 +60,13 @@ class ChannelStats:
time_of_peak_s: Optional[float] = None # seconds (relative to trigger; can be negative)
peak_accel_g: Optional[float] = None # g (geo channels only)
peak_disp_in: Optional[float] = None # in (geo channels only)
# When BW writes "OORANGE" (Out Of Range — truncated) for a PPV
# value, the true peak exceeded the channel's full-scale range.
# We substitute the range max (e.g. 10.000 in/s for Normal range)
# as a lower bound, and flag here so downstream UI / alerts know
# to render "> 10 in/s" or "saturated" instead of trusting the
# value as an exact measurement.
ppv_saturated: bool = False
@dataclass
@@ -69,6 +76,11 @@ class MicStats:
pspl_dbl: Optional[float] = None # dB(L)
zc_freq_hz: Optional[float] = None
time_of_peak_s: Optional[float] = None
# Set when BW writes "OORANGE" for PSPL — mic exceeded its
# measurement range. pspl_dbl gets the conservative upper bound
# 140 dBL (typical NL-43 max; some units cap at 148). Consumers
# should render "> 140 dB(L)" or similar when this flag is set.
pspl_saturated: bool = False
@dataclass
@@ -92,6 +104,21 @@ class MonitorLogEntry:
description: Optional[str] = None
# BW saturation marker — appears in PPV / Peak Vector Sum / similar
# numeric fields when the underlying measurement exceeded the
# channel's full-scale range (e.g., a geophone reading > 10 in/s at
# Normal range, or a mic exceeding its sensitivity ceiling). Treated
# as "≥ range_max" + a saturated flag rather than discarded.
# Appears as: ``"Tran PPV : OORANGE in/s"``
_OORANGE_MARKERS = ("OORANGE", "OUT OF RANGE")
def _is_oorange(value: str) -> bool:
"""True when a BW numeric field is an Out-Of-Range saturation marker."""
s = value.strip().upper()
return any(m in s for m in _OORANGE_MARKERS)
@dataclass
class BwAsciiReport:
"""Structured representation of one BW per-event ASCII export."""
@@ -144,6 +171,12 @@ class BwAsciiReport:
# ── Vector sum ──────────────────────────────────────────────────────────
peak_vector_sum_ips: Optional[float] = None
peak_vector_sum_time_s: Optional[float] = None
# Saturation flag — set when BW writes "OORANGE" for the PVS. We
# then substitute sqrt(3) * geo_range_ips as a conservative upper
# bound (the theoretical maximum PVS when all 3 geo channels are
# simultaneously at full-scale). Consumers should display this as
# ">{value} in/s" or similar.
peak_vector_sum_saturated: bool = False
# Histograms additionally have an absolute date+time for the PVS
# (it occurred at a specific interval). Waveform reports show
# only the relative-time value above.
@@ -486,9 +519,17 @@ def parse_report(text: Union[str, bytes], *, parse_samples: bool = False) -> BwA
):
ch_name, stat = key.split(" ", 1)
cs = report.channels.setdefault(ch_name, ChannelStats())
if stat == "PPV":
if _is_oorange(value):
# Channel saturated — substitute range max as lower
# bound; flag so downstream UI can render "> 10 in/s".
cs.ppv_ips = report.geo_range_ips
cs.ppv_saturated = True
else:
cs.ppv_ips = _parse_number(value)
else:
num = _parse_number(value)
if stat == "PPV": cs.ppv_ips = num
elif stat == "ZC Freq": cs.zc_freq_hz = num
if stat == "ZC Freq": cs.zc_freq_hz = num
elif stat == "Time of Peak": cs.time_of_peak_s = num
elif stat == "Peak Acceleration": cs.peak_accel_g = num
elif stat == "Peak Displacement": cs.peak_disp_in = num
@@ -532,8 +573,22 @@ def parse_report(text: Union[str, bytes], *, parse_samples: bool = False) -> BwA
# ── Vector Sum ───────────────────────────────────────────────────────
elif key == "Peak Vector Sum":
if _is_oorange(value):
# PVS saturated — conservative upper bound is
# sqrt(3) * geo_range_ips (all 3 channels at full-scale).
# Real PVS could be lower (channels rarely peak
# simultaneously) but never higher within the range.
if report.geo_range_ips is not None:
import math as _math
report.peak_vector_sum_ips = _math.sqrt(3) * report.geo_range_ips
report.peak_vector_sum_saturated = True
else:
report.peak_vector_sum_ips = _parse_number(value)
elif key == "Peak Vector Sum Time":
# BW writes the PVS-time label with a typo: "Peak Vector Sum TimeSum"
# (looks like Sum got appended twice). Accept both forms. Confirmed
# against actual BW output on 2026-05-27 — every PVS-time line in
# the field examples (T190, T438, K557) uses the typo'd label.
elif key in ("Peak Vector Sum Time", "Peak Vector Sum TimeSum"):
report.peak_vector_sum_time_s = _parse_number(value)
_pvs_time_raw = value
elif key == "Peak Vector Sum Date":
@@ -558,6 +613,11 @@ def parse_report(text: Union[str, bytes], *, parse_samples: bool = False) -> BwA
elif key == "Microphone":
report.mic.weighting = value
elif key == "MicL PSPL":
if _is_oorange(value):
# Mic saturated — substitute conservative upper bound 140 dBL.
report.mic.pspl_dbl = 140.0
report.mic.pspl_saturated = True
else:
report.mic.pspl_dbl = _parse_number(value)
# Mirror onto the "MicL" entry in channels so callers querying
# `channels["MicL"].ppv_ips` see something — but it's dB(L), not
+11 -1
View File
@@ -120,7 +120,12 @@ def _bw_report_to_dict(report: BwAsciiReport) -> dict:
"peak_disp_in": cs.peak_disp_in,
}
# Drop all-None entries — keeps the JSON tidy for partial reports.
return {k: v for k, v in out.items() if v is not None}
out = {k: v for k, v in out.items() if v is not None}
# Saturation flag (only present when True) — signals that ppv_ips
# is the channel range max (a lower bound), not an exact reading.
if getattr(cs, "ppv_saturated", False):
out["ppv_saturated"] = True
return out
def _sc(ch_name: str) -> dict:
sc = report.sensor_check.get(ch_name)
@@ -175,11 +180,16 @@ def _bw_report_to_dict(report: BwAsciiReport) -> dict:
# (the interval at which it occurred); waveform events
# only have the time_s offset.
"when": report.peak_vector_sum_when.isoformat() if report.peak_vector_sum_when else None,
# Set when BW reported the PVS as OORANGE — value is the
# conservative upper bound sqrt(3) * geo_range_ips, not
# an exact peak.
"saturated": bool(getattr(report, "peak_vector_sum_saturated", False)),
},
},
"mic": {
"weighting": report.mic.weighting,
"pspl_dbl": report.mic.pspl_dbl,
"pspl_saturated": bool(getattr(report.mic, "pspl_saturated", False)),
"zc_freq_hz": report.mic.zc_freq_hz,
"time_of_peak_s": report.mic.time_of_peak_s,
},
+58
View File
@@ -385,6 +385,64 @@ def test_user_notes_extra_lines_beyond_four_are_dropped():
assert "L5" not in r.user_note_labels.values()
def test_oorange_marker_treated_as_saturation():
"""BW writes 'OORANGE' (Out Of Range — truncated) when a channel
exceeds its full-scale. Verify ppv_ips falls back to geo_range_ips
+ saturated flag is set, mirroring the real T190LD5Q.LK0W,
T438L713.RY0W, and K557L3YM.OE0W events from prod 2026-05-27.
"""
txt = """\
"Event Type : Full Waveform"
"Serial Number : BE18190"
"Geo Range : 10.000 in/s"
"Tran PPV : 2.140 in/s"
"Vert PPV : OORANGE in/s"
"Long PPV : 2.830 in/s"
"Peak Vector Sum : OORANGE in/s"
"Peak Vector Sum TimeSum : 0.007 s"
"MicL PSPL : OORANGE "
"""
r = parse_report(txt)
# Tran/Long parse normally
assert r.channels["Tran"].ppv_ips == 2.14
assert r.channels["Tran"].ppv_saturated is False
assert r.channels["Long"].ppv_ips == 2.83
# Vert saturated → range max + flag
assert r.channels["Vert"].ppv_ips == 10.0
assert r.channels["Vert"].ppv_saturated is True
# PVS saturated → sqrt(3) * range_max as upper bound + flag
import math
assert r.peak_vector_sum_ips == pytest.approx(math.sqrt(3) * 10.0)
assert r.peak_vector_sum_saturated is True
# Mic saturated → 140 dBL conservative upper bound + flag
assert r.mic.pspl_dbl == 140.0
assert r.mic.pspl_saturated is True
# PVS time still parses despite the BW typo'd label "TimeSum"
assert r.peak_vector_sum_time_s == pytest.approx(0.007)
def test_real_oorange_event_t190_parses():
"""End-to-end against the real T190LD5Q.LK0W ASCII file pulled from
a Windows watcher PC on 2026-05-27. This is the canonical example
of the parser-PPV-miss bug we fixed in this iteration."""
fixture_path = (
Path(__file__).parent.parent / "example-events" /
"ascii-5-27-26" / "T190LD5Q_LK0W_ASCII.TXT"
)
if not fixture_path.exists():
pytest.skip("real ASCII fixture not present (local-only)")
r = parse_report_file(fixture_path)
assert r.serial == "BE18190"
assert r.geo_range_ips == 10.0
# Tran reads cleanly, Vert was OORANGE
assert r.channels["Tran"].ppv_ips == pytest.approx(2.14)
assert r.channels["Vert"].ppv_ips == 10.0
assert r.channels["Vert"].ppv_saturated is True
assert r.channels["Long"].ppv_ips == pytest.approx(2.83)
assert r.peak_vector_sum_saturated is True
assert r.peak_vector_sum_time_s == pytest.approx(0.007)
def test_real_histogram_fixture_populates_sensor_location():
"""End-to-end: the histogram fixture uses 'Seis. Location:' — must
successfully populate sensor_location via position-based parsing."""