report_pdf: PDF histogram aggregation + fix footer/x-axis overlap

Two issues spotted on a histogram event PDF:

1. Footer scale ("Time — /div  Amplitude Geo: X in/s/div  Mic: Y
   psi(L)/div") was overlapping horizontally with the x-axis tick
   labels (0, 20, 40, 60...).  Both rendered on the same Y row.
   Fix: bumped gridspec bottom margin from 0.06 → 0.12, moved the
   footer text from y=0.045 → y=0.030 (below the tick labels), moved
   the page-bottom Created/Event line from y=0.015 → y=0.005.
   Trigger legend on waveforms moved 0.030 → 0.018.  Everything
   stacks cleanly now without collision.

2. PDF was showing the raw codec output (~150+ bars per histogram)
   instead of BW's per-interval aggregation.  Why: the aggregation
   I'd added to /db/events/{id}/waveform.json wasn't replicated in
   the PDF gather path.  Now: gather_report_data does the same
   max-per-group aggregation when bw_report.histogram.n_intervals is
   populated, AND derives per-interval HH:MM:SS labels from the
   start time + interval_size_s.  Result: histogram PDFs now match
   BW's display (one bar per BW interval, x-axis labeled with actual
   times) — same fix as the modal chart, applied to the PDF.

For events ingested BEFORE the parser extension (no histogram block
in their sidecar), aggregation is a no-op — they still render with
per-block bars + interval-index x-axis (but the overlap fix applies
to them too).  Re-forwarding repopulates the histogram block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

README.md

@@ -567,3 +567,4 @@ Implementation steps (concrete):
 - [ ] RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred).
 - [ ] **NULL-timestamp duplicate-row dedup.**  A small handful of events (2 known on prod as of 2026-05-22) have `events.timestamp IS NULL` because the codec couldn't extract a timestamp from the binary footer.  The `UNIQUE(serial, timestamp)` constraint doesn't fire on `NULL` (SQL semantics: `NULL ≠ NULL`), so every `--force` backfill INSERTs a new row instead of UPSERTing the existing one.  Cleanup: a one-shot SQL query that keeps only the newest row per `(serial, blastware_filename)` and deletes the rest.  Longer-term: extend the unique key to `(serial, COALESCE(timestamp, blastware_filename))` or reject inserts with NULL timestamp.
 - [ ] **Histogram body sub-format with `byte[5] != 0`.**  ~3 events on prod (`T190LD5Q.LD0H`, `O121L4L1.GU0H`) use a histogram body my walker doesn't recognize — the first block has `byte[5] = 0x01` or `0x07` instead of `0x00`, and the entire body lacks the `1e 0a 00 00` tail signature.  Codec returns 0 valid blocks; their DB PVS comes from the bw_report ASCII overlay (which BW computed from the same binary, so the DB columns are correct).  Only the `.h5` waveform plot is empty.  Cracking the sub-format would unlock the plot.  Needs binary+ASCII pairs from a few `byte[5]!=0` events; same RE approach as the K558 case.
+- [ ] **Sensor-check waveform extraction from the BW binary.**  BW's Event Report PDFs include a narrow panel on the right side of the waveform plot showing each channel's response to the sensor self-check signal (a damped sinusoid for geo, sawtooth-at-test-freq for mic).  Our parser captures the test RESULTS (`test_freq_hz`, `test_ratio`, `test_amplitude_mv`, `test_results` pass/fail) and the PDF + modal display them as text — but BW's per-sample sensor-check waveform isn't accessible to us today.  Two paths to add it:  (a) RE the binary to find where the sensor-check samples are stored — could be a section before STRT, after the footer, or in a separate sub-record; protocol reference doesn't currently mention it.  (b) If samples aren't in the binary, synthesize a representative waveform from the test parameters (damped sinusoid at `test_freq_hz` with damping from `test_ratio`).  Path (a) is the honest answer; path (b) is decorative.  Until either lands, the text-only sensor-check display in the report is fine.

sfm/event_browser.html

@@ -717,8 +717,9 @@ function renderWaveform(data) {
     // up AND down).  Mic + histograms keep default auto-scale (always
     // positive values; zero at the bottom).
     let yBounds = {};
-    const isGeoWaveform = !isHistogram && ch !== 'MicL';
+    const isGeo = ch !== 'MicL';
-    if (isGeoWaveform) {
+    if (isGeo && !isHistogram) {
+      // Waveform geo: symmetric around zero for full shape detail.
       let absMax = 0;
       for (const v of values) {
         const a = Math.abs(v);
@@ -726,13 +727,25 @@ function renderWaveform(data) {
       }
       const padded = (absMax || 1) * 1.10;
       yBounds = { min: -padded, max: padded };
+    } else if (isGeo && isHistogram) {
+      // Histogram geo: enforce minimum chart range so quiet events
+      // look quiet (matches BW's near-fixed-scale convention).
+      const HIST_GEO_MIN_INS = 0.05;
+      let p = 0;
+      for (const v of values) { const a = Math.abs(v); if (a > p) p = a; }
+      yBounds = { min: 0, max: Math.max(p * 1.10, HIST_GEO_MIN_INS) };
     } else if (ch === 'MicL' && micUnit === 'dBL') {
-      // Baseline at noise-floor minimum (matches what we floored
+      // Mic dBL: baseline at noise-floor minimum, top at peak + 5 dB.
-      // null/quiet samples to), top at peak + 5 dB headroom.
       const peakDbl = (typeof peak === 'number' && isFinite(peak))
         ? peak + 5
         : 100;
       yBounds = { min: MIC_DBL_FLOOR, max: Math.max(peakDbl, MIC_DBL_FLOOR + 20) };
+    } else if (ch === 'MicL' && isHistogram && micUnit === 'psi') {
+      // Mic histogram in psi: same minimum-range treatment as geo.
+      const HIST_MIC_MIN_PSI = 0.001;
+      let p = 0;
+      for (const v of values) { const a = Math.abs(v); if (a > p) p = a; }
+      yBounds = { min: 0, max: Math.max(p * 1.10, HIST_MIC_MIN_PSI) };
     }
 
     const chart = new Chart(canvas, {

sfm/report_pdf.py

@@ -285,6 +285,43 @@ def gather_report_data(
         except Exception as exc:
             log.warning("gather_report_data: hdf5 read failed: %s", exc)
 
+    # ── Histogram aggregation ──
+    # Codec emits ~N per-block samples (typically 1/sec); BW reports
+    # one bar per configured interval (1 min / 5 min / etc.).  When
+    # bw_report.histogram.n_intervals is populated (events ingested
+    # with the parser extension), group max-per-group to match.  Also
+    # derives per-interval timestamps for the x-axis.  No-op for
+    # waveform events or when n_intervals is missing.
+    if rd.is_histogram and rd.histogram_n_intervals and rd.histogram_n_intervals >= 1:
+        n = int(rd.histogram_n_intervals)
+        for ch, vals in list(rd.channels.items()):
+            if not vals:
+                continue
+            per_group = len(vals) // n
+            remainder = len(vals) % n
+            agg: list = []
+            offset = 0
+            for i in range(n):
+                grp_size = per_group + (1 if i < remainder else 0)
+                if grp_size > 0:
+                    grp = vals[offset:offset + grp_size]
+                    agg.append(max((abs(v) for v in grp if v is not None), default=0))
+                    offset += grp_size
+                else:
+                    agg.append(0)
+            rd.channels[ch] = agg
+        # Derive per-interval HH:MM:SS labels if we have the start time + size
+        if rd.histogram_start_str and rd.histogram_interval_size_s and not rd.histogram_interval_times:
+            try:
+                import datetime as _dt
+                start = _dt.datetime.fromisoformat(rd.histogram_start_str)
+                rd.histogram_interval_times = [
+                    (start + _dt.timedelta(seconds=(i + 1) * rd.histogram_interval_size_s)).strftime("%H:%M:%S")
+                    for i in range(n)
+                ]
+            except Exception:
+                pass
+
     return rd
 
 
@@ -308,16 +345,18 @@ def render_event_report_pdf(rd: ReportData) -> bytes:
     else:
         _render_waveform_layout(fig, rd)
 
-    # Footer (common to both layouts) — Created date + Xmark-like attribution.
+    # Page footer (common to both layouts) — Created date + event id.
+    # Pushed to the very page bottom so it doesn't collide with the
+    # waveform footer scale / trigger legend lines just above.
     fig.text(
-        0.07, 0.015,
+        0.07, 0.005,
         f"Created: {rd.server_received_at or '—'}  •  seismo-relay",
-        fontsize=7, color="#888", ha="left",
+        fontsize=6, color="#888", ha="left",
     )
     fig.text(
-        0.93, 0.015,
+        0.93, 0.005,
         f"Event {rd.event_id[:8] if rd.event_id else '—'}",
-        fontsize=7, color="#888", ha="right",
+        fontsize=6, color="#888", ha="right",
     )
 
     buf = io.BytesIO()
@@ -331,10 +370,13 @@ def _render_waveform_layout(fig, rd: ReportData) -> None:
 
     Stats table includes Time (Rel. to Trig), Peak Accel, Peak Disp.
     Left margin sized to fit the channel labels (MicL/Long/Vert/Tran).
+    Extra bottom margin reserves space for x-axis tick labels +
+    "Amplitude Geo: X in/s/div Mic: Y psi(L)/div" footer + trigger
+    legend without overlap.
     """
     gs = fig.add_gridspec(
         nrows=4, ncols=1,
-        left=0.11, right=0.94, top=0.97, bottom=0.06,
+        left=0.11, right=0.94, top=0.97, bottom=0.12,
         height_ratios=[1.7, 2.0, 1.8, 5.5],
         hspace=0.35,
     )
@@ -355,11 +397,13 @@ def _render_histogram_layout(fig, rd: ReportData) -> None:
 
     No USBM compliance chart (it's a waveform-only concept).  Stats table
     uses Date + Time-of-peak instead of relative-time + accel + disp.
-    Left margin sized to fit the channel labels.
+    Left margin sized to fit the channel labels.  Extra bottom margin
+    leaves room for the x-axis time labels + footer scale legend
+    without overlap.
     """
     gs = fig.add_gridspec(
         nrows=4, ncols=1,
-        left=0.11, right=0.94, top=0.97, bottom=0.06,
+        left=0.11, right=0.94, top=0.97, bottom=0.12,
         height_ratios=[1.8, 0.9, 1.7, 5.6],
         hspace=0.35,
     )
@@ -718,12 +762,12 @@ def _draw_waveform_subplot(fig, gridspec_cell, rd: ReportData) -> None:
             geo_amp_div = f"{(amax * 1.1 * 2) / 10:.3f}"
             break
     fig.text(
-        0.07, 0.045,
+        0.11, 0.030,
         f"Time(Seconds) {div_s:.2f} sec/div   Amplitude Geo: {geo_amp_div} in/s/div   Mic: 0.001 psi(L)/div",
         fontsize=7, color="#444", ha="left",
     )
     fig.text(
-        0.07, 0.030,
+        0.11, 0.018,
         "Trigger = ▶━━━━━ ━━━━━━◀",
         fontsize=7, color="#444", ha="left",
     )
@@ -789,7 +833,7 @@ def _draw_histogram_subplot(fig, gridspec_cell, rd: ReportData) -> None:
             geo_amp_div = f"{amax / 5:.3f}"
             break
     fig.text(
-        0.07, 0.045,
+        0.11, 0.030,
         f"Time {interval_str} /div   Amplitude Geo: {geo_amp_div} in/s/div   Mic: 0.001 psi(L)/div",
         fontsize=7, color="#444", ha="left",
     )

sfm/sfm_webapp.html

@@ -2748,8 +2748,9 @@ function _renderScWaveform(data) {
     //   - Mic (always positive sound pressure) + histograms (per-interval
     //     peaks, always positive): default auto-scale, zero at the bottom.
     let yBounds = {};
-    const isGeoWaveform = !isHistogram && ch !== 'MicL';
+    const isGeo = ch !== 'MicL';
-    if (isGeoWaveform) {
+    if (isGeo && !isHistogram) {
+      // Waveform geo: symmetric around zero, full zoom to shape detail.
       let absMax = 0;
       for (const v of values) {
         const a = Math.abs(v);
@@ -2757,13 +2758,31 @@ function _renderScWaveform(data) {
       }
       const padded = (absMax || 1) * 1.10;
       yBounds = { min: -padded, max: padded };
+    } else if (isGeo && isHistogram) {
+      // Histogram geo: enforce a minimum chart range so a quiet
+      // 0.005 in/s event renders as ~10% of chart height instead of
+      // filling the panel.  Matches BW's near-fixed-scale convention
+      // (their footer is "Geo: 0.002 in/s/div" — a chart-relative scale,
+      // not auto-zoom).
+      const HIST_GEO_MIN_INS = 0.05;
+      let peak = 0;
+      for (const v of values) { const a = Math.abs(v); if (a > peak) peak = a; }
+      yBounds = { min: 0, max: Math.max(peak * 1.10, HIST_GEO_MIN_INS) };
     } else if (ch === 'MicL' && micUnit === 'dBL') {
-      // Pin baseline at the chart floor (which matches what we flooded
+      // Mic in dBL — pin baseline at noise-floor minimum (where we floored
-      // null/quiet samples to), top at the actual peak + a few dB headroom.
+      // quiet samples), top at actual peak + a few dB headroom.
       const peakDbl = (typeof chPeak === 'number' && isFinite(chPeak))
         ? chPeak + 5
         : 100;
       yBounds = { min: MIC_DBL_FLOOR, max: Math.max(peakDbl, MIC_DBL_FLOOR + 20) };
+    } else if (ch === 'MicL' && isHistogram && micUnit === 'psi') {
+      // Mic histogram in psi — same minimum-range treatment as geo.
+      // 0.001 psi ≈ 110 dBL — typical "loud" mic peak.  Quiet events
+      // sit near the bottom.
+      const HIST_MIC_MIN_PSI = 0.001;
+      let peak = 0;
+      for (const v of values) { const a = Math.abs(v); if (a > peak) peak = a; }
+      yBounds = { min: 0, max: Math.max(peak * 1.10, HIST_MIC_MIN_PSI) };
     }
 
     _scCharts[ch] = new Chart(canvas, {