Commit Graph
148 Commits
Author SHA1 Message Date
serversdownandClaude Opus 4.8 8d3cdba1b5 feat(h5): standardize sensor-check into the .h5 (schema v2); SFM reads it
Make the sensor self-check a first-class part of the standardized decoded event
so SFM stops decoding it at report time — device-agnostic, per the store's
decoder→standardized-.h5→SFM model.

  * Event gains a `sensor_check` field; both decoders attach the traces where
    they set raw_samples — series-3 in event_file_io.read_blastware_file
    (minimateplus.sensor_check), series-4 in waveform_store's IDF path
    (micromate.sensor_check).  Covers ingest and backfill (both re-decode).
  * event_hdf5 bumps schema_version 1→2 and writes an optional /sensor_check
    group (raw counts, int32, per channel present).  read_event_hdf5 returns
    it; plot_json_from_hdf5 carries it as a top-level key.  Old v1 files still
    read cleanly (no group → None), so nothing breaks before the backfill.
  * gather_report_data reads sensor_check_waveforms from the .h5 and drops the
    report-time series-3 decode — the report no longer reaches into a decoder,
    and a series-4 event now lights up the same strip automatically.

Stored as raw counts (a shape diagnostic, rendered fit-to-box): the per-series
count scale differs and a physical mic unit is ill-defined, so conversion would
add complexity for no display benefit — easy to add later if a numeric use
appears.

Tests: .h5 roundtrip + backward-compat + plot_json + real series-3 decode
attaches to the Event.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-16 06:49:55 +00:00
serversdownandClaude Opus 4.8 2a747f6893 fix(report): attach sensor-check strip to the waveform panel; move "0.0"
Match Blastware's layout, measured off the reference PDF: the sensor-check
strip shares a border with the main waveform panel (no gap between them), and
the per-lane "0.0" baseline labels sit to the RIGHT of the strip. Previously
the strip floated with a gap and the "0.0" label overprinted the strip's left
edge. Purely layout — the traces and decode are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 05:48:31 +00:00
serversdownandClaude Opus 4.8 2c5c20cfd7 fix(report): fit sensor-check mini-plots to their boxes
The sensor-check strip used a symmetric ±max scale, so the one-sided geophone
ring-downs (a dip to ~-990 with the baseline at 0) sat in the bottom half of
each mini-box with the top half blank — visibly off next to Blastware. Scale
each mini-plot to its actual data range with a small pad instead, and draw a
faint zero baseline, so the ring-downs and the mic pulse train fill their boxes
the way BW draws them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 05:39:53 +00:00
serversdownandClaude Opus 4.8 ab9d84fde6 feat(report): render the sensor-check strip + report polish
Wire the decoded sensor self-check waveforms (previous commit) onto the event
report PDF, and fold in two related waveform-panel cleanups.

Sensor-check strip (matches Blastware):
  * ReportData gains sensor_check_waveforms; gather_report_data decodes it from
    the retained raw BW binary (store.paths_for) at report time — no ingest or
    .h5 change, waveform events only.
  * _draw_waveform_subplot now draws a narrow right-hand strip of per-channel
    mini-plots (MicL pulse train + Long/Vert/Tran ring-downs) aligned to the
    lanes, captioned "Sensor Check".
  * stats table gains the "Frequency" / "Overswing Ratio" sub-rows under Sensor
    Check (7.5/7.7/7.3 Hz, 3.6/3.3/3.7), formatted to 1 decimal like BW; values
    come from the already-parsed sensor_check scalars.

Cleanups (pre-existing, in the same panel):
  * fix the stacked-lane y-tick collision — adjacent lanes' -1.0 / 1.0 labels
    overprinted at the shared boundary; prune the extreme ticks (MaxNLocator
    prune="both") so each lane shows clean interior ticks only.
  * fix the header serial+firmware line running off the right page edge —
    tighter right-column indent + BW's slightly smaller 7.5pt header.

Tests: sensor-check + compliance + geo-scale + fft all green (15). The
test_bw_ascii_report failures are pre-existing (gitignored decode-re fixtures
absent in this worktree), unrelated to this change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 05:33:38 +00:00
serversdownandClaude Opus 4.8 dc74c97ade feat(report): size the USBM compliance chart to match Blastware
The compliance chart on the event-report PDF was correctly drawn but far
too small ("it's tiny") — a ~2.6in square dropped between the mic and
stats rows.  Resize + reposition it to match Blastware's Event Report,
measured directly off a BW reference PDF (n844lqhbzt0w) rasterized with
fitz: the chart data box now spans figure fractions x[0.489,0.951]
y[0.502,0.867] — a ~3.9in square running from just under the header down
through the stats band, hard against the right page margin, exactly as BW
draws it.  Title updated to BW's "USBM RI8507 And OSMRE".

To clear room for the BW-sized chart (waveform layout only):
  * _draw_stats_table gains bbox_width/col_widths/fontsize params; the
    waveform layout packs the Tran/Vert/Long table into the left ~0.42 so
    its columns no longer sit under the chart.  Histogram layout keeps the
    wider defaults (byte-identical output; it has no compliance chart).
  * the mic block's long "Channel Test Passed (Freq … Amp … mv)" line gets
    a tighter indent + one-point-smaller font so it ends before the chart's
    left edge instead of running behind it (_kv gains a fontsize param).
  * the Peak Vector Sum line left-aligns under the compacted table (one pt
    smaller) so it clears the chart's bottom-left tick labels.

Chart placement centralized in the _COMPLIANCE_BOX constant. No change to
the compliance math, the scatter, or the histogram report.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-14 22:56:44 +00:00
serversdownandClaude Opus 4.8 95f926c318 feat(report): enlarge the compliance chart to a full upper-right panel
The chart was cramped into the short mic band (~2in) and rendered tiny. Move it
to its own large square panel (_draw_compliance_panel) spanning the mic + stats
rows on the right, clear of the stats columns — matching Blastware's Event
Report proportions. _draw_mic_and_usbm now draws only the mic block.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-14 20:33:09 +00:00
serversdownandClaude Opus 4.8 6ad4fd73dd fix(compliance): square plot box (set_box_aspect) so the chart isn't squashed
The compliance chart sits in the short, wide mic-and-USBM band on the event
report; without a fixed aspect matplotlib stretched it wide-and-short. Force a
square plot box, which is how log-log compliance charts are conventionally drawn.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-14 20:21:17 +00:00
serversdownandClaude Opus 4.8 db818f716c feat(report): draw the USBM RI8507 compliance chart on the event report
Replace the "[compliance chart coming soon]" placeholder in
_draw_mic_and_usbm with a real inset axes calling
sfm.compliance.draw_compliance_chart on rd.channels / rd.sample_rate_sps
(the full-rate in/s waveform samples). Title updated "USBM RI8507 And OSMRE"
→ "USBM RI8507" — we draw only the RI8507 lines (Drywall 0.75 + plaster 0.50);
the OSMRE overlay is dropped by choice.

Waveform events only (the histogram layout has no USBM chart). Falls back to a
"(no waveform data)" note when samples are unavailable. Closes the 1.0
compliance-chart blocker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-14 20:18:29 +00:00
serversdownandClaude Opus 4.8 dad35e47fe feat(compliance): USBM RI8507/OSMRE compliance chart + reference doc
sfm/compliance.py renders the velocity-vs-frequency blasting compliance chart
Blastware draws on its Event Report:
- limit_at()/limit_curve() — the RI8507 Fig B-1 / 30 CFR 816.67 curve as data
  (Drywall 0.75 + plaster 0.50 lines): 0.030in low-freq bound, plateau, 0.008in
  rising diagonal to a 2.0 in/s cap at ~40 Hz, drawn continuous.
- channel_compliance_points() — the per-cycle (freq, peak-velocity) scatter by
  the zero-crossing method (matches Blastware; cloud ceiling = channel PPV).
- draw_compliance_chart() — matplotlib rendering (both lines + scatter, BW tick
  scales + channel markers).

Verified against 7 BE12844 Blastware reports. docs/ri8507_compliance_curve.md
captures the curve construction, the SHM basis, and the scatter method.

Not yet wired into report_pdf.py — that placeholder is the next step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-14 18:34:18 +00:00
serversdownandClaude Opus 5 726c2ce1b5 fix(series4): Thor/Micromate decoder is now per-sample exact
Verified against Thor's own CSV exports, which carry a per-sample
four-column block beside every binary (CSV/<name>.IDFW.csv). Those 1,012
paired files were in the corpus all along; the decoder had been pinned to
a superseded walker on the stated grounds that "Thor has no ASCII ground
truth in the corpus and its geo scaling is separately suspect". Both
premises were false.

  IDFW per-sample exact      39.1%  -> 100.000% (1,057,536/1,057,536)
  IDFW files fully exact     0/153  -> 153/153
  IDFW PPV median error      -3.32% -> -0.002%
  IDFH within 2% of Thor PPV 51.1%  -> 100.0% (858/858)
  prod IDFW, 8 units         -3.3%  -> -0.001%

Four independent root causes:

- Geo LSB was 0.0003, the 4-dp *display rounding* of the real
  0.000310308 mistaken for the LSB, so every series-4 geophone sample
  read 3.3% low. Pinned to +-6e-11 by intersecting 991,415 rounding
  constraints; corroborated by the +-full-scale seed (+-32226) left in
  unwritten IDFH slots. IDFH had a separate, also wrong, 10.0/32768.

- IDFH histograms were capped at 250 intervals: the segment validator
  required the interval counter's high byte to be zero, but the counter
  is a uint16 cumulative index, so every segment past interval 255 was
  rejected. Runs over ~4 hours lost their tail, often the peak.
  540/858 corpus files affected.

- Record mode 00 00 (raw int16, 10-byte header) was unhandled and fell
  through the dispatch, silently dropping each channel's first 512
  samples -- the long-standing "loud events truncate" symptom.
  MODE_ABSOLUTE is now also accepted as a segment-0 preamble.

- The body-offset search matched 00 02 00 *inside* record headers,
  selecting a candidate part-way down the chain and decoding a
  rotation-shifted body. It now anchors on record headers and takes the
  chain head (6 ms/file).

Also fixes the separately tracked "UM-series decodes ~1000x low" bug.
Series-3 re-verified unchanged at 14,338/14,338 exact after the shared
waveform_codec change.

Known open: 41/575 prod IDFW files (7%, mostly UM12947/UM20147) decode
with unequal channel lengths and also fail metadata extraction -- a
different header variant with no Thor export in the store.

NOTE: this is a codec change; the Thor store owes a regeneration via
scripts/backfill_thor_events.py.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-10 18:06:14 +00:00
serversdownandClaude Opus 4.8 91b9b4578c fix(pdf): shared geo Y scale across Long/Vert/Tran (was per-trace)
The event-report waveform plot scaled each geo lane to its own peak, so a small
channel filled its lane looking as big as a large one — and the "Geo: X in/s/div"
footer only reflected whichever channel was checked first, so its div value was
wrong for the other two. Now all three geo lanes share ONE symmetric scale =
max |sample| across them (padded, 0.05 in/s floor), matching the event modal and
BW's single amp/div; the footer reflects that shared scale. Mic keeps its own psi
scale. Big events are unchanged (e.g. BE12844 stays 0.185 in/s/div).

Test-first: tests/test_report_pdf_geo_scale.py (shared scale + floor), 2 tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-07 23:26:42 +00:00
serversdownandClaude Opus 5 9ceff65bfb fix(sfm): read the serial family prefix from the file, enabling BlastMates
The BW filename encodes only the serial NUMBER — `<letter><3 digits>`, so
`L895…` is 10895 and nothing more. The two-letter family prefix is not in it:
"BE" is a MiniMate Plus, "BA" a BlastMate. Both are Series III and their
files are byte-identical in every way that matters — all 1,493 BlastMate
binaries in the DL2 archive decode through the existing codec at 100%, same
four channels — so the serial string was the only thing standing between SFM
and BlastMate support.

Two sites synthesised the prefix and got it wrong:

- waveform_store `_serial_from_bw_filename` returned f"BE{num}" on import, so
  a BlastMate event was filed under a unit that does not exist, silently, and
  Terra-View read it straight through. Split into
  `_serial_number_from_bw_filename` (the number, which the filename really
  does carry) and a new `_serial_from_bw_bytes` that reads the serial out of
  the body and accepts it only when its numeric part agrees with the
  filename. save_imported_bw now prefers hint -> body -> filename guess.
  Verified against real archive bytes for BA9229, BA10060, BA10895, BA15957
  and BE9558/BE11529/BE18003.

- client `_decode_0a_partial_header` searched for a literal b"BE" in the
  monitor-log partial record. On a BlastMate that returns -1 and skips the
  whole block, so the geo threshold went missing along with the serial. Now
  matches any two-letter prefix, and requires the NUL terminator — stricter
  than the bare two-byte search it replaces.

Nothing to migrate: no BlastMate events are in prod. The archive's BA units
last recorded 2018-10 (BA9229, BA15957), 2023-08 (BA10895) and 2023-11
(BA10060), and the prod backfill only reaches back to ~May 2025.

21 tests. Suite: 309 passed, same 16 pre-existing failures as at HEAD
(15 missing ASCII fixtures + one peak_values assertion, all untouched here).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
2026-09-06 07:58:00 +00:00
serversdownandClaude Opus 4.8 c0cf6547d9 feat(ft): optional false_trigger_reason ("offset" etc.) as an FT subtype
A reason records *why* an event is a false trigger. It is optional (plain
FT flags still record no reason) and is a subtype of the FT flag: setting a
reason implies false_trigger=1, and the reason is cleared whenever FT ends
up 0 (confirm-real, clear-FT, set_false_trigger(false)). Twin propagation
carries the reason to the histogram/waveform twin alongside the FT flag.

New nullable `false_trigger_reason TEXT` column (schema + _migrate ADD
COLUMN only — not the Migration-1 rebuild). 7 tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-03 20:53:57 +00:00
serversdownandClaude Opus 4.8 3554d00583 feat(offset): DC-offset detector productionized into the shape pipeline
Productionizes the validated scratch/offset_scan3.py: a DC offset (baseline
shifted off zero — sensor bumped/settled/drifted) is |median(pre-trigger)| >= 5
counts (0.025 in/s) AND flat across pre/mid/end thirds (spread <= 0.02); a
transient moves one third and is rejected by the spread test.

- shape_metrics: offset_from_samples / offset_from_h5 (reads .h5 samples +
  pretrig_samples attr; range-aware via the .h5's in/s float samples)
- events schema: shape_offset / _axis / _pre / _spread (via _SCHEMA + the
  _migrate ADD COLUMN loop only; NOT the Migration-1 rebuild), threaded through
  insert + upsert mirroring shape_*
- ingest: computed at all three waveform_store save paths alongside shape
- backfill_event_shape: also computes + stores (and stale-clears) offset
- exposed via /db/events automatically (SELECT *)

Gating to waveforms is done downstream in terra-view ft_suspicion (mirrors how
shape is ignored for histograms), not at the SFM call sites. 13 new tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-02 04:47:01 +00:00
serversdownandClaude Opus 4.8 c8c4ec2b9f fix(sfm): /health reports the real service version, not a stale 0.1.0
terra-view's SFM Admin page (/admin/sfm) displays whatever /health returns for
`version`. That was hardcoded to "0.1.0" and never bumped, so the page showed
0.1.0 while the service was actually 0.26.0. Point both /health and the FastAPI
OpenAPI version at the release-bumped TOOL_VERSION (single source of truth), so
they can't drift again. Adds httpx-free regression tests (call health() directly).

Note: minimateplus.__version__ is separately stale at 0.1.0 — left as-is here
(nothing user-facing reads it; touching the package __init__ risks import order).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-08-28 20:39:51 +00:00
serversdownandClaude Opus 4.8 75ac610c61 fix(twins): interval-based histogram/waveform matching in find_twins (#102 sub-task 2)
A real trigger is recorded twice — as a triggered waveform (stamped at the
trigger instant) and inside the scheduled histogram whose interval contains it
(stamped at the 7am/7pm interval start). The two twins routinely differ by
HOURS, so the old ±5-minute window in find_twins silently missed them — which
broke review propagation (flagging one twin left its twin unflagged).

Twins are now matched by: same serial + identical peak_vector_sum + OPPOSITE
record type + the waveform's timestamp falling within the histogram's interval
(bounded by the next same-serial histogram). Matching keys off record timestamps
(not call-in/received times, which drift with field connectivity). window_seconds
is retained but ignored.

Rewrote test_find_twins + test_twin_propagation for the new contract (incl. the
75-min-apart UM12947 case, cross-type exclusion, containing-interval selection,
open-ended latest interval). Full suite: 264 passed; the 16 failures are
pre-existing (missing gitignored fixtures + a v0.26.0 codec case), unchanged
from baseline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-08-28 05:23:30 +00:00
serversdownandClaude Opus 5 b2ef02ebcc chore(release): v0.26.0 — series-3 decode correctness
Two body-model rewrites, a systematic scale error affecting every
geophone reading the system ever produced, a recovered file format, and
two artifact-hygiene bugs where stale files outlived the decodes that
made them.

  - geo full scale is 32000 ADC counts, not 32768 (every reading 2.34% low)
  - the waveform body is a record chain, not a tag stream
  - the histogram block is big-endian, with a terminal tail
  - sub-minute intervals pack several per block (415 files recovered)
  - three more defects found by a full-corpus sweep, each masking the next
  - stale .h5 files and stale shape_* columns are now cleared, not left

All 11,603 series-3 binaries in the production snapshot pass every check.
Ground truth: 1,211/1,211 histograms exact per-interval, 75/75 waveform
sample counts exact, multi-interval fixture exact on all 45,680 values.

Also corrects a changelog note that went stale within the same day: the
"3 of 75 events still truncate" item was resolved by the record-chain
rewrite, and the remaining open items are now listed explicitly.

CLAUDE.md gains a "Where things stand" block at the top — the header had
been reading v0.21.0, four releases behind, which is the first thing you
see when picking the project back up.

Tests: 259 passed; the 16 failures are pre-existing (gitignored
fixtures) and unchanged from baseline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
2026-08-27 16:35:47 +00:00
serversdownandClaude Opus 5 686ab6e7a6 fix(codec): geo full scale is 32000 counts; 4 walker framing cases; channel-id from header
Two independent bugs, both found by diffing 75 production events against
their preserved Blastware ASCII exports (<store>/<serial>/<file>_ASCII.TXT).

1. Geo full scale was wrong — every geophone reading was 2.34% low.
   The codec emits geo samples in 16-count units with a documented LSB of
   exactly 0.005 in/s, and decoded_to_adc_counts multiplies by 16, so one
   ADC count is 0.005/16 in/s and 10.000 in/s is 10.0/(0.005/16) = 32000
   counts.  sfm/event_hdf5.py and minimateplus/event_file_io.py both
   divided by 32768 (2^15), scaling every sample and derived peak down by
   1 - 32000/32768.  The error scales with amplitude, so it was invisible
   on quiet events and worst on the loud ones that matter for compliance.
   Mic is unaffected (it back-solves its scale from the device peak).

   216 per-channel comparisons: 32768 -> 151/216 exact, worst error 0.238
   in/s on a 10 in/s event; 32000 -> 216/216 exact, worst 0.005 = 1 LSB.

2. walk_body silently truncated channels on four unhandled framing cases.
   An unrecognised tag ends the walk and decode_waveform_v2 returns
   whatever it got, so this surfaced as short channels, never an error:
     - wide-NN RLE `0X NN` (runs longer than 252 samples)
     - `30 NN` with NN > 0x10 (the old cap was arbitrary)
     - variable-width `40 NN` headers: NN counts previous-channel
       continuation deltas, so the header is 2*NN + 16 bytes; `40 01`
       and `40 03` occur alongside `40 02`
     - tagless segment headers: no `40 NN` tag at all, just the 14-byte
       tail [field2:2][len:2][channel_id:4][marker:2][anchors:4]

Also: the header field documented as a "monotonic uint32 LE counter" is
really [channel_id][00][00][segment_index], with 0x46=Tran 0x47=Vert
0x48=Long 0x49=MicL — verified on 1697/1697 segment headers, zero
disagreements.  decode_waveform_v2 now takes the channel from that field
instead of rotation position, which was fragile: one missed header
desynced every channel after it.

parse_segment_header now returns n_prev_deltas/prev_deltas/marker/
anchors/channel/segment_index; the old fixed_pattern (02 00 00 01)
conflated the 2-byte marker with the first anchor.

Ground-truth corpus, end to end through the production path:
  exact 37 -> 72, truncated 23 -> 3, full-length value errors 15 -> 0.
Store-wide, 729 of 1388 series-3 waveform events decode differently and
728 gain samples; the scale fix changes float values on all of them, so
stored .h5 files need regenerating.

Still open: 3 events truncate at a header variant with a variable-width
prefix (2/4/6 bytes) before the channel id and an `01 00` marker.
Documented in docs/instantel_protocol_reference.md with byte offsets.

+20 tests.  No regressions: the byte-exact fixture suite still passes and
the full-suite failure list is unchanged from baseline (16 pre-existing
failures from gitignored fixtures).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
2026-08-25 08:11:11 +00:00
serversdown 37043a47e9 fix(review): quick FT path enforces 3-state exclusivity + twin propagation; changelog + twin caveat
set_false_trigger now clears reviewed_real when flagging false_trigger=True
(mirrors update_event_review's exclusivity), and the quick
PATCH /db/events/{id}/false_trigger endpoint now calls
propagate_review_to_twins after the flag write, matching the sidecar PATCH
path's try/except-with-log.warning pattern. Previously the quick path could
leave both flags set and never touched twins.

Also corrects the v0.25.0 CHANGELOG bullet (exclusivity was not actually
enforced on the quick path until this commit) and adds a caveat comment on
find_twins about rare clamped/saturated-PVS false-positive twin matches.
2026-08-25 00:55:38 +00:00
serversdown 4a581e0e67 chore(release): v0.25.0 — reviewed_real + twin review-propagation 2026-08-25 00:45:33 +00:00
serversdown 7aae0208f8 feat(review): propagate false_trigger/reviewed_real to twins on sidecar PATCH 2026-08-25 00:42:27 +00:00
serversdown 5ffa92ab87 feat(db): find_twins (serial + identical PVS + time window) 2026-08-25 00:38:49 +00:00
serversdown f73c8eec91 feat(db): update_event_review mirrors reviewed_real + enforces 3-state exclusivity 2026-08-25 00:34:25 +00:00
serversdown 23e4f585a8 fix(db): keep reviewed_real out of the Migration-1 rebuild table (positional SELECT *); regression test 2026-08-25 00:31:39 +00:00
serversdown d9cc5f1780 feat(db): reviewed_real column on events (+ auto-migrate) 2026-08-25 00:26:22 +00:00
serversdown ac67e83bcf chore(release): v0.24.0 — waveform-shape metrics on events 2026-08-22 06:12:01 +00:00
serversdown e64e3bcd3e feat(ingest): compute shape from the written .h5 in all save paths 2026-08-22 06:01:43 +00:00
serversdown 54c4182023 feat(db): insert_events persists shape_* from waveform record 2026-08-22 05:55:25 +00:00
serversdown a894b001b1 feat(db): shape_* columns on events (+ auto-migrate) 2026-08-22 05:50:45 +00:00
serversdown 2539f903de feat(shape): crest-factor + points-near-peak waveform metrics 2026-08-22 05:42:36 +00:00
serversdownandClaude Opus 4.8 aebb5644bd chore(release): v0.23.0 — per-channel ZC frequency in events store
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-08-04 21:42:41 +00:00
serversdownandClaude Opus 4.8 1744eb1803 feat(db): persist per-channel ZC freq on insert/upsert
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-07-29 17:27:36 +00:00
serversdown 248276d141 feat(db): add per-channel ZC-freq columns to events (+ migration) 2026-07-29 17:13:59 +00:00
serversdown 6e2994acdd fix(api): threadpool the /db/restore handler (drop async) 2026-07-02 17:50:28 +00:00
serversdown b56eb8c692 feat(api): /db/snapshot, /db/waveforms/recent.zip, gated /db/restore 2026-07-02 06:53:24 +00:00
serversdown 280b3d25ec fix(db): WAL-safe safety backup + tests for validate-first and zip-traversal guard 2026-07-02 06:45:17 +00:00
serversdown 7edd0a1265 feat(db): gated WAL-safe restore of seismo_relay.db + waveforms 2026-07-02 06:35:24 +00:00
serversdown 916e6fcabb feat(db): zip recent events' waveform files 2026-07-02 06:30:11 +00:00
serversdown 31660d24e9 feat(db): WAL-safe seismo_relay.db snapshot helper 2026-07-02 06:22:46 +00:00
serversdown 1ed86244d0 fix(thor-events): add parallel field for mic psi. Now shows mic in dbl and psi. (psi for charts) 2026-06-01 18:27:24 +00:00
serversdown b2c565f217 fix(idf_waveforms): _find_waveform_body_offset() — scans every 00 02 00 magic past offset 0x0E00, runs decode_waveform_v2 on each candidate, picks the one that returns the most samples. Validated on 483 prod IDFW files: 0 preamble-only events (was ~50%), 355/483 fully decode, 126/483 partial (BW codec walker-stops-early on loud events — known issue).
IDFH now synthesises a 1-sample-per-interval array from the binary intervals and writes an .h5 so the existing renderer works unchanged. Each "sample" is the per-interval peak ADC count → h5_value = count × geo_fs/32768 yields the right bar height.
2026-05-31 20:51:09 +00:00
serversdownandClaude Opus 4.7 23e83908c2 report_pdf: fix PVS overlapping stats table, drop NA caption
Two related fixes to the per-channel stats block:

1. Pin the stats table's position via an explicit bbox= on
   ax.table() so the bottom edge is at a known axes-fraction Y.
   The previous loc="upper left" + tbl.scale(1, 1.4) combo let
   matplotlib choose row heights based on text size, which made the
   table extend further below the axes than the hard-coded PVS line
   at y=-0.08 expected.  Result was the "Peak Vector Sum X in/s"
   string landing horizontally inside the Peak Displacement row.

   With bbox=[0, 1-N*0.12, 0.80, N*0.12] the table is pinned to a
   precise rectangle (12% axes-fraction per row × N rows tall).
   _draw_stats_table now stashes the bottom Y on the axes for the
   PVS helper to reference, so the geometry stays in sync.

2. Center PVS horizontally (ha="center" at x=0.5 instead of ha="left"
   at x=0).  The previous left-edge alignment put PVS at the same
   X as the label column, which read as "off-center" once the rest
   of the stats data was column-aligned further right.

3. Drop the "NA: Not Applicable" caption.  It existed to explain
   "—" placeholder cells, but "—" is universally understood and the
   caption was always visually squished against the PVS line below.
   Less cruft on the page; one fewer position to manage.

Verified against a real BE12599 histogram event (5 data rows) and
a real UM12947 IDFW waveform event (6 data rows) — both layouts
clear the table cleanly with no overlap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 22:17:43 +00:00
serversdownandClaude Opus 4.7 bee118506b fix(idf): decode from in-memory bytes during ingest
Bug shipped in v0.21.0: save_imported_idf called read_idf_file()
with `source_path` (a bare filename like "UM12947_….IDFW") BEFORE
writing the binary to disk.  The codec did Path(path).read_bytes()
which resolved relative to /app and hit FileNotFoundError.  The
error was caught + logged as a warning, and ingest fell back to
.txt-only — events still landed in the DB but lost the bw_report
block + .h5 waveform that the codec was supposed to produce.

Observed during a full re-forward from thor-watcher on 2026-05-29:
every Thor event logged "binary codec failed for X: [Errno 2] No
such file or directory" and got binary_decoded=False.

Fix:
- read_idf_file() gains a `data: Optional[bytes]` kwarg.  When
  supplied, skips the disk read and decodes the provided bytes
  directly.  `path` stays required (used for filename in error
  messages + .IDFH vs .IDFW suffix detection); only the read is
  conditional.  Backward compatible — existing positional callers
  (CLI scripts, tests) continue to work unchanged.
- save_imported_idf passes `data=idf_bytes` since the bytes are
  already in memory from the multipart upload.  Filesystem write
  still happens at step 5 of the existing flow; codec just no
  longer depends on it.

Verified end-to-end against UM11719_20231219162723.IDFW from the
example-data corpus: ingest endpoint returns inserted=1, log line
shows binary_decoded=True + h5=...IDFW.h5, no warnings.

Re-forward existing Thor events from thor-watcher after deploy to
backfill the bw_report block — UPSERT preserves review state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 20:09:54 +00:00
serversdownandClaude Opus 4.7 defd17d9c2 sfm_webapp: harmonize "Received by server at" → "Time received"
Matches Terra-View's event-modal relabel from the same iteration.
Wording was already clearer here than in Terra-View's "Captured at",
but using identical text across both surfaces means operators see the
same label whether they're in the native modal or the standalone
webapp.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 19:51:58 +00:00
serversdown 9fd52ddabb feat: add thor report generation, pdf generation. 2026-05-29 19:03:06 +00:00
serversdown 9b71ead44b series 4 codec work, inital decode success 2026-05-29 06:33:13 +00:00
serversdownandClaude Opus 4.7 6a73523e4d ui: surface per-channel ZC Freq (and ">100") in event modals
The PDF report shows per-channel ZC Freq alongside PPV in the stats
block, but neither modal exposed it.  Now that the sidecar projection
carries zc_freq_hz + zc_freq_above_range, plumb them through:

- sfm_webapp.html: inline suffix on existing Peaks cells, e.g.
  "Tran  0.04500 in/s · >100 Hz".  Empty suffix when no ZC is
  available (legacy events without a preserved .TXT).

- event_browser.html: new ZC Freq column on the per-channel stats
  table.  Required adding a parallel sidecar fetch in loadEvent()
  (waveform.json alone doesn't carry bw_report).  Fetch failure is
  non-fatal — falls back to "—" in the new column.

Above-range ZC peaks (BW ">100 Hz") render with a literal ">"
prefix mirroring the PDF, so operators don't have to generate the
PDF to see when a channel hit the zero-crossing ceiling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:47:37 +00:00
serversdownandClaude Opus 4.7 780b45a371 feat: render ">100" for above-range ZC Freq instead of "—"
BW writes ">100 Hz" for ZC Freq when the zero-crossing algorithm sees a
peak too fast to count — the device's reporting ceiling is 100 Hz on
V10.72.  Our parser fell back to None via _parse_number (which requires
a leading digit), so the PDF rendered "—" where BW shows ">100".

Mirrors the OORANGE/saturated pattern already used for PPV and PSPL:
parser stores the threshold (100.0) on zc_freq_hz + sets a new
zc_freq_above_range flag.  Projection carries the flag through to the
sidecar; PDF renderer prepends ">" when set.

Affects both per-channel stats tables (waveform + histogram variants)
and the mic block's ZC Freq row.

Verified on the real T190LD5Q.LK0W fixture: Tran zc_freq_hz=100.0
above_range=True; Vert/Long (normal values) above_range=False; "N/A"
still produces zc_freq_hz=None which renders as "—" (unchanged).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:38:49 +00:00
serversdownandClaude Opus 4.7 f6abe3caa0 fix(report_pdf): histogram geo channels share nice-quantized y-axis
Two related visual bugs on histogram PDFs:

1. Per-channel auto-scale meant Tran/Vert/Long had different y-axes
   (e.g. 0-0.015, 0-0.025, 0-0.020) — bars looked taller on the
   channel that happened to be quietest.  Not directly comparable.

2. Footer "Amplitude Geo: X in/s/div" was just amax/5 of the FIRST
   geo channel with data, with no LSB quantization — producing
   nonsense like 0.003 in/s/div when the geophone LSB is 0.005.

Fix: compute a single shared geo y-axis range from max(Tran,Vert,Long),
quantize the per-division step to BW's 1-2-5 sequence rounded to the
0.005 LSB (0.005, 0.01, 0.025, 0.05, 0.1, 0.25, ...), apply the same
ylim + ticks to all three geo subplots, and use that same step for the
footer label.  MicL stays on its own auto-scale (different units).

Verified across edge cases including the reported event
(geo max 0.025 → 0.005/div, top 0.025), small PVS events, and large
blast amplitudes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:22:20 +00:00
serversdownandClaude Opus 4.7 ad2702d4bf fix(report_pdf): add missing histogram_interval_size_s field
The histogram-interval-times derivation block at line 314 references
rd.histogram_interval_size_s, but the field wasn't declared on the
ReportData dataclass — only the string form histogram_interval_size
was.  Result: every PDF render of a histogram event raised
AttributeError → 500 from /db/events/{id}/report.pdf.

Cause: when the histogram aggregation block was inlined into
gather_report_data, the seconds-numeric counterpart that the
projection already carries (bw_report.histogram.interval_size_s) was
never wired into the dataclass.  Waveform PDFs weren't affected
because the offending line is gated on is_histogram.

Fix: add the field, read it from the projection alongside the other
histogram keys.  No-op for waveform events (the field stays None and
the gate skips it).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 18:07:41 +00:00