backfill_sidecars.py skipped the .h5 write when a file produced no
samples, with the stated intent of not replacing it with an empty
placeholder. That silently preserved output from a superseded decoder.
After the record-chain fix, 415 histogram files stopped decoding (216 on
BE18193, 199 on BE9440) but kept .h5 files whose peaks ran up to 400x
the device-reported PPV. Those were feeding charts and the
false-trigger detector with nothing marking them. The .h5 is now
removed in that case and the run reports stale_h5_removed.
Store-wide effect, series-3, decoded peak vs device-reported PPV:
waveform 1307/1307 (100%), mean abs ratio error 0.00000
histogram 4434/4435 (100%)
Both were 99% with a tail of 18 and 25 wrong files respectively.
The 415 files are a genuine unmapped format variant, not a regression:
their bodies open `00 00 00 01 0a 00` (valid block header, marker 0a at
[4], block_ctr 256) but block[28:32] matches neither known tail, and no
stride from 8 to 64 bytes places a marker at [4] consistently. Bodies
are very large (one is 360,573 bytes). They were previously being
decoded by the WAVEFORM codec, which accepted them and returned garbage
- so the gap pre-dates today's work; the fix only exposed it. Logged as
an open question in the protocol reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Supersedes the segment-header model entirely, including the fixes made
earlier today. Found via multi-agent structural analysis of the 25 files
that stalled the walker, then verified independently.
Records are self-delimiting: off+2 is a uint16 BE length, next_record =
off + 2 + len, and the chain ends on a record whose chan_id is 0x06.
off+8 carries a 3-valued mode enum:
02 00 14-byte header, 2 anchors, then CUMULATIVE delta blocks
01 00 10-byte header, no anchors, blocks are ABSOLUTE values
00 03 10-byte header, NO TAGS AT ALL - raw 12-bit packed absolute
`40 NN` is an ordinary int16 BE data block (2*NN + 2), never a header.
Reading it as a 2*NN + 16 header is what made walks drift — the
"variable-prefix segment descriptors" reported earlier today were not a
format feature, just walker drift of exactly
4 - (old_stop - true_record_start), on all 25 affected files.
Measured on the production snapshot:
all four channels equal length 156/1388 -> 1388/1388
ASCII sample-count exact 72/75 -> 75/75
ASCII fully exact 70/75 -> 73/75
device PPV waveform (live) 1288/1306 -> 1306/1306 (mean err 0.00000)
device PPV histogram (live) 4434/4459 -> 4458/4459
Also eliminates the walker-over-read class: 24 of those 35 files were
histograms that read_blastware_file fed to the waveform codec first; the
old walker accepted them and returned garbage (one yielded 98,923
"intervals"), while the record-chain decoder returns None so they fall
through to histogram_codec.
00 03 records are DECODED, not skipped. Skipping them silently shifts
the time base of everything after them on that channel — BE9558/
K558LOF2.820W had MicL displaced by exactly 512 samples with nothing
marking the gap.
Footer detection now prefers the 0e 08 candidate whose body yields a
chain terminating on 0x06; the signature can occur inside a sample
stream. Blast radius 1 file of 1388.
The superseded model survives as decode_waveform_legacy, pinned by
micromate/idf_file.py: its Thor IDFW body-offset search trial-decodes
candidates and keeps whichever yields the most samples, so the new
decoder returning None where the old returned garbage changes that
heuristic's winner. Deferred until that search uses the record chain.
Tests: 253 passed (+11), failure list unchanged from baseline. The 9
tests pinning the superseded model are retargeted at
decode_waveform_legacy, which still implements it.
NOTE: stored .h5 files need regenerating — nearly all get longer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
backfill_event_shape.py skipped rows whose .h5 produced no shape and left
the previously stored value in place. A stale shape outlives the decode
it came from and silently feeds the false-trigger detector.
Found while re-running the backfill after the histogram codec fix: 493
rows in the prod snapshot were carrying shape metrics that no longer
matched their .h5 — e.g. BE17353/S353LDOK.XZ0H held crest_factor from a
223-sample decode while its .h5 holds a single interval. These predate
today's work (present in the pre-32000 snapshot), so this is pre-existing
behaviour rather than fallout from the codec fixes.
Now NULLs shape_crest_factor / shape_near_peak_count / shape_sample_count
/ shape_axis in that case and reports a `cleared_stale` count. Verified
on the snapshot: 493 cleared, 0 stale rows remaining, 11570 rows matching
their .h5 exactly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Two errors in the series-3 histogram block model, both found by diffing
against the per-interval data table in the preserved Blastware ASCII
exports (1211 files in the prod snapshot — far stronger ground truth
than the header PPV used previously).
1. The block is uniformly BIG-ENDIAN. Peaks and half-periods are uint16
BE (T_peak [5:7], T_halfperiod [7:9], V_peak [9:11], V_halfperiod
[11:13], L_peak [13:15], L_halfperiod [15:17], M_peak [17:19],
M_halfperiod [19:21]); only block_ctr [2:4] is little-endian.
The old uint8-peak model silently CLIPPED any peak above 1.275 in/s:
the final interval of BE18193/T193LQ9K.OE0H reads 8.270 in/s in BW's
export (1654 counts = 0x0676) and decoded as 0x76 = 118 = 0.590.
The byte documented as a per-channel "annotation" was never an
annotation — it is the half-period's high byte, which is exactly why
it was non-zero on the sub-Hz intervals BW renders as "<1.0".
The marker is block[4] alone. Testing [4:6] as a uint16 LE marker
forced block[5] == 0, which is what capped the peak at one byte.
2. The final block of each stream carries tail 9c 06 00 42 instead of
1e 0a 00 00, and holds arbitrary bytes at [21:23]. Rejecting it
dropped the last interval of nearly every histogram — frequently the
interval holding the event peak, so the file's PPV read low.
Verified end to end through the production path: 1211/1211 histograms
decode exactly (interval count + every per-interval peak), plus 842,442
per-interval frequency comparisons with zero mismatches. Previously
1 of 1196 files was fully correct.
decode_histogram_body_full records expose `is_terminal` in place of the
removed `annotations` tuple. +6 tests. No regressions: full-suite
failure list unchanged from baseline.
NOTE: stored histogram .h5 files need regenerating to pick this up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Re-measured properly. The first pass compared h5 max against the ASCII
header PPV and reported "26% of channels miss the peak". The histogram
ASCII actually carries a full per-interval data table (Tran/Vert/Long
peak + freq + PVS per interval), which is real ground truth, so the
comparison should have been per-interval from the start.
Per-interval result, n=1196 series-3 histograms:
- decoded VALUES are right: 1031/1196 (86%) match within 1 LSB across
the overlapping prefix
- the interval COUNT is short in 1195 of 1196 files: median 1 missing,
1088 short by 1-2, 65 by 3-10, 39 by 11-100, 3 by >100 (max 205)
- decoded max falls below the device PPV in 169/1196 files (14%), not
26% — that happens when a dropped interval held the peak
So it is a termination bug in histogram_codec.decode_histogram_body,
the same family as the waveform-walker truncation fixed earlier today,
rather than mis-decoded interval values. Series-3 only; there is no
preserved series-4 ASCII in the snapshot to compare against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Operator report: attaching a different geophone to an affected unit
makes the offset go away. That rules out the unit's analog front-end
and any stored per-channel zero constant (a constant lives in the unit
and would survive a sensor swap).
The stored data agrees — MicL, a separate transducer on its own cable,
shows no offset during either episode (|mean|/peak 0.17 and 0.02) while
the geo channels on the same unit at the same moment are pinned.
Two distinct sensor-side patterns recorded:
BE18438 Vert 0.97, Tran 0.16, Long 0.18 -> one conductor pair
BE9558 Long 0.99, Tran 0.90, Vert 0.81 -> shared return / ground
Candidate mechanisms narrowed to three, since a geophone coil is passive
and cannot generate sustained DC: galvanic corrosion at a connector or
splice (matches the ~46 mV referred to the ADC input), a leakage path to
shield, or changed coil DC resistance interacting with the amplifier's
input bias current.
Also records the confound: swapping a sensor requires a monitoring
restart, and these units run Sensor Check "Before monitoring", so the
restart re-zeros too. The swap does not cleanly separate "new sensor"
from "the restart re-zeroed it". Controls and the single best
measurement (open-circuit DC across the suspect connector) documented.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Brings the protocol reference, CLAUDE.md and the codec RE status doc up
to date with everything confirmed in this pass.
instantel_protocol_reference.md
- Changelog row for the five findings.
- S7.6.1: scope table showing the 32000 scale correction applies to
series-3 waveform, series-3 histogram and series-4 Thor alike, with
the measured before/after ratios for each.
- S15: closed "Full channel ID mapping in SUB 5A stream" — resolved by
the segment-header channel id ([channel][00][00][segment], 0x46=Tran
0x47=Vert 0x48=Long 0x49=MicL, 1697/1697 verified). Four new open
questions: variable-prefix segment descriptors, the histogram codec
missing peak intervals (26% of channels), UM-series IDF decoding
~1000x low, and the Thor per-count LSB residual.
- NEW Appendix E — Known Device Faults. Documents the field-observed
"offset" fault: symptom, why it floods the ACH queue (pedestal
exceeds the unit's own geo trigger level), the episode table, the
detection rule that works, what the data rules out (not the
geophone, not the battery, not environmental, not condensation),
and the two remaining candidate mechanisms with the test that
separates them. Explicitly flags that it is NOT a decode artifact,
since that mistake has already been made once.
CLAUDE.md
- Body-codec section: the four framing cases and the channel-id
finding, with the corpus result.
- "What's NOT solved": replaced the stale walker-edge-cases bullet
with the four genuinely open items.
waveform_codec_re_status.md
- Scale scope table matching the protocol reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
274 series-3 waveform events across 6 episodes on 2 units (BE9558,
BE18438) whose dominant geo axis sits pinned at a DC offset above that
unit's own geo trigger level — the "offset" hardware fault that makes a
unit retrigger continuously and flood the ACH queue.
Detection rule: dominant-axis |mean|/peak > 0.7 AND |mean| >= 0.9 x the
unit's geo trigger level. Bare |mean|/peak is useless on quiet events —
a trace at the 0.010 in/s noise floor clears any ratio threshold.
Not a decode artifact: these reproduce exactly in Blastware's own ASCII
export. Kept as the starting point for the archive-wide analysis.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
The scale lives in _samples_to_float, which every event passes through
regardless of source codec, so waveforms, histograms and Thor IDF events
were all 2.34% low — not just waveforms. Verified after regeneration:
series-3 histogram peaks vs ASCII reports now median 1.0000 across 1137
comparisons (0.9766 under 32768); series-4 peaks vs device peaks moved
from median 0.960 to 0.983 across 1468.
The four block-framing fixes remain waveform-only; histogram_codec is
untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
Two independent bugs, both found by diffing 75 production events against
their preserved Blastware ASCII exports (<store>/<serial>/<file>_ASCII.TXT).
1. Geo full scale was wrong — every geophone reading was 2.34% low.
The codec emits geo samples in 16-count units with a documented LSB of
exactly 0.005 in/s, and decoded_to_adc_counts multiplies by 16, so one
ADC count is 0.005/16 in/s and 10.000 in/s is 10.0/(0.005/16) = 32000
counts. sfm/event_hdf5.py and minimateplus/event_file_io.py both
divided by 32768 (2^15), scaling every sample and derived peak down by
1 - 32000/32768. The error scales with amplitude, so it was invisible
on quiet events and worst on the loud ones that matter for compliance.
Mic is unaffected (it back-solves its scale from the device peak).
216 per-channel comparisons: 32768 -> 151/216 exact, worst error 0.238
in/s on a 10 in/s event; 32000 -> 216/216 exact, worst 0.005 = 1 LSB.
2. walk_body silently truncated channels on four unhandled framing cases.
An unrecognised tag ends the walk and decode_waveform_v2 returns
whatever it got, so this surfaced as short channels, never an error:
- wide-NN RLE `0X NN` (runs longer than 252 samples)
- `30 NN` with NN > 0x10 (the old cap was arbitrary)
- variable-width `40 NN` headers: NN counts previous-channel
continuation deltas, so the header is 2*NN + 16 bytes; `40 01`
and `40 03` occur alongside `40 02`
- tagless segment headers: no `40 NN` tag at all, just the 14-byte
tail [field2:2][len:2][channel_id:4][marker:2][anchors:4]
Also: the header field documented as a "monotonic uint32 LE counter" is
really [channel_id][00][00][segment_index], with 0x46=Tran 0x47=Vert
0x48=Long 0x49=MicL — verified on 1697/1697 segment headers, zero
disagreements. decode_waveform_v2 now takes the channel from that field
instead of rotation position, which was fragile: one missed header
desynced every channel after it.
parse_segment_header now returns n_prev_deltas/prev_deltas/marker/
anchors/channel/segment_index; the old fixed_pattern (02 00 00 01)
conflated the 2-byte marker with the first anchor.
Ground-truth corpus, end to end through the production path:
exact 37 -> 72, truncated 23 -> 3, full-length value errors 15 -> 0.
Store-wide, 729 of 1388 series-3 waveform events decode differently and
728 gain samples; the scale fix changes float values on all of them, so
stored .h5 files need regenerating.
Still open: 3 events truncate at a header variant with a variable-width
prefix (2/4/6 bytes) before the channel id and an `01 00` marker.
Documented in docs/instantel_protocol_reference.md with byte offsets.
+20 tests. No regressions: the byte-exact fixture suite still passes and
the full-suite failure list is unchanged from baseline (16 pre-existing
failures from gitignored fixtures).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
set_false_trigger now clears reviewed_real when flagging false_trigger=True
(mirrors update_event_review's exclusivity), and the quick
PATCH /db/events/{id}/false_trigger endpoint now calls
propagate_review_to_twins after the flag write, matching the sidecar PATCH
path's try/except-with-log.warning pattern. Previously the quick path could
leave both flags set and never touched twins.
Also corrects the v0.25.0 CHANGELOG bullet (exclusivity was not actually
enforced on the quick path until this commit) and adds a caveat comment on
find_twins about rare clamped/saturated-PVS false-positive twin matches.
7-task TDD plan: shape DSP module (crest factor + points-near-peak), shape_*
columns + auto-migrate, insert_events persistence, ingest population in the
save paths, backfill script, /db/events exposure + v0.24.0 bump. Phase B
(terra-view scoring/UI/review) gets its own plan once this feed is live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
Full-snapshot bundle (SFM side): document the /db/snapshot,
/db/waveforms/recent.zip and gated /db/restore endpoints. Bump package
version 0.21.1 -> 0.22.0 (pyproject), README version badge + history, and
CHANGELOG. TOOL_VERSION (codec/output provenance) left at 0.21.1 on purpose
— the event codec is unchanged, so existing sidecars must not be flagged
stale for re-backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyT5m9xppYmoVGmCCVVV6X
Two gaps in backfill_thor_events.py that left old Thor events showing
stale charts after a v0.21.1 backfill pass:
1. IDFH events were skipped from .h5 regeneration (the "have decoded
samples" gate was IDFW-only). Histograms kept their pre-v0.21.1
.h5 — written from raw_samples = None, which the renderer turned
into a near-empty bar chart, or for older events the dB(L)-as-pseudo-
psi mic scale that produced "107.7 psi" peaks (atomic-bomb level
instead of footstep level). Fix: synthesise the same 1-sample-per-
interval array save_imported_idf v0.21.1 uses (peak ADC count per
channel per interval) so the renderer's bar-chart grouping has
data to work with.
2. The IDFW h5 path didn't merge binary_peaks.mic_pspl_psi onto the
IdfEvent before to_minimateplus_event(). The live save_imported_idf
does this merge — without it, IdfEvent.from_report() only sees the
.txt's dB(L) value, the bridge falls back to the dBL→psi formula
(instead of the binary-accurate 2.14e-6 psi/count value), and the
h5 writer's per-count mic factor lands on a less-correct value.
Fix: same merge the live ingest does (lift res.event.peaks.mic_pspl_psi
onto idf_event.peaks before the bridge call).
Verified against UM6047_20250804190047.IDFH (250-interval prod
histogram): 250 intervals decode, mic_pspl_psi = 2.78e-5 (was being
treated as dB(L)=107.7 in the old h5).
Operator: re-run after deploy. `docker compose exec sfm python
scripts/backfill_thor_events.py` is idempotent — the existing version
check still skips events already at the new TOOL_VERSION, and review
state + captured_at are preserved on the second pass.
IDFH now synthesises a 1-sample-per-interval array from the binary intervals and writes an .h5 so the existing renderer works unchanged. Each "sample" is the per-interval peak ADC count → h5_value = count × geo_fs/32768 yields the right bar height.
Refreshes the bw_report sidecar block + .h5 waveform files for Thor
events ingested before the v0.21.0 adapter wiring + the bee1185 codec
fix. Those events landed with extensions.idf_report only (no
bw_report, no .h5 for IDFW) — symptom on the UI side: the modal chart
404'd on /waveform.json and the PDF rendered from DB-only fields
without sensor self-check, full per-channel breakdown, or mic dB(L).
Walks <store>/<serial>/<filename>:
- Reads the existing sidecar (preserves review state + captured_at)
- Re-runs read_idf_file() on the binary bytes (passes data=
kwarg so codec doesn't try the broken bare-path Path.read_bytes)
- Reads extensions.idf_report from the existing sidecar
- Runs build_bw_report_from_idf adapter
- Writes refreshed sidecar with bw_report + bumped tool_version,
preserving review block and original captured_at
- For IDFW: regenerates .h5 by bridging IdfEvent.from_report ->
to_minimateplus_event -> write_event_hdf5 (mirrors save_imported_idf
steps 4-7)
- IDFH events skip .h5 (histograms have no per-sample data)
Skips events already at current TOOL_VERSION with bw_report present.
--force overrides. --skip-hdf5 limits to sidecar-only refresh.
--dry-run for preview.
Validated against the prod-snap waveform store: 3,815 Thor sidecars
refreshed cleanly with 0 errors, 462 IDFW .h5 files written, 2 skipped
(binaries with no sidecar — backfill doesn't conjure events from
nothing). Verified one originally-broken IDFW event now serves
waveform.json (200, 168KB) and a fully populated PDF (119KB vs the
previous 56KB sparse output).
Operator workflow on prod:
docker exec <sfm-container> python3 /app/scripts/backfill_thor_events.py --dry-run
# Inspect counts, then for real:
docker exec <sfm-container> python3 /app/scripts/backfill_thor_events.py
Idempotent — re-running it is a no-op once everything's at the current
TOOL_VERSION.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes to the per-channel stats block:
1. Pin the stats table's position via an explicit bbox= on
ax.table() so the bottom edge is at a known axes-fraction Y.
The previous loc="upper left" + tbl.scale(1, 1.4) combo let
matplotlib choose row heights based on text size, which made the
table extend further below the axes than the hard-coded PVS line
at y=-0.08 expected. Result was the "Peak Vector Sum X in/s"
string landing horizontally inside the Peak Displacement row.
With bbox=[0, 1-N*0.12, 0.80, N*0.12] the table is pinned to a
precise rectangle (12% axes-fraction per row × N rows tall).
_draw_stats_table now stashes the bottom Y on the axes for the
PVS helper to reference, so the geometry stays in sync.
2. Center PVS horizontally (ha="center" at x=0.5 instead of ha="left"
at x=0). The previous left-edge alignment put PVS at the same
X as the label column, which read as "off-center" once the rest
of the stats data was column-aligned further right.
3. Drop the "NA: Not Applicable" caption. It existed to explain
"—" placeholder cells, but "—" is universally understood and the
caption was always visually squished against the PVS line below.
Less cruft on the page; one fewer position to manage.
Verified against a real BE12599 histogram event (5 data rows) and
a real UM12947 IDFW waveform event (6 data rows) — both layouts
clear the table cleanly with no overlap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bug shipped in v0.21.0: save_imported_idf called read_idf_file()
with `source_path` (a bare filename like "UM12947_….IDFW") BEFORE
writing the binary to disk. The codec did Path(path).read_bytes()
which resolved relative to /app and hit FileNotFoundError. The
error was caught + logged as a warning, and ingest fell back to
.txt-only — events still landed in the DB but lost the bw_report
block + .h5 waveform that the codec was supposed to produce.
Observed during a full re-forward from thor-watcher on 2026-05-29:
every Thor event logged "binary codec failed for X: [Errno 2] No
such file or directory" and got binary_decoded=False.
Fix:
- read_idf_file() gains a `data: Optional[bytes]` kwarg. When
supplied, skips the disk read and decodes the provided bytes
directly. `path` stays required (used for filename in error
messages + .IDFH vs .IDFW suffix detection); only the read is
conditional. Backward compatible — existing positional callers
(CLI scripts, tests) continue to work unchanged.
- save_imported_idf passes `data=idf_bytes` since the bytes are
already in memory from the multipart upload. Filesystem write
still happens at step 5 of the existing flow; codec just no
longer depends on it.
Verified end-to-end against UM11719_20231219162723.IDFW from the
example-data corpus: ingest endpoint returns inserted=1, log line
shows binary_decoded=True + h5=...IDFW.h5, no warnings.
Re-forward existing Thor events from thor-watcher after deploy to
backfill the bw_report block — UPSERT preserves review state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Matches Terra-View's event-modal relabel from the same iteration.
Wording was already clearer here than in Terra-View's "Captured at",
but using identical text across both surfaces means operators see the
same label whether they're in the native modal or the standalone
webapp.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Documents two commits that landed on dev since v0.20.0:
9b71ead series 4 codec work, initial decode success
micromate/idf_file.read_idf_file() decodes both IDFW
(waveform; 87-99% sample fidelity reusing
decode_waveform_v2 at offset 0x0f1f) and IDFH (histogram;
dedicated segment-based decoder, all 859 corpus files
decode, 181,071 intervals total).
9fd52dd feat: add thor report generation, pdf generation
micromate/idf_to_bw_report.py adapter projects parsed
Thor data into the bw_report sidecar shape so Thor
events flow through sfm/report_pdf.py without a
separate renderer. Wired into save_imported_idf.
Net effect: a Thor event ingested via /db/import/idf_file now
lands with the same fidelity as a BW event, gets a per-event PDF
on demand, and renders in Terra-View's modal chart using the same
plotting code as a BW event.
Roadmap items closed:
- Binary .IDFW / .IDFH codec (was pending)
- Series IV (Thor IDF) binary codec reverse-engineering
Companion: Terra-View v0.13.0 ships in parallel and closes Phase 1
of the SFM integration. No API changes in seismo-relay for that
piece — Terra-View just consumes existing endpoints better.
Bumps:
- pyproject.toml 0.20.0 → 0.21.0
- minimateplus.event_file_io.TOOL_VERSION 0.20.0 → 0.21.0
(any subsequent backfill_sidecars.py --force will re-stamp
existing sidecars; expected + harmless)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes out the Event-Report PDF iteration started in v0.17.x and ships
the parser fixes the real-world events were tripping over.
Today's additions on top of the pre-v0.20 unreleased body:
- Server-wide display TZ via the TZ env var (default America/New_York
on prod). Affects server logs, the PDF report's "Created" footer,
matplotlib datetime axes. DB columns stay UTC. Dockerfile now
installs tzdata.
- ZC Freq "above-range" handling — parser stores 100.0 +
zc_freq_above_range flag for BW's ">100 Hz" marker. Renders as
>100 in the PDF stats table, both modals (inline on webapp Peaks,
new column on event-browser table).
- scripts/backfill_sidecars.py --reparse-txt — re-runs the current
parser against the preserved _ASCII.TXT and overwrites the
sidecar's bw_report block. Lets parser fixes reach old events
without re-forwarding. Validated end-to-end against ~10k prod
events.
Fixes shipped today:
- histogram_interval_size_s missing from ReportData → every
histogram PDF render 500'd.
- Histogram PDF geo channels now share a nice-quantized y-axis
(0.005-LSB-aware 1-2-5 step sequence) instead of auto-scaling
per channel + inventing sub-LSB "0.003 in/s/div" footer labels.
Roadmap delta: closes the BW ASCII parser "PPV-miss on some TXT
formats", "histogram-specific structural fields", and ">100 Hz value
parsing" items. Adds a new entry for the byte[5]==0 histogram body
sub-format observed on S353 events.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
serversdownandClaude Opus 5