serversdown/seismo-relay
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(scratch): dedupe the DL2 Sent/ mirror; correct the recovered-file count

Browse Source 
The DL2 export keeps a byte-identical `Sent/` copy of its root, so walking it
counts every binary twice: 127,035 histogram paths are 63,535 distinct files,
and 13,077 waveform paths are 6,577. offset_scan.py now keeps the first
occurrence of each basename.

Corrects the previous commit's changelog claim of 8 recovered files — it is 4:
K440HJCN.3C0H and K557IF1U.8K0H (stride 252), T191HVNP.0S0H (92), T193L0XM.CI0H
(612). Still zero regressions. The per-unit breakdown reading exactly 2-2-2-2
should have given the doubling away.

The 14,338-exact verification result is unaffected: ASCII exports are not
mirrored (14,340 paths, 14,340 distinct names), and the harness enumerates
those rather than the binaries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
serversdown
2026-08-28 20:19:30 +00:00
co-authored by Claude Opus 5
parent 14e997b20c
commit 4839ddfa0e
2 changed files with 21 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+12 -7
View File
@@ -19,14 +19,19 @@ All notable changes to seismo-relay are documented here.
false positives that once mis-dispatched 9,082 files, is unchanged.
Found by decoding the full DL2 archive against its preserved Blastware ASCII
exports. Across 127,035 histogram binaries the fix recovers **8 files** (strides
92, 252 and 612; units BE18193, BE18191, BE9557, BE9440) with **zero** files
regressed. Verification over all 14,340 archive pairs goes 14,337 → 14,338 exact,
the only remainder being two series-4 IDF files that belong to a different codec.
exports. Across **63,535 unique** histogram binaries the fix recovers **4 files** —
`K440HJCN.3C0H` and `K557IF1U.8K0H` (stride 252), `T191HVNP.0S0H` (92) and
`T193L0XM.CI0H` (612) — with **zero** files regressed. Verification over all
14,340 archive pairs goes 14,337 → 14,338 exact, the only remainder being two
series-4 IDF files that belong to a different codec.
⚠ Prod stores hold `.h5` files generated before this fix. The 8 affected events
stay empty until `backfill_sidecars.py` is re-run — not urgent at 8 files, and
worth folding into the next backfill rather than doing one for this alone.
(The DL2 export keeps a byte-identical `Sent/` mirror of its root, so a naive
walk double-counts every binary — 127,035 paths are 63,535 distinct files. The
ASCII exports are *not* mirrored, so the 14,340 pair count is already distinct.)
⚠ Prod stores hold `.h5` files generated before this fix. Those 4 events stay
empty until `backfill_sidecars.py` is re-run — not worth a two-hour prod backfill
on its own; fold it into the next one.
- **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real
trigger is recorded twice — as a triggered waveform (stamped at the trigger instant)