diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ab9bf9..dd3626e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,14 +19,19 @@ All notable changes to seismo-relay are documented here. false positives that once mis-dispatched 9,082 files, is unchanged. Found by decoding the full DL2 archive against its preserved Blastware ASCII - exports. Across 127,035 histogram binaries the fix recovers **8 files** (strides - 92, 252 and 612; units BE18193, BE18191, BE9557, BE9440) with **zero** files - regressed. Verification over all 14,340 archive pairs goes 14,337 → 14,338 exact, - the only remainder being two series-4 IDF files that belong to a different codec. + exports. Across **63,535 unique** histogram binaries the fix recovers **4 files** — + `K440HJCN.3C0H` and `K557IF1U.8K0H` (stride 252), `T191HVNP.0S0H` (92) and + `T193L0XM.CI0H` (612) — with **zero** files regressed. Verification over all + 14,340 archive pairs goes 14,337 → 14,338 exact, the only remainder being two + series-4 IDF files that belong to a different codec. - ⚠ Prod stores hold `.h5` files generated before this fix. The 8 affected events - stay empty until `backfill_sidecars.py` is re-run — not urgent at 8 files, and - worth folding into the next backfill rather than doing one for this alone. + (The DL2 export keeps a byte-identical `Sent/` mirror of its root, so a naive + walk double-counts every binary — 127,035 paths are 63,535 distinct files. The + ASCII exports are *not* mirrored, so the 14,340 pair count is already distinct.) + + ⚠ Prod stores hold `.h5` files generated before this fix. Those 4 events stay + empty until `backfill_sidecars.py` is re-run — not worth a two-hour prod backfill + on its own; fold it into the next one. - **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real trigger is recorded twice — as a triggered waveform (stamped at the trigger instant) diff --git a/scratch/offset_scan.py b/scratch/offset_scan.py index 842bdcf..1b4e377 100644 --- a/scratch/offset_scan.py +++ b/scratch/offset_scan.py @@ -107,8 +107,15 @@ def main(): ap.add_argument("--out", required=True) a = ap.parse_args() - files = [p for p in Path(a.dir).rglob("*") if p.is_file() and _WAVE_RE.search(p.name)] - files.sort() + # The DL2 export keeps a byte-identical `Sent/` mirror of the root, so + # enumerate paths but keep only the first occurrence of each basename — + # otherwise every event is counted twice. + seen = set() + files = [] + for q in sorted(Path(a.dir).rglob("*")): + if q.is_file() and _WAVE_RE.search(q.name) and q.name not in seen: + seen.add(q.name) + files.append(q) if a.limit: files = files[: a.limit] print(f"waveform binaries to scan: {len(files)}", flush=True)