v0.27.0 Decoder fixes, offset exploration and testing. #34
+12
-7
@@ -19,14 +19,19 @@ All notable changes to seismo-relay are documented here.
|
|||||||
false positives that once mis-dispatched 9,082 files, is unchanged.
|
false positives that once mis-dispatched 9,082 files, is unchanged.
|
||||||
|
|
||||||
Found by decoding the full DL2 archive against its preserved Blastware ASCII
|
Found by decoding the full DL2 archive against its preserved Blastware ASCII
|
||||||
exports. Across 127,035 histogram binaries the fix recovers **8 files** (strides
|
exports. Across **63,535 unique** histogram binaries the fix recovers **4 files** —
|
||||||
92, 252 and 612; units BE18193, BE18191, BE9557, BE9440) with **zero** files
|
`K440HJCN.3C0H` and `K557IF1U.8K0H` (stride 252), `T191HVNP.0S0H` (92) and
|
||||||
regressed. Verification over all 14,340 archive pairs goes 14,337 → 14,338 exact,
|
`T193L0XM.CI0H` (612) — with **zero** files regressed. Verification over all
|
||||||
the only remainder being two series-4 IDF files that belong to a different codec.
|
14,340 archive pairs goes 14,337 → 14,338 exact, the only remainder being two
|
||||||
|
series-4 IDF files that belong to a different codec.
|
||||||
|
|
||||||
⚠ Prod stores hold `.h5` files generated before this fix. The 8 affected events
|
(The DL2 export keeps a byte-identical `Sent/` mirror of its root, so a naive
|
||||||
stay empty until `backfill_sidecars.py` is re-run — not urgent at 8 files, and
|
walk double-counts every binary — 127,035 paths are 63,535 distinct files. The
|
||||||
worth folding into the next backfill rather than doing one for this alone.
|
ASCII exports are *not* mirrored, so the 14,340 pair count is already distinct.)
|
||||||
|
|
||||||
|
⚠ Prod stores hold `.h5` files generated before this fix. Those 4 events stay
|
||||||
|
empty until `backfill_sidecars.py` is re-run — not worth a two-hour prod backfill
|
||||||
|
on its own; fold it into the next one.
|
||||||
|
|
||||||
- **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real
|
- **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real
|
||||||
trigger is recorded twice — as a triggered waveform (stamped at the trigger instant)
|
trigger is recorded twice — as a triggered waveform (stamped at the trigger instant)
|
||||||
|
|||||||
@@ -107,8 +107,15 @@ def main():
|
|||||||
ap.add_argument("--out", required=True)
|
ap.add_argument("--out", required=True)
|
||||||
a = ap.parse_args()
|
a = ap.parse_args()
|
||||||
|
|
||||||
files = [p for p in Path(a.dir).rglob("*") if p.is_file() and _WAVE_RE.search(p.name)]
|
# The DL2 export keeps a byte-identical `Sent/` mirror of the root, so
|
||||||
files.sort()
|
# enumerate paths but keep only the first occurrence of each basename —
|
||||||
|
# otherwise every event is counted twice.
|
||||||
|
seen = set()
|
||||||
|
files = []
|
||||||
|
for q in sorted(Path(a.dir).rglob("*")):
|
||||||
|
if q.is_file() and _WAVE_RE.search(q.name) and q.name not in seen:
|
||||||
|
seen.add(q.name)
|
||||||
|
files.append(q)
|
||||||
if a.limit:
|
if a.limit:
|
||||||
files = files[: a.limit]
|
files = files[: a.limit]
|
||||||
print(f"waveform binaries to scan: {len(files)}", flush=True)
|
print(f"waveform binaries to scan: {len(files)}", flush=True)
|
||||||
|
|||||||
Reference in New Issue
Block a user