fix(backfill): remove stale .h5 when nothing decodes; log the 415-file histogram variant
backfill_sidecars.py skipped the .h5 write when a file produced no samples, with the stated intent of not replacing it with an empty placeholder. That silently preserved output from a superseded decoder. After the record-chain fix, 415 histogram files stopped decoding (216 on BE18193, 199 on BE9440) but kept .h5 files whose peaks ran up to 400x the device-reported PPV. Those were feeding charts and the false-trigger detector with nothing marking them. The .h5 is now removed in that case and the run reports stale_h5_removed. Store-wide effect, series-3, decoded peak vs device-reported PPV: waveform 1307/1307 (100%), mean abs ratio error 0.00000 histogram 4434/4435 (100%) Both were 99% with a tail of 18 and 25 wrong files respectively. The 415 files are a genuine unmapped format variant, not a regression: their bodies open `00 00 00 01 0a 00` (valid block header, marker 0a at [4], block_ctr 256) but block[28:32] matches neither known tail, and no stride from 8 to 64 bytes places a marker at [4] consistently. Bodies are very large (one is 360,573 bytes). They were previously being decoded by the WAVEFORM codec, which accepted them and returned garbage - so the gap pre-dates today's work; the fix only exposed it. Logged as an open question in the protocol reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -8,6 +8,15 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`backfill_sidecars.py` now removes a stale `.h5` when nothing decodes.**
|
||||
It previously skipped the write "so we don't replace whatever's there with an
|
||||
empty placeholder", which silently preserved output from a superseded
|
||||
decoder. After the record-chain fix, 415 histogram files stopped decoding (an
|
||||
unmapped block variant on BE18193 and BE9440) but kept `.h5` files whose peaks
|
||||
ran up to **400× the device's own reported PPV** — garbage feeding the charts
|
||||
and the false-trigger detector with nothing marking it. Reports a
|
||||
`stale_h5_removed` count.
|
||||
|
||||
- **The series-3 waveform body is a RECORD CHAIN, not a tag stream — this
|
||||
supersedes the segment-header model, including the fixes made earlier the
|
||||
same day.**
|
||||
|
||||
Reference in New Issue
Block a user