feat(histogram): decode multi-interval blocks — recovers 415 files
Sub-minute histogram intervals are packed several to a block so that
every block still covers exactly one minute of data:
interval intervals/block stride
1 minute 1 32 <- the standard big-endian block
15 s 4 92
2 s 30 612
stride = 12 + n * 20
Block = [00][segment][ctr uint16 LE][0a][00], then n x 20-byte records of
8 x uint16 LITTLE-endian values (T_peak, T_halfp, V_peak, V_halfp,
L_peak, L_halfp, M_peak, M_halfp) plus a 2-word tail whose first word is
0000 on every real interval, then a 6-byte block trailer.
The standard 32-byte block is BIG-endian; this variant is LITTLE-endian.
The tail-word check matters: a session ending mid-block leaves buffer
garbage in the remaining interval slots, which decoded as peaks
thousands of times the real value. Stride detection also requires at
least 2 records, since a 1-record block would have stride 32 and
collide with the standard block.
Recovers 415 files that decoded to nothing: 216 on BE18193 (2 s
intervals) and 199 on BE9440 (15 s). Before decoding to nothing they
were being accepted by the WAVEFORM codec, which returned garbage
peaking up to 400x the device-reported PPV.
Ground truth BE9440/K440L3AQ.T70H (5,710 intervals) matches its
Blastware ASCII export exactly: 17,130/17,130 geo peaks, 22,840/22,840
frequencies, 5,710/5,710 mic dB(L). Across all 455 affected files,
1,354/1,365 channel peaks (99.2%) match the device-reported PPV; the 11
that don't are under-reads on BE9440 where the walk stops early.
Fixture (binary + ASCII) saved under tests/fixtures/, which is
gitignored per repo practice — the ground-truth test skips when absent.
Tests: 258 passed, failure list unchanged from baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -8,6 +8,34 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Sub-minute histogram intervals are packed several to a block — 415 files
|
||||
recovered.** The device always writes one minute of data per block, so a
|
||||
shorter interval just means more intervals in a longer block:
|
||||
|
||||
| interval | intervals/block | stride |
|
||||
|---|---|---|
|
||||
| 1 min | 1 | 32 (the standard block) |
|
||||
| 15 s | 4 | 92 |
|
||||
| 2 s | 30 | 612 |
|
||||
|
||||
`stride = 12 + n * 20`. Each 20-byte record carries 8 × uint16
|
||||
**little**-endian values — peak and half-period per channel — plus a 2-word
|
||||
tail whose first word is `0000` on every real interval (a session ending
|
||||
mid-block leaves buffer garbage in the remaining slots, which decoded as
|
||||
peaks thousands of times the real value until that check was added).
|
||||
**The standard 32-byte block is big-endian; this variant is not.**
|
||||
|
||||
These 415 files (216 on BE18193 at 2 s intervals, 199 on BE9440 at 15 s)
|
||||
previously decoded to nothing at all — and before that were being accepted
|
||||
by the *waveform* codec, which returned garbage peaking up to 400× the
|
||||
device-reported PPV.
|
||||
|
||||
Ground truth `BE9440/K440L3AQ.T70H` — 5,710 intervals — matches its
|
||||
Blastware ASCII export on **17,130/17,130** geo peaks, **22,840/22,840**
|
||||
frequencies and **5,710/5,710** mic dB(L) values. Across all 455 affected
|
||||
files, **1,354/1,365 (99.2%)** channel peaks match the device-reported PPV;
|
||||
the 11 that don't are under-reads on BE9440 where the walk stops early.
|
||||
|
||||
- **`backfill_sidecars.py` now removes a stale `.h5` when nothing decodes.**
|
||||
It previously skipped the write "so we don't replace whatever's there with an
|
||||
empty placeholder", which silently preserved output from a superseded
|
||||
|
||||
Reference in New Issue
Block a user