fix(codec): geo full scale is 32000 counts; 4 walker framing cases; channel-id from header
Two independent bugs, both found by diffing 75 production events against
their preserved Blastware ASCII exports (<store>/<serial>/<file>_ASCII.TXT).
1. Geo full scale was wrong — every geophone reading was 2.34% low.
The codec emits geo samples in 16-count units with a documented LSB of
exactly 0.005 in/s, and decoded_to_adc_counts multiplies by 16, so one
ADC count is 0.005/16 in/s and 10.000 in/s is 10.0/(0.005/16) = 32000
counts. sfm/event_hdf5.py and minimateplus/event_file_io.py both
divided by 32768 (2^15), scaling every sample and derived peak down by
1 - 32000/32768. The error scales with amplitude, so it was invisible
on quiet events and worst on the loud ones that matter for compliance.
Mic is unaffected (it back-solves its scale from the device peak).
216 per-channel comparisons: 32768 -> 151/216 exact, worst error 0.238
in/s on a 10 in/s event; 32000 -> 216/216 exact, worst 0.005 = 1 LSB.
2. walk_body silently truncated channels on four unhandled framing cases.
An unrecognised tag ends the walk and decode_waveform_v2 returns
whatever it got, so this surfaced as short channels, never an error:
- wide-NN RLE `0X NN` (runs longer than 252 samples)
- `30 NN` with NN > 0x10 (the old cap was arbitrary)
- variable-width `40 NN` headers: NN counts previous-channel
continuation deltas, so the header is 2*NN + 16 bytes; `40 01`
and `40 03` occur alongside `40 02`
- tagless segment headers: no `40 NN` tag at all, just the 14-byte
tail [field2:2][len:2][channel_id:4][marker:2][anchors:4]
Also: the header field documented as a "monotonic uint32 LE counter" is
really [channel_id][00][00][segment_index], with 0x46=Tran 0x47=Vert
0x48=Long 0x49=MicL — verified on 1697/1697 segment headers, zero
disagreements. decode_waveform_v2 now takes the channel from that field
instead of rotation position, which was fragile: one missed header
desynced every channel after it.
parse_segment_header now returns n_prev_deltas/prev_deltas/marker/
anchors/channel/segment_index; the old fixed_pattern (02 00 00 01)
conflated the 2-byte marker with the first anchor.
Ground-truth corpus, end to end through the production path:
exact 37 -> 72, truncated 23 -> 3, full-length value errors 15 -> 0.
Store-wide, 729 of 1388 series-3 waveform events decode differently and
728 gain samples; the scale fix changes float values on all of them, so
stored .h5 files need regenerating.
Still open: 3 events truncate at a header variant with a variable-width
prefix (2/4/6 bytes) before the channel id and an `01 00` marker.
Documented in docs/instantel_protocol_reference.md with byte offsets.
+20 tests. No regressions: the byte-exact fixture suite still passes and
the full-suite failure list is unchanged from baseline (16 pre-existing
failures from gitignored fixtures).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -6,6 +6,99 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Geophone full scale is 32000 ADC counts, not 32768 — every geo reading was
|
||||
2.3% low.** The verified body codec emits geo samples in 16-count units whose
|
||||
documented LSB is exactly 0.005 in/s, and `decoded_to_adc_counts` multiplies
|
||||
by 16, so one ADC count is `0.005/16` in/s and Normal range (10.000 in/s) is
|
||||
`10.0 / (0.005/16)` = **32000** counts. Both `sfm/event_hdf5.py` and
|
||||
`minimateplus/event_file_io.py` divided by 32768, scaling every geophone
|
||||
sample and every derived peak down by `1 - 32000/32768` = **2.34%**.
|
||||
|
||||
Measured against 216 per-channel comparisons with preserved Blastware ASCII
|
||||
exports: **32768 → 151/216 exact** (worst error 0.238 in/s on a 10 in/s
|
||||
event); **32000 → 216/216 exact**, worst error 0.005 in/s (exactly 1 LSB —
|
||||
pure quantization). The error scales with amplitude, so it was invisible on
|
||||
quiet events and worst on the loud ones that matter for compliance.
|
||||
|
||||
The mic path is unaffected — it back-solves its own per-count factor from the
|
||||
device-reported peak.
|
||||
|
||||
- **Series-3 waveform codec: four block-framing cases caused silent channel
|
||||
truncation.** `walk_body` hit its unknown-tag `break` mid-stream and every
|
||||
channel decoded after that point came out short — typically Vert/Long/MicL,
|
||||
sometimes at a third of their true length, with no error raised.
|
||||
|
||||
- **Wide-NN RLE `0X NN`** — the 12-bit NN encoding already handled for
|
||||
`1X NN` / `2X NN` also applies to the `00 NN` RLE tag. Runs longer than
|
||||
252 samples must use the wide form (e.g. `01 0c` = 268 repeats).
|
||||
- **`30 NN` with NN > 0x10** — the `0 < NN <= 0x10` guard was arbitrary;
|
||||
data-section `30 NN` blocks reach at least NN = 0x18. The length formula
|
||||
(`NN × 1.5 + 2`) was already correct.
|
||||
- **Variable-width `40 NN` segment headers** — NN is the *count of
|
||||
previous-channel continuation deltas*, so the header is `2 × NN + 16`
|
||||
bytes and every field after the deltas shifts by `2 × NN`. Only `40 02`
|
||||
(20 bytes) was handled; `40 01` (18) and `40 03` (22) both occur.
|
||||
- **Tagless segment headers** — a segment header can appear with no
|
||||
`40 NN` tag at all: just the 14-byte tail
|
||||
`[field2:2][len:2][channel_id:4][marker:2][anchors:4]`. This is the NN=0
|
||||
case (no continuation deltas needed, so no tag and no delta bytes). It is
|
||||
where the walk stopped in 7 of the 8 events still truncating after the
|
||||
first three fixes.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Segment channel now comes from the header's own channel-id byte** rather
|
||||
than from rotation position. The field previously documented as a
|
||||
"monotonic uint32 LE counter" is really `[channel][00][00][segment_index]`
|
||||
with `0x46`=Tran `0x47`=Vert `0x48`=Long `0x49`=MicL — verified on
|
||||
**1697 of 1697** segment headers across the ground-truth corpus with zero
|
||||
disagreements. Rotation-by-position is kept only as a fallback for unknown
|
||||
ids; it was fragile because a single missed or extra header (exactly what
|
||||
tagless headers caused) desynced every channel after it.
|
||||
|
||||
- **`parse_segment_header` return shape** — now `n_prev_deltas`,
|
||||
`prev_deltas`, `marker`, `anchors`, `channel`, `segment_index` in place of
|
||||
the fixed-offset `anchor_bytes` / `fixed_pattern` / `tail` keys. The old
|
||||
`fixed_pattern` (`02 00 00 01`) conflated the 2-byte constant marker with
|
||||
the first anchor. `counter` is retained as the raw uint32 of the id field.
|
||||
|
||||
### Verification
|
||||
|
||||
Against the 75 ground-truth events (BW binary paired with its preserved
|
||||
`_ASCII.TXT` export), decoding end-to-end through the production path:
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| exact (full length, within 1 LSB) | 37 | **72** |
|
||||
| truncated | 23 | **3** |
|
||||
| full length, value error > 2 LSB | 15 | **0** |
|
||||
|
||||
Worst remaining error among the 72: 0.0050 in/s = exactly 1 LSB.
|
||||
|
||||
No regressions — the byte-exact fixture suite still passes, and the full-suite
|
||||
failure list is unchanged from baseline (16 pre-existing failures from
|
||||
gitignored fixtures).
|
||||
|
||||
### Notes
|
||||
|
||||
- **The "DC offset" symptom is _not_ a decode bug.** Events whose geo trace
|
||||
sits at a constant level instead of oscillating around zero
|
||||
(dominant-axis `|mean| / peak` >> 0) reproduce *exactly* in Blastware's own
|
||||
ASCII export — e.g. `BE12599/N599LQD7.8E0W` Tran reads mean +0.345,
|
||||
min +0.335, max +0.355 in both. It is a known recurring hardware fault (the
|
||||
operators call it an "offset"): the affected channel's baseline exceeds the
|
||||
unit's own geo trigger level, so the unit retriggers continuously and floods
|
||||
the ACH queue with garbage events. Store-wide it affects 2 units of 21 across
|
||||
6 episodes; see `scratch/offset_candidates.csv` and the project memory notes.
|
||||
|
||||
- **Still open:** 3 of 75 ground-truth events truncate at a segment-header
|
||||
variant with a variable-width prefix before the channel-id field (2, 4 or 6
|
||||
bytes observed) and an `01 00` marker instead of `02 00`. See the protocol
|
||||
reference, "Unmapped: variable-prefix segment descriptors". Examples:
|
||||
`BE12599/N599LPNB.JF0W` at body offset 1155, `BE9558/K558LOF2.820W` at 1485.
|
||||
|
||||
---
|
||||
|
||||
## v0.25.0 — 2026-08-25
|
||||
|
||||
Reference in New Issue
Block a user