fix(codec): geo full scale is 32000 counts; 4 walker framing cases; channel-id from header
Two independent bugs, both found by diffing 75 production events against
their preserved Blastware ASCII exports (<store>/<serial>/<file>_ASCII.TXT).
1. Geo full scale was wrong — every geophone reading was 2.34% low.
The codec emits geo samples in 16-count units with a documented LSB of
exactly 0.005 in/s, and decoded_to_adc_counts multiplies by 16, so one
ADC count is 0.005/16 in/s and 10.000 in/s is 10.0/(0.005/16) = 32000
counts. sfm/event_hdf5.py and minimateplus/event_file_io.py both
divided by 32768 (2^15), scaling every sample and derived peak down by
1 - 32000/32768. The error scales with amplitude, so it was invisible
on quiet events and worst on the loud ones that matter for compliance.
Mic is unaffected (it back-solves its scale from the device peak).
216 per-channel comparisons: 32768 -> 151/216 exact, worst error 0.238
in/s on a 10 in/s event; 32000 -> 216/216 exact, worst 0.005 = 1 LSB.
2. walk_body silently truncated channels on four unhandled framing cases.
An unrecognised tag ends the walk and decode_waveform_v2 returns
whatever it got, so this surfaced as short channels, never an error:
- wide-NN RLE `0X NN` (runs longer than 252 samples)
- `30 NN` with NN > 0x10 (the old cap was arbitrary)
- variable-width `40 NN` headers: NN counts previous-channel
continuation deltas, so the header is 2*NN + 16 bytes; `40 01`
and `40 03` occur alongside `40 02`
- tagless segment headers: no `40 NN` tag at all, just the 14-byte
tail [field2:2][len:2][channel_id:4][marker:2][anchors:4]
Also: the header field documented as a "monotonic uint32 LE counter" is
really [channel_id][00][00][segment_index], with 0x46=Tran 0x47=Vert
0x48=Long 0x49=MicL — verified on 1697/1697 segment headers, zero
disagreements. decode_waveform_v2 now takes the channel from that field
instead of rotation position, which was fragile: one missed header
desynced every channel after it.
parse_segment_header now returns n_prev_deltas/prev_deltas/marker/
anchors/channel/segment_index; the old fixed_pattern (02 00 00 01)
conflated the 2-byte marker with the first anchor.
Ground-truth corpus, end to end through the production path:
exact 37 -> 72, truncated 23 -> 3, full-length value errors 15 -> 0.
Store-wide, 729 of 1388 series-3 waveform events decode differently and
728 gain samples; the scale fix changes float values on all of them, so
stored .h5 files need regenerating.
Still open: 3 events truncate at a header variant with a variable-width
prefix (2/4/6 bytes) before the channel id and an `01 00` marker.
Documented in docs/instantel_protocol_reference.md with byte offsets.
+20 tests. No regressions: the byte-exact fixture suite still passes and
the full-suite failure list is unchanged from baseline (16 pre-existing
failures from gitignored fixtures).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
This commit is contained in:
@@ -99,8 +99,14 @@ def test_hdf5_round_trip_preserves_metadata(tmp_path: Path):
|
||||
|
||||
|
||||
def test_hdf5_samples_in_physical_units_normal_range(tmp_path: Path):
|
||||
"""Vert hits ADC full-scale (32767) → with Normal range FS=10 in/s,
|
||||
the HDF5 sample value should be ≈ 10 * 32767/32768 in/s."""
|
||||
"""Vert hits 32767 ADC counts → with Normal range FS=10 in/s that is
|
||||
``10 * 32767/32000`` in/s.
|
||||
|
||||
Geo full scale is 32000 counts, not 32768 (see
|
||||
test_geo_full_scale_count_is_32000), so 32767 counts sits slightly
|
||||
ABOVE nominal full scale -- the ADC has headroom past 10.000 in/s,
|
||||
which is why Blastware reports peaks like 10.14 in/s. This test
|
||||
previously asserted the 32768 scale and was wrong by 2.3%."""
|
||||
ev = _make_event_with_samples()
|
||||
h5 = tmp_path / "n.h5"
|
||||
event_hdf5.write_event_hdf5(h5, ev, serial="BE11529", geo_range="normal")
|
||||
@@ -110,7 +116,7 @@ def test_hdf5_samples_in_physical_units_normal_range(tmp_path: Path):
|
||||
assert vert.dtype.name == "float32"
|
||||
assert max(abs(v) for v in vert) > 9.99 # full-scale ≈ 10.0
|
||||
# The dirac was at n//2 → 32767 ADC counts.
|
||||
expected_peak = 10.0 * 32767 / 32768
|
||||
expected_peak = 10.0 * 32767 / 32000
|
||||
assert abs(max(vert) - expected_peak) < 1e-3
|
||||
|
||||
|
||||
@@ -122,7 +128,7 @@ def test_hdf5_samples_in_physical_units_sensitive_range(tmp_path: Path):
|
||||
data = event_hdf5.read_event_hdf5(h5)
|
||||
|
||||
vert = data["samples"]["Vert"]
|
||||
expected_peak = 1.250 * 32767 / 32768
|
||||
expected_peak = 1.250 * 32767 / 32000
|
||||
assert abs(max(vert) - expected_peak) < 1e-4
|
||||
|
||||
|
||||
@@ -294,3 +300,35 @@ if __name__ == "__main__":
|
||||
failed += 1
|
||||
print(f"\n{passed} passed, {failed} failed")
|
||||
sys.exit(0 if failed == 0 else 1)
|
||||
|
||||
|
||||
# ── Geophone full-scale count ───────────────────────────────────────────────
|
||||
|
||||
def test_geo_full_scale_count_is_32000():
|
||||
"""Geo full scale is 32000 ADC counts, not 32768.
|
||||
|
||||
The verified body codec emits geo samples in 16-count units with a
|
||||
documented LSB of exactly 0.005 in/s, and ``decoded_to_adc_counts``
|
||||
multiplies by 16 — so one ADC count is 0.005/16 in/s and Normal range
|
||||
(10.000 in/s) is 10.0 / (0.005/16) = 32000 counts.
|
||||
|
||||
Using 32768 made every geophone reading 2.3% low (1 - 32000/32768).
|
||||
Confirmed 2026-08-25 against 216 channel comparisons with preserved
|
||||
Blastware ASCII exports: 32000 → 216/216 exact within 1 LSB;
|
||||
32768 → 151/216, worst error 0.238 in/s on a 10 in/s event.
|
||||
"""
|
||||
from sfm.event_hdf5 import _GEO_INT16_FS
|
||||
assert _GEO_INT16_FS == 32000.0
|
||||
|
||||
|
||||
def test_samples_to_float_lsb_is_exactly_5_milli_ips():
|
||||
"""One decoder unit (= 16 ADC counts) must be exactly 0.005 in/s."""
|
||||
from sfm.event_hdf5 import _samples_to_float
|
||||
out = _samples_to_float([16], 10.0)
|
||||
assert abs(float(out[0]) - 0.005) < 1e-9
|
||||
|
||||
|
||||
def test_samples_to_float_full_scale_count_maps_to_full_scale():
|
||||
from sfm.event_hdf5 import _samples_to_float
|
||||
assert abs(float(_samples_to_float([32000], 10.0)[0]) - 10.0) < 1e-4
|
||||
assert abs(float(_samples_to_float([32000], 1.25)[0]) - 1.25) < 1e-5
|
||||
|
||||
Reference in New Issue
Block a user