update to 0.26.0. Big chonking update including 0.23, 0.24, and 0.25 as well. #33
@@ -8,6 +8,62 @@ All notable changes to seismo-relay are documented here.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **The series-3 waveform body is a RECORD CHAIN, not a tag stream — this
|
||||
supersedes the segment-header model, including the fixes made earlier the
|
||||
same day.**
|
||||
|
||||
Records are self-delimiting. `off+2` is a `uint16 BE` length and
|
||||
`next_record = off + 2 + len`; the chain ends on a record whose `chan_id` is
|
||||
`0x06`. `off+8` carries a 3-valued mode enum:
|
||||
|
||||
| mode | header | data section |
|
||||
|---|---|---|
|
||||
| `02 00` | 14 B | anchors, then **cumulative deltas** |
|
||||
| `01 00` | 10 B | no anchors, **absolute** values |
|
||||
| `00 03` | 10 B | **no tags at all** — raw 12-bit packed absolute |
|
||||
|
||||
**`40 NN` is an ordinary int16 BE data block** (`2*NN + 2`), never a segment
|
||||
header. Reading it as a `2*NN + 16` header is what made walks drift — and the
|
||||
"variable-prefix segment descriptors" reported earlier today were not a format
|
||||
feature at all, just walker drift of exactly
|
||||
`4 - (old_stop - true_record_start)` on all 25 affected files.
|
||||
|
||||
Measured against the production snapshot:
|
||||
|
||||
| | before | after |
|
||||
|---|---|---|
|
||||
| all four channels equal length | 156 / 1388 | **1388 / 1388** |
|
||||
| ASCII sample-count exact | 72 / 75 | **75 / 75** |
|
||||
| ASCII fully exact | 70 / 75 | **73 / 75** |
|
||||
| device PPV, waveform (live decode) | 1288 / 1306 | **1306 / 1306** |
|
||||
| device PPV, histogram (live decode) | 4434 / 4459 | **4458 / 4459** |
|
||||
|
||||
Mean absolute PPV ratio error on waveforms is now 0.00000. The 2 remaining
|
||||
ASCII imperfections differ by exactly 1 LSB on samples sitting at the
|
||||
±10.000 in/s rail.
|
||||
|
||||
**This also eliminated the walker-over-read class.** 24 of those 35 files
|
||||
were histograms that `read_blastware_file` fed to the *waveform* codec first;
|
||||
the old walker accepted them and returned garbage (one yielded 98,923
|
||||
"intervals"), while the record-chain decoder correctly returns `None` so they
|
||||
fall through to `histogram_codec`.
|
||||
|
||||
`00 03` records are decoded rather than skipped. Skipping them does not merely
|
||||
lose samples — it silently shifts the time base of everything after them on
|
||||
that channel (observed on `BE9558/K558LOF2.820W`, MicL displaced by exactly
|
||||
512 samples with nothing marking the gap).
|
||||
|
||||
Footer detection now prefers whichever `0e 08` candidate yields a chain
|
||||
terminating on `0x06`, since the signature can occur inside a sample stream.
|
||||
Blast radius: 1 file of 1,388.
|
||||
|
||||
The superseded model is retained as `decode_waveform_legacy` and pinned by
|
||||
`micromate/idf_file.py`, whose Thor IDFW body-offset search trial-decodes
|
||||
candidate offsets and keeps whichever yields the most samples — the new
|
||||
decoder correctly returns `None` where the old one returned garbage, which
|
||||
changes that heuristic's winner. Switching Thor over is deferred until that
|
||||
search is reworked to use the record chain directly.
|
||||
|
||||
- **Series-3 histogram block is uniformly big-endian, and the stream's final
|
||||
block has its own tail — the codec was clipping large peaks and dropping the
|
||||
last interval of nearly every histogram.**
|
||||
|
||||
@@ -223,6 +223,29 @@ custom delta + RLE + variable-width codec.
|
||||
`NN + 2` for int8 blocks). Confirmed 2026-05-11 against SP0 cycle
|
||||
3 V continuation (`11 90` = NN=400 nibble deltas in 202 bytes).
|
||||
|
||||
### ⚠ SUPERSEDED 2026-08-25 — the body is a RECORD CHAIN
|
||||
|
||||
Everything in this section below about `40 NN` segment headers, tagless
|
||||
headers, variable header widths and channel rotation describes a model that
|
||||
is **wrong**. The body is a chain of self-delimiting per-channel records:
|
||||
|
||||
off+2 len uint16 BE -> next_record = off + 2 + len
|
||||
off+4 chan_id 0x46 Tran / 0x47 Vert / 0x48 Long / 0x49 MicL / 0x06 = END
|
||||
off+8 mode 02 00 = deltas+anchors (14B hdr)
|
||||
01 00 = ABSOLUTE values (10B hdr)
|
||||
00 03 = raw 12-bit absolute, NO TAGS (10B hdr)
|
||||
|
||||
`40 NN` is an ordinary int16 BE data block (`2*NN + 2`), never a header. The
|
||||
"variable prefix" of 0/2/4/6/8 bytes was walker drift, exactly
|
||||
`4 - (old_stop - true_record_start)`.
|
||||
|
||||
All four channels now come out equal length in **1388/1388** files (was
|
||||
156/1388); ASCII sample-count exact **75/75**, fully exact **73/75**; device
|
||||
PPV on a live decode **1306/1306** waveform, **4458/4459** histogram.
|
||||
|
||||
The old model survives as `decode_waveform_legacy` because
|
||||
`micromate/idf_file.py` pins it for Thor IDFW body-offset search.
|
||||
|
||||
### Framing cases added 2026-05-11 → 2026-08-25
|
||||
|
||||
Four more block-framing cases, each of which had been causing **silent
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
|
||||
| Date | Section | Change |
|
||||
|---|---|---|
|
||||
| 2026-08-25 (3) | S7.6.1, S15 | **THE WAVEFORM BODY IS A RECORD CHAIN, NOT A TAG STREAM — supersedes the segment-header model entirely.** Records are self-delimiting: `off+2` is a uint16 BE length and `next = off + 2 + len`; the chain ends on a record whose chan_id is `0x06`. `off+8` holds a 3-valued mode enum - `02 00` (14-byte header, anchors, cumulative deltas), `01 00` (10-byte, no anchors, ABSOLUTE values), `00 03` (10-byte, no tags at all, raw 12-bit absolute). **`40 NN` is an ordinary int16 BE data block of length 2*NN+2**, never a header; reading it as a 2*NN+16 header is what made walks drift, and the 'variable prefix' of 0/2/4/6/8 bytes reported earlier the same day was walker drift, exactly `4 - (old_stop - true_record_start)`. Verified: chain terminates on `06` in 1387/1388 files; all four channels equal length in **1388/1388** (was 156/1388); ASCII sample-count exact 72/75 -> **75/75**, fully exact 70/75 -> **73/75**; device PPV on a live decode **1306/1306** waveform (mean abs ratio error 0.00000) and **4458/4459** histogram. Also eliminated the walker-over-read class: 24 of those files were histograms the waveform codec was wrongly accepting. The superseded model is retained as `decode_waveform_legacy` because `micromate/idf_file.py` pins it for Thor IDFW body-offset search. |
|
||||
| 2026-08-25 (2) | S7.6.2, S15 | **HISTOGRAM BLOCK IS BIG-ENDIAN + terminal block tail.** The 32-byte histogram block's per-channel fields are uint16 **big-endian** (`T_peak` [5:7], `T_halfperiod` [7:9], `V_peak` [9:11], `V_halfperiod` [11:13], `L_peak` [13:15], `L_halfperiod` [15:17], `M_peak` [17:19], `M_halfperiod` [19:21]); only `block_ctr` [2:4] is little-endian. The marker is `block[4]` alone - the previous uint16 LE marker test at [4:6] forced `block[5] == 0` and thereby capped every geo peak at 255 counts (1.275 in/s), silently clipping larger peaks. The byte previously documented as a per-channel "annotation" is the high byte of the big-endian half-period, which is why it was non-zero exactly on sub-Hz intervals. Separately, the **final block of each stream carries tail `9c 06 00 42`** rather than `1e 0a 00 00` and holds arbitrary bytes at [21:23]; rejecting it dropped the last interval of nearly every histogram. Verified against 1211 production histograms paired with their Blastware ASCII exports: 1211/1211 decode exactly, plus 842,442 per-interval frequency comparisons with zero mismatches (previously 1 of 1196 files fully correct). |
|
||||
| 2026-08-25 | §7.6.1, §15, Appendix E (NEW) | **BODY CODEC + SCALE PASS — five findings, all verified against 75 production events paired with their preserved Blastware ASCII exports.** (1) **Geo full scale is 32000 ADC counts, not 32768** — one decoder unit (16 counts) is exactly 0.005 in/s, so 10.000 in/s = 32000 counts. Consumers dividing by 32768 read every geophone sample and derived peak **2.34% low**; the error scales with amplitude so it was invisible on quiet events and worst on loud ones. 216 per-channel comparisons: 32768 → 151/216 exact (worst 0.238 in/s on a 10 in/s event); 32000 → 216/216 exact, worst 1 LSB. Affects waveforms, histograms and series-4 alike. (2) **Wide-NN RLE `0X NN`** — the 12-bit NN encoding already known for `1X`/`2X` also applies to the `00 NN` RLE tag (runs > 252 samples). (3) **`30 NN` is not capped at NN=0x10** — data-section blocks reach at least 0x18; the length formula was already right. (4) **`40 NN` segment headers are variable width** — NN counts the previous-channel continuation deltas, so the header is `2*NN + 16` bytes; `40 01` and `40 03` occur alongside `40 02`. A header can also appear **tagless** (the NN=0 case): just the 14-byte tail. (5) **The header field documented as a "monotonic uint32 LE counter" is really `[channel_id][00][00][segment_index]`** with 0x46=Tran 0x47=Vert 0x48=Long 0x49=MicL — verified on 1697/1697 segment headers, zero disagreements. Decoders should take the channel from this field, not from rotation position. Items (2)–(5) each caused **silent channel truncation**: an unhandled tag ends the walk and the decoder returns short channels with no error. Corpus result end-to-end: exact 37 → 72, truncated 23 → 3, full-length value errors 15 → 0. New Appendix E documents the field-observed "offset" device fault. |
|
||||
| 2026-05-20 | §2, §3, §4.2, §5.1, §5.3, §6, §7.5b, §7.6.1, §7.6.3, §7.6.4, §7.7.2, §7.7.3, §7.7.5, §7.8.4, §7.8.7, §7.9, §8, §11, §12, §13, §14, §15, Appendix D | **DOC AUDIT PASS — accuracy sweep against `CLAUDE.md` + `minimateplus/` code.** Fixed: (1) S3 frames terminate on bare ETX, not DLE+ETX — §2/§3 rewritten. (2) §3 payload layout corrected — byte[1]=flags, byte[2]=SUB (was wrongly labelled DLE/ADDR). (3) §4.2 — probe responses do NOT carry data length; lengths are hardcoded `DATA_LENGTHS` constants. (4) §5.1 — removed stale duplicate "SUB 1C = TRIGGER CONFIG READ" row; SUB 0A lengths corrected from `0x30/0x26` to `0x46/0x2C` (real event / boundary marker). (5) §5.3 — added missing write-frame format (BW_CMD-only doubling, DLE-aware checksum, offset formula, ack format, SUB 71 chunk parameters). (6) §6 — fixed "SUB 06 → channel config read" → event storage range. (7) §7.5b / §8 — added the 10-byte `sub_code=0x03` continuous-mode timestamp variant alongside the 9-byte single-shot layout; peak vector sum location corrected from "fixed offset 87" to `tran_pos − 12` (label-relative). (8) §7.6 / §7.6.1 / §7.6.3 / §7.6.4 — switched compliance-anchor convention from the 10-byte form to the canonical 6-byte `\xbe\x80\x00\x00\x00\x00`; recording_mode confirmed at anchor−8 in BOTH read and write (was wrongly listed as anchor−3 write / anchor−4 read); sample_rate at anchor−6, histogram_interval at anchor−4, record_time at anchor+6; geo_range row added at channel_label+33. (9) §7.7.2 — token byte position corrected from `params[6]` to `params[7]`. (10) §7.8.4 — fi==9 skip marked FIXED (already removed from code); chunk-count totals updated. (11) §7.8.7 — TODO replaced with current state of `_decode_a5_metadata_into`. (12) §7.9 — Histogram Interval upgraded ❓ → ✅. (13) §11 — POLL example wire bytes corrected; SUB 5A row added to checksum table. (14) §13 — device-under-test updated for current primary unit (BE11529 / S338.17). (15) §14 — TCP Idle Timeout fixed (0→2 min); Data Forwarding Timeout units clarified. (16) §15 (renumbered from second §14) — open-question items already resolved in CLAUDE.md closed out. (17) Appendix D — extension taxonomy rewritten to reflect the AB0T timestamp encoding (D.5.2/D.5.3); EXTENSION REFUTED warning replaced with the resolved encoding. |
|
||||
@@ -1268,36 +1269,79 @@ re-deriving the whole production store:
|
||||
The series-4 figure is closer to correct but not exact — the Thor
|
||||
per-count LSB is its own open question (see §15).
|
||||
|
||||
###### Unmapped: variable-prefix segment descriptors ❓ OPEN
|
||||
###### SUPERSEDED 2026-08-25 — the body is a RECORD CHAIN, not a tag stream
|
||||
|
||||
Three of 75 ground-truth production events still truncate. In each,
|
||||
the walk reaches a segment header whose channel-id field is preceded by
|
||||
a **variable-width prefix** — 2, 4 or 6 bytes have all been observed,
|
||||
where the standard tagless form always has 4 (``field2`` + ``len``).
|
||||
These records also carry the ``01 00`` marker rather than ``02 00``,
|
||||
and appear packed back-to-back with little or no sample data between
|
||||
them.
|
||||
Everything above about ``40 NN`` segment headers, tagless headers, variable
|
||||
header widths and channel rotation describes a model that is **wrong**. It
|
||||
produced nearly-correct output because the block table happens to tile the
|
||||
data sections correctly, but the framing is not what the device writes.
|
||||
|
||||
The ``01 00`` marker is *not* simply an anchor count: records carrying
|
||||
it have been seen with both 2-byte and 4-byte anchor fields in the same
|
||||
file, so the prefix width and the marker are not yet reconciled.
|
||||
|
||||
The decoder stops cleanly at these rather than emitting garbage.
|
||||
Examples: ``BE12599/N599LPNB.JF0W`` at body offset 1155 (2-byte
|
||||
prefix), ``BE12599/N599LPWJ.980W`` at 849 (6-byte prefix),
|
||||
``BE9558/K558LOF2.820W`` at 1485.
|
||||
|
||||
Examples from event-c (1 sec single-shot):
|
||||
The body is a chain of **self-delimiting per-channel records**:
|
||||
|
||||
```
|
||||
Segment header 1 (offset 235):
|
||||
40 02 | 00 00 00 00 | 0a 4b 01 1e | 47 00 00 00 | 02 00 00 01 | 00 01
|
||||
^counter=0x47
|
||||
Segment header 2 (offset 523):
|
||||
40 02 | ff fe ff fe | 13 f5 01 06 | 48 00 00 00 | 02 00 00 01 | 00 02
|
||||
^counter=0x48 (+1)
|
||||
off+0 field2 uint16 purpose unknown (not a length, not a checksum)
|
||||
off+2 len uint16 BE next_record = off + 2 + len <- authoritative
|
||||
off+4 chan_id 0x46 Tran / 0x47 Vert / 0x48 Long / 0x49 MicL
|
||||
0x06 = END OF WAVEFORM STREAM
|
||||
off+5 0x00
|
||||
off+6 0x00
|
||||
off+7 segment index
|
||||
off+8 mode 2 bytes, a 3-valued enum
|
||||
off+10 anchors 2 x int16 BE, ABSOLUTE -- present ONLY when mode == 02 00
|
||||
```
|
||||
|
||||
**Mode enum**, all three ground-truth verified:
|
||||
|
||||
| mode | header | data section |
|
||||
|---|---|---|
|
||||
| ``02 00`` | 14 bytes | anchors emitted, then blocks are **cumulative deltas** |
|
||||
| ``01 00`` | 10 bytes | no anchors, blocks carry **absolute** sample values |
|
||||
| ``00 03`` | 10 bytes | **no tags at all** — raw 12-bit packed absolute samples |
|
||||
|
||||
Census over the 1,388 production series-3 waveform binaries:
|
||||
``02 00`` x 32,617, ``01 00`` x 74, ``00 03`` x 71.
|
||||
|
||||
**``40 NN`` is an ordinary int16 BE data block** of length ``2*NN + 2``
|
||||
(1 <= NN <= 8), never a header. The superseded model read it as a header of
|
||||
length ``2*NN + 16``, which is exactly why walks drifted: the "variable prefix"
|
||||
of 0/2/4/6/8 bytes reported earlier was walker drift, precisely
|
||||
``4 - (old_stop - true_record_start)``, on all 25 affected files.
|
||||
|
||||
Block table for data sections (``NN = ((tag_hi & 0x0F) << 8) | tag_lo``):
|
||||
|
||||
| tag | length | samples |
|
||||
|---|---|---|
|
||||
| ``0X NN`` | 2 | NN (RLE hold — holds the previous value in BOTH delta and absolute modes) |
|
||||
| ``1X NN`` | NN/2 + 2 | NN (4-bit nibble) |
|
||||
| ``2X NN`` | NN + 2 | NN (int8) |
|
||||
| ``30 NN`` | NN*1.5 + 2 | NN (12-bit packed) |
|
||||
| ``40 NN`` | 2*NN + 2 | NN (int16 BE) |
|
||||
|
||||
The ``30 NN`` "trailer length = NN*4" fallback must NOT be applied inside a
|
||||
record — it corrupts records whose ``30 NN`` sits near a boundary.
|
||||
|
||||
**The preamble is segment 0's implicit Tran record.** ``body[1:3]`` carries
|
||||
the same mode pair: ``00 02 00`` (1,387 of 1,388 files) means two int16 BE
|
||||
anchors at ``body[3:7]`` then delta blocks; ``00 00 03`` (1 file,
|
||||
``BE13121/O121L4L1.KF0W``) means raw 12-bit absolute from ``body[3]``, which
|
||||
cannot be block-walked — the first record must be located by scanning.
|
||||
|
||||
**Verification.** The length chain terminates on a ``0x06`` record in 1,387 of
|
||||
1,388 files (the exception has an ambiguous footer signature inside its sample
|
||||
stream). All four channels come out at identical length in **1,388/1,388**,
|
||||
against 156/1,388 under the superseded model. Against the 75 events with a
|
||||
preserved Blastware ASCII export: sample-count exact **72/75 -> 75/75**, fully
|
||||
exact **70/75 -> 73/75** (the 2 remaining differ by exactly 1 LSB on samples
|
||||
sitting at the +-10.000 in/s rail). Against device-reported PPVs on a live
|
||||
decode: waveform **1306/1306** exact with mean absolute ratio error 0.00000;
|
||||
histogram **4458/4459**.
|
||||
|
||||
This also eliminated the walker-over-read class entirely. 24 of those 35
|
||||
"histogram" over-reads were histogram files that ``read_blastware_file`` fed to
|
||||
the *waveform* codec first; the old walker accepted them and returned garbage
|
||||
(one produced 98,923 "intervals"), while the record-chain decoder correctly
|
||||
returns None so they fall through to ``histogram_codec``.
|
||||
|
||||
##### Trailer
|
||||
|
||||
The trailer (after the last segment's data) is a sequence of 32-byte
|
||||
@@ -3037,7 +3081,7 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger
|
||||
| **ACH inbound server — RESOLVED.** `bridges/ach_server.py` implements full inbound ACH pipeline. `--clear-after-download` flag for delete-after-upload workflow. Post-erase key-reuse detection via `max_downloaded_key` high-water mark. | RESOLVED | 2026-04-11 | |
|
||||
| **Sensor Check dropdown byte location** — byte offset in 1A compliance config payload for the "Sensor Check: Before monitoring / After each event / Disabled" setting is NOT YET LOCATED. Confirmed: unit always runs with "Before monitoring" set. Need a capture with "Disabled" to diff. | MEDIUM | 2026-04-08 | Still open |
|
||||
| **RV55 DCD/DTR default** — newer Sierra Wireless RV55 firmware does not assert DCD/DTR by default, so the MiniMate Plus never detects TCP disconnect and stays idle instead of resuming monitoring. Root cause: RV55 ACEmanager `DCD Control` setting. Workaround not yet found. | MEDIUM | 2026-04-11 | Still open |
|
||||
| **Variable-prefix segment descriptors** — 3 of 75 ground-truth events still truncate. The walk reaches a segment header whose channel-id field is preceded by a *variable-width* prefix (2, 4 or 6 bytes observed; the standard tagless form always has 4), carrying an `01 00` marker instead of `02 00`. The marker is **not** simply an anchor count — `01 00` records appear with both 2- and 4-byte anchor fields in the same file, so prefix width and marker are not yet reconciled. Examples: `BE12599/N599LPNB.JF0W` @1155, `BE12599/N599LPWJ.980W` @849, `BE9558/K558LOF2.820W` @1485. | MEDIUM | 2026-08-25 | Still open |
|
||||
| ~~**Variable-prefix segment descriptors**~~ - **RESOLVED 2026-08-25:** there is no variable prefix. The body is a chain of self-delimiting records (see S7.6.1); the 0/2/4/6/8-byte prefix was walker drift from reading `40 NN` as a segment header. All 25 affected files now chain cleanly to the `06` terminator. | RESOLVED | 2026-08-25 | Resolved 2026-08-25 |
|
||||
| ~~**Histogram codec drops trailing intervals (series-3)**~~ - **RESOLVED 2026-08-25.** Two errors, both in the block model. (1) The block is **uniformly big-endian**: peaks and half-periods are uint16 BE (`T_peak` [5:7], `T_halfperiod` [7:9], `V_peak` [9:11], ...); only `block_ctr` [2:4] is LE. The old uint8-peak + "annotation"-byte model silently clipped any peak above 1.275 in/s, and the "annotation" byte was really the half-period high byte - non-zero exactly on the sub-Hz intervals BW renders `<1.0`. The marker is `block[4]` alone; testing [4:6] as a uint16 LE marker forced `block[5] == 0`, which is what capped the peak at one byte. (2) The **final block of the stream carries tail `9c 06 00 42`** instead of `1e 0a 00 00`, with arbitrary bytes at [21:23]; rejecting it dropped the last interval of nearly every histogram - often the one holding the event peak. Verified on 1211 production histograms vs their BW ASCII exports: **1211/1211 exact** (interval count + every per-interval peak) and 842,442 frequency comparisons with zero mismatches; was 1/1196. | RESOLVED | 2026-08-25 | Resolved 2026-08-25 |
|
||||
| **Micromate (UM-series) IDF decode is ~1000x low** — e.g. `UM11402_20260406130113.IDFW` decodes a Tran peak of 0.0009 in/s against a device-reported 1.1168. Distinct from the Thor IDF path, which decodes sanely. Suspect a different per-count LSB or a body offset that does not hold for UM-series files. | MEDIUM | 2026-08-25 | Still open |
|
||||
| **Thor IDF per-count LSB** — after the 32000 geo full-scale correction, series-4 Thor peaks sit at a median 0.983 of the device-reported peak (was 0.960 under 32768). Closer, but the residual ~1.7% suggests Thor uses its own per-count LSB rather than the BW 16-count/0.005 in/s convention. A code comment in `sfm/waveform_store.py` claims Thor's LSB is 0.0003 in/s, which would predict Thor reading *high* — the measurement shows the opposite, so that comment is unverified. | LOW | 2026-08-25 | Still open |
|
||||
|
||||
@@ -1,3 +1,22 @@
|
||||
> ## SUPERSEDED 2026-08-25 — the body is a RECORD CHAIN
|
||||
>
|
||||
> The tag-dispatch model described in this document — `40 NN` segment headers,
|
||||
> tagless headers, channel rotation — is **wrong**. It produced nearly-correct
|
||||
> output only because the block table happens to tile the data sections.
|
||||
>
|
||||
> The body is a chain of self-delimiting per-channel records: `off+2` is a
|
||||
> uint16 BE length, `next = off + 2 + len`, and the chain ends on a record whose
|
||||
> chan_id is `0x06`. A 3-valued mode enum at `off+8` selects delta / absolute /
|
||||
> raw-12-bit semantics. `40 NN` is an ordinary int16 BE data block.
|
||||
>
|
||||
> See the record-chain section of `docs/instantel_protocol_reference.md` §7.6.1
|
||||
> and the implementation in `minimateplus/waveform_codec.py`.
|
||||
>
|
||||
> Result: all four channels equal length in 1388/1388 files (was 156/1388);
|
||||
> ASCII sample-count exact 75/75, fully exact 73/75; device PPV 1306/1306.
|
||||
>
|
||||
> This document is retained as the reasoning trail.
|
||||
|
||||
# Waveform body codec — FULLY DECODED (2026-05-11)
|
||||
|
||||
This is the **clean working note** for the body-codec reverse-engineering
|
||||
|
||||
+13
-1
@@ -47,7 +47,19 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Optional, Union
|
||||
|
||||
from minimateplus.waveform_codec import decode_waveform_v2
|
||||
# Thor IDFW bodies are pinned to the SUPERSEDED tag-dispatch decoder.
|
||||
#
|
||||
# _find_waveform_body_offset() trial-decodes every candidate offset and keeps
|
||||
# whichever yields the most samples. The series-3 record-chain decoder
|
||||
# correctly returns None where the legacy walker returned garbage, which
|
||||
# changes that heuristic's winner on 33 of 577 files. The net effect measured
|
||||
# 2026-08-25 was positive (all-channels-equal 8/577 -> 506/577, mean abs PPV
|
||||
# error 0.228 -> 0.173 in/s) but Thor has no ASCII ground truth in the corpus
|
||||
# and its geo scaling is separately suspect, so the switch is deferred until
|
||||
# the body-offset search is reworked to use the record chain directly.
|
||||
from minimateplus.waveform_codec import (
|
||||
decode_waveform_legacy as decode_waveform_v2,
|
||||
)
|
||||
|
||||
from .models import IdfEvent, IdfPeaks, IdfReport
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ from typing import Optional, Union
|
||||
from .models import Event, PeakValues, ProjectInfo, Timestamp
|
||||
from . import blastware_file as _bw # avoid circular reference at module load
|
||||
from .bw_ascii_report import BwAsciiReport
|
||||
from . import waveform_codec as _wc
|
||||
from .waveform_codec import decode_waveform_v2, decoded_to_adc_counts
|
||||
from .histogram_codec import decode_histogram_body
|
||||
|
||||
@@ -843,7 +844,13 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
|
||||
# Footer: locate the 0e 08 marker, validating the year is in a sane range.
|
||||
body_start = _bw._WAVEFORM_HEADER_SIZE + 21
|
||||
footer_pos = -1
|
||||
# The 0e 08 + plausible-year footer signature can occur inside the sample
|
||||
# stream. Collect every candidate and prefer the first whose body yields a
|
||||
# waveform record chain terminating on the 0x06 marker; fall back to the
|
||||
# first candidate otherwise. Blast radius measured 2026-08-25: changes the
|
||||
# chosen footer on exactly 1 of 1,388 series-3 waveform files
|
||||
# (BE17353/S353L4O5.OX0W, false positive at 3800, real footer at 8576).
|
||||
footer_candidates = []
|
||||
pos = body_start
|
||||
while True:
|
||||
pos = raw.find(b"\x0e\x08", pos)
|
||||
@@ -851,10 +858,24 @@ def read_blastware_file(path: Union[str, Path]) -> Event:
|
||||
break
|
||||
yr = (raw[pos + 4] << 8) | raw[pos + 5]
|
||||
if 2015 <= yr <= 2050:
|
||||
footer_pos = pos
|
||||
break
|
||||
footer_candidates.append(pos)
|
||||
pos += 1
|
||||
|
||||
footer_pos = -1
|
||||
for cand in footer_candidates:
|
||||
cand_body = raw[body_start:cand]
|
||||
try:
|
||||
recs = _wc.walk_records(cand_body)
|
||||
except Exception:
|
||||
recs = []
|
||||
if recs:
|
||||
tail = recs[-1]["end"]
|
||||
if tail + 5 <= len(cand_body) and cand_body[tail + 4] == _wc.STREAM_END_ID:
|
||||
footer_pos = cand
|
||||
break
|
||||
if footer_pos < 0 and footer_candidates:
|
||||
footer_pos = footer_candidates[0]
|
||||
|
||||
if footer_pos < 0 and len(raw) >= 26:
|
||||
footer_pos = len(raw) - 26
|
||||
if footer_pos < body_start:
|
||||
|
||||
@@ -441,8 +441,16 @@ def decode_tran_initial(body: bytes) -> Optional[List[int]]:
|
||||
return out
|
||||
|
||||
|
||||
def decode_waveform_v2(body: bytes) -> Optional[dict]:
|
||||
def decode_waveform_legacy(body: bytes) -> Optional[dict]:
|
||||
"""
|
||||
SUPERSEDED 2026-08-25 — the tag-dispatch / segment-header model.
|
||||
|
||||
Retained because ``micromate/idf_file.py`` trial-decodes Thor IDFW bodies
|
||||
at many candidate offsets and keeps whichever yields the most samples;
|
||||
the record-chain decoder returns None where this one returned garbage,
|
||||
which shifts that heuristic's winner. Thor is pinned here until its own
|
||||
body-offset search is reworked. Do not use for series-3.
|
||||
|
||||
Decode the body into per-channel sample arrays.
|
||||
|
||||
Status (2026-05-11 evening — channel-rotation hypothesis CONFIRMED):
|
||||
@@ -673,3 +681,231 @@ def decode_a5_frames(a5_frames) -> Optional[dict]:
|
||||
if decoded is None:
|
||||
return None
|
||||
return decoded_to_adc_counts(decoded)
|
||||
|
||||
|
||||
# ── Record-chain body model (CONFIRMED 2026-08-25) ──────────────────────────
|
||||
#
|
||||
# The body is NOT a flat tag-dispatch stream with ``40 NN`` segment headers.
|
||||
# It is a chain of self-delimiting per-channel RECORDS:
|
||||
#
|
||||
# off+0 field2 uint16 purpose unknown (not a length, not a checksum)
|
||||
# off+2 len uint16 BE next_record = off + 2 + len <- authoritative
|
||||
# off+4 chan_id 0x46 Tran / 0x47 Vert / 0x48 Long / 0x49 MicL
|
||||
# 0x06 = end of waveform stream
|
||||
# off+5 0x00
|
||||
# off+6 0x00
|
||||
# off+7 segment index
|
||||
# off+8 mode 2 bytes, a 3-valued enum (see below)
|
||||
# off+10 anchors 2 x int16 BE, ABSOLUTE — present only when mode is 02 00
|
||||
#
|
||||
# Mode semantics, all ground-truth verified:
|
||||
# 02 00 14-byte header; emit the 2 anchors, then blocks are CUMULATIVE deltas
|
||||
# 01 00 10-byte header; no anchors; blocks carry ABSOLUTE sample values
|
||||
# 00 03 10-byte header; NO TAGS AT ALL — the data section is raw 12-bit
|
||||
# packed ABSOLUTE samples (6 bytes -> 4 samples)
|
||||
#
|
||||
# ``40 NN`` is an ordinary int16 BE DATA block (length 2*NN + 2), never a header.
|
||||
# The previous model read it as a variable-width segment header of length
|
||||
# 2*NN + 16, which is why walks drifted and channels came out unequal.
|
||||
#
|
||||
# Verified over the 1,388 series-3 waveform binaries in the production
|
||||
# snapshot: the length chain terminates on a 0x06 record in 1,387 of them (the
|
||||
# exception has an ambiguous footer, handled by the caller), and all four
|
||||
# channels come out at identical length in 1,388/1,388 — against 156/1,388
|
||||
# under the superseded model. Against the 75 events with a preserved
|
||||
# Blastware ASCII export: sample-count exact 72/75 -> 75/75, fully exact
|
||||
# 70/75 -> 73/75.
|
||||
|
||||
CHANNEL_IDS = {0x46: "Tran", 0x47: "Vert", 0x48: "Long", 0x49: "MicL"}
|
||||
STREAM_END_ID = 0x06
|
||||
|
||||
MODE_DELTA = (0x02, 0x00)
|
||||
MODE_ABSOLUTE = (0x01, 0x00)
|
||||
MODE_RAW12 = (0x00, 0x03)
|
||||
_MODES = (MODE_DELTA, MODE_ABSOLUTE, MODE_RAW12)
|
||||
|
||||
|
||||
def _u16(b: bytes, p: int) -> int:
|
||||
return (b[p] << 8) | b[p + 1]
|
||||
|
||||
|
||||
def _i16(b: bytes, p: int) -> int:
|
||||
v = _u16(b, p)
|
||||
return v - 0x10000 if v >= 0x8000 else v
|
||||
|
||||
|
||||
def data_block_len(body: bytes, p: int) -> Tuple[Optional[int], Optional[int]]:
|
||||
"""``(byte_length, n_samples)`` of the data block at *p*, or ``(None, None)``.
|
||||
|
||||
Data-section blocks only — there is no segment-header tag in this model.
|
||||
``30 NN`` has no trailer-length fallback here; that fallback corrupted
|
||||
records whose ``30 NN`` sat near a record boundary.
|
||||
"""
|
||||
if p + 2 > len(body):
|
||||
return None, None
|
||||
t0, t1 = body[p], body[p + 1]
|
||||
hi = t0 & 0xF0
|
||||
nn = ((t0 & 0x0F) << 8) | t1
|
||||
if hi == 0x40: # int16 BE data block
|
||||
return (None, None) if (nn == 0 or nn > 0x08) else (2 * nn + 2, nn)
|
||||
if nn == 0 or nn % 4:
|
||||
return None, None
|
||||
if hi == 0x00:
|
||||
return 2, nn # RLE hold
|
||||
if hi == 0x10:
|
||||
return nn // 2 + 2, nn # 4-bit nibble
|
||||
if hi == 0x20:
|
||||
return nn + 2, nn # int8
|
||||
if hi == 0x30:
|
||||
return nn * 3 // 2 + 2, nn # 12-bit packed
|
||||
return None, None
|
||||
|
||||
|
||||
def unpack12(data: bytes) -> List[int]:
|
||||
"""Raw 12-bit packed samples: 6 bytes -> 4 signed values."""
|
||||
out: List[int] = []
|
||||
for g in range(len(data) // 6):
|
||||
hi = (data[6 * g] << 8) | data[6 * g + 1]
|
||||
for k in range(4):
|
||||
x = (((hi >> (12 - 4 * k)) & 0xF) << 8) | data[6 * g + 2 + k]
|
||||
out.append(x - 0x1000 if x >= 0x800 else x)
|
||||
return out
|
||||
|
||||
|
||||
def is_record(body: bytes, p: int) -> bool:
|
||||
"""True if a per-channel record header starts at *p*."""
|
||||
return (p + 10 <= len(body)
|
||||
and body[p + 4] in CHANNEL_IDS
|
||||
and body[p + 5] == 0x00 and body[p + 6] == 0x00
|
||||
and 8 <= _u16(body, p + 2) <= len(body) - p
|
||||
and (body[p + 8], body[p + 9]) in _MODES)
|
||||
|
||||
|
||||
def find_first_record(body: bytes) -> Optional[int]:
|
||||
"""Offset of the first record, or None.
|
||||
|
||||
Under the normal ``00 02 00`` preamble the leading bytes are segment-0's
|
||||
Tran blocks, so walk them. Under the ``00 00 03`` preamble that data is
|
||||
raw 12-bit with no tags at all and cannot be block-walked — scan instead.
|
||||
"""
|
||||
if len(body) >= 3 and (body[1], body[2]) == MODE_RAW12:
|
||||
scan_from = 3
|
||||
else:
|
||||
i = 7
|
||||
while i < len(body):
|
||||
if is_record(body, i):
|
||||
nxt = i + 2 + _u16(body, i + 2)
|
||||
if nxt + 5 <= len(body) and (is_record(body, nxt)
|
||||
or body[nxt + 4] == STREAM_END_ID):
|
||||
return i
|
||||
length, _ = data_block_len(body, i)
|
||||
if length is None:
|
||||
return None
|
||||
i += length
|
||||
return None
|
||||
for i in range(scan_from, max(scan_from, len(body) - 10)):
|
||||
if is_record(body, i):
|
||||
nxt = i + 2 + _u16(body, i + 2)
|
||||
if nxt + 5 <= len(body) and (is_record(body, nxt)
|
||||
or body[nxt + 4] == STREAM_END_ID):
|
||||
return i
|
||||
return None
|
||||
|
||||
|
||||
def walk_records(body: bytes, first: Optional[int] = None) -> List[dict]:
|
||||
"""Follow the length chain from *first* to the ``0x06`` terminator."""
|
||||
if first is None:
|
||||
first = find_first_record(body)
|
||||
out: List[dict] = []
|
||||
if first is None:
|
||||
return out
|
||||
p, seen = first, set()
|
||||
while p is not None and p + 10 <= len(body):
|
||||
if p in seen:
|
||||
break
|
||||
seen.add(p)
|
||||
cid = body[p + 4]
|
||||
if cid == STREAM_END_ID or cid not in CHANNEL_IDS:
|
||||
break
|
||||
length = _u16(body, p + 2)
|
||||
if length < 8 or p + 2 + length > len(body):
|
||||
break
|
||||
out.append({"offset": p, "channel": CHANNEL_IDS[cid],
|
||||
"segment_index": body[p + 7],
|
||||
"mode": (body[p + 8], body[p + 9]),
|
||||
"end": p + 2 + length})
|
||||
p += 2 + length
|
||||
return out
|
||||
|
||||
|
||||
def decode_waveform_v2(body: bytes) -> Optional[dict]:
|
||||
"""Decode a Blastware waveform body into per-channel sample arrays.
|
||||
|
||||
Returns ``{"Tran": [...], "Vert": [...], "Long": [...], "MicL": [...]}``
|
||||
in 16-count units (LSB = 0.005 in/s at Normal range), or None if *body*
|
||||
is not a decodable waveform body.
|
||||
|
||||
Implements the record-chain model documented above.
|
||||
"""
|
||||
if len(body) < 8 or body[0] != 0x00:
|
||||
return None
|
||||
preamble = (body[1], body[2])
|
||||
if preamble not in (MODE_DELTA, MODE_RAW12):
|
||||
return None
|
||||
first = find_first_record(body)
|
||||
if first is None:
|
||||
return None
|
||||
|
||||
out: dict = {c: [] for c in ("Tran", "Vert", "Long", "MicL")}
|
||||
|
||||
def run(channel: str, start: int, end: int, absolute: bool) -> None:
|
||||
cur = out[channel][-1] if out[channel] else 0
|
||||
i = start
|
||||
while i < end:
|
||||
length, nn = data_block_len(body, i)
|
||||
if length is None or i + length > end:
|
||||
return # stop this record; the chain resyncs at end
|
||||
hi = body[i] & 0xF0
|
||||
if hi == 0x00:
|
||||
vals = [None] * nn
|
||||
elif hi == 0x10:
|
||||
vals = []
|
||||
for k in range(nn):
|
||||
byte = body[i + 2 + k // 2]
|
||||
v = (byte >> 4) if k % 2 == 0 else (byte & 0xF)
|
||||
vals.append(v - 16 if v >= 8 else v)
|
||||
elif hi == 0x20:
|
||||
vals = [v - 256 if v >= 128 else v
|
||||
for v in body[i + 2:i + 2 + nn]]
|
||||
elif hi == 0x30:
|
||||
vals = unpack12(body[i + 2:i + length])
|
||||
else:
|
||||
vals = [_i16(body, i + 2 + 2 * k) for k in range(nn)]
|
||||
for v in vals:
|
||||
if v is None:
|
||||
pass # RLE hold, in delta AND absolute modes
|
||||
elif absolute:
|
||||
cur = v
|
||||
else:
|
||||
cur += v
|
||||
out[channel].append(cur)
|
||||
i += length
|
||||
|
||||
# Segment 0 is an implicit Tran record carried in the preamble.
|
||||
if preamble == MODE_DELTA:
|
||||
out["Tran"].extend([_i16(body, 3), _i16(body, 5)])
|
||||
run("Tran", 7, first, absolute=False)
|
||||
else:
|
||||
out["Tran"].extend(unpack12(body[3:first]))
|
||||
|
||||
for rec in walk_records(body, first):
|
||||
ch, off, mode, end = (rec["channel"], rec["offset"],
|
||||
rec["mode"], rec["end"])
|
||||
if mode == MODE_DELTA:
|
||||
out[ch].extend([_i16(body, off + 10), _i16(body, off + 12)])
|
||||
run(ch, off + 14, end, absolute=False)
|
||||
elif mode == MODE_ABSOLUTE:
|
||||
run(ch, off + 10, end, absolute=True)
|
||||
elif mode == MODE_RAW12:
|
||||
out[ch].extend(unpack12(body[off + 10:end]))
|
||||
return out
|
||||
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
|
||||
from minimateplus.waveform_codec import (
|
||||
WaveformBlock,
|
||||
decode_waveform_legacy,
|
||||
decode_tran_initial,
|
||||
decode_waveform_v2,
|
||||
decoded_to_adc_counts,
|
||||
@@ -548,9 +549,18 @@ def test_walk_body_wide_rle_block():
|
||||
assert blocks[0].length == 2
|
||||
|
||||
|
||||
# NOTE (2026-08-25): the four tests below assert the SUPERSEDED tag-dispatch
|
||||
# model — `40 NN` as a variable-width segment header, tagless headers, channel
|
||||
# from rotation. The body format is really a chain of self-delimiting records
|
||||
# (see the record-chain section of waveform_codec.py), so `decode_waveform_v2`
|
||||
# no longer behaves this way. They are retargeted at `decode_waveform_legacy`,
|
||||
# which still implements the old model and is pinned by micromate/idf_file.py
|
||||
# for Thor IDFW bodies.
|
||||
|
||||
|
||||
def test_decode_wide_rle_repeats_full_run():
|
||||
"""A wide RLE run repeats the running value NN times, not NN & 0xFF."""
|
||||
decoded = decode_waveform_v2(_synth(b"\x01\x0c"))
|
||||
decoded = decode_waveform_legacy(_synth(b"\x01\x0c"))
|
||||
# 2 preamble anchors + 268 repeats
|
||||
assert len(decoded["Tran"]) == 2 + 268
|
||||
assert set(decoded["Tran"]) == {0}
|
||||
@@ -589,7 +599,7 @@ def test_segment_header_anchors_track_header_width(nn, hdr_len):
|
||||
data[2 * nn + 8 : 2 * nn + 10] = b"\x02\x00"
|
||||
data[2 * nn + 10 : 2 * nn + 12] = (7).to_bytes(2, "big") # anchor 0
|
||||
data[2 * nn + 12 : 2 * nn + 14] = (9).to_bytes(2, "big") # anchor 1
|
||||
decoded = decode_waveform_v2(_synth(bytes([0x40, nn]) + bytes(data)))
|
||||
decoded = decode_waveform_legacy(_synth(bytes([0x40, nn]) + bytes(data)))
|
||||
assert decoded["Vert"][:2] == [7, 9]
|
||||
|
||||
|
||||
@@ -629,7 +639,7 @@ def test_tagless_header_carries_full_14_bytes_as_data():
|
||||
|
||||
def test_tagless_header_anchors_and_channel_id():
|
||||
"""Anchors decode from data[10:14]; the channel comes from the id byte."""
|
||||
decoded = decode_waveform_v2(_synth(_tagless(chan_id=0x48, a0=11, a1=13)))
|
||||
decoded = decode_waveform_legacy(_synth(_tagless(chan_id=0x48, a0=11, a1=13)))
|
||||
assert decoded["Long"][:2] == [11, 13] # 0x48 → Long, not rotation
|
||||
assert decoded["Vert"] == []
|
||||
|
||||
@@ -642,10 +652,146 @@ def test_segment_channel_comes_from_id_not_rotation(chan_id, name):
|
||||
would put the second one on the next channel and corrupt both."""
|
||||
body = _synth(_tagless(chan_id=chan_id, seg=1, a0=5, a1=6),
|
||||
_tagless(chan_id=chan_id, seg=2, a0=7, a1=8))
|
||||
decoded = decode_waveform_v2(body)
|
||||
decoded = decode_waveform_legacy(body)
|
||||
# Tran additionally carries the body preamble's 2 anchors (both 0 here).
|
||||
expected = [0, 0, 5, 6, 7, 8] if name == "Tran" else [5, 6, 7, 8]
|
||||
assert decoded[name] == expected
|
||||
for other in ("Tran", "Vert", "Long", "MicL"):
|
||||
if other != name:
|
||||
assert decoded[other] == ([0, 0] if other == "Tran" else [])
|
||||
|
||||
|
||||
# ── Record-chain body model (2026-08-25) ────────────────────────────────────
|
||||
#
|
||||
# The body is a chain of self-delimiting per-channel records, not a flat
|
||||
# tag-dispatch stream. Verified over 1,388 production series-3 waveform
|
||||
# binaries: the chain terminates on a 0x06 record in 1,387 of them and all
|
||||
# four channels come out at identical length in 1,388/1,388 (was 156/1,388).
|
||||
# Against the 75 events with a preserved Blastware ASCII export: sample-count
|
||||
# exact 72/75 -> 75/75, fully exact 70/75 -> 73/75.
|
||||
|
||||
from minimateplus.waveform_codec import ( # noqa: E402
|
||||
CHANNEL_IDS,
|
||||
MODE_ABSOLUTE,
|
||||
MODE_DELTA,
|
||||
MODE_RAW12,
|
||||
STREAM_END_ID,
|
||||
data_block_len,
|
||||
find_first_record,
|
||||
is_record,
|
||||
unpack12,
|
||||
walk_records,
|
||||
)
|
||||
|
||||
|
||||
def _rec(chan_id, mode, payload, seg=0, field2=b"\x00\x00", anchors=None):
|
||||
"""Build one self-delimiting record."""
|
||||
head = bytearray()
|
||||
head += bytes([chan_id, 0x00, 0x00, seg])
|
||||
head += bytes(mode)
|
||||
if anchors is not None:
|
||||
for a in anchors:
|
||||
head += int(a).to_bytes(2, "big", signed=True)
|
||||
body = bytes(head) + payload
|
||||
return field2 + (len(body) + 2).to_bytes(2, "big") + body
|
||||
|
||||
|
||||
def _terminator():
|
||||
return b"\x00\x00" + (8).to_bytes(2, "big") + bytes([STREAM_END_ID, 0, 0, 0, 0, 0])
|
||||
|
||||
|
||||
def _body(*records, preamble=b"\x00\x02\x00", seg0=b"\x00\x00\x00\x00"):
|
||||
return preamble + seg0 + b"".join(records) + _terminator()
|
||||
|
||||
|
||||
def test_forty_nn_is_a_data_block_not_a_segment_header():
|
||||
"""`40 NN` is an int16 BE data block of length 2*NN + 2.
|
||||
|
||||
The superseded model read it as a segment header of length 2*NN + 16,
|
||||
which is what made walks drift and channels come out unequal.
|
||||
"""
|
||||
assert data_block_len(b"\x40\x02\x00\x01\x00\x02", 0) == (6, 2)
|
||||
assert data_block_len(b"\x40\x08" + bytes(16), 0) == (18, 8)
|
||||
# NN > 8 is not a data block
|
||||
assert data_block_len(b"\x40\x0c" + bytes(24), 0) == (None, None)
|
||||
|
||||
|
||||
def test_record_chain_is_followed_by_length_not_by_tag_sniffing():
|
||||
payload = b"\x00\x04" # RLE hold x4
|
||||
body = _body(_rec(0x47, MODE_DELTA, payload, anchors=(3, 5)))
|
||||
recs = walk_records(body)
|
||||
assert len(recs) == 1
|
||||
assert recs[0]["channel"] == "Vert"
|
||||
assert recs[0]["mode"] == MODE_DELTA
|
||||
|
||||
|
||||
def test_chain_terminates_on_channel_id_06():
|
||||
body = _body(_rec(0x47, MODE_DELTA, b"\x00\x04", anchors=(1, 1)),
|
||||
_rec(0x48, MODE_DELTA, b"\x00\x04", anchors=(2, 2)))
|
||||
assert [r["channel"] for r in walk_records(body)] == ["Vert", "Long"]
|
||||
assert not is_record(body, len(body) - 10) # the terminator is not a record
|
||||
|
||||
|
||||
def test_mode_delta_emits_anchors_then_accumulates():
|
||||
# two anchors, then an int8 block of +1,+1,+1,+1
|
||||
body = _body(_rec(0x47, MODE_DELTA, b"\x20\x04\x01\x01\x01\x01",
|
||||
anchors=(10, 11)))
|
||||
d = decode_waveform_v2(body)
|
||||
assert d["Vert"] == [10, 11, 12, 13, 14, 15]
|
||||
|
||||
|
||||
def test_mode_absolute_replaces_rather_than_accumulates():
|
||||
"""mode `01 00`: no anchors, and block values are ABSOLUTE samples."""
|
||||
body = _body(_rec(0x48, MODE_ABSOLUTE, b"\x20\x04\x0a\x0b\x0c\x0d"))
|
||||
d = decode_waveform_v2(body)
|
||||
assert d["Long"] == [10, 11, 12, 13], "01 00 blocks are absolute, not deltas"
|
||||
|
||||
|
||||
def test_mode_absolute_rle_holds_the_previous_value():
|
||||
body = _body(_rec(0x48, MODE_ABSOLUTE, b"\x20\x04\x07\x07\x07\x07\x00\x04"))
|
||||
d = decode_waveform_v2(body)
|
||||
assert d["Long"] == [7, 7, 7, 7, 7, 7, 7, 7]
|
||||
|
||||
|
||||
def test_mode_raw12_has_no_tags_at_all():
|
||||
"""mode `00 03`: the whole data section is raw 12-bit absolute samples.
|
||||
|
||||
Decoding these matters for the time base — skipping the record would
|
||||
displace every later sample on that channel (observed on
|
||||
BE9558/K558LOF2.820W, MicL shifted by exactly 512).
|
||||
"""
|
||||
packed = bytes([0x01, 0x23, 0x04, 0x05, 0x06, 0x07]) # 4 samples
|
||||
body = _body(_rec(0x49, MODE_RAW12, packed))
|
||||
d = decode_waveform_v2(body)
|
||||
assert d["MicL"] == unpack12(packed)
|
||||
assert len(d["MicL"]) == 4
|
||||
|
||||
|
||||
def test_unpack12_sign_extends():
|
||||
assert unpack12(bytes([0x00, 0x00, 0x01, 0x02, 0x03, 0x04])) == [1, 2, 3, 4]
|
||||
# high nibble 0x8 -> negative
|
||||
assert unpack12(bytes([0x80, 0x00, 0x00, 0x00, 0x00, 0x00]))[0] == -2048
|
||||
|
||||
|
||||
def test_channel_comes_from_the_record_id():
|
||||
for cid, name in CHANNEL_IDS.items():
|
||||
body = _body(_rec(cid, MODE_ABSOLUTE, b"\x20\x04\x01\x02\x03\x04"))
|
||||
d = decode_waveform_v2(body)
|
||||
assert d[name][-4:] == [1, 2, 3, 4], f"{name} misrouted"
|
||||
|
||||
|
||||
def test_raw12_preamble_is_scanned_not_block_walked():
|
||||
"""A `00 00 03` preamble carries raw 12-bit data from body[3] with no tags,
|
||||
so find_first_record must scan rather than block-walk. One production file
|
||||
has this (BE13121/O121L4L1.KF0W); block-walking returns None on it."""
|
||||
packed = bytes([0x00, 0x00, 0x01, 0x02, 0x03, 0x04])
|
||||
body = _body(_rec(0x47, MODE_DELTA, b"\x00\x04", anchors=(1, 1)),
|
||||
preamble=b"\x00\x00\x03", seg0=packed)
|
||||
assert find_first_record(body) == 3 + len(packed)
|
||||
d = decode_waveform_v2(body)
|
||||
assert d["Tran"] == unpack12(packed)
|
||||
|
||||
|
||||
def test_returns_none_when_no_record_chain():
|
||||
assert decode_waveform_v2(b"\x00\x02\x00" + bytes(40)) is None
|
||||
assert decode_waveform_v2(b"") is None
|
||||
|
||||
Reference in New Issue
Block a user