seismo-relay/docs/idf_protocol_reference.md

# IDF Protocol Reference — Thor / Micromate Series IV

Starting-point reference for reverse-engineering Instantel's Micromate
Series IV event-file format.  Sibling to
[instantel_protocol_reference.md](instantel_protocol_reference.md) (the
Series III "Rosetta Stone") — this doc holds what we know so far and
the open questions still to crack.

> ⚠ **The "Status (2026-05-28)" block below is SUPERSEDED.**  Its geo LSB
> (0.0003), its IDFH scale (`/32768 × 10`), its fixed body offset (`0x0f1f`)
> and its "87–99% byte-exact / loud events truncate" caveat were all wrong or
> incomplete.  See **[Verified against Thor's own exports
> (2026-09-10)](#verified-against-thors-own-exports-2026-09-10)** — the
> decoder is now per-sample exact on 1,057,536/1,057,536 samples.  The block
> is kept only for the reverse-engineering trail.

**Status (2026-05-28, SUPERSEDED):** ASCII text sidecar fully decoded (1,014
sample files round-trip).  **Thor IDFW** binary now decodes via
`micromate.idf_file.read_idf_file()` — reuses the BW segment-rotated
block codec verbatim at fixed body offset `0x0f1f`; metadata (serial,
timestamp, sample_rate, record_time, calibration_date) extracted from
the binary header.  Sample fidelity is 87–99% byte-exact on quiet
events; loud events hit the BW codec's known walker-stops-early
limitation.  Residual ~3% drift on per-sample deltas (likely a
Thor-specific 12-bit delta refinement not yet modelled).

**Thor IDFH histograms also decoded.**  Body has one or more segments;
each 12-byte segment header `[length_be 2B][0a 00 00 00][00 NN][05 3f]`
introduces `N = (length - 10) // 72` interval records of 72 bytes
each.  Each interval = 4 × 16-byte per-channel records:
`[int16 min][int16 max][int16 ??][uint16 halfp][2B 00][uint16 ??][2B 00][uint16 ??]`.
Geo peak `= max(|min|, |max|) / 32768 × 10` in/s (matches sidecar
~1.8%); freq `= 512 / halfp` Hz (None for halfp ≤ 5 → ">100"
sentinel).  Corpus: **all 859 Thor IDFH files decode, 181,071
intervals**.  Wired through `read_idf_file()` →
`save_imported_idf()` → sidecar's `extensions.idf_intervals`.

**Note on the BE9439 outliers in the example corpus:** Two files
(`BE9439_20200713131747.IDFW` and `BE9439_20200713124251.IDFH`) are
**Series III Blastware** binaries, not Thor.  Provenance: TMI tried
to use Thor to manage auto-call-homes for Series III units; the
experiment didn't work out, but it did leave a few BW event files
in Thor's per-serial directory structure with `.IDFW`/`.IDFH`
extensions — Thor's forwarder applied its own naming convention to
the BW bodies it was relaying.  Their header `10 00 01 80 00 00
Instantel STRT ff fe <end_key> <start_key>` is the BW SUB 5A STRT
record, not a Thor body preamble.  The reader detects them by
signature and raises `NotImplementedError` pointing callers at
`read_blastware_file()`, which extracts BW-format peaks from them.

**Still NYI for Thor IDFH:** per-channel `int16 field4` (possibly
time-of-peak); the two uint16 fields (probably PVS contributions);
8-byte interval tail (PVS data); mic dB(L) exact conversion constant.

## Verified against Thor's own exports (2026-09-10)

**The series-4 decoder is now per-sample exact.**  1,057,536 / 1,057,536
geophone samples across all 153 genuine Thor waveform files reproduce Thor's
own CSV export exactly; histogram peaks land within 2% on 858/858 files
(median error −0.004%).

### Ground truth — it was there all along

Thor writes `TXT/`, `CSV/`, `XML/` and `PDF/` exports beside every binary:

```
<serial dir>/UM13981_20220207084555.IDFW
<serial dir>/CSV/UM13981_20220207084555.IDFW.csv
```

The **CSV carries a per-sample block** — four columns (Tran, Vert, Long, Mic)
in in/s and psi, after the 2-column report header.  That is the series-4
equivalent of Blastware's `_ASCII.TXT` exports, and it gives 1,012 paired
files (152 IDFW + 860 IDFH).  Earlier notes in this file and in
`micromate/idf_file.py` asserted "Thor has no ASCII ground truth in the
corpus"; that was wrong, and it is why the decoder sat pinned to a
superseded walker with a scaling constant nobody could check.

Harness: `scratch/verify_thor_against_csv.py`.

### Geo LSB = 0.000310308 in/s per count (NOT 0.0003)

The old 0.0003 was read off the smallest non-zero sample in the exports —
but that is Thor's **4-decimal display rounding of the LSB, not the LSB**.
It read every series-4 geophone sample **3.3% low**.  The quantisation
ladder gives it away: counts 1..6 export as 0.0003, 0.0006, 0.0009, 0.0012,
0.0016, 0.0019 — an LSB of exactly 0.0003 would end 0.0015, 0.0018.

Each exported sample constrains the LSB to the window that rounds to its
printed value.  Intersecting 991,415 such constraints gives

```
LSB ∈ [0.000310307933, 0.000310308057]      width 1.2e-10
```

so `_GEO_LSB_IPS = 0.000310308`, i.e. full scale 10.0 in/s = **32226.05
counts**.  Corroboration: an IDFH interval that never recorded keeps its
min/max accumulator at its ±full-scale seed, and that seed is
`(min=+32226, max=-32226)`.  ⚠ The tempting closed form `10.0/32226` is
very slightly wrong — it lands 4.5e-10 above the feasible window and loses
78 boundary samples while never winning one.  **Series III uses 32000 counts
for the same 10.0 in/s, so the two generations do not share a scale.**

Independently confirmed on 8 production units (UM6047, UM11402, UM11719,
UM12947, UM13981, UM14133, UM20146, UM20147): every unit's median PPV error
against its device-reported peak moved from −3.3% to within ±0.03%.  It is a
global constant, not a per-unit calibration.

### IDFH segment header: the counter is a uint16, and it is cumulative

```
[length_be 2B][0a 00 00 00][counter_be 2B][05 3f]
```

`counter` is the **0-based cumulative index of the last interval in the
segment** — 9, 19, 29, ... for the usual 10-intervals-per-segment layout
(`length` = 730).

The validator used to require `counter`'s high byte to be `0x00`.  That
silently **capped every histogram at 250 intervals**: once the cumulative
counter passed 255 the high byte went non-zero and every later segment was
rejected.  Any run longer than ~4 hours lost its tail — frequently the part
holding the event peak, so the file's PPV read low.  **540 of 858 corpus
files were affected**; fixing it moved histogram peaks from 48.3% to 93.8%
within 0.5% of Thor's reported PPV.

### Unwritten interval slots carry a ±full-scale seed

An interval the device reserved but never wrote keeps `min = +32226`,
`max = -32226` on all four channels — `min > max`, impossible for real data.
Decoded naively it yields a 10.0 in/s peak on every channel and, being a
max-over-intervals, poisons the whole file's PPV.  Rare but real: exactly 1
of 497,611 corpus intervals, and it inflated that file's Long PPV from
0.0081 to 10.0 in/s.  The inversion is all-or-nothing across channels (0
partial cases), so requiring every channel to be inverted is a safe test.

### Record mode `00 00` — raw int16 absolute (MODE_RAW16)

The record chain's mode field at `off+8` takes a fourth value:

| mode | meaning | header |
|---|---|---|
| `02 00` | deltas + two int16 anchors | 14 B |
| `01 00` | absolute, tagged blocks | 10 B |
| `00 03` | raw 12-bit absolute, untagged | 10 B |
| **`00 00`** | **raw int16 BE absolute, untagged** | **10 B** |

A `MODE_RAW16` record with `length = 1032` carries exactly
`(1032 - 8) / 2 = 512` samples and reproduced Thor's export **512/512
exactly** on first test.  Thor uses it for segment 0 (the pre-trigger
window) on some events.  Before this mode existed the record fell through
the dispatch unhandled, so the channel silently lost its first 512 samples —
which is what produced the "loud events truncate" symptom.

`MODE_ABSOLUTE` is also valid as a **preamble** (the implicit segment-0 Tran
record); its tagged blocks start at `body[3]`, not `body[7]`, because its
header is 10 bytes rather than 14.

### Body offset is not fixed at 0x0f1f — and 0x0f1f is really a record + 7

A "body offset" is `<record start> + 7`, so that `body[0]` is the segment
index and `body[1:3]` is the mode.  The canonical `0x0f1f` is simply the
record at `0x0f18`.

Searching for the literal preamble `00 02 00` finds only MODE_DELTA bodies,
and worse, it **matches the `[seg][mode]` bytes inside any record header**,
so the scan could pick a candidate part-way down the chain.  That decodes a
plausible-looking but rotation-shifted body which drops each channel's
segment 0 — the real cause of the remaining truncations.

`_find_waveform_body_offset()` now anchors on record headers (the
`<channel_id> 00 00` signature at `+4`, validated with `is_record()`),
takes the **chain head** — a record no other record's length field points at
— and trial-decodes `head + 7`, preferring the candidate where all four
channels come out the same length.

⚠ Do **not** scan for candidate preambles instead: `MODE_RAW16` is
`00 00`, so every run of three zero bytes looks like a body start and each
costs a full trial decode (~0.5 s/file measured, vs 6 ms/file now).

### `40 NN` is not capped at NN=8 (2026-09-11)

`data_block_len()` rejected any `40 NN` int16 block with `NN > 0x08`.  The cap
had no evidence behind it — every corpus available when it was written used
only NN ∈ {1, 2, 3, 4, 8}, so it was never exercised.  Loud events use much
wider blocks:

| corpus | `40 NN` values | walker stops |
|---|---|---|
| first + 3-channel corpora | 1, 2, 3, 4, 8 | none |
| UM12947 2025-07..09 | 2, 4, 8, **12, 16, 20 … 196** | every value > 8 |

Because `walk_body`/`run` stop at the first unrecognised tag rather than
raising, this surfaced as **silently short channels** — e.g. Tran 1812 /
Vert 2132 / Long 2324 on a file whose export has 2324 for all three.  The
real bound is the buffer (and the caller's record end), not a magic constant.

Verified against Thor's exports for UM12947 (2025-07-14 … 2025-09-25, 167
waveforms): length mismatches **22 → 0**, and **1,476,242 / 1,476,249**
samples exact.

⚠ These events are **not** truncated recordings, which was the competing
hypothesis — the exports carry the full sample count.

**The 7 residual samples are Thor's rounding, not ours.**  Each differs by
exactly one 4th-decimal tick (e.g. decoded 3.3551 vs export 3.3550).
Intersecting the per-sample rounding constraints over this corpus is
**infeasible** — the binding pair (count 2013 → 0.6247, count 4351 → 1.3501)
contradict by 2.3e-11, i.e. 7e-5 relative.  No single linear LSB can
reproduce every printed value, so Thor is not doing plain round-half-up on
`count × LSB`.  Do not retune `_GEO_LSB_IPS` to chase these; it is already
pinned to ~1e-11.

### Mic-disabled units are a distinct shape (2026-09-10, second corpus)

Some units run with the microphone disabled — **3 channels, not 4** — and that
changes two structural things.  Confirmed on the `9-10-26-csv-req` corpus
(UM11402, UM12947, UM20147): 139/139 waveforms and 877/877 histograms.

**Waveform: the body starts earlier.**  A 3-channel unit has a shorter fixed
header and puts its record chain head at **`0x0dba`**, below the old
`_BODY_SCAN_FLOOR` of `0x0E00`.  The head was therefore invisible to the scan,
which fell through to the *Vert* segment-0 record and decoded a body shifted
one position around the channel rotation.  The signature is unmistakable:

```
Tran 3072 / Vert 2560 / Long 3072 / MicL 0      <- Vert exactly 512 short
```

46 of 139 files in that corpus were affected; all 46 became per-sample exact
once the floor dropped to `0x0C00`.  Note the body-offset scoring also had to
stop requiring four channels — `len(lengths) >= 3`, not `== 4`, or `equal` is
permanently False for these events and the pick falls back to raw sample count.

**Histogram: the interval record is 56 bytes, not 72.**

```
interval_size = 16 × n_channels + 8        (72 for 4 channels, 56 for 3)
```

It is **not a constant**, and it cannot be inferred from `length` alone.
Derive the interval count from the segment counter — it is cumulative, so
`n = counter - previous_counter` — and then `stride = (length - 10) / n`.
`n_channels` follows from `(stride - 8) / 16`.

Assuming 72 read 7 intervals out of each 10-interval segment and then walked
off alignment into garbage that decoded as ~10 in/s peaks — inflating those
files' PPV by up to 191,000%.  Fixing it moved the second corpus from 56.6% to
**100.0%** of histograms within 2% of Thor's reported PPV, and recovered 4
files that previously decoded no intervals at all.

### What is still open

- ~~23 of 575 production IDFW files~~ — **RESOLVED 2026-09-11.** Production
  IDFW is now **575/575** with zero truncations and zero decode failures
  (median PPV error −0.0007%).  See "`40 NN` is not capped at NN=8" above.

- Mic → psi scale is still the rough `2.14e-6` regression, not derived.
- Per-channel `int16 field4` in the IDFH interval record (possibly
  time-of-peak) and the 8-byte tail (PVS data) remain undecoded.

⚠ **Thor's histogram PPV has a display floor of 0.0050 in/s.**  In the
production store 6,080 sidecar PPV values are exactly 0.0050 (next most
common value: 275 occurrences), and **41.4% of IDFH sidecars report a
component PPV larger than their own vector sum** — geometrically impossible.
On those quiet files the decoder's ~0.0025 in/s is *more* accurate than the
reference; do not "fix" the decoder to match it.

### Codec breakthroughs (2026-05-28)

- **Body offset is a fixed `0x0f1f`** across 151/154 corpus IDFW
  files.  Preceded by a 4-byte record-type marker (`46 00 00 00`)
  + magic preamble `00 02 00 [Tran[0] BE] [Tran[1] BE]`.
- **Sample stream is BW's segment-rotated block codec verbatim.**
  Thor reuses `10 NN` (nibble), `20 NN` (int8), `00 NN` (RLE),
  `30 NN` (packed12), `40 02` (segment header) tags with the same
  semantics.  Channel rotation Tran→Vert→Long→MicL.
- **Geo LSB = 0.0003 in/s** (not BW's 0.005), because Thor's 16-bit
  ADC range maps to 10 in/s without the 16-count BW quantization step.
- **Mic ≈ 2.14×10⁻⁶ psi/count** (rough scale; refine after channel
  block calibration constants are decoded).
- **BW compliance anchor `\xbe\x80\x00\x00\x00\x00` reappears at
  IDFW offset 0x952** — sample_rate at anchor−6 (uint16 BE),
  record_time at anchor+6 (float32 BE), same layout as BW.
- **Event timestamp at offset 0x97A** — 8 bytes `[day][month]
  [year_be][unk][hour][min][sec]`.  Stop-time mirrors at 0x982.
- **Serial as null-terminated ASCII at 0x14E**.
- **Calibration date** at 0x194–0x197 (day, month, year_be).
- Per-sample residual drift of ~3% suggests Thor encodes int8/nibble
  deltas with an extra refinement bit that BW doesn't carry —
  unsolved; errors resync within a few samples so cumulative impact
  is small.

---

## File model

### Filename convention

```
<SERIAL>_<YYYYMMDDHHMMSS>.<KIND>
```

- **SERIAL** — literal device serial, two-letter prefix + numeric
  suffix.  Examples seen: `UM11719`, `UM13981`, `UM20147`, `BE9439`.
  Unlike Series III BW filenames (`M529LK44.AB0`, base-36 stem),
  Series IV filenames carry the serial in plain text.
- **YYYYMMDDHHMMSS** — 14-char ASCII timestamp in **device local
  time** (no timezone marker).
- **KIND** — `IDFH` for histograms, `IDFW` for waveforms.

The `.IDFH.txt` / `.IDFW.txt` ASCII sidecar lives in a `TXT/`
**subfolder** of the unit's directory, not alongside the binary.
This pairing convention is encoded in
`event_forwarder.idf_report_path()`.

### Directory layout

```
C:\THORDATA\
└── <Project>\
    └── <UM####>\                  ← unit serial dir
        ├── UM12345_20260520100000.MLG     ← monitor log (not events)
        ├── UM12345_20260520100000.IDFH    ← histogram event (binary)
        ├── UM12345_20260520100000.IDFW    ← waveform event (binary)
        ├── UM12345_20260520100000.IDFW.CDB ← cache-DB variant (skip)
        ├── TXT\
        │   ├── UM12345_20260520100000.IDFH.txt    ← histogram ASCII sidecar
        │   └── UM12345_20260520100000.IDFW.txt    ← waveform  ASCII sidecar
        ├── CSV\, HTML\, PDF\, XML\        ← operator-facing derived exports
        └── ...
```

The `.IDFW.CDB` files share the binary's basename but appear to be a
separate cache/database variant.  Their first 8 bytes match the
**old**-firmware Thor signature (see below) regardless of which
signature the paired `.IDFW` uses.  Purpose unknown; sizes vary
wildly (observed 123 B → 40,491 B).  Thor-watcher's forwarder
deliberately skips them.

### Sample corpus

The `thor-watcher/example-data/THORDATA_example/` tree carries
**1,014 paired .IDFW / .IDFH + .txt files** spanning 2020–2023
across nine units (UM11719, UM13981, UM20147, …, plus BE9439 from
2020).  This is the reverse-engineering ground truth.

---

## ASCII sidecar (`.IDFW.txt` / `.IDFH.txt`) — fully decoded

Shape: plain text, one `"Key : Value"` line per metadata field,
followed for waveforms by a tab-separated sample table headed by
the literal line `Waveform Data Channels`.  Parsed by
[`micromate/idf_ascii_report.py`](../micromate/idf_ascii_report.py).
See [`micromate/models.py`](../micromate/models.py) for the typed
`IdfReport` shape.

### Notable conventions

- **Units are native to Thor** — geophone in **in/s**, microphone in
  **dB(L)** (not psi like Series III BW reports), frequency in Hz,
  acceleration in g, displacement in in.
- **Below-threshold readings** appear as the literal string
  `<0.005 in/s` (155 occurrences in the sample corpus) — the parser
  strips the `<` and treats the numeric remainder as the value.
- **Out-of-range / not-measured** values appear as `N/A` — parser
  drops the field rather than letting the string leak into a numeric
  column.
- **Firmware string** observed: `Micromate ISEE 11.0AK`.
- **TitleString1..4** are operator-defined free-text slots; Thor's
  default labels map them to Location / Client / Company / Notes,
  which the parser surfaces as `project` / `client` / `operator` /
  `notes`.
- **Histogram sidecars** use `HistogramStartDate` / `HistogramStartTime`
  in place of waveform's `EventDate` / `EventTime`.  Parser falls
  through to either.
- **Histogram tabular block** lacks the `Waveform Data Channels`
  marker; instead it's a multi-line column header followed by
  per-interval rows (`<date> <time> <tran-ppv> <freq> ...`).  Parser
  silently ignores lines after the metadata block since they lack a
  colon-separated `key : value` shape (the timestamps DO contain
  colons but produce garbage keys that don't collide with any
  recognised field).

---

## Binary header signatures (observed)

Hex dump of the first 32 bytes across 1,014 sample files reveals
**two distinct file signatures**, both anchored by the literal
ASCII string `"\x00Instantel\x00"` at offset 6–16:

### Signature A — newer firmware (1,012 files, 99.8% of corpus)

```
00000000: 0012 0100 0000 496e 7374 616e 7465 6c00   ......Instantel.
00000010: 0000 a695 002e b500 4f70 6572 6174 6f72   ........Operator
                                ^^^^^^^^^^^^^^^^
                                operator/title string starts at 0x18
```

Header bytes 0–5: `00 12 01 00 00 00`.  Followed immediately by the
8-byte ASCII tag, then 6 unknown bytes, then ASCII operator-supplied
strings (Operator name, etc.) and on through the project / client /
title strings.  No `STRT` record observed in this layout.

### Signature B — older firmware (2 files: BE9439 from 2020)

```
00000000: 1000 0180 0000 496e 7374 616e 7465 6c00   ......Instantel.
00000010: 072c 0012 0300 5354 5254 fffe 0111 2340   .,....STRT....#@
                          ^^^^^^^^^                ^^^^^^^^^
                          STRT magic               4-byte end_key
00000020: 0111 0000 2e5f 00ac 4600 0000 0200 0000   ....._..F.......
          ^^^^^^^^^             ^^^
          4-byte start_key      0x46 (BW WAVEHDR record-type marker)
```

Header bytes 0–5: `10 00 01 80 00 00`.  The structure after the
`Instantel` magic is **byte-for-byte identical to a BW SUB 5A
probe-response STRT record** as documented in
[instantel_protocol_reference.md → "SUB 5A — STRT record encodes
end_offset"](instantel_protocol_reference.md).  Specifically:

| Offset | Bytes               | Meaning (per BW reference)          |
|--------|---------------------|--------------------------------------|
| 0x14   | `53 54 52 54`       | `STRT` magic                         |
| 0x18   | `ff fe`             | STRT sentinel                        |
| 0x1A   | `01 11 23 40`       | `end_key` (4 bytes)                  |
| 0x1E   | `01 11 00 00`       | `start_key` (4 bytes)                |
| 0x26   | `46`                | `0x46` waveform-record type marker   |

**Hypothesis:** Older Micromate firmware writes a wrapped BW-format
event into the `.IDFW` file — essentially the same on-disk shape as
a Series III device, with the new filename convention applied at
export time.  Newer firmware (signature A) abandoned the
BW-compatible layout for an Instantel-specific format.

If that hypothesis holds, the 2 signature-B files can already be
parsed via `minimateplus/event_file_io.read_blastware_file()` — worth
testing.  The 1,012 signature-A files are the real reverse-engineering
target.

### `.IDFW.CDB` cache files

Always carry signature B (`10 00 01 80 ...`), even when the paired
`.IDFW` carries signature A.  Plausible explanation: the CDB is an
internal Thor cache-database export that retains the legacy BW-style
record layout regardless of the user-facing `.IDFW` format version.
Not currently consumed by the forwarder.

---

## File-size patterns (Signature A, the main target)

Survey of 1,012 signature-A files:

| Event type   | Typical size      | Source of variance                           |
|--------------|-------------------|----------------------------------------------|
| `.IDFW` 2-sec | 9,200 – 10,500 B | Operator-supplied strings (TitleString1..4) of varying length |
| `.IDFH`       | 2,944 – 4,076 B  | Histogram interval count (record duration / interval) |

**Naive arithmetic for 2-sec waveform:**
- 4 channels × 2 sec × 1024 sps = 8,192 samples
- At 2 bytes/sample (int16) = 16,384 sample bytes → file would be > 16 KB
- Observed: ~9–10 KB
- → samples are likely **1 byte each** (int8 quantised), **or** stored
  with bit-packing / delta encoding, **or** only one channel's
  full-rate samples are stored with the others reconstructed
  arithmetically.  Verifying this is the **first RE milestone**.

Project-string–length variance (~1 KB across the corpus) is consistent
with the file carrying a single copy of each TitleString1..4 plus
operator + setup-name as null-padded ASCII regions.

---

## Open questions

The reverse-engineering targets, roughly in dependency order:

1. **Sample encoding (signature A)** — int8? int16 LE/BE? Bit-packed?
   Delta-coded?  Per-channel interleaved or sequential blocks?
2. **Header field layout (signature A)** — where do sample_rate,
   record_time, channel count, and per-channel peaks live in the
   binary?  The ASCII sidecar gives the device-authoritative values,
   so binary fields can be confirmed by diff.
3. **Operator-string offsets** — `Operator` at 0x18 is the first
   visible string in signature-A files; the rest (project, client,
   notes, setup) follow.  Need to map exact offsets and null-padding
   conventions.
4. **Signature-B → BW codec compatibility** — does
   `minimateplus/event_file_io.read_blastware_file()` actually parse
   the 2 BE9439 signature-B files as-is?  If yes, the OLD-format
   ingest is free.
5. **`.IDFW.CDB` purpose** — is it an internal Thor cache, a
   ring-buffer dump, or something else?  Worth a single small effort
   to characterise so we know what we're skipping.
6. **Footer / checksum** — every BW event file has a footer; does
   IDF?  Where does the per-channel sample block end?

---

## Reverse-engineering playbook (when we start)

The Series III BW codec took ~2 months of MITM wire captures
because we didn't have ground-truth metadata.  Thor's situation is
**substantially better**:

- **Ground truth is on disk.**  Every binary in `example-data/`
  has a paired `.IDFW.txt` carrying the full decoded sample table
  (`Waveform Data Channels` block — see any sample file in
  `thor-watcher/example-data/.../TXT/`).  Aligning binary bytes
  to the table's float-per-row values gives an immediate per-byte
  hypothesis test.
- **Cross-event diffing.**  1,012 signature-A samples from 9 units
  spanning 4 years means any field that varies between events is
  immediately localisable.  Fields that are constant across all
  files (firmware ID, channel labels, format-version word) are also
  immediately localisable by complementary search.
- **No protocol surface.**  Files at rest, not a wire dialect.  No
  DLE stuffing, no inner-frame parsing, no probe/data two-step.

Suggested first session (2-4 hours): hand-decode `UM11719_20231219162723.IDFW`
(10,290 bytes) against its `TXT/UM11719_20231219162723.IDFW.txt`
sample table (the 2-sec waveform at 1024 sps × 4 channels = 8,192
sample rows).  Find the first per-channel sample value (`0.0003` in
the Tran column at t=0) in the binary.  Confirms sample encoding.
Everything else flows from there.

---

## Code seams ready to receive the codec

When the codec lands, it goes into
[`micromate/idf_file.py`](../micromate/idf_file.py) (currently a
stub raising `NotImplementedError`).  Public API:

```python
from micromate import IdfEvent
from micromate.idf_file import read_idf_file

event: IdfEvent = read_idf_file(Path("UM11719_20231219163444.IDFW"))
# event.peaks.transverse_ips, event.timestamp, event.raw_samples, ...
```

The ingest pipeline (`WaveformStore.save_imported_idf`) currently
builds the `IdfEvent` from the `.txt` parser only.  Once
`read_idf_file()` works, the binary becomes authoritative; the
`.txt` parser drops to fast-path metadata cross-check.  Operators
who don't enable Thor's TXT exporter still get fully populated
events.

---

## See also

- [instantel_protocol_reference.md](instantel_protocol_reference.md) — Series III BW protocol reference (the Rosetta Stone).  STRT record format, DLE framing, BW filename encoding.
- [`micromate/idf_ascii_report.py`](../micromate/idf_ascii_report.py) — `.txt` sidecar parser.
- [`micromate/models.py`](../micromate/models.py) — `IdfEvent`, `IdfReport` typed dataclasses.
- [`micromate/idf_file.py`](../micromate/idf_file.py) — placeholder for the binary codec.
- [`thor-watcher/example-data/THORDATA_example/`](../../thor-watcher/example-data/) — 1,014 paired binary + .txt files for codec validation.