feat: first try at building waveform binary files.
This commit is contained in:
@@ -105,6 +105,8 @@
|
||||
| 2026-04-11 | §14.6 | **RESOLVED — ACH Session Lifecycle is no longer "Future".** `bridges/ach_server.py` fully implements inbound ACH: POLL handshake, device info, event download. State tracked via `ach_state.json` (key-based, with `max_downloaded_key` for post-erase detection). `--clear-after-download` flag added for the standard delete-after-upload workflow. |
|
||||
| 2026-04-17 | §7.6.2, §14 | **RESOLVED — Float 6.206053 at channel_label+28 is the ADC-to-velocity scale factor.** Confirmed from Series III Interface Handbook §4.5 formula: `Range (×1) = 1.61133 V / Sensitivity (V/unit)`. For the standard Instantel geophone at Normal range (10.000 in/s): Sensitivity = 1.61133 / 10 = 0.161133 V/(in/s). The stored value is the **inverse sensitivity** = 1/0.161133 = **6.206053 (in/s)/V**. Cross-check: 1.61133 V × 6.206053 = 10.000 in/s ✅. The firmware uses it as: `PPV (in/s) = ADC_voltage (V) × 6.206053`. Value is identical on all Instantel standard geophones — it is a hardware/firmware constant, NOT a user-configurable setting. Do NOT write this field. Open question §14 item "Max Geo Range float 6.2061" is now **RESOLVED**. |
|
||||
| 2026-04-20 | §7.6.4 (NEW), §7.9, Appendix B | **CONFIRMED — Recording Mode byte location.** Three targeted captures (4-20-26) confirmed `recording_mode` at anchor−8 in both the E5 read payload and the BW write payload (6-byte anchor `\xbe\x80\x00\x00\x00\x00`). BW write payload and E5 read payload are **byte-identical** around the anchor region — Blastware round-trips the wire-encoded E5 bytes verbatim with only the target field modified. Anchor position varies by ±1 depending on whether recording_mode = 0x03 (Histogram), because E5 wire-encodes `0x03` as the inner DLE+ETX pair `\x10\x03` (2 bytes), which S3FrameParser preserves as two literal bytes in `compliance_raw`. Enum: `0x00`=Single Shot, `0x01`=Continuous, `0x03`=Histogram, `0x04`=Histogram+Continuous. `0x02` value not yet observed. The byte at anchor−9 is `0x00` for Single Shot / Continuous, and `0x10` for Histogram (DLE prefix from E5 encoding) and Histogram+Continuous (actual config byte). See §7.6.4 for full details. |
|
||||
| 2026-04-21 | Appendix D (NEW) | **NEW — Blastware .N00 and .MLG file formats fully decoded.** `minimateplus/blastware_file.py` implements `write_n00()` and `write_mlg()`. N00 file format confirmed: 22B header + 21B STRT record + variable body + 26B footer. Body reconstructed from A5 bulk waveform stream frames with per-frame skip amounts (probe=7+strt_pos+21, A5[1]=13, A5[2+]=12, terminator=11) and DLE strip rule (strip `0x10` before `{0x02,0x03,0x04}`, keep following byte). Footer extracted verbatim from terminator frame's last 26 bytes. Split-pair edge case: when `frame.data[-1]==0x10` and `chk_byte∈{0x02,0x03,0x04}`, reunite both bytes before stripping and always remove trailing chk_byte (`stripped[:-1]`) — chk_byte is checksum, not payload. STRT record must be copied verbatim from A5[0]; bytes [10:20] are device-specific and cannot be reconstructed from Event fields. `write_n00` verified byte-perfect against `M529LIY6.N00` from 4-3-26-multi_event capture. MLG format: 308B header + N×292B records; CRC algorithm unknown (write as 0x0000). |
|
||||
| 2026-04-21 | Appendix D §D.5 (NEW) | **NEW — Blastware filename stem encoding confirmed; extension taxonomy partially decoded.** Stem is a 4-character uppercase base-36 encoding of `floor((event_local_time − 1985-01-01T00:00:00) / 1296)`, where 1296 = 36² seconds ≈ 21.6 minutes per unit. Epoch = January 1, 1985 (Instantel founding year). Confirmed against 6 independent events (April 1–9, 2026): all 6 stems (LIY6, LJ31, LJ8V, LJDY×3) match exactly; epoch estimate within ±7 minutes of midnight across all samples. Third char is always `'0'`. Serial prefix = `"M"` + last 3 decimal digits of serial. Multiple events within the same 21.6-minute window share a stem; their extension distinguishes them. Extension taxonomy: `.N00`=single-shot (compliance_raw recording_mode=0x00), `.9T0`=continuous (recording_mode=0x01) confirmed. `.490`, `.5K0`, `.980`, `.ML0` observed but not decoded — binary analysis shows they are structurally identical to `.9T0` files in all metadata regions (the A5 body's session-start compliance config reflects the state at session start, not at per-event capture time). Extension likely encodes the capture-time recording mode × sample rate combination, but cannot be determined from file body alone without capture-time compliance data. **DLE-shift note for reading recording_mode from file body:** the 0x10 constant at logical anchor−7 gets stripped by `_strip_inner_frame_dles` when sample_rate_HI = 0x04 (1024 sps), shifting recording_mode from logical anchor−8 to file position anchor−7. For sample_rate ≠ 1024 (0x08 or 0x10 as HI byte), no stripping occurs and recording_mode remains at file[anchor−8]. |
|
||||
| 2026-04-21 | §7.6.2, §5.3 | **CORRECTED — compliance_raw contains wire-encoded bytes, NOT logical bytes.** S3FrameParser appends DLE+ETX inner-frame pairs as two literal bytes to the frame body. Any `0x03` values in the compliance config appear in `compliance_raw` as `\x10\x03` (two bytes), not as a single `0x03`. The previous claim "S3FrameParser handles this transparently so compliance_raw contains logical (destuffed) bytes" was wrong. Consequence: `compliance_raw` is the wire-encoded E5 payload; anchor-relative reads work correctly because the anchor position automatically accounts for any DLE-encoded bytes before it. For write-back, round-tripping `compliance_raw` verbatim sends the correct wire bytes to the device. **DLE ETX escaping in write frames:** Blastware escapes `0x03` bytes in write frame data as `\x10\x03` on wire; our `build_bw_write_frame` does not (writes data raw). Device is confirmed to accept raw writes for all tested modes — likely uses the offset/length field for write frame framing, not ETX scanning. |
|
||||
| 2026-04-20 | §7.6.2, §7.9, Appendix B | **CONFIRMED — Geophone maximum range / sensitivity selector byte location.** Two targeted captures (4-20-26, geo sensitivity folder): one at Normal 10.000 in/s, one at Sensitive 1.250 in/s. E5 read payload diff: exactly 3 bytes differ at channel_label+33 for Tran/Vert/Long. Values: `0x00`=Normal 10.000 in/s, `0x01`=Sensitive 1.250 in/s. Same offset applies to the SUB 71 write payload (which is the same 2126-byte E5-format buffer round-tripped verbatim). **`channel_label+20` reads `0x01` in ALL captures regardless of range setting — it is NOT this field.** Previous hypothesis (uint8 at Tran+20, 0x01=Normal) was WRONG. Stored as `geo_range` in `ComplianceConfig`. Encoded to all three geo channel blocks (Tran/Vert/Long) at label+33. |
|
||||
| 2026-04-20 | §5.1, §5.3, §7.12 (NEW) | **NEW — Auto Call Home config protocol confirmed from 4-20-26 call home settings captures.** SUB 0x2C (Call Home Config READ, response 0xD3, data offset 0x7C=124) and SUB 0x7E/0x7F (WRITE + CONFIRM, response 0x81/0x80) confirmed. Write payload = read payload (125 bytes) + `\x00\x00` (127 bytes total). **DLE-escaped ETX at raw[117:119]:** the device returns logical value 0x03 (num_retries=3) as `\x10\x03` on the wire — S3FrameParser preserves both bytes as two literals, causing a +1 byte shift for all subsequent fields. Write frame sends these bytes verbatim (device interprets `\x10\x03` as literal value 3). Field map confirmed from 10-frame BW TX diff. See §7.12 for full layout. |
|
||||
@@ -2245,6 +2247,223 @@ Semantic Interpretation <- settings, events, responses
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Appendix D — Blastware Binary File Formats (.N00 / .MLG)
|
||||
|
||||
> ✅ CONFIRMED 2026-04-21 — all fields verified by binary diff of reconstructed vs reference
|
||||
> files from the 4-3-26-multi_event capture (M529LIY6.N00, BE11529.MLG).
|
||||
|
||||
### D.1 Common File Header (22 bytes)
|
||||
|
||||
All Blastware files (regardless of type) share an 18-byte prefix followed by a 4-byte type tag.
|
||||
|
||||
| Offset | Length | Value | Description |
|
||||
|---|---|---|---|
|
||||
| 0x00 | 6 | `10 00 01 80 00 00` | Fixed prefix |
|
||||
| 0x06 | 10 | `Instantel\x00` | ASCII string |
|
||||
| 0x10 | 2 | `07 2c` | Fixed suffix |
|
||||
| 0x12 | 4 | varies | File type tag (see below) |
|
||||
|
||||
**Total header: 22 bytes.**
|
||||
|
||||
**Type tags:**
|
||||
|
||||
| Extension | Type tag | Description |
|
||||
|---|---|---|
|
||||
| `.N00` | `00 12 03 00` | Single-shot waveform event |
|
||||
| `.MLG` | `22 01 0e a0` | Monitor log |
|
||||
|
||||
Blastware identifies file type by extension, not by type tag alone.
|
||||
|
||||
### D.2 Timestamp Encoding (Blastware files)
|
||||
|
||||
All timestamps in N00 and MLG files use an **8-byte big-endian format**:
|
||||
|
||||
| Byte | Field |
|
||||
|---|---|
|
||||
| 0 | day (uint8) |
|
||||
| 1 | month (uint8) |
|
||||
| 2–3 | year (uint16 BE) |
|
||||
| 4 | `0x00` (reserved) |
|
||||
| 5 | hour (uint8) |
|
||||
| 6 | minute (uint8) |
|
||||
| 7 | second (uint8) |
|
||||
|
||||
Example: `01 04 07 ea 00 00 1c 08` → April 1, 2026, 00:28:08.
|
||||
|
||||
Note: this differs from the 8-byte protocol timestamp (`[day][sub_code][month][year_HI][year_LO][0x00][hour][min][sec]` = 9 bytes) used in the device's on-wire 0C waveform records. The file format uses a compact 8-byte layout without the `sub_code` byte.
|
||||
|
||||
### D.3 N00 File Format — Single-Shot Waveform Event
|
||||
|
||||
**File layout:** `[22B header] [21B STRT record] [body bytes] [26B footer]`
|
||||
|
||||
#### D.3.1 STRT Record (21 bytes)
|
||||
|
||||
The STRT record immediately follows the 22-byte header.
|
||||
|
||||
| Offset | Length | Field | Notes |
|
||||
|---|---|---|---|
|
||||
| 0 | 4 | `STRT` | ASCII literal |
|
||||
| 4 | 2 | `ff fe` | Fixed |
|
||||
| 6 | 4 | event key (key4) | 4-byte waveform key |
|
||||
| 10 | 4 | device-specific | NOT a repeat of key4 — device-internal field |
|
||||
| 14 | 6 | device-specific | NOT zero-padded — device-internal fields |
|
||||
| 20 | 1 | rectime | uint8 seconds |
|
||||
|
||||
**Critical:** The STRT record must be copied verbatim from A5[0].data[7+strt_pos:] — bytes [10:20] contain device-specific values that cannot be reconstructed from protocol-level Event fields alone.
|
||||
|
||||
#### D.3.2 Body Bytes (variable)
|
||||
|
||||
The body is reconstructed from the raw A5 bulk waveform stream frames by stripping DLE framing markers and taking the appropriate slice of each frame's data section.
|
||||
|
||||
**Per-frame contribution (from `frame.data`):**
|
||||
|
||||
| Frame | Skip amount | Notes |
|
||||
|---|---|---|
|
||||
| A5[0] (probe) | `7 + strt_pos_in_w0 + 21` | Skip frame.data prefix + STRT record |
|
||||
| A5[1] | 13 | 7-byte prefix + 6-byte first-chunk header |
|
||||
| A5[2..N] | 12 | 7-byte prefix + 5-byte chunk header |
|
||||
| Terminator (page_key=0x0000) | 11 | 7-byte prefix + 4-byte terminator header |
|
||||
|
||||
**DLE strip rule:** For each frame's contribution (`frame.data[skip:]`), strip any `0x10` byte immediately followed by `0x02`, `0x03`, or `0x04`. Only the `0x10` is stripped; the following byte is kept as payload.
|
||||
|
||||
**Split-pair edge case:** When `frame.data[-1] == 0x10` AND `frame.chk_byte ∈ {0x02, 0x03, 0x04}`, the S3FrameParser split a DLE+XX pair at the payload/checksum boundary. Reunite the bytes before stripping (`relevant + bytes([chk_byte])`), then always remove the trailing chk_byte from the result (`stripped[:-1]`) — chk_byte is the wire checksum, never payload.
|
||||
|
||||
**Body/footer split:** Accumulate all frame contributions (data frames + terminator) into `all_bytes`. Then:
|
||||
- `body = all_bytes[:-26]` (variable length)
|
||||
- `footer = all_bytes[-26:]` (always 26 bytes — extracted from terminator content)
|
||||
|
||||
#### D.3.3 Footer (26 bytes)
|
||||
|
||||
The footer terminates the N00 file. Its bytes come directly from the terminator A5 frame's inner content — do NOT reconstruct from event metadata.
|
||||
|
||||
| Offset | Length | Field | Notes |
|
||||
|---|---|---|---|
|
||||
| 0 | 2 | `0e 08` | Fixed marker |
|
||||
| 2 | 8 | ts1 | Start timestamp (8B big-endian) |
|
||||
| 10 | 8 | ts2 | Stop timestamp (8B big-endian) |
|
||||
| 18 | 6 | `00 01 00 02 00 00` | Fixed |
|
||||
| 24 | 2 | CRC | 2-byte CRC — algorithm unconfirmed |
|
||||
|
||||
**CRC:** The 2-byte CRC at footer[24:26] has an unconfirmed algorithm. In M529LIY6.N00 it reads `fe da`. Attempts to match CRC-16/CCITT, CRC-16/IBM, CRC-32 (truncated), and 40+ polynomial/init combinations all failed. The writer copies it verbatim from the terminator frame.
|
||||
|
||||
### D.4 MLG File Format — Monitor Log
|
||||
|
||||
**File layout:** `[308B header] [N × 292B records]`
|
||||
|
||||
#### D.4.1 MLG Header (308 bytes)
|
||||
|
||||
| Offset | Length | Field | Notes |
|
||||
|---|---|---|---|
|
||||
| 0x00 | 22 | common header | prefix + `22 01 0e a0` type tag |
|
||||
| 0x16 | 16 | unknown | observed as zeros in BE11529.MLG |
|
||||
| 0x2A | 8 | serial number | null-padded ASCII (e.g. `"BE11529"`) |
|
||||
| 0x32 | remainder | zero pad | pads to 308 bytes total |
|
||||
|
||||
#### D.4.2 MLG Record (292 bytes each)
|
||||
|
||||
| Offset | Length | Field | Notes |
|
||||
|---|---|---|---|
|
||||
| 0 | 2 | CRC | 2-byte CRC — algorithm unconfirmed; write as `00 00` |
|
||||
| 2 | 4 | `22 01 0e 80` | Record marker |
|
||||
| 6 | 8 | ts1 | Start timestamp (8B big-endian) |
|
||||
| 14 | 8 | ts2 | Stop timestamp (8B big-endian); zeros if no stop |
|
||||
| 22 | 4 | flags | Record type flags (see below) |
|
||||
| 26 | 10 | serial | Null-padded ASCII serial number |
|
||||
| 36 | variable | text | Type-dependent content |
|
||||
| — | remainder | zero pad | pads to 292 bytes total |
|
||||
|
||||
**Record flags:**
|
||||
|
||||
| Value | Meaning |
|
||||
|---|---|
|
||||
| `ff ff 00 00` | Monitoring start with no stop recorded |
|
||||
| `01 00 02 00` | Triggered event (has ts1 + ts2) |
|
||||
| `02 00 00 00` | Monitoring interval (has ts1 + ts2) |
|
||||
|
||||
**Text content for triggered events (`flags = 01 00 02 00`):**
|
||||
|
||||
| Byte | Field |
|
||||
|---|---|
|
||||
| 0 | `0x08` |
|
||||
| 1–8 | ts1 copy (8B big-endian) |
|
||||
| 9+ | `"Geo: X.XXX in/s\x00"` ASCII geo threshold |
|
||||
|
||||
#### D.4.3 MLG CRC
|
||||
|
||||
The 2-byte CRC at record[0:2] uses an unconfirmed algorithm. Tested against CRC-16/CCITT, CRC-16/IBM, CRC-32 (truncated), word sums, XOR variants, and 40+ polynomial/init combinations — none matched. The writer emits `00 00`. Blastware may reject files with incorrect CRCs (impact on import unknown — TODO: test).
|
||||
|
||||
### D.5 Filename Encoding ✅ CONFIRMED 2026-04-21
|
||||
|
||||
Blastware assigns waveform filenames of the form `M<serial3><stem><ext>`, where:
|
||||
|
||||
#### D.5.1 Serial Prefix
|
||||
|
||||
`"M"` + last 3 decimal digits of the device serial number.
|
||||
|
||||
Example: serial `"BE11529"` → prefix `"M529"`.
|
||||
|
||||
#### D.5.2 Stem — 4-character base-36 timestamp encoding
|
||||
|
||||
```
|
||||
stem_int = floor((event_local_time − 1985-01-01T00:00:00_local) / 1296)
|
||||
stem = 4-character uppercase base-36 string of stem_int
|
||||
```
|
||||
|
||||
- **Unit:** 1296 seconds = 36² seconds ≈ 21.6 minutes per stem increment
|
||||
- **Epoch:** January 1, 1985, 00:00:00 local time (Instantel founding year)
|
||||
- **Alphabet:** `"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"` (digits then uppercase letters)
|
||||
- **Collision:** Events within the same 21.6-minute window share a stem; their extension distinguishes them
|
||||
|
||||
Confirmed against 6 events (April 1–9, 2026):
|
||||
|
||||
| Stem | Event time | Epoch estimate |
|
||||
|---|---|---|
|
||||
| LIY6 | 2026-04-01 00:28 | 1985-01-01 00:23 local |
|
||||
| LJ31 | 2026-04-03 15:20 | 1985-01-01 00:22 local |
|
||||
| LJ8V | 2026-04-06 18:52 | 1985-01-01 00:25 local |
|
||||
| LJDY | 2026-04-09 12:46 | 1985-01-01 00:23 local |
|
||||
|
||||
All 6 stems match exactly. Epoch estimates converge within ±7 minutes of midnight Jan 1 1985.
|
||||
|
||||
#### D.5.3 Extension taxonomy
|
||||
|
||||
Third character of extension is always `'0'`. File type is identified by extension, not by the type tag in the header (all waveform extensions share type tag `00 12 03 00`).
|
||||
|
||||
| Extension | Recording mode | Sample rate | Status |
|
||||
|---|---|---|---|
|
||||
| `.N00` | Single Shot (0x00) | 1024 sps | ✅ CONFIRMED |
|
||||
| `.9T0` | Continuous (0x01) | 1024 sps | ✅ CONFIRMED |
|
||||
| `.490` | ? | ? | ❓ observed from M529LJ8V.490 |
|
||||
| `.5K0` | ? | ? | ❓ observed from M529LJDY.5K0 |
|
||||
| `.980` | ? | ? | ❓ observed from M529LJDY.980 |
|
||||
| `.ML0` | ? | ? | ❓ observed from M529LJDY.ML0 (167s duration; possibly Histogram) |
|
||||
|
||||
**Why 5 extensions for "Continuous"?** Binary analysis of all 6 example files shows that `.9T0`, `.490`, `.5K0`, `.980`, `.ML0` are byte-for-byte identical in all metadata regions (compliance anchor block, channel descriptor blocks `Tran/Vert/Long/MicL`). The A5 frame 7 body reflects the **session-start** compliance config, not the per-event capture config. All 5 files show recording_mode=0x01 and sample_rate=1024 in the body. The extension must therefore encode the **capture-time** compliance state — likely a combination of recording mode, sample rate, and possibly mic units or other options. This cannot be determined from file body alone without capture-time compliance data from the 0C record sub_code and the actual waveform sample count.
|
||||
|
||||
**DLE-shift offset note for reading recording_mode from N00/9T0 body:**
|
||||
|
||||
The compliance block in the file body has been through `_strip_inner_frame_dles`. The 0x10 constant at logical `anchor−7` (between recording_mode and sample_rate_HI) gets stripped when sample_rate_HI = `0x04` (1024 sps), because `0x10` precedes `0x04 ∈ {0x02,0x03,0x04}`. After stripping, the anchor shifts left by 1, so:
|
||||
|
||||
| 1024 sps (strip occurs) | 2048 or 4096 sps (no strip) |
|
||||
|---|---|
|
||||
| `file[anc−7]` = recording_mode | `file[anc−8]` = recording_mode |
|
||||
| `file[anc−6:anc−4]` = sample_rate | `file[anc−6:anc−4]` = sample_rate |
|
||||
|
||||
For 1024 sps files, the expected file bytes around the anchor are:
|
||||
```
|
||||
file[anc−9]: mode_prefix (0x00 for Single Shot/Continuous; 0x10 for Histogram)
|
||||
file[anc−8]: 0x00 (was recording_mode, but shifted away — now reads 0x00 for mode_prefix)
|
||||
file[anc−7]: recording_mode (0x00=Single Shot, 0x01=Continuous, etc.)
|
||||
file[anc−6]: 0x04 (sample_rate_HI for 1024 sps)
|
||||
file[anc−5]: 0x00 (sample_rate_LO)
|
||||
file[anc−4]: histogram_interval_HI
|
||||
file[anc−3]: histogram_interval_LO
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*All findings reverse-engineered from live RS-232 bridge captures.*
|
||||
*Cross-referenced from 2026-03-02 with Instantel MiniMate Plus Operator Manual (716U0101 Rev 15).*
|
||||
*This is a living document — append changelog entries and timestamps as new findings are confirmed or corrected.*
|
||||
Reference in New Issue
Block a user