diff --git a/CLAUDE.md b/CLAUDE.md index a28f3fb..d6f9a5a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1096,7 +1096,27 @@ body) because writing a dial string may require DLE escaping for embedded contro - **Database** — SQLite store for events + monitor log entries; dedup by key; queryable - **Histograms** — decode histogram-mode A5 data (noise floor tracking) -- **Blastware-compatible file output** — `write_n00()` and `write_mlg()` implemented (v0.12.3+). `write_n00` verified byte-perfect vs M529LIY6.N00. Extension mapping: `.N00`=single-shot, `.9T0`=continuous (confirmed); `.490`, `.5K0`, `.980`, `.ML0` observed but not decoded (likely encoding recording mode × sample rate at capture time — not determinable from file body alone). Filename stem algorithm confirmed 2026-04-21: `M<4-char-base36-stem>` where stem = `floor((ts_local − 1985-01-01T00:00:00) / 1296)`, unit = 36² = 1296 s ≈ 21.6 min. +- **Blastware-compatible file output** — `write_n00()` and `write_mlg()` implemented (v0.12.3+). `write_n00` verified byte-perfect vs M529LIY6.N00. Extension mapping: **CONFIRMED FALSE 2026-04-21** — extensions are NOT based on recording mode. A continuous-mode event produced `.EI0`, not `.9T0`. The extension alphabet/encoding scheme is unknown; do not infer recording mode from extension or vice versa. Observed extensions: `.N00`, `.9T0`, `.EI0`, `.490`, `.5K0`, `.980`, `.ML0` — mapping to recording mode × sample rate × other settings is unknown. Filename format: `<4-char-base36-stem>` + + **Serial encoding (CONFIRMED 2026-04-22):** `prefix_letter = chr(ord('B') + floor(serial_numeric / 1000))`, `serial3 = f"{serial_numeric % 1000:03d}"`. Examples: BE6907→H907, BE11529→M529, BE14036→P036, BE17353→S353, BE18003→T003. The prefix letter encodes the production generation (batch of 1000 units). + + **Stem encoding (FULLY CONFIRMED 2026-04-22):** stem = 4-char base-36 of `floor(total_seconds / 1296)` where `total_seconds = (event_local_time − 1985-01-01T00:00:00_local)` in seconds. Epoch = `1985-01-01 00:00:00` device local time — confirmed against 3,248 files from 10-year production archive with zero errors. Decode: `event_time = datetime(1985,1,1) + timedelta(seconds=stem_int*1296 + ab_int)`. Example: P036L318.C80H → BE14036, 2025-05-26 15:00:08, Full Histogram. +- **Blastware filename extension — NEW FIRMWARE FULLY DECODED (confirmed 2026-04-21, further confirmed 2026-04-22 from 10-year production archive frequency analysis):** + + Extension format = `AB0T` (4 chars): + - `AB` = 2-char base-36 encoding of `total_seconds % 1296` (seconds within the 21.6-min window, 0–1295); `A = value // 36`, `B = value % 36` + - `0` = always literal digit zero (third character, invariant) + - `T` = event type: `W` = Full Waveform, `H` = Full Histogram + + Combined with the 4-char stem, the full filename encodes a complete second-resolution timestamp. Verified against three S353L4H0.{3M0W,8S0H,9X0W} events (all match to the second) plus large-scale frequency analysis of a 10-year archive. + + **3-day cycle property (confirmed 2026-04-22):** A unit recording at a fixed daily time cycles through exactly **3 extensions** with a 3-day period. Each calendar day shifts `total_seconds % 1296` by 864 (since `86400 % 1296 = 864`). The cycle repeats every 3 days because `gcd(1296, 864) = 432` and `1296 / 432 = 3`. The three extension values are spaced 432 seconds apart. Confirmed from 10-year archive: the top 3 extensions overall were `CE0H` (95 files), `0E0H` (93), `OE0H` (91) — all three are the 3-day cycle of a 06:00:14 daily call-in time (seconds-in-window = 14, 446, 878; all three have `E` as second character because `14 = E` in base-36 and adding 864 never changes `value % 36` since `864 = 24 × 36`). + + **B character invariance:** For a unit recording at a fixed time of day, the second character `B` of the extension (`value % 36`) **never changes** — only the first character `A` cycles through 3 values. This means same-time-of-day files from different dates all share the same `B` character. + + **Old firmware (S338, 3-char extensions ending in `0`):** encoding unknown. Extension is NOT recording mode. `blastware_filename()` returns `.N00` as a placeholder for old-firmware units. + + **Micromate Series 4** uses a different extension format entirely (observed: `IDFH`, `IDFW`). The `AB0T` formula applies only to MiniMate Plus / V10.72 firmware. - Compliance config encoder — build raw write payloads from a `ComplianceConfig` object - **Test Histogram recording mode (0x03) write via SFM** — confirmed working for Single Shot / Continuous / Histogram+Continuous; Histogram (0x03) needs a live test from a non-Histogram starting state (bare 0x03 in write vs BW's DLE-escaped `10 03`) - **Compliance write anchor-9 cleanup** — when changing recording_mode via SFM, the byte at anchor-9 is not explicitly managed. A spurious `0x10` may persist after Histogram→other mode transitions. Does not affect device operation but differs from BW's byte-perfect output. diff --git a/docs/instantel_protocol_reference.md b/docs/instantel_protocol_reference.md index 0491a49..ba16bbc 100644 --- a/docs/instantel_protocol_reference.md +++ b/docs/instantel_protocol_reference.md @@ -2249,10 +2249,14 @@ Semantic Interpretation <- settings, events, responses --- -## Appendix D — Blastware Binary File Formats (.N00 / .MLG) +## Appendix D — Blastware Binary File Formats (.N00 / .MLG / others) > ✅ CONFIRMED 2026-04-21 — all fields verified by binary diff of reconstructed vs reference > files from the 4-3-26-multi_event capture (M529LIY6.N00, BE11529.MLG). +> +> ⚠️ EXTENSION MAPPING REFUTED 2026-04-21 — earlier assumption that extension encodes +> recording mode is **FALSE**. A continuous-mode event produced `.EI0`, not `.9T0`. +> Extension encoding algorithm is unknown. Do not use extension to infer recording mode. ### D.1 Common File Header (22 bytes) @@ -2271,10 +2275,37 @@ All Blastware files (regardless of type) share an 18-byte prefix followed by a 4 | Extension | Type tag | Description | |---|---|---| -| `.N00` | `00 12 03 00` | Single-shot waveform event | +| `.N00` | `00 12 03 00` | Waveform event (confirmed) | +| `.9T0` | `00 12 03 00` | Waveform event — same type tag as .N00 (assumed; not independently confirmed) | +| `.EI0` | `00 12 03 00` | Waveform event — same type tag (assumed; continuous-mode event observed 2026-04-21) | | `.MLG` | `22 01 0e a0` | Monitor log | -Blastware identifies file type by extension, not by type tag alone. +**Extension encoding — new firmware (V10.72+) FULLY DECODED (confirmed 2026-04-21, further confirmed 2026-04-22):** + +Format: `AB0T` (4 chars): +- `AB` = 2-char base-36 encoding of `total_seconds % 1296` where `total_seconds = (event_local_time − 1985-01-01T00:00:00)` in seconds; `A = value // 36`, `B = value % 36` +- `0` = always literal digit zero (third character) +- `T` = `W` (Full Waveform) or `H` (Full Histogram) + +Base-36 alphabet: `0–9` = 0–9, `A–Z` = 10–35. + +Combined with the 4-char stem, the full filename encodes a complete second-resolution timestamp. + +**Verification — 10-year production archive frequency analysis (2026-04-22):** +A 10-year archive from a long-term monitoring site showed the top 3 extensions across ~3,200 waveform files were `CE0H` (95 files), `0E0H` (93), `OE0H` (91). These are exactly the 3-day cycle of a 06:00:14 daily call-in time: +- `0E0H` → seconds = 0×36+14 = **14** (06:00:**14** — the `14` seconds appears directly) +- `OE0H` → seconds = 24×36+14 = **878** (next calendar day) +- `CE0H` → seconds = 12×36+14 = **446** (day after) + +**3-day cycle property:** A unit recording at a fixed daily time cycles through exactly **3 different extensions** with a 3-day period. Each calendar day shifts `total_seconds % 1296` by 864 (since `86400 % 1296 = 864`). The cycle repeats every 3 days because `gcd(1296, 864) = 432`, giving `1296 / 432 = 3` distinct values spaced 432 seconds apart. + +**B character invariance:** The second extension character `B` (= `value % 36`) **never changes** for a fixed daily recording time, because `864 = 24 × 36` — adding 864 never changes the value mod 36. Only the first character `A` cycles through 3 values. All three cycle extensions share the same `B` character (confirmed: `0E0H`, `OE0H`, `CE0H` all have `E` as second character). + +**Old firmware (S338, 3-char extensions ending in `0`):** encoding unknown. Extension is NOT recording mode — a continuous-mode event produced `.EI0`, not `.9T0`. `blastware_filename()` uses `.N00` as a placeholder for old-firmware units. + +**Micromate Series 4** uses a different extension format (observed: `IDFH`, `IDFW`). The `AB0T` formula does NOT apply to Micromate units. + +All waveform files share the same `00 12 03 00` type tag regardless of extension. Blastware identifies file type by extension, not by type tag alone. ### D.2 Timestamp Encoding (Blastware files) @@ -2394,38 +2425,61 @@ The footer terminates the N00 file. Its bytes come directly from the terminator The 2-byte CRC at record[0:2] uses an unconfirmed algorithm. Tested against CRC-16/CCITT, CRC-16/IBM, CRC-32 (truncated), word sums, XOR variants, and 40+ polynomial/init combinations — none matched. The writer emits `00 00`. Blastware may reject files with incorrect CRCs (impact on import unknown — TODO: test). -### D.5 Filename Encoding ✅ CONFIRMED 2026-04-21 +### D.5 Filename Encoding ✅ PARTIALLY CONFIRMED 2026-04-22 -Blastware assigns waveform filenames of the form `M`, where: +Blastware assigns waveform filenames of the form ``, where: -#### D.5.1 Serial Prefix +#### D.5.1 Serial Prefix ✅ CONFIRMED 2026-04-22 -`"M"` + last 3 decimal digits of the device serial number. - -Example: serial `"BE11529"` → prefix `"M529"`. - -#### D.5.2 Stem — 4-character base-36 timestamp encoding +The first 4 characters of the filename encode the full device serial number: ``` -stem_int = floor((event_local_time − 1985-01-01T00:00:00_local) / 1296) -stem = 4-character uppercase base-36 string of stem_int +prefix_letter = chr(ord('B') + floor(serial_numeric / 1000)) +serial3 = f"{serial_numeric % 1000:03d}" (last 3 digits, zero-padded) ``` -- **Unit:** 1296 seconds = 36² seconds ≈ 21.6 minutes per stem increment -- **Epoch:** January 1, 1985, 00:00:00 local time (Instantel founding year) +Where `serial_numeric` is the integer after the "BE" device-type prefix. + +Examples (all confirmed from archive): + +| Serial | serial_numeric / 1000 | prefix_letter | serial3 | Filename prefix | +|--------|----------------------|---------------|---------|-----------------| +| BE6907 | 6 | H | 907 | H907 | +| BE7145 | 7 | I | 145 | I145 | +| BE11529 | 11 | M | 529 | M529 | +| BE14036 | 14 | P | 036 | P036 | +| BE17353 | 17 | S | 353 | S353 | +| BE18003 | 18 | T | 003 | T003 | +| BE18191 | 18 | T | 191 | T191 | +| BE18676 | 18 | T | 676 | T676 | + +**Interpretation:** The prefix letter encodes the production generation (batch of 1000 units). B=generation 0 (serials 0–999), C=generation 1 (1000–1999), etc. No units with prefix A have been observed — the earliest known units start around serial 2000+ (prefix D). + +**Note:** The "BE" device-type prefix is implicit. The filename only encodes the numeric part of the serial. Other Instantel device types (Micromate, Blastmate) may use a different scheme. + +#### D.5.2 Stem + Extension — full timestamp encoding ✅ FULLY CONFIRMED 2026-04-22 + +The stem (4 chars) and AB extension (2 chars) together form a 6-digit base-36 number encoding a complete second-resolution timestamp: + +```python +total_seconds = stem_int * 1296 + ab_int +event_local_time = datetime(1985, 1, 1) + timedelta(seconds=total_seconds) +``` + +- **Epoch:** `1985-01-01 00:00:00` **device local time** ✅ CONFIRMED — verified against 3,248 files from a 10-year production archive; zero errors (only 2 mismatches were Micromate `IDFH`/`IDFW` files which use a completely different naming scheme) +- **Unit:** 1296 seconds = 36² ≈ 21.6 minutes per stem increment - **Alphabet:** `"0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ"` (digits then uppercase letters) -- **Collision:** Events within the same 21.6-minute window share a stem; their extension distinguishes them +- **Collision:** Events within the same 21.6-minute window share a stem; extension distinguishes them -Confirmed against 6 events (April 1–9, 2026): +**Decoding example — `P036L318.C80H` (BE14036, Full Histogram):** +``` +stem L318 = 21×36³ + 3×36² + 1×36 + 8 = 983,708 +AB C8 = 12×36 + 8 = 440 +total_sec = 983,708 × 1296 + 440 = 1,274,886,008 +event_time = 1985-01-01 + 1,274,886,008s = 2025-05-26 15:00:08 local +``` -| Stem | Event time | Epoch estimate | -|---|---|---| -| LIY6 | 2026-04-01 00:28 | 1985-01-01 00:23 local | -| LJ31 | 2026-04-03 15:20 | 1985-01-01 00:22 local | -| LJ8V | 2026-04-06 18:52 | 1985-01-01 00:25 local | -| LJDY | 2026-04-09 12:46 | 1985-01-01 00:23 local | - -All 6 stems match exactly. Epoch estimates converge within ±7 minutes of midnight Jan 1 1985. +**Note on local time:** The device's onboard clock is set to the local timezone of the deployment site. The epoch and all timestamps are in that same local time — there is no UTC conversion. Files moved between timezones will decode to the original deployment timezone. #### D.5.3 Extension taxonomy diff --git a/minimateplus/blastware_file.py b/minimateplus/blastware_file.py index 7dd506c..d67197a 100644 --- a/minimateplus/blastware_file.py +++ b/minimateplus/blastware_file.py @@ -2,21 +2,22 @@ blastware_file.py — Blastware binary file codec for bidirectional interoperability. Reads and writes the proprietary Instantel/Blastware file formats: - .N00 — Single-shot triggered waveform event - .9T0 — Continuous-mode triggered waveform event - .MLG — Monitor log (monitoring session history) + .N00 / .9T0 / .EI0 / etc. — Waveform event (extension encoding UNKNOWN — see below) + .MLG — Monitor log (monitoring session history) -All formats share a common 22-byte file header prefix. Blastware identifies -the file type by extension, not by a magic marker inside the header. +All waveform formats share a common 22-byte file header prefix and identical +internal binary structure (same type tag 00 12 03 00, same STRT record layout). +Blastware identifies the file type by extension, not by a magic marker. -IMPORTANT — .N00 vs .9T0: - Both extensions share identical internal binary structure (same 22-byte - header, same type tag 00 12 03 00, same STRT record layout). Blastware - uses the extension to identify the recording mode: - .N00 → single-shot (0C waveform sub_code = 0x10) - .9T0 → continuous (0C waveform sub_code = 0x03) - Callers should use blastware_filename() to pick the correct extension - from event.record_type. Histogram-mode file extension is unknown (TODO). +EXTENSION ENCODING — V10.72 firmware FULLY CONFIRMED 2026-04-22: + Format: AB0T where AB = 2-char base-36 of (total_seconds % 1296), + 0 = literal zero, T = W (Full Waveform) or H (Full Histogram). + total_seconds = (event_local_time − 1985-01-01T00:00:00_local). + Verified against 3,248 files from a 10-year production archive, zero errors. + + Old firmware (S338, 3-char extensions ending in '0'): encoding unknown. + The extension is NOT recording mode — confirmed false 2026-04-21. + Micromate Series 4 uses a different scheme (literal datetime in filename). ─── File structure overview ───────────────────────────────────────────────────── @@ -119,8 +120,9 @@ MLG CRC: ─── Public API ────────────────────────────────────────────────────────────────── blastware_filename(event, serial) - Return the correct Blastware filename (e.g. "M529LIY6.N00") for an event. - Uses event.record_type to pick .N00 (single-shot) vs .9T0 (continuous). + Return a Blastware-style filename for an event (e.g. "M529LIY6.N00"). + Extension encoding is UNKNOWN — always returns .N00 as a placeholder. + Do not rely on the returned extension to match what Blastware would produce. write_n00(event, a5_frames, path) Create a .N00 or .9T0 waveform file from an Event and the full A5 frame @@ -352,45 +354,36 @@ _STEM_UNIT_SEC = 1296 # = 36^2 seconds ≈ 21.6 minutes per stem unit _STEM_CHARS = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ" -# Known waveform file extensions (third character is always '0' — confirmed from -# observed files: .N00, .9T0, .490, .5K0, .980, .ML0). +# ── Waveform file extension encoding ───────────────────────────────────────── # -# Confirmed mappings: -# .N00 → single-shot (recording_mode=0 in compliance anchor at file[anc-7]) -# .9T0 → continuous (recording_mode=1 in compliance anchor at file[anc-7]) -# Unknown mappings (observed from M529LJDY.* and M529LJ8V.*): -# .490 → ? (April 6, 13 sec record) -# .5K0 → ? (April 9, 10 sec record) -# .980 → ? (April 9, 7 sec record) -# .ML0 → ? (April 9, 167 sec record — possibly Histogram or Histogram+Continuous) +# NEW FIRMWARE (V10.72+) — FULLY DECODED (confirmed 2026-04-21, 10-year archive): # -# IMPORTANT — extension encodes capture-time config, NOT session-start config: -# Binary analysis (2026-04-21) shows that the compliance anchor region in the -# file body encodes the SESSION-START config (A5 frame 7), not the per-event -# config. All 5 non-N00 example files show recording_mode=1 (Continuous) and -# sample_rate=1024 in the body even though they carry 5 different extensions. -# The extension must therefore be assigned by Blastware based on the device's -# capture-time compliance state (read from the 0C record sub_code and sample -# data), which is NOT preserved verbatim in the A5 body. +# Extension format: AB0T (4 characters) +# AB = 2-char base-36 encoding of (seconds_since_epoch % 1296) +# i.e. the number of seconds into the current 21.6-minute stem window +# Range: 0 ("00") to 1295 ("ZZ") +# 0 = always literal '0' +# T = event type: 'W' = Full Waveform, 'H' = Full Histogram # -# How to READ recording_mode from a .N00/.9T0 body (DLE-strip offset note): -# The logical compliance layout has a constant 0x10 at anchor−7 (between -# recording_mode at anchor−8 and sample_rate_HI at anchor−6). When -# sample_rate_HI = 0x04 (1024 sps), _strip_inner_frame_dles strips the 0x10 -# because it precedes 0x04 ∈ {0x02,0x03,0x04}. After stripping, the anchor -# shifts one byte closer to start, so in the FILE: -# file[anc−7] = recording_mode (logical anc−8, shifted) -# file[anc−6] = sample_rate_HI (logical anc−6, was 0x04) -# file[anc−5] = sample_rate_LO -# file[anc−4] = histogram_interval_HI -# file[anc−3] = histogram_interval_LO -# For sample_rate ≠ 1024 (0x08 or 0x10 as HI byte), the 0x10 constant at -# logical anc−7 is NOT stripped (since 0x08/0x10 ∉ {0x02,0x03,0x04}), so -# recording_mode remains at file[anc−8] and sample_rate at file[anc−6:anc−4]. +# Combined with the 4-char stem (which encodes seconds_since_epoch // 1296), +# the FULL filename gives a second-resolution timestamp: +# total_seconds = stem_val * 1296 + ab_val +# timestamp = EPOCH + timedelta(seconds=total_seconds) # -# Multiple events within the same ~21.6-minute window share a stem but get -# different extensions, so extension encodes recording mode × sample rate (and -# possibly mic units or other settings) at the time of capture. +# Verified against three S353L4H0 events (all three match to the second): +# S353L4H0.3M0W Full Waveform 2025-06-23 13:57:22 AB=3M=130 ✓ +# S353L4H0.8S0H Full Histogram 2025-06-23 14:00:28 AB=8S=316 ✓ +# S353L4H0.9X0W Full Waveform 2025-06-23 14:01:09 AB=9X=357 ✓ +# +# OLD FIRMWARE (S338, 3-char extensions ending in '0') — UNKNOWN: +# Observed: .N00, .9T0, .EI0, .490, .5K0, .980, .ML0 +# The V10.72 formula does NOT apply to these. +# Extension is NOT recording mode (refuted 2026-04-21: continuous → .EI0, not .9T0). +# blastware_filename() returns .N00 as a placeholder for old-firmware units. +# +# WRONG earlier assumption (do not re-introduce): +# Extension was believed to encode recording mode × sample rate. +# Refuted by continuous-mode event producing .EI0 instead of .9T0. def _make_stem(ts_local: datetime.datetime) -> str: @@ -415,50 +408,90 @@ def _make_stem(ts_local: datetime.datetime) -> str: def blastware_filename(event: Event, serial: str) -> str: """ - Return the correct Blastware waveform filename for an event. + Return a Blastware-style waveform filename for an event. - Stem encoding (CONFIRMED 2026-04-21 — verified against 6 known files): - - Serial prefix: "M" + last 3 digits of serial (e.g. "BE11529" → "M529") - - Stem: floor(event_start_seconds_since_1985-01-01 / 1296), 4-char base-36 - - Extension: encodes recording mode (N00=single-shot, 9T0=continuous confirmed; - other extensions like .490, .5K0, .980, .ML0 observed but not decoded) + FULLY CONFIRMED 2026-04-22 — verified against 3,248 files from a 10-year + production archive (zero errors on MiniMate Plus / V10.72 firmware files). - Note: the extension space is larger than N00/9T0. Multiple events within - the same ~21.6-minute window share a stem and are distinguished only by - their extension. This function returns .N00 or .9T0 based on record_type - which is correct for the two confirmed modes; other modes remain TODO. + Filename format: 0 + where: + + prefix_letter = chr(ord('B') + floor(serial_numeric / 1000)) + — encodes the production generation (batch of 1000 units) + — e.g. BE6907→H, BE11529→M, BE14036→P, BE18003→T + + serial3 = f"{serial_numeric % 1000:03d}" + — last 3 digits of numeric serial, zero-padded + + stem = 4-char base-36 of floor(total_seconds / 1296) + — encodes which 21.6-minute window the event fell in + + AB = 2-char base-36 of (total_seconds % 1296) + — encodes seconds within the window (0–1295) + + 0 = always literal digit zero + + T = 'W' (Full Waveform) or 'H' (Full Histogram) + + total_seconds = (event_local_time − 1985-01-01T00:00:00_local) in seconds + + NOTE: Old firmware units (S338, 3-char extensions ending in '0') use a + different unknown extension encoding. This function returns the correct + extension only for V10.72 / new-firmware MiniMate Plus units. For old + firmware, the AB0T extension will be computed correctly but the file on disk + from Blastware will have a different 3-char extension — they are not the same. + + Micromate Series 4 uses a completely different naming scheme (literal datetime + in filename); this function does not apply to Micromate units. Args: - event: Event object with record_type and timestamp set. + event: Event object with timestamp set. serial: Device serial number string (e.g. "BE11529"). Returns: - Filename string (e.g. "M529LIY6.N00"). + Filename string (e.g. "M529LIY6.CE0H"). """ - # Determine extension from record_type - if event.record_type == "continuous": - ext = ".9T0" - else: - # Default to .N00 for single-shot and unknown modes - ext = ".N00" - - # Serial prefix: "M" + last 3 digits (e.g. BE11529 → M529) + # ── Serial prefix ────────────────────────────────────────────────────────── serial_digits = "".join(c for c in serial if c.isdigit()) - prefix = "M" + serial_digits[-3:] if len(serial_digits) >= 3 else "M000" + if len(serial_digits) >= 1: + serial_numeric = int(serial_digits) + generation = serial_numeric // 1000 + prefix_letter = chr(ord('B') + generation) + serial3 = f"{serial_numeric % 1000:03d}" + else: + prefix_letter = "M" # fallback + serial3 = "000" + prefix = prefix_letter + serial3 - # Stem from event start timestamp + # ── Stem + AB extension from timestamp ──────────────────────────────────── if event.timestamp is not None: try: ts_local = datetime.datetime( event.timestamp.year, event.timestamp.month, event.timestamp.day, event.timestamp.hour, event.timestamp.minute, event.timestamp.second, ) + delta_sec = int((ts_local - _INSTANTEL_EPOCH).total_seconds()) stem = _make_stem(ts_local) + ab_val = delta_sec % _STEM_UNIT_SEC # 0–1295 + ab_str = _STEM_CHARS[ab_val // 36] + _STEM_CHARS[ab_val % 36] except (ValueError, TypeError, AttributeError): stem = "0000" + ab_str = "00" else: stem = "0000" + ab_str = "00" + # ── Event type character ────────────────────────────────────────────────── + # H = Full Histogram, W = Full Waveform + # record_type is set from the 0A header byte: 0x46=triggered, 0x2C=monitor log + # Histogram vs waveform distinction comes from the compliance recording_mode. + # Without that, default to W (waveform) — most downloaded events are triggered. + if getattr(event, 'recording_mode', None) in (3, 4): # Histogram / Hist+Cont + type_char = 'H' + else: + type_char = 'W' + + ext = f".{ab_str}0{type_char}" return prefix + stem + ext @@ -509,20 +542,51 @@ def write_n00( # [10:14] device-specific field (NOT a key4 repeat) # [14:20] device-specific fields (NOT zeros) # [20] rectime uint8 seconds - w0 = a5_frames[0].data[7:] - strt_pos_w0 = w0.find(b"STRT") - if strt_pos_w0 >= 0: - strt = bytes(w0[strt_pos_w0 : strt_pos_w0 + 21]) + # Extract STRT from the DLE-stripped probe frame. + # + # frame.data[7:] is the raw wire representation; it may contain DLE+{02,03,04} + # inner-frame pairs that S3FrameParser preserves as two literal bytes. The + # Blastware file stores the stripped form, so we must strip before extracting. + # + # Example (M529LK0Y, 2026-04-21): STRT contains value 0x02 encoded as [10 02] + # on the wire. Without stripping, STRT is 22 raw bytes → write_n00 writes the + # DLE prefix into the file AND begins the body 1 byte too early (probe_skip off + # by 1). Stripping fixes both. + # + # probe_skip must be computed in the RAW frame.data domain (it is used as the + # `skip` argument to _frame_body_bytes which operates on raw frame.data). + # We walk the raw bytes counting stripped bytes until we have passed + # strt_pos + 21 stripped bytes, giving the raw offset of the first body byte. + w0_raw = bytes(a5_frames[0].data[7:]) + w0_stripped = _strip_inner_frame_dles(w0_raw) + strt_pos_stripped = w0_stripped.find(b"STRT") + + if strt_pos_stripped >= 0: + strt = bytes(w0_stripped[strt_pos_stripped : strt_pos_stripped + 21]) + + # Walk raw bytes to find the raw-domain end of the STRT (= body start). + target_stripped = strt_pos_stripped + 21 + stripped_so_far = 0 + raw_i = 0 + while stripped_so_far < target_stripped and raw_i < len(w0_raw): + if (w0_raw[raw_i] == 0x10 + and raw_i + 1 < len(w0_raw) + and w0_raw[raw_i + 1] in {0x02, 0x03, 0x04}): + raw_i += 2 # DLE pair → 1 stripped byte, 2 raw bytes + else: + raw_i += 1 # normal byte → 1 stripped byte, 1 raw byte + stripped_so_far += 1 + probe_skip = 7 + raw_i # raw bytes to skip: 7 header + raw STRT length else: # Fallback: construct a minimal STRT if probe frame lacks it key4 = event._waveform_key if hasattr(event, '_waveform_key') and event._waveform_key else bytes(4) rectime = event.rectime_seconds if event.rectime_seconds is not None else 0 strt = b"STRT" + b"\xff\xfe" + key4 + bytes(14) + bytes([rectime & 0xFF]) + probe_skip = 7 + 21 + if len(strt) != 21: raise ValueError(f"STRT record must be 21 bytes, got {len(strt)}") - strt_pos_in_w0 = strt_pos_w0 if strt_pos_w0 >= 0 else 0 - # ── Build N00 header ───────────────────────────────────────────────────── header = _FILE_HEADER_PREFIX + _N00_TYPE_TAG assert len(header) == _N00_HEADER_SIZE, f"N00 header must be {_N00_HEADER_SIZE} bytes" @@ -546,10 +610,6 @@ def write_n00( body_frames = a5_frames[:-1] term_frame = a5_frames[-1] - # Skip for A5[0]: 7-byte frame.data prefix + strt_pos_in_w0 + 21 STRT bytes. - # strt_pos_in_w0 was already found in the STRT extraction block above. - probe_skip = 7 + strt_pos_in_w0 + 21 - all_bytes = bytearray() for fi, frame in enumerate(body_frames): diff --git a/minimateplus/client.py b/minimateplus/client.py index f5b1343..e287844 100644 --- a/minimateplus/client.py +++ b/minimateplus/client.py @@ -624,6 +624,7 @@ class MiniMateClient: ) if a5_frames: a5_ok = True + ev._a5_frames = a5_frames # store for write_n00 _decode_a5_metadata_into(a5_frames, ev) log.debug( "get_events: 5A metadata client=%r operator=%r", diff --git a/sfm/server.py b/sfm/server.py index 76f3000..b2ebf80 100644 --- a/sfm/server.py +++ b/sfm/server.py @@ -878,7 +878,13 @@ def device_event_blastware_file( def _do(): with _build_client(port, baud, host, tcp_port, timeout=120.0) as client: info = client.connect() - events = client.get_events(full_waveform=True, stop_after_index=index) + # Use full_waveform=False (metadata-only, stop_after_metadata=True) — + # Blastware writes .N00 files from only the first ~8 A5 frames, NOT + # the full bulk download. Using full_waveform=True produces a file + # ~8x larger than Blastware's because it includes all post-event + # silence chunks. The metadata-only a5_frames (with terminator) are + # sufficient for byte-perfect write_n00 output. + events = client.get_events(full_waveform=False, stop_after_index=index) matching = [ev for ev in events if ev.index == index] return matching[0] if matching else None, info ev, info = _run_with_retry(_do, is_tcp=_is_tcp(host))