Compare commits
3 Commits
4448c74f6c
...
a684d3e642
| Author | SHA1 | Date | |
|---|---|---|---|
|
a684d3e642 | ||
|
|
22d4023ea0 | ||
|
|
a5a21a6c32 |
2
.gitignore
vendored
2
.gitignore
vendored
@@ -1 +1 @@
|
||||
/bridge/captures
|
||||
/bridges/captures/
|
||||
@@ -35,7 +35,7 @@ from typing import Optional
|
||||
|
||||
import serial
|
||||
|
||||
VERSION = "v0.5.0"
|
||||
VERSION = "v0.5.1"
|
||||
|
||||
DLE = 0x10
|
||||
STX = 0x02
|
||||
|
||||
@@ -40,6 +40,9 @@
|
||||
| 2026-03-02 | §14 Open Questions | `0x082A` hypothesis refined: 2090 decimal. At 1024 sps, 2 sec record = 2048 samples. Possible that 0x082A = total samples including 0.25s pre-trigger (256 samples) at some adjusted rate. Needs capture with different record time. |
|
||||
| 2026-03-02 | §14 Open Questions | **NEW items added:** Trigger sample width (default=2), Auto Window (1-9 sec), Aux Trigger (enabled/disabled) — all confirmed settings from operator manual not yet mapped in protocol. |
|
||||
| 2026-03-02 | §14 Open Questions | Monitoring LCD Cycle resolved — removed from open questions. |
|
||||
| 2026-03-03 | §2 Frame Structure | **UPDATED:** Documented BW/S3 framing asymmetry. BW uses bare STX (`0x02`); S3 uses DLE+STX (`0x10 0x02`). ETX initially believed symmetric — see correction below. |
|
||||
| 2026-03-03 | §2 Frame Structure | **CORRECTED:** ETX is also asymmetric. BW uses bare ETX (`0x03`); S3 uses DLE+ETX (`0x10 0x03`). Confirmed via checksum validation: 91/98 BW frames pass with bare `0x03` as terminator. All `10 03` sequences in `raw_bw.bin` are in-payload data, never followed by `41 02` (next frame start). Full confirmed grammar: BW=`02`...`03`, S3=`10 02`...`10 03`. Both sides stuff literal `0x10` as `10 10`. This is the formally confirmed link-layer grammar. |
|
||||
| 2026-03-03 | Appendix A | **CORRECTED:** Previous entry stated logger strips DLE from ETX. This was wrong — applied to older logger only. `s3_bridge v0.5.0+` is lossless. Section rewritten to reflect current flat raw dump format. |
|
||||
| 2026-03-02 | Appendix A | **CORRECTED:** Previous entry stated logger strips DLE from ETX. This was wrong — it applied to an older logger version. `s3_bridge v0.5.0` confirmed to preserve raw wire bytes including `0x10 0x03` intact. HxD inspection of new capture confirmed `10 03` present in S3→BW record payloads. |
|
||||
| 2026-03-02 | Appendix A | **UPDATED:** New capture architecture: two flat raw wire dumps per session (`raw_s3.bin`, `raw_bw.bin`), one per direction, no record wrapper. Replaces structured `.bin` format for parser input. |
|
||||
| 2026-03-02 | Appendix A | **PARSER:** Deterministic DLE state machine implemented (`s3_parser.py`). Three states: `IDLE → IN_FRAME → AFTER_DLE`. Replaces heuristic global scanning. Properly handles DLE stuffing (`10 10` → literal `10`). Only complete STX→ETX pairs counted as frames. |
|
||||
@@ -62,38 +65,70 @@
|
||||
|
||||
## 2. Frame Structure
|
||||
> ⚠️ **2026-02-26 — CORRECTED:** Previous version incorrectly identified `0x41` as STX and `0x02`/`0x03` as bare frame delimiters. The protocol uses proper **DLE framing**. See below.
|
||||
> ⚠️ **2026-03-03 — UPDATED:** Frame start AND end are both asymmetric by direction. See confirmed grammar below.
|
||||
|
||||
Every message follows this structure:
|
||||
### Confirmed Link-Layer Grammar ✅ CONFIRMED — 2026-03-03
|
||||
|
||||
The two sides of the connection use **fully asymmetric framing**. DLE stuffing applies on both sides.
|
||||
|
||||
| Direction | STX (frame start) | ETX (frame end) | Stuffing | Notes |
|
||||
|---|---|---|---|---|
|
||||
| S3 → BW (device) | `0x10 0x02` (DLE+STX) | `0x10 0x03` (DLE+ETX) | `0x10` → `0x10 0x10` | Full DLE framing |
|
||||
| BW → S3 (Blastware) | `0x02` (bare STX) | `0x03` (bare ETX) | `0x10` → `0x10 0x10` | Bare delimiters, DLE stuffing only |
|
||||
|
||||
**Evidence:**
|
||||
- 91/98 BW frames validate checksum when parsed with bare `0x03` as ETX
|
||||
- All `10 03` sequences in `raw_bw.bin` are in-payload data — none are followed by `41 02` (next frame start)
|
||||
- `10 03` appearing in BW payload is always `10 10 03` origin (stuffed DLE + literal `03`) — the S3 device correctly parses this via its own state machine without false ETX detection
|
||||
- S3 captures consistently terminate with `10 03` confirmed via HxD
|
||||
|
||||
**Practical impact for parsers:**
|
||||
- Parser on `raw_s3.bin`: trigger on `10 02`, terminate on `10 03`
|
||||
- Parser on `raw_bw.bin`: trigger on bare `02`, terminate on bare `03`
|
||||
- Both parsers must handle `10 10` → literal `10` unstuffing
|
||||
- ETX detection must be state-machine-aware (not raw byte search) to avoid false matches on stuffed sequences
|
||||
|
||||
### Frame Structure by Direction
|
||||
|
||||
**S3 → BW (device responses):**
|
||||
```
|
||||
[ACK] [DLE+STX] [PAYLOAD...] [CHECKSUM] [DLE+ETX]
|
||||
0x41 0x10 0x02 N bytes 1 byte 0x10 0x03
|
||||
```
|
||||
|
||||
**BW → S3 (Blastware commands):**
|
||||
```
|
||||
[ACK] [STX] [PAYLOAD...] [CHECKSUM] [ETX]
|
||||
0x41 0x02 N bytes 1 byte 0x03
|
||||
```
|
||||
|
||||
### Special Byte Definitions
|
||||
|
||||
| Token | Raw Bytes | Meaning | Certainty |
|
||||
|---|---|---|---|
|
||||
| ACK | `0x41` (ASCII `'A'`) | Acknowledgment / ready token. Standalone single byte. Sent before every frame by both sides. | ✅ CONFIRMED |
|
||||
| DLE | `0x10` | Data Link Escape. Prefixes the next byte to give it special meaning. | ✅ CONFIRMED — 2026-02-26 |
|
||||
| STX | `0x10 0x02` | DLE+STX = Start of frame (two-byte sequence) | ✅ CONFIRMED — 2026-02-26 |
|
||||
| ETX | `0x10 0x03` | DLE+ETX = End of frame (two-byte sequence) | ✅ CONFIRMED — 2026-02-26 |
|
||||
| CHECKSUM | 1 byte | 8-bit sum of de-stuffed payload bytes, modulo 256. Sits between payload and DLE+ETX. | ✅ CONFIRMED |
|
||||
| DLE | `0x10` | Data Link Escape. Used for stuffing on both sides; also prefixes STX/ETX on S3 side only. | ✅ CONFIRMED — 2026-02-26 |
|
||||
| STX (S3) | `0x10 0x02` | DLE+STX = Start of frame sent by device | ✅ CONFIRMED — 2026-02-26 |
|
||||
| STX (BW) | `0x02` | Bare STX = Start of frame sent by Blastware | ✅ CONFIRMED — 2026-03-03 |
|
||||
| ETX (S3) | `0x10 0x03` | DLE+ETX = End of frame sent by device | ✅ CONFIRMED — 2026-02-26 |
|
||||
| ETX (BW) | `0x03` | Bare ETX = End of frame sent by Blastware | ✅ CONFIRMED — 2026-03-03 |
|
||||
| CHECKSUM | 1 byte | 8-bit sum of de-stuffed payload bytes, modulo 256. Sits between payload and ETX. | ✅ CONFIRMED |
|
||||
|
||||
### DLE Byte Stuffing Rule
|
||||
> ✅ CONFIRMED — 2026-02-26
|
||||
|
||||
Any `0x10` byte appearing **naturally in the payload data** is escaped by doubling it: `0x10` → `0x10 0x10`. This prevents the parser from confusing real data with frame control sequences.
|
||||
Any `0x10` byte appearing **naturally in the payload data** is escaped by doubling it: `0x10` → `0x10 0x10`. This applies on **both sides** of the connection.
|
||||
|
||||
- **Transmit:** Replace every `0x10` in payload with `0x10 0x10`
|
||||
- **Receive:** Replace every `0x10 0x10` in the frame body with a single `0x10`
|
||||
|
||||
| Sequence on wire | Meaning |
|
||||
|---|---|
|
||||
| `0x10 0x02` | Frame START — only valid at beginning |
|
||||
| `0x10 0x03` | Frame END |
|
||||
| `0x10 0x10` | Escaped literal `0x10` byte in payload data |
|
||||
| Any other `0x10 0xXX` | Protocol error / undefined |
|
||||
| Sequence on wire | S3 context | BW context |
|
||||
|---|---|---|
|
||||
| `0x10 0x02` | Frame START | Stuffed `0x10` + payload `0x02` |
|
||||
| `0x10 0x03` | Frame END | Stuffed `0x10` + payload `0x03` |
|
||||
| `0x10 0x10` | Escaped literal `0x10` | Escaped literal `0x10` |
|
||||
| `0x02` | Payload byte | Frame START |
|
||||
| `0x03` | Payload byte | Frame END |
|
||||
|
||||
### Frame Parser Notes
|
||||
|
||||
@@ -595,14 +630,17 @@ Timestamps are 6-byte sequences appearing in event headers and waveform keys.
|
||||
## 10. DLE Byte Stuffing
|
||||
> ✅ **CONFIRMED — 2026-02-26** (previously ❓ SPECULATIVE)
|
||||
|
||||
This protocol uses standard **DLE (Data Link Escape) byte stuffing**, a classical technique used in protocols like IBM BISYNC dating to the 1970s.
|
||||
This protocol uses standard **DLE (Data Link Escape) byte stuffing**, a classical technique used in protocols like IBM BISYNC dating to the 1970s. Both sides stuff literal `0x10` bytes as `0x10 0x10`. The framing delimiters differ by direction — see §2.
|
||||
|
||||
### Parser State Machine
|
||||
### Parser State Machine — S3→BW direction (device responses)
|
||||
|
||||
Trigger on DLE+STX, terminate on DLE+ETX.
|
||||
|
||||
```
|
||||
IDLE:
|
||||
receive 0x41 → emit ACK event, stay IDLE
|
||||
receive 0x10 → goto WAIT_STX
|
||||
receive anything → discard, stay IDLE
|
||||
|
||||
WAIT_STX:
|
||||
receive 0x02 → frame started, goto IN_FRAME
|
||||
@@ -615,8 +653,27 @@ IN_FRAME:
|
||||
ESCAPE:
|
||||
receive 0x03 → frame complete — validate checksum, process buffer, goto IDLE
|
||||
receive 0x10 → append single 0x10 to buffer, goto IN_FRAME (stuffed literal)
|
||||
receive 0x02 → error (nested STX), goto IDLE
|
||||
receive anything → error, goto IDLE
|
||||
receive anything → append DLE + byte to buffer (recovery), goto IN_FRAME
|
||||
```
|
||||
|
||||
### Parser State Machine — BW→S3 direction (Blastware commands)
|
||||
|
||||
Trigger on bare STX, terminate on bare ETX. DLE only appears in stuffing context.
|
||||
|
||||
```
|
||||
IDLE:
|
||||
receive 0x41 → emit ACK event, stay IDLE
|
||||
receive 0x02 → frame started, goto IN_FRAME
|
||||
receive anything → discard, stay IDLE
|
||||
|
||||
IN_FRAME:
|
||||
receive 0x10 → goto ESCAPE
|
||||
receive 0x03 → frame complete — validate checksum, process buffer, goto IDLE
|
||||
receive any byte → append to buffer, stay IN_FRAME
|
||||
|
||||
ESCAPE:
|
||||
receive 0x10 → append single 0x10 to buffer, goto IN_FRAME (stuffed literal)
|
||||
receive anything → append DLE + byte to buffer (recovery), goto IN_FRAME
|
||||
```
|
||||
|
||||
---
|
||||
@@ -757,23 +814,25 @@ Build in this order — each step is independently testable:
|
||||
---
|
||||
|
||||
## Appendix A — s3_bridge Capture Format
|
||||
> ✅ **CONFIRMED — 2026-02-26**
|
||||
> ⚠️ **This section describes tooling behavior, not protocol semantics.**
|
||||
> **2026-03-03 — CORRECTED:** Previous version of this section incorrectly stated that `s3_bridge` strips DLE from ETX. This applied to an older logger version only. `s3_bridge v0.5.0+` is confirmed lossless. See Appendix C for full validation details.
|
||||
|
||||
> ⚠️ **This behavior is not part of the Instantel protocol. It is an artifact of the bridge logger implementation.**
|
||||
### Current Format (v0.5.0+) ✅ CONFIRMED — 2026-03-03
|
||||
|
||||
The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger makes one modification:
|
||||
As of `s3_bridge v0.5.0`, captures are produced as **two flat raw wire dump files per session**:
|
||||
|
||||
| Wire sequence | In .bin file | Notes |
|
||||
|---|---|---|
|
||||
| `0x10 0x03` (DLE+ETX) | `0x03` | DLE stripped from end-of-frame marker |
|
||||
| All other bytes | Unchanged | ACK, DLE+STX, stuffed payload, checksum all preserved verbatim |
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `raw_s3.bin` | All bytes transmitted by S3 (device → Blastware), in order |
|
||||
| `raw_bw.bin` | All bytes transmitted by BW (Blastware → device), in order |
|
||||
|
||||
**Practical impact for parsing `.bin` files:**
|
||||
- Frame end: scan for bare `0x03` (not `0x10 0x03`)
|
||||
- Checksum: the byte immediately before the bare `0x03` is the checksum
|
||||
- Everything else (ACK detection, DLE+STX, payload de-stuffing) works as documented in §10
|
||||
Every byte on the wire is written verbatim — no modification, no record headers, no timestamps. `0x10 0x03` (DLE+ETX) is preserved intact.
|
||||
|
||||
> ⚠️ This means checksums cannot be verified on frames where the stuffed payload ends in `0x10` — that trailing `0x10` would normally be the DLE prefix of ETX, but the logger strips it, making the frame boundary ambiguous in that edge case. In practice this has not been observed in captured data.
|
||||
**Practical impact for parsing:**
|
||||
- `raw_s3.bin`: trigger on `0x10 0x02`, terminate on `0x10 0x03` (DLE+ETX)
|
||||
- `raw_bw.bin`: trigger on bare `0x02`, terminate on bare `0x03`
|
||||
- Both: handle `0x10 0x10` → literal `0x10` unstuffing
|
||||
- ETX detection must be state-machine-aware on both sides to avoid false matches on stuffed sequences
|
||||
|
||||
---
|
||||
|
||||
@@ -846,9 +905,9 @@ The `.bin` files produced by `s3_bridge` are **not raw wire bytes**. The logger
|
||||
The earlier stripping behavior applied to a previous logger version. v0.5.0 is confirmed lossless with respect to wire bytes.
|
||||
|
||||
**Confirmed wire framing:**
|
||||
- Frame start: `0x10 0x02` (DLE STX) ✅
|
||||
- Frame end: `0x10 0x03` (DLE ETX) ✅
|
||||
- DLE stuffing: `0x10 0x10` in payload = literal `0x10` ✅
|
||||
- S3→BW: frame start `0x10 0x02`, frame end `0x10 0x03` ✅
|
||||
- BW→S3: frame start `0x02`, frame end `0x03` ✅
|
||||
- Both sides: DLE stuffing `0x10 0x10` = literal `0x10` ✅
|
||||
|
||||
### C.2 Capture Architecture (Current)
|
||||
|
||||
@@ -875,6 +934,8 @@ STATE_AFTER_DLE — last byte was 0x10, awaiting qualifier
|
||||
|
||||
**Transitions:**
|
||||
|
||||
**S3→BW parser states:**
|
||||
|
||||
| Current State | Byte | Action | Next State |
|
||||
|---|---|---|---|
|
||||
| IDLE | `10 02` | Begin new frame | IN_FRAME |
|
||||
@@ -883,7 +944,19 @@ STATE_AFTER_DLE — last byte was 0x10, awaiting qualifier
|
||||
| IN_FRAME | `10` | — | AFTER_DLE |
|
||||
| AFTER_DLE | `10` | Append literal `0x10` | IN_FRAME |
|
||||
| AFTER_DLE | `03` | Frame complete, emit | IDLE |
|
||||
| AFTER_DLE | other | Treat as payload (recovery) | IN_FRAME |
|
||||
| AFTER_DLE | other | Append DLE + byte (recovery) | IN_FRAME |
|
||||
|
||||
**BW→S3 parser states:**
|
||||
|
||||
| Current State | Byte | Action | Next State |
|
||||
|---|---|---|---|
|
||||
| IDLE | `02` | Begin new frame | IN_FRAME |
|
||||
| IDLE | any | Discard | IDLE |
|
||||
| IN_FRAME | `03` | Frame complete, emit | IDLE |
|
||||
| IN_FRAME | `10` | — | AFTER_DLE |
|
||||
| IN_FRAME | other | Append to payload | IN_FRAME |
|
||||
| AFTER_DLE | `10` | Append literal `0x10` | IN_FRAME |
|
||||
| AFTER_DLE | other | Append DLE + byte (recovery) | IN_FRAME |
|
||||
|
||||
**Properties:**
|
||||
- Does not scan globally for `10 02`
|
||||
@@ -894,9 +967,9 @@ STATE_AFTER_DLE — last byte was 0x10, awaiting qualifier
|
||||
### C.4 Observed Traffic (Validation Captures)
|
||||
|
||||
**`raw_bw.bin`** (Blastware → S3):
|
||||
- 7 complete frames via state machine
|
||||
- Mostly small command/control frames, several zero-length payloads
|
||||
- Bare `0x02` used as STX (asymmetric — BW does not use DLE STX)
|
||||
- 98 complete frames via state machine (bare STX + bare ETX mode)
|
||||
- 91/98 checksums validate; 7 failures are large frames containing in-payload `10 03` sequences that a naive scanner misreads as ETX
|
||||
- Bare `0x02` STX and bare `0x03` ETX confirmed; DLE used for stuffing only
|
||||
- Contains project metadata strings: `"Standard Recording Setup.set"`, `"Claude test2"`, `"Location #1 - Brians House"`
|
||||
|
||||
**`raw_s3.bin`** (S3 → Blastware):
|
||||
|
||||
98
parsers/bw_frames.jsonl
Normal file
98
parsers/bw_frames.jsonl
Normal file
@@ -0,0 +1,98 @@
|
||||
{"index": 0, "start_offset": 0, "end_offset": 21, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 1, "start_offset": 21, "end_offset": 42, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 2, "start_offset": 42, "end_offset": 63, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 3, "start_offset": 63, "end_offset": 84, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 4, "start_offset": 84, "end_offset": 105, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 5, "start_offset": 105, "end_offset": 126, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 6, "start_offset": 126, "end_offset": 147, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 7, "start_offset": 147, "end_offset": 168, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 8, "start_offset": 168, "end_offset": 189, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 9, "start_offset": 189, "end_offset": 210, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 10, "start_offset": 210, "end_offset": 231, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 11, "start_offset": 231, "end_offset": 252, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 12, "start_offset": 252, "end_offset": 273, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 13, "start_offset": 273, "end_offset": 294, "payload_len": 17, "payload_hex": "1000150000000000000000000000000025", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 14, "start_offset": 294, "end_offset": 315, "payload_len": 17, "payload_hex": "10001500000a000000000000000000002f", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 15, "start_offset": 315, "end_offset": 427, "payload_len": 108, "payload_hex": "10006800005a00000000000000000000005809000000010107cb00061e00010107cb00140000000000173b00000000000000000000000000000100000000000100000000000000010001000000000000000000000000000000000064000000000000001effdc0000100200c8", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 16, "start_offset": 427, "end_offset": 448, "payload_len": 17, "payload_hex": "1000730000000000000000000000000083", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 17, "start_offset": 448, "end_offset": 1497, "payload_len": 1045, "payload_hex": "1000710010040000000000000000000000082a6400001004100400003c0000be800000000040400000001003000f000000073dbb457a3db956e1000100015374616e64617264205265636f7264696e672053657475702e7365740000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000050726f6a6563743a0000000000000000000000000000544553542000000000000000000000000000000000000000000000000000000000000000000000000000436c69656e743a000000000000000000000000000000436c6175646520746573743200000000000000000000000000000000000000000000000000000000000055736572204e616d653a00000000000000000000000054657272612d4d656368616e69637320496e632e202d20422e204861727269736f6e000000000000000053656973204c6f633a000000000000000000000000004c6f636174696f6e202331202d20427269616e7320486f75736500000000000000000000000000000000457874656e646564204e6f74657300000000000000000a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 18, "start_offset": 1497, "end_offset": 2574, "payload_len": 1073, "payload_hex": "1000710010040000001004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000015472616e000000010050000f0028001510021003011004001003000040c697fd00003f19999a696e2e00400000002f730000000156657274000000010050000f0028001510021003011004001003000040c697fd00003f19999a696e2e00400000002f73000000014c6f6e67000000010050000f0028001510021003011004001003000040c697fd00003f19999a696e2e00400000002f73000000004d69634c000000100200c80032000a000a1002d501db000500003d38560800003c1374bc707369003cac0831284c29000010025472616e320000010050000f0028001510021003011004001003000040c697fd00003f000000696e2e00400000002f73000000100256657274320000010050000f0028001510021003011004001003000040c697fd00003f000000696e2e00400000002f7300000010024c6f6e67320000010050000f0028001510021003011004001003000040c697fd00003f000000696e2e00400000002f73000000004d69634c1002", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 19, "start_offset": 2574, "end_offset": 2641, "payload_len": 63, "payload_hex": "10007100002c00000800000000000000320000100200c80032000a000a1002d501db000500003d38560800003c23d70a707369003cac0831284c29007cea32", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 20, "start_offset": 2641, "end_offset": 2662, "payload_len": 17, "payload_hex": "1000720000000000000000000000000082", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 21, "start_offset": 2662, "end_offset": 2711, "payload_len": 45, "payload_hex": "10008200001c00000000000000000000001ad5000001080affffffffffffffffffffffffffffffffffff00009e", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 22, "start_offset": 2711, "end_offset": 2732, "payload_len": 17, "payload_hex": "1000830000000000000000000000000093", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 23, "start_offset": 2732, "end_offset": 2957, "payload_len": 221, "payload_hex": "1000690000ca0000000000000000000000c8080000010001000100010001000100010010020001001e0010020001000a000a4576656e742053756d6d617279205265706f7274000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002580000801018c76af", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 24, "start_offset": 2957, "end_offset": 2978, "payload_len": 17, "payload_hex": "1000740000000000000000000000000084", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 25, "start_offset": 2978, "end_offset": 2999, "payload_len": 17, "payload_hex": "1000720000000000000000000000000082", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 26, "start_offset": 2999, "end_offset": 3020, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 27, "start_offset": 3020, "end_offset": 3041, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 28, "start_offset": 3041, "end_offset": 3062, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 29, "start_offset": 3062, "end_offset": 3083, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 30, "start_offset": 3083, "end_offset": 3104, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 31, "start_offset": 3104, "end_offset": 3125, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 32, "start_offset": 3125, "end_offset": 3146, "payload_len": 17, "payload_hex": "1000150000000000000000000000000025", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 33, "start_offset": 3146, "end_offset": 3167, "payload_len": 17, "payload_hex": "10001500000a000000000000000000002f", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 34, "start_offset": 3167, "end_offset": 3188, "payload_len": 17, "payload_hex": "1000010000000000000000000000000011", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 35, "start_offset": 3188, "end_offset": 3209, "payload_len": 17, "payload_hex": "10000100009800000000000000000000a9", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 36, "start_offset": 3209, "end_offset": 3230, "payload_len": 17, "payload_hex": "1000080000000000000000000000000018", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 37, "start_offset": 3230, "end_offset": 3251, "payload_len": 17, "payload_hex": "1000080000580000000000000000000070", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 38, "start_offset": 3251, "end_offset": 3272, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 39, "start_offset": 3272, "end_offset": 3293, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 40, "start_offset": 3293, "end_offset": 3314, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 41, "start_offset": 3314, "end_offset": 3335, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 42, "start_offset": 3335, "end_offset": 3356, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 43, "start_offset": 3356, "end_offset": 3377, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 44, "start_offset": 3377, "end_offset": 3398, "payload_len": 17, "payload_hex": "1000010000000000000000000000000011", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 45, "start_offset": 3398, "end_offset": 3419, "payload_len": 17, "payload_hex": "10000100009800000000000000000000a9", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 46, "start_offset": 3419, "end_offset": 3440, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 47, "start_offset": 3440, "end_offset": 3461, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 48, "start_offset": 3461, "end_offset": 3482, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 49, "start_offset": 3482, "end_offset": 3503, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 50, "start_offset": 3503, "end_offset": 3524, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 51, "start_offset": 3524, "end_offset": 3545, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 52, "start_offset": 3545, "end_offset": 3566, "payload_len": 17, "payload_hex": "1000150000000000000000000000000025", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 53, "start_offset": 3566, "end_offset": 3587, "payload_len": 17, "payload_hex": "10001500000a000000000000000000002f", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 54, "start_offset": 3587, "end_offset": 3608, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 55, "start_offset": 3608, "end_offset": 3629, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 56, "start_offset": 3629, "end_offset": 3650, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 57, "start_offset": 3650, "end_offset": 3671, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 58, "start_offset": 3671, "end_offset": 3692, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 59, "start_offset": 3692, "end_offset": 3713, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 60, "start_offset": 3713, "end_offset": 3734, "payload_len": 17, "payload_hex": "1000150000000000000000000000000025", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 61, "start_offset": 3734, "end_offset": 3755, "payload_len": 17, "payload_hex": "10001500000a000000000000000000002f", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 62, "start_offset": 3755, "end_offset": 3776, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 63, "start_offset": 3776, "end_offset": 3797, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 64, "start_offset": 3797, "end_offset": 3818, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 65, "start_offset": 3818, "end_offset": 3839, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 66, "start_offset": 3839, "end_offset": 3860, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 67, "start_offset": 3860, "end_offset": 3881, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 68, "start_offset": 3881, "end_offset": 3902, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 69, "start_offset": 3902, "end_offset": 3923, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 70, "start_offset": 3923, "end_offset": 3944, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 71, "start_offset": 3944, "end_offset": 3965, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 72, "start_offset": 3965, "end_offset": 3986, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 73, "start_offset": 3986, "end_offset": 4007, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 74, "start_offset": 4007, "end_offset": 4028, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 75, "start_offset": 4028, "end_offset": 4049, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 76, "start_offset": 4049, "end_offset": 4070, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 77, "start_offset": 4070, "end_offset": 4091, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 78, "start_offset": 4091, "end_offset": 4112, "payload_len": 17, "payload_hex": "10005b000000000000000000000000006b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 79, "start_offset": 4112, "end_offset": 4133, "payload_len": 17, "payload_hex": "10005b000030000000000000000000009b", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 80, "start_offset": 4133, "end_offset": 4154, "payload_len": 17, "payload_hex": "1000010000000000000000000000000011", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 81, "start_offset": 4154, "end_offset": 4175, "payload_len": 17, "payload_hex": "10000100009800000000000000000000a9", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 82, "start_offset": 4175, "end_offset": 4196, "payload_len": 17, "payload_hex": "10002e000000000000000000000000003e", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 83, "start_offset": 4196, "end_offset": 4217, "payload_len": 17, "payload_hex": "10002e00001a0000000000000000000058", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 84, "start_offset": 4217, "end_offset": 4238, "payload_len": 17, "payload_hex": "1000010000000000000000000000000011", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 85, "start_offset": 4238, "end_offset": 4259, "payload_len": 17, "payload_hex": "10000100009800000000000000000000a9", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 86, "start_offset": 4259, "end_offset": 4280, "payload_len": 17, "payload_hex": "10001a000000000000000000006400008e", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 87, "start_offset": 4280, "end_offset": 4302, "payload_len": 18, "payload_hex": "10001a001004000000000000000064000092", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 88, "start_offset": 4302, "end_offset": 4325, "payload_len": 19, "payload_hex": "10001a00100400000010040000000064000096", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 89, "start_offset": 4325, "end_offset": 4346, "payload_len": 17, "payload_hex": "10001a00002a00000800000000640000c0", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 90, "start_offset": 4346, "end_offset": 4367, "payload_len": 17, "payload_hex": "1000090000000000000000000000000019", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 91, "start_offset": 4367, "end_offset": 4388, "payload_len": 17, "payload_hex": "1000090000ca00000000000000000000e3", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 92, "start_offset": 4388, "end_offset": 4409, "payload_len": 17, "payload_hex": "1000080000000000000000000000000018", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 93, "start_offset": 4409, "end_offset": 4430, "payload_len": 17, "payload_hex": "1000080000580000000000000000000070", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 94, "start_offset": 4430, "end_offset": 4451, "payload_len": 17, "payload_hex": "1000010000000000000000000000000011", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 95, "start_offset": 4451, "end_offset": 4472, "payload_len": 17, "payload_hex": "10000100009800000000000000000000a9", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 96, "start_offset": 4472, "end_offset": 4493, "payload_len": 17, "payload_hex": "1000080000000000000000000000000018", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
{"index": 97, "start_offset": 4493, "end_offset": 4514, "payload_len": 17, "payload_hex": "1000080000580000000000000000000070", "trailer_hex": "", "checksum_valid": null, "checksum_type": null, "checksum_hex": null}
|
||||
BIN
parsers/raw_bw.bin
Normal file
BIN
parsers/raw_bw.bin
Normal file
Binary file not shown.
BIN
parsers/raw_s3.bin
Normal file
BIN
parsers/raw_s3.bin
Normal file
Binary file not shown.
@@ -1,232 +1,364 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
s3_parse.py — parse Instantel/Series3-like DLE-framed serial captures from a raw .bin logger.
|
||||
s3_parser.py — Unified Instantel frame parser (S3 + BW).
|
||||
|
||||
Assumptions (based on your HxD patterns):
|
||||
- Frames are delimited by DLE STX (0x10 0x02) ... DLE ETX (0x10 0x03)
|
||||
- Inside payload, a literal 0x10 is escaped as 0x10 0x10
|
||||
- After ETX, there may be a trailer (often CRC16, maybe + seq/flags)
|
||||
Modes:
|
||||
- s3: DLE STX (10 02) ... DLE ETX (10 03)
|
||||
- bw: ACK+STX (41 02) ... ETX (03)
|
||||
|
||||
Stuffing:
|
||||
- Literal 0x10 in payload is stuffed as 10 10 in both directions.
|
||||
|
||||
Checksums:
|
||||
- BW frames appear to use more than one checksum style depending on message type.
|
||||
Small frames often validate with 1-byte SUM8.
|
||||
Large config/write frames appear to use a 2-byte CRC16 variant.
|
||||
|
||||
In BW mode we therefore validate candidate ETX positions using AUTO checksum matching:
|
||||
- SUM8 (1 byte)
|
||||
- CRC16 variants (2 bytes), both little/big endian
|
||||
If any match, we accept the ETX as a real frame terminator.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import List, Optional, Tuple
|
||||
from typing import Callable, Dict, List, Optional, Tuple
|
||||
|
||||
DLE = 0x10
|
||||
STX = 0x02
|
||||
ETX = 0x03
|
||||
EOT = 0x04
|
||||
ACK = 0x41
|
||||
|
||||
__version__ = "0.2.2"
|
||||
|
||||
# How the capture was produced:
|
||||
# - Raw serial captures include DLE+ETX (`0x10 0x03`).
|
||||
# - The s3_bridge `.bin` logger strips the DLE byte from ETX, so frames end with a
|
||||
# bare `0x03`. See docs/instantel_protocol_reference.md §Appendix A.
|
||||
ETX_MODE_AUTO = "auto"
|
||||
ETX_MODE_RAW = "raw" # expect DLE+ETX
|
||||
ETX_MODE_STRIPPED = "stripped" # expect bare ETX
|
||||
|
||||
@dataclass
|
||||
class Frame:
|
||||
index: int
|
||||
start_offset: int
|
||||
end_offset: int
|
||||
payload_raw: bytes # as captured between STX..ETX, still escaped
|
||||
payload: bytes # unescaped
|
||||
trailer: bytes # bytes immediately after ETX (length chosen by user)
|
||||
crc_match: Optional[str] # best-guess CRC type if verified, else None
|
||||
payload_raw: bytes # de-stuffed bytes between STX..ETX (includes checksum bytes at end)
|
||||
payload: bytes # payload without checksum bytes
|
||||
trailer: bytes
|
||||
checksum_valid: Optional[bool]
|
||||
checksum_type: Optional[str]
|
||||
checksum_hex: Optional[str]
|
||||
|
||||
def unescape_dle(payload_escaped: bytes) -> bytes:
|
||||
"""Convert DLE-stuffing: 0x10 0x10 => 0x10 (literal DLE)."""
|
||||
out = bytearray()
|
||||
i = 0
|
||||
n = len(payload_escaped)
|
||||
while i < n:
|
||||
b = payload_escaped[i]
|
||||
if b == DLE:
|
||||
if i + 1 < n and payload_escaped[i + 1] == DLE:
|
||||
out.append(DLE)
|
||||
i += 2
|
||||
continue
|
||||
# If we see a single DLE not followed by DLE inside payload,
|
||||
# keep it as-is (conservative) — could be real data or malformed capture.
|
||||
out.append(b)
|
||||
i += 1
|
||||
return bytes(out)
|
||||
|
||||
# ---- CRC helpers (we don't know which one yet, so we try a few) ----
|
||||
# ------------------------
|
||||
# Checksum / CRC helpers
|
||||
# ------------------------
|
||||
|
||||
def checksum8_sum(data: bytes) -> int:
|
||||
"""SUM8: sum(payload) & 0xFF"""
|
||||
return sum(data) & 0xFF
|
||||
|
||||
|
||||
def crc16_ibm(data: bytes) -> int:
|
||||
# CRC-16/IBM (aka ARC) poly=0xA001 (reflected 0x8005), init=0x0000
|
||||
# CRC-16/IBM (aka ARC) poly=0xA001, init=0x0000, refin/refout true
|
||||
crc = 0x0000
|
||||
for b in data:
|
||||
crc ^= b
|
||||
for _ in range(8):
|
||||
if crc & 1:
|
||||
crc = (crc >> 1) ^ 0xA001
|
||||
else:
|
||||
crc >>= 1
|
||||
crc = (crc >> 1) ^ 0xA001 if (crc & 1) else (crc >> 1)
|
||||
return crc & 0xFFFF
|
||||
|
||||
|
||||
def crc16_ccitt_false(data: bytes) -> int:
|
||||
# CRC-16/CCITT-FALSE poly=0x1021, init=0xFFFF, no reflection
|
||||
# CRC-16/CCITT-FALSE poly=0x1021, init=0xFFFF, refin/refout false
|
||||
crc = 0xFFFF
|
||||
for b in data:
|
||||
crc ^= (b << 8)
|
||||
for _ in range(8):
|
||||
if crc & 0x8000:
|
||||
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
|
||||
else:
|
||||
crc = (crc << 1) & 0xFFFF
|
||||
crc = ((crc << 1) ^ 0x1021) & 0xFFFF if (crc & 0x8000) else (crc << 1) & 0xFFFF
|
||||
return crc
|
||||
|
||||
|
||||
def crc16_x25(data: bytes) -> int:
|
||||
# CRC-16/X-25 poly=0x1021, init=0xFFFF, refin/refout true, xorout=0xFFFF
|
||||
# CRC-16/X-25 poly=0x8408 (reflected), init=0xFFFF, xorout=0xFFFF
|
||||
crc = 0xFFFF
|
||||
for b in data:
|
||||
crc ^= b
|
||||
for _ in range(8):
|
||||
if crc & 1:
|
||||
crc = (crc >> 1) ^ 0x8408
|
||||
else:
|
||||
crc >>= 1
|
||||
crc = (crc >> 1) ^ 0x8408 if (crc & 1) else (crc >> 1)
|
||||
return (crc ^ 0xFFFF) & 0xFFFF
|
||||
|
||||
CRC_FUNCS = {
|
||||
"CRC-16/IBM": crc16_ibm,
|
||||
"CRC-16/CCITT-FALSE": crc16_ccitt_false,
|
||||
"CRC-16/X-25": crc16_x25,
|
||||
|
||||
CRC16_FUNCS: Dict[str, Callable[[bytes], int]] = {
|
||||
"CRC16_IBM": crc16_ibm,
|
||||
"CRC16_CCITT_FALSE": crc16_ccitt_false,
|
||||
"CRC16_X25": crc16_x25,
|
||||
}
|
||||
|
||||
def parse_frames(blob: bytes, trailer_len: int) -> List[Frame]:
|
||||
|
||||
def _try_validate_sum8(body: bytes) -> Optional[Tuple[bytes, bytes, str]]:
|
||||
"""
|
||||
body = payload + chk8
|
||||
Returns (payload, chk_bytes, type) if valid, else None
|
||||
"""
|
||||
if len(body) < 1:
|
||||
return None
|
||||
payload = body[:-1]
|
||||
chk = body[-1]
|
||||
if checksum8_sum(payload) == chk:
|
||||
return payload, bytes([chk]), "SUM8"
|
||||
return None
|
||||
|
||||
|
||||
def _try_validate_crc16(body: bytes) -> Optional[Tuple[bytes, bytes, str]]:
|
||||
"""
|
||||
body = payload + crc16(2 bytes)
|
||||
Try multiple CRC16 types and both endian interpretations.
|
||||
Returns (payload, chk_bytes, type) if valid, else None
|
||||
"""
|
||||
if len(body) < 2:
|
||||
return None
|
||||
payload = body[:-2]
|
||||
chk_bytes = body[-2:]
|
||||
|
||||
given_le = int.from_bytes(chk_bytes, "little", signed=False)
|
||||
given_be = int.from_bytes(chk_bytes, "big", signed=False)
|
||||
|
||||
for name, fn in CRC16_FUNCS.items():
|
||||
calc = fn(payload)
|
||||
if calc == given_le:
|
||||
return payload, chk_bytes, f"{name}_LE"
|
||||
if calc == given_be:
|
||||
return payload, chk_bytes, f"{name}_BE"
|
||||
return None
|
||||
|
||||
|
||||
def validate_bw_body_auto(body: bytes) -> Optional[Tuple[bytes, bytes, str]]:
|
||||
"""
|
||||
Try to interpret the tail of body as a checksum in several ways.
|
||||
Return (payload, checksum_bytes, checksum_type) if any match; else None.
|
||||
"""
|
||||
# Prefer SUM8 first (it fits small frames and is cheap)
|
||||
hit = _try_validate_sum8(body)
|
||||
if hit:
|
||||
return hit
|
||||
|
||||
# Then CRC16 variants
|
||||
hit = _try_validate_crc16(body)
|
||||
if hit:
|
||||
return hit
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ------------------------
|
||||
# S3 MODE (DLE framed)
|
||||
# ------------------------
|
||||
|
||||
def parse_s3(blob: bytes, trailer_len: int) -> List[Frame]:
|
||||
frames: List[Frame] = []
|
||||
|
||||
STATE_IDLE = 0
|
||||
STATE_IN_FRAME = 1
|
||||
STATE_AFTER_DLE = 2
|
||||
IDLE = 0
|
||||
IN_FRAME = 1
|
||||
AFTER_DLE = 2
|
||||
|
||||
state = STATE_IDLE
|
||||
payload_raw = bytearray()
|
||||
state = IDLE
|
||||
body = bytearray()
|
||||
start_offset = 0
|
||||
idx = 0
|
||||
|
||||
i = 0
|
||||
n = len(blob)
|
||||
|
||||
print(">>> CLEAN RAW STATE MACHINE ACTIVE <<<")
|
||||
|
||||
while i < n:
|
||||
b = blob[i]
|
||||
|
||||
if state == STATE_IDLE:
|
||||
# look for DLE STX
|
||||
if state == IDLE:
|
||||
if b == DLE and i + 1 < n and blob[i + 1] == STX:
|
||||
print("FRAME START at", i)
|
||||
start_offset = i
|
||||
payload_raw = bytearray()
|
||||
state = STATE_IN_FRAME
|
||||
body.clear()
|
||||
state = IN_FRAME
|
||||
i += 2
|
||||
continue
|
||||
|
||||
elif state == STATE_IN_FRAME:
|
||||
elif state == IN_FRAME:
|
||||
if b == DLE:
|
||||
state = STATE_AFTER_DLE
|
||||
state = AFTER_DLE
|
||||
i += 1
|
||||
continue
|
||||
else:
|
||||
payload_raw.append(b)
|
||||
body.append(b)
|
||||
|
||||
elif state == STATE_AFTER_DLE:
|
||||
else: # AFTER_DLE
|
||||
if b == DLE:
|
||||
# escaped literal DLE
|
||||
payload_raw.append(DLE)
|
||||
state = STATE_IN_FRAME
|
||||
body.append(DLE)
|
||||
state = IN_FRAME
|
||||
i += 1
|
||||
continue
|
||||
|
||||
elif b == ETX:
|
||||
print("FRAME END at", i)
|
||||
# end of frame
|
||||
if b == ETX:
|
||||
end_offset = i + 1
|
||||
|
||||
# capture trailer
|
||||
trailer_start = i + 1
|
||||
trailer_end = trailer_start + trailer_len
|
||||
trailer = blob[trailer_start:trailer_end]
|
||||
|
||||
# For S3 mode we don't assume checksum type here yet.
|
||||
frames.append(Frame(
|
||||
index=idx,
|
||||
start_offset=start_offset,
|
||||
end_offset=end_offset,
|
||||
payload_raw=bytes(payload_raw),
|
||||
payload=bytes(payload_raw),
|
||||
payload_raw=bytes(body),
|
||||
payload=bytes(body),
|
||||
trailer=trailer,
|
||||
crc_match=None
|
||||
checksum_valid=None,
|
||||
checksum_type=None,
|
||||
checksum_hex=None
|
||||
))
|
||||
|
||||
idx += 1
|
||||
state = STATE_IDLE
|
||||
state = IDLE
|
||||
i = trailer_end
|
||||
continue
|
||||
|
||||
else:
|
||||
# unexpected sequence: DLE followed by non-DLE/non-ETX
|
||||
# treat both bytes as data (robust recovery)
|
||||
payload_raw.append(DLE)
|
||||
payload_raw.append(b)
|
||||
state = STATE_IN_FRAME
|
||||
i += 1
|
||||
continue
|
||||
# Unexpected DLE + byte → treat as literal data
|
||||
body.append(DLE)
|
||||
body.append(b)
|
||||
state = IN_FRAME
|
||||
i += 1
|
||||
continue
|
||||
|
||||
i += 1
|
||||
|
||||
print("Frames parsed:", len(frames))
|
||||
return frames
|
||||
|
||||
def best_crc_match(payload: bytes, trailer: bytes, little_endian: bool) -> Optional[str]:
|
||||
"""Try to interpret first 2 trailer bytes as CRC16 and see which algorithm matches."""
|
||||
if len(trailer) < 2:
|
||||
return None
|
||||
given = int.from_bytes(trailer[:2], byteorder="little" if little_endian else "big", signed=False)
|
||||
matches = []
|
||||
for name, fn in CRC_FUNCS.items():
|
||||
calc = fn(payload)
|
||||
if calc == given:
|
||||
matches.append(name)
|
||||
if len(matches) == 1:
|
||||
return matches[0]
|
||||
if len(matches) > 1:
|
||||
return " / ".join(matches)
|
||||
return None
|
||||
|
||||
# ------------------------
|
||||
# BW MODE (ACK+STX framed, bare ETX)
|
||||
# ------------------------
|
||||
|
||||
def parse_bw(blob: bytes, trailer_len: int, validate_checksum: bool) -> List[Frame]:
|
||||
frames: List[Frame] = []
|
||||
|
||||
IDLE = 0
|
||||
IN_FRAME = 1
|
||||
AFTER_DLE = 2
|
||||
|
||||
state = IDLE
|
||||
body = bytearray()
|
||||
start_offset = 0
|
||||
idx = 0
|
||||
|
||||
i = 0
|
||||
n = len(blob)
|
||||
|
||||
while i < n:
|
||||
b = blob[i]
|
||||
|
||||
if state == IDLE:
|
||||
# Frame start signature: ACK + STX
|
||||
if b == ACK and i + 1 < n and blob[i + 1] == STX:
|
||||
start_offset = i
|
||||
body.clear()
|
||||
state = IN_FRAME
|
||||
i += 2
|
||||
continue
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if state == IN_FRAME:
|
||||
if b == DLE:
|
||||
state = AFTER_DLE
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if b == ETX:
|
||||
# Candidate end-of-frame.
|
||||
# Accept ETX if the next bytes look like a real next-frame start (ACK+STX),
|
||||
# or we're at EOF. This prevents chopping on in-payload 0x03.
|
||||
next_is_start = (i + 2 < n and blob[i + 1] == ACK and blob[i + 2] == STX)
|
||||
at_eof = (i == n - 1)
|
||||
|
||||
if not (next_is_start or at_eof):
|
||||
# Not a real boundary -> payload byte
|
||||
body.append(ETX)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
trailer_start = i + 1
|
||||
trailer_end = trailer_start + trailer_len
|
||||
trailer = blob[trailer_start:trailer_end]
|
||||
|
||||
chk_valid = None
|
||||
chk_type = None
|
||||
chk_hex = None
|
||||
payload = bytes(body)
|
||||
|
||||
if validate_checksum:
|
||||
hit = validate_bw_body_auto(payload)
|
||||
if hit:
|
||||
payload, chk_bytes, chk_type = hit
|
||||
chk_valid = True
|
||||
chk_hex = chk_bytes.hex()
|
||||
else:
|
||||
chk_valid = False
|
||||
|
||||
frames.append(Frame(
|
||||
index=idx,
|
||||
start_offset=start_offset,
|
||||
end_offset=i + 1,
|
||||
payload_raw=bytes(body),
|
||||
payload=payload,
|
||||
trailer=trailer,
|
||||
checksum_valid=chk_valid,
|
||||
checksum_type=chk_type,
|
||||
checksum_hex=chk_hex
|
||||
))
|
||||
idx += 1
|
||||
state = IDLE
|
||||
i = trailer_end
|
||||
continue
|
||||
|
||||
# Normal byte
|
||||
body.append(b)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# AFTER_DLE
|
||||
if b == DLE:
|
||||
body.append(DLE) # 10 10 => literal 10
|
||||
else:
|
||||
# Robust recovery: treat as literal DLE + byte
|
||||
body.append(DLE)
|
||||
body.append(b)
|
||||
state = IN_FRAME
|
||||
i += 1
|
||||
|
||||
return frames
|
||||
|
||||
|
||||
# ------------------------
|
||||
# CLI
|
||||
# ------------------------
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description="Parse DLE-framed serial capture .bin into frames (and guess CRC).")
|
||||
ap.add_argument("binfile", type=Path, help="Path to capture .bin file")
|
||||
ap.add_argument("--trailer-len", type=int, default=2, help="Bytes to capture after DLE ETX (default: 2)")
|
||||
ap.add_argument("--crc", action="store_true", help="Attempt CRC match using first 2 trailer bytes")
|
||||
ap.add_argument("--crc-endian", choices=["little", "big"], default="little", help="CRC endian when reading trailer")
|
||||
ap = argparse.ArgumentParser(description="Parse Instantel S3/BW binary captures.")
|
||||
ap.add_argument("binfile", type=Path)
|
||||
ap.add_argument("--mode", choices=["s3", "bw"], default="s3")
|
||||
ap.add_argument("--trailer-len", type=int, default=0)
|
||||
ap.add_argument("--no-checksum", action="store_true")
|
||||
ap.add_argument("--out", type=Path, default=None)
|
||||
|
||||
ap.add_argument("--out", type=Path, default=None, help="Write JSONL output to this file")
|
||||
args = ap.parse_args()
|
||||
|
||||
print(f"s3_parser v{__version__}")
|
||||
|
||||
blob = args.binfile.read_bytes()
|
||||
frames = parse_frames(blob, trailer_len=args.trailer_len)
|
||||
|
||||
little = (args.crc_endian == "little")
|
||||
if args.crc:
|
||||
for f in frames:
|
||||
f.crc_match = best_crc_match(f.payload, f.trailer, little_endian=little)
|
||||
if args.mode == "s3":
|
||||
frames = parse_s3(blob, args.trailer_len)
|
||||
else:
|
||||
frames = parse_bw(blob, args.trailer_len, validate_checksum=not args.no_checksum)
|
||||
|
||||
# Summary
|
||||
total = len(frames)
|
||||
crc_hits = sum(1 for f in frames if f.crc_match) if args.crc else 0
|
||||
print(f"Frames found: {total}")
|
||||
if args.crc:
|
||||
print(f"CRC matches: {crc_hits} ({(crc_hits/total*100.0):.1f}%)" if total else "CRC matches: 0")
|
||||
print("Frames found:", len(frames))
|
||||
|
||||
# Emit JSONL
|
||||
def to_hex(b: bytes) -> str:
|
||||
return b.hex()
|
||||
|
||||
@@ -239,7 +371,9 @@ def main() -> None:
|
||||
"payload_len": len(f.payload),
|
||||
"payload_hex": to_hex(f.payload),
|
||||
"trailer_hex": to_hex(f.trailer),
|
||||
"crc_match": f.crc_match,
|
||||
"checksum_valid": f.checksum_valid,
|
||||
"checksum_type": f.checksum_type,
|
||||
"checksum_hex": f.checksum_hex,
|
||||
}
|
||||
lines.append(json.dumps(obj))
|
||||
|
||||
@@ -247,11 +381,11 @@ def main() -> None:
|
||||
args.out.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
print(f"Wrote: {args.out}")
|
||||
else:
|
||||
# Print first few only (avoid spewing your terminal)
|
||||
for line in lines[:10]:
|
||||
print(line)
|
||||
if len(lines) > 10:
|
||||
print(f"... ({len(lines) - 10} more)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
main()
|
||||
Binary file not shown.
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user