serversdown/seismo-relay
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
744473888344ae221e6185801e6eb35c0fa65015
seismo-relay/CHANGELOG.md
T

38 KiB
Raw Blame History

Changelog

All notable changes to seismo-relay are documented here.

v0.14.1 — 2026-05-04

Fixed

  • read_bulk_waveform_stream — event-N probe counter off-by-0x46. Continuation events (start_key[2:4] != 0) were being probed at counter start_offset + 0x0046 instead of just start_offset. In the iteration walk, cur_key from 1F is already the off=0x46 WAVEHDR record key, so the earlier formula effectively double-counted the WAVEHDR offset. The probe landed one WAVEHDR past the actual event start, the response no longer contained the STRT record at byte 17, parse_strt_end_offset returned None, and the chunk loop fell back to the max_chunks=128 cap — walking ~110 chunks of post-event circular-buffer garbage. Verified against the 5-1-26 "copy 2nd address" and 5-4-26 BW 2-sec event captures: BW probes counter=0x2238 with key=01112238 and STRT is present at byte 17 of the response (end_offset=0x417E).
  • CLAUDE.md / docs/instantel_protocol_reference.md — corrected the event-N section to clarify that start_key in those formulas is the off=0x46 key, not the off=0x2C boundary key, and removed the spurious +0x46 from the chunk-walk pseudocode.

v0.12.6 — 2026-05-01

Fixed

  • blastware_file.py — waveform frame classification — A5 frame classification for waveform-only vs header-only frames now uses frame.record_type instead of frame index. Only waveform frames (0x46) are written to the file body; metadata frames are skipped. Fixes spurious data corruption from incorrectly classified frames.

  • s3_analyzer.py — A5/5A frame naming — Bulk waveform stream frames (SUB 5A response) are now correctly labeled "A5" in analyzer output instead of being conflated with other multi-frame responses (SUB A4, E5, etc.).

  • S3FrameParser — frame terminator detection — Corrected the bare ETX terminator detection. Frame termination is now correctly identified by a standalone ETX=0x03 byte, not by the DLE+ETX sequence (which is part of the payload when it appears within a frame).

v0.13.2 — 2026-05-01

Fixed

  • _extract_record_type — third 0C-record header format ("short", 8 bytes). A live SFM download against BE11529 produced files named M5290000.000 (zero-stamped) because the 0C waveform record's first bytes were 01 05 07 ea ... — neither the 9-byte single-shot layout (0x10 at byte 1) nor the 10-byte continuous layout (0x10 at bytes 0 and 2). Investigation showed this is a third format observed in the wild: an 8-byte header with no marker bytes at all ([day][month][year_BE:2][unknown][hour][min][sec]). The detection logic now scans the year (uint16 BE) at byte 2 / byte 3 / byte 4 and picks whichever offset returns a sensible year (2015–2050) — each format has the year at a unique position so this disambiguates cleanly.
    • New format → event.record_type = "Waveform (Short)", Timestamp.from_short_record().
    • Existing single-shot and continuous parsers unchanged.
    • The user's event from May 1, 2026 13:21:37 now correctly resolves to a filename like M529LKIQ.G10 instead of M5290000.000.

Added

  • Timestamp.from_short_record(data) — decodes the 8-byte header.
  • _detect_record_format(data) — internal helper returning "single_shot" / "continuous" / "short" / None via year-position scan.

v0.13.1 — 2026-05-01

Fixed

  • _extract_record_type — Continuous-mode record headers misclassified as Unknown. In single-shot mode the 0C waveform record's 9-byte header puts the sub_code marker 0x10 at byte 1, with the day at byte 0. In Continuous mode the header is 10 bytes with the marker at byte 0 and byte 2, and the day at byte 1. Previous logic only inspected byte 1 and treated any value other than 0x10 / 0x03 as "Unknown", which prevented event.timestamp from being populated for any continuous-mode event whose day-of-month wasn't exactly 3 or 16. As a downstream effect, blastware_filename() saw event.timestamp == None, fell back to stem="0000" / ab="00", and produced filenames like M5290000.000. Discovered from a live SFM run on BE11529 in continuous mode (day-of-month = 5). Now disambiguates by checking BOTH byte 0 and byte 2: if both are 0x10, it's the 10-byte continuous header; else if byte 1 is 0x10, it's the 9-byte single-shot header. Day-of-month no longer matters.

    Superseded by v0.13.2 — the user's actual record uses a third 8-byte format with no 0x10 markers, which v0.13.1 still misclassified.

v0.13.0 — 2026-05-01

Fixed

  • SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec. read_bulk_waveform_stream was walking the chunk counter past the actual end of the event, picking up post-event circular-buffer garbage that corrupted reconstructed Blastware files for any waveform > ~1 sec. The loop now extracts the event's end_offset from the STRT record at data[23:27] of the probe response and stops the chunk walk when the next counter would step past it. Verified against three BW MITM captures (4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7 bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.

Added

  • framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter) — computes the corrected SUB 5A TERM frame's (offset_word, params) per the formula confirmed across all 3 BW captures. Not yet wired into read_bulk_waveform_stream (the legacy TERM is still used to preserve the existing blastware_file.write_blastware_file frame-structure expectations); available for the next iteration that switches to BW's 0x0200 chunk step.
  • framing.parse_strt_end_offset(a5_data) — extracts the event-end pointer from the STRT record in an A5 response payload.

Documentation

  • CLAUDE.md and docs/instantel_protocol_reference.md extensively rewritten to reflect the corrected SUB 5A protocol. See:
    • CLAUDE.md "SUB 5A — chunk counter formula (REWRITTEN 2026-05-01)"
    • CLAUDE.md "SUB 5A — STRT record encodes end_offset"
    • CLAUDE.md "SUB 5A — TERM frame formula"
    • CLAUDE.md "SUB 5A — fixed metadata pages 0x1002 and 0x1004"
    • CLAUDE.md "SUB 0A — WAVEHDR response length distinguishes events from boundaries" (0x46 = real event, 0x2C = boundary marker)
    • protocol reference §7.8.5 / §7.8.6 / §7.8.7 / §7.8.8
  • The previous chunk-counter formula (max(key4[2:4], 0x0400) + (chunk-1) * 0x0400) is now marked DEPRECATED and explicitly tagged WRONG with pointers to the new sections, so future work doesn't re-derive it.

Known minor diffs vs Blastware (deferred to a follow-up)

  • We still use the OLD 0x0400 chunk step rather than BW's 0x0200; switching also requires updating blastware_file.write_blastware_file's skip values and "extra chunk after metadata" logic, which depends on a fresh capture to verify.
  • We still use the legacy fixed offset_word=0x005A TERM frame rather than BW's end_offset - next_boundary formula, for the same reason.
  • Two fixed metadata pages at counter 0x1002 and 0x1004 are not yet read explicitly; under the current 0x0400 walk their content is reachable via the sample chunk that covers buffer addresses [0x1000, 0x1400).

v0.12.5 — 2026-04-21

Added

  • seismo_lab.py — Download tab — New fourth tab for live wire-byte capture during event downloads. Captures both BW→device and device→S3 frames in real time, allowing inspection of the 5A bulk stream chunk sequence and frame-by-frame analysis without needing a bridge or MITM proxy. Files are saved with user-specified labels for easy tracking.

Changed

  • s3_bridge.py — raw captures always-on by default — --raw-bw and --raw-s3 now default to "auto" instead of None. Every bridge session automatically generates timestamped raw_bw_<ts>.bin and raw_s3_<ts>.bin files alongside the .bin/.log session files. Pass --raw-bw "" (explicit empty string) to disable if needed.

  • gui_bridge.py — raw capture checkboxes pre-checked — Both "BW→S3 raw" and "S3→BW raw" checkboxes start checked. Path fields are empty by default (bridge auto-names the files). Unchecking a box passes --raw-bw "" to explicitly disable capture.

  • Bridge tab — TCP mode added — Serial/TCP radio toggle allows connection via cellular modem (RV50/RV55) instead of direct RS-232. Supports multi-capture design (simultaneous Bridge + Analyzer + Download sessions).

  • ach_server.py — TX capture added (raw_tx_<ts>.bin) — Every ACH inbound session now saves both directions: raw_rx_<ts>.bin (device → us, S3 side, as before) and raw_tx_<ts>.bin (us → device, BW side). Both files are usable in the Analyzer. TX bytes are buffered in memory until startup handshake succeeds (same as RX), preventing scanner probes from creating empty files.

v0.12.4 — 2026-04-21 (protocol analysis / docs only — no code changes)

Discovered

  • compliance_raw is wire-encoded, not logical bytes — read_compliance_config() returns bytes that include DLE prefix bytes (0x10) before any 0x03 values (because S3FrameParser preserves DLE+ETX inner-frame pairs as two literal bytes). The previous CLAUDE.md claim that "S3FrameParser handles this transparently so compliance_raw contains logical bytes" was wrong.

  • anchor-9 behavior per recording mode (confirmed from 4-20-26 BW write captures):

    • Single Shot (0x00) / Continuous (0x01): anchor-9 = 0x00
    • Histogram (0x03): anchor-9 = 0x10 — the E5 DLE prefix for the 0x03 recording_mode byte
    • Histogram+Continuous (0x04): anchor-9 = 0x10 — an actual stored config byte for this mode Anchor position shifts by ±1 when recording_mode = 0x03 due to the extra DLE byte; the dynamic anchor search (buf.find(ANCHOR, 0, 150)) handles this correctly without code changes.

  • Write frame ETX escaping — BW escapes 0x03 bytes in write frame data as 0x10 0x03 on the wire. Our build_bw_write_frame sends data bytes raw without ETX escaping. Device accepts our raw writes for all tested modes. Hypothesis: device write parser uses the offset/length field for frame boundaries, not ETX scanning, making ETX escaping optional. Histogram mode (recording_mode = 0x03) write via SFM from a non-Histogram starting state not yet tested.

  • BW write payload vs E5 read payload are byte-identical around the anchor region (confirmed by comparing 3-11-26 BW TX and S3 captures). BW does NOT strip DLE prefix bytes before writing; it round-trips the wire-encoded bytes verbatim with only the modified fields changed.

  • Capture folder content catalogued — see CLAUDE.md "BW capture reference" table for a summary of all available protocol captures and their contents.

v0.12.3 — 2026-04-20

Added

  • Auto Call Home config protocol — Full read/write/decode/encode pipeline for the device's Remote Access → Setup Unit ACH settings, confirmed from 4-20-26 call home settings captures.

    Protocol (new):

    • SUB 0x2C — Call Home Config READ (response 0xD3); two-step read; data offset 0x7C = 124; raw payload 125 bytes (1-byte longer than DATA_LENGTH due to DLE-escaped \x10\x03 at raw[117:119] representing num_retries = 3)
    • SUB 0x7E — Call Home Config WRITE (response 0x81); 127-byte payload (125-byte read payload + \x00\x00); offset = data[1]+2 = 0x7E; write format (DLE-aware checksum)
    • SUB 0x7F — Call Home WRITE CONFIRM (response 0x80); no data

    Field map (confirmed from 10-frame BW TX diff):

    • raw[5] — auto_call_home_enabled (bool)
    • raw[6:46] — dial_string (40-byte null-padded ASCII)
    • raw[87] — after_event_recorded (bool)
    • raw[91] — at_specified_times (bool)
    • raw[93] — time1_enabled / raw[101] — time1_hour / raw[102] — time1_min
    • raw[95] — time2_enabled / raw[105] — time2_hour / raw[106] — time2_min
    • raw[117:119] — \x10\x03 (DLE-escaped 0x03 = num_retries value 3)
    • raw[120] — time_between_retries_sec / raw[122] — wait_for_connection_sec / raw[124] — warm_up_time_sec

    Library (minimateplus/):

    • models.py — CallHomeConfig dataclass (14 fields; raw bytes preserved for round-trip writes)
    • protocol.py — SUB_CALL_HOME = 0x2C, SUB_CALL_HOME_WRITE = 0x7E, SUB_CALL_HOME_CONFIRM = 0x7F; read_call_home_config(), write_call_home_config()
    • client.py — get_call_home_config(), set_call_home_config(), _decode_call_home_config() (handles DLE prefix at raw[117]), _encode_call_home_config() (patches in-place; raises ValueError if hour/min = 3)

    REST API (sfm/server.py):

    • GET /device/call_home — reads and decodes call home config from device
    • POST /device/call_home — reads, patches specified fields, writes back to device
    • CallHomeConfigBody Pydantic model with 9 optional writable fields

    Web UI (sfm/sfm_webapp.html):

    • New "Call Home" tab with enable flag, dial string (read-only), after-event trigger, at-specified-times flag, two time slots (enable + HH:MM each), and read-only retry settings (num_retries, time_between_retries_sec, wait_for_connection_sec, warm_up_time_sec)
    • "Read from Device", "Write to Device", "Clear Form" action buttons
    • Client-side guard: rejects hour or minute value equal to 3 with a clear message explaining the DLE-encoding limitation

v0.12.2 — 2026-04-20

Added / Fixed

  • Geophone sensitivity / maximum range field confirmed — 4-20-26 geo sensitivity captures (1.25 in/s vs 10 in/s) diffed across all three SUB 71 write chunks and both E5 read payloads. The geo_range uint8 field per channel is now fully confirmed:

    • E5 read offset: channel_label + 33; SUB 71 write offset: channel_label + 29
    • 0x00 = Normal 10.000 in/s (standard gain); 0x01 = Sensitive 1.250 in/s (high gain)
    • Correction: previous hypothesis (channel_label+20, 0x01=Normal) was wrong. channel_label+20 reads 0x01 on ALL captures regardless of range — not this field.
    • _decode_compliance_config_into: read offset corrected from tran_pos+20 → tran_pos+33
    • _encode_compliance_config: added geo_range parameter; writes to Tran/Vert/Long at +29
    • apply_config: added geo_range parameter
    • POST /device/config: added geo_range to DeviceConfigBody
    • Web UI Config tab: added "Maximum Range — Geo" select (Normal / Sensitive)
    • Web UI Device tab: added "Max Range (geo)" row to compliance table

  • recording_mode + histogram_interval_sec confirmed and implemented (4-20-26 captures)

    • recording_mode: uint8 at anchor−8 (E5 read) / anchor−7 (write); enum: 0x00=Single Shot, 0x01=Continuous, 0x03=Histogram, 0x04=Histogram+Continuous
    • histogram_interval_sec: uint16 BE seconds at anchor−4; same offset in read & write; valid: 2, 5, 15, 60, 300, 900 (matching Blastware dropdown: 2s, 5s, 15s, 1m, 5m, 15m)
    • Both fields added to ComplianceConfig, _decode_compliance_config_into, _encode_compliance_config, apply_config, REST API body, and web UI

v0.12.1 — 2026-04-16

Added

  • sfm/server.py — _LiveCache — in-memory live device cache that eliminates redundant TCP round-trips between web requests. Plain Python dict + threading.Lock, no extra dependencies.

    Cache strategy per endpoint:

    Endpoint Strategy
    GET /device/info Indefinite; invalidated by POST /device/config
    GET /device/events Count-probe fast path — poll()+count_events() (~2 s); returns cached data if event count is unchanged; full download only when new events are detected
    GET /device/monitor/status 30-second TTL; invalidated immediately on monitor start/stop
    GET /device/event/{idx}/waveform Permanent per-index (waveforms are immutable once recorded)

  • ?force=true query param on all cached endpoints — bypasses cache and forces a fresh read from the device.

  • Cache invalidation hooks — POST /device/config marks device info and events stale; POST /device/monitor/start and /stop evict the monitor status entry immediately so the next status poll reflects the actual device state.

v0.12.0 — 2026-04-13

Added

  • sfm/server.py — _LiveCache — in-memory live device cache, eliminating redundant TCP round-trips between requests. No extra dependencies (plain Python dict + threading.Lock). Replaces the SQLAlchemy-based sfm/cache.py experiment from the feature/intelligent-caching branch.

    Cache behaviour by endpoint:

    Endpoint Cache strategy
    GET /device/info Indefinite; invalidated by POST /device/config
    GET /device/events Count-probe fast path: quick poll()+count_events() (~2s); return cache if count matches; full download only when new events detected
    GET /device/monitor/status 30-second TTL; invalidated by monitor start/stop
    GET /device/event/{idx}/waveform Permanent per-index (waveforms are immutable)

  • ?force=true param on all four cached endpoints — bypasses cache and re-reads from device.

  • POST /device/config cache invalidation — marks device info + events dirty so the next read reflects the new compliance config.

  • POST /device/monitor/start / stop cache invalidation — evicts the monitor status cache entry immediately so the next poll returns the updated state.

Removed

  • sfm/cache.py — SQLAlchemy-based cache from the experimental caching branch. Its logic has been ported to the sqlite3-native _LiveCache class above. sqlalchemy is no longer a dependency.

v0.11.0 — 2026-04-13

Added

  • sfm/database.py — SeismoDb — SQLite persistence layer for all ACH data. Three tables, all unit-keyed by serial number:

    • ach_sessions — one row per inbound call-home: serial, timestamp, peer IP, events_downloaded, monitor_entries, duration_seconds
    • events — one row per triggered waveform event: serial, waveform_key (dedup), timestamp, Tran/Vert/Long/VectorSum/Mic PPV, project/client/operator/sensor_location strings, sample_rate, record_type, false_trigger flag
    • monitor_log — one row per monitoring interval: serial, waveform_key (dedup), start_time, stop_time, duration_seconds, geo_threshold_ips
    • WAL mode, per-request connections — safe for the single-writer / occasional-reader ACH server pattern
    • Deduplication by (serial, waveform_key) UNIQUE constraint — re-runs and repeat call-homes never produce duplicate rows

  • ach_server.py — DB integration — after each successful call-home, writes new events and monitor log entries to seismo_relay.db then records the session in ach_sessions. DB write failures are logged as warnings and do not abort the session.

  • sfm/server.py — DB read endpoints:

    • GET /db/units — distinct serials with last_seen, total_events, total_monitor_entries
    • GET /db/events — query events with serial / date range / false_trigger filters
    • GET /db/monitor_log — query monitoring intervals
    • GET /db/sessions — query ACH call-home sessions
    • PATCH /db/events/{id}/false_trigger — flag/unflag false triggers (for review UI)

Architecture

  • seismo-relay DB is unit-keyed only — no project concepts. Project aggregation is terra-view's responsibility via UnitAssignment / DeploymentRecord + date range queries against the SFM DB endpoints.
  • DB file lives at bridges/captures/seismo_relay.db by default.

v0.10.0 — 2026-04-11

Added

  • MiniMateClient.get_monitor_log_entries(skip_keys=None) — browse-mode walk (1E → 0A → 1F) that collects partial records (0x2C record type) from the device's event list without triggering a full waveform download (no 0C or 5A). Returns list[MonitorLogEntry]. Each entry represents one continuous monitoring interval where no threshold was exceeded.

  • _decode_0a_partial_header(raw_data, index, key4) in client.py — decodes a SUB 0x0A response payload whose record type is 0x2C. Extracts:

    • start_time / stop_time — two consecutive timestamps; auto-detects 9-byte (sub_code=0x10, single-shot) vs 10-byte (sub_code=0x03, continuous) format from raw_data[11]. Handles a 1-byte gap between the two timestamps that occurs when ts1 and ts2 share the same minute:second.
    • serial — device serial string found via b"BE" anchor scan.
    • geo_threshold_ips — trigger level found via b"Geo: " anchor scan.

  • MonitorLogEntry dataclass in models.py — new model for partial records: index, key, start_time, stop_time, serial, geo_threshold_ips, raw_header, and a duration_seconds property.

  • read_waveform_header() return value extended — now returns (data_rsp.data, length) (full payload) instead of (data_rsp.data[11:11+length], length). Callers get the complete payload including the record-type byte at position 0. Full records use raw_data[11:11+length] as before; partial records are detected by raw_data[0] == 0x2C.

  • ACH server: monitor log collection — after get_events(), calls get_monitor_log_entries(skip_keys=seen_keys) and saves new entries to monitor_log.json in the session directory. Monitor log keys are included in downloaded_keys for state persistence (no re-processing on next call-home).

  • _monitor_log_entry_to_dict() in ach_server.py — serialises a MonitorLogEntry to a JSON-compatible dict with ISO-format timestamps.

Protocol / Documentation

  • SUB 0x0A partial record (0x2C) format confirmed (✅ 4-11-26 MITM capture, 12 frames):

    • Record type 0x2C at raw_data[0]; length < 64 bytes.
    • Two timestamps at raw_data[11:] — start and stop of the monitoring interval.
    • ASCII metadata region after timestamps: BE<serial>\x00Geo: <float> in/s.
    • Edge case: 1-byte separator between timestamps when ts1 and ts2 share minute:second.
    • 10-byte timestamp format (sub_code=0x03) signalled by raw_data[11] == 0x10.

  • Key reuse detection for monitor log entries — monitor log keys are tracked alongside event keys in ach_state.json so the ACH server does not re-process them after a call-home cycle.

v0.9.0 — 2026-04-11

Added

  • MiniMateClient.list_event_keys() — fast browse-mode walk (1E → 0A → 1F, no waveform download) that returns the list of event key hex strings currently stored on the device. Used by the ACH server as a cheap pre-check before deciding whether to call get_events().

  • get_events(skip_waveform_for_keys=set(...)) — new optional parameter. For any key in the set the function performs only 0A + 1F(browse) instead of the full 1E-arm → 0C → POLL×3 → 5A sequence. Eliminates redundant waveform downloads on repeat call-homes when the device still holds previously downloaded events.

  • MiniMateClient.delete_all_events() — erases all events from device memory using the confirmed 4-step sequence:

    • SUB 0xA3 begin_erase_all — initiate erase (token=0xFE) → ack 0x5C
    • SUB 0x1C read_monitor_status — intermediate status read (Blastware-required)
    • SUB 0x06 read_event_storage_range — verify storage state (token=0xFE) → 36-byte response
    • SUB 0xA2 confirm_erase_all — commit erase (token=0xFE) → ack 0x5D

    All four steps confirmed from 4-11-26 MITM capture of a live Blastware ACH session. After a successful call, the device's event counter resets to 0x01110000.

  • MiniMateProtocol erase methods: begin_erase_all(), confirm_erase_all(), read_event_storage_range() added to protocol.py with documented SUB constants SUB_ERASE_ALL_BEGIN = 0xA3 and SUB_ERASE_ALL_CONFIRM = 0xA2.

  • bridges/ach_mitm.py — transparent TCP-to-TCP MITM proxy. Listens for inbound unit connections, connects upstream to a real Blastware ACH server, and saves both directions to raw_bw_<ts>.bin / raw_s3_<ts>.bin files matching the existing capture format. Used to capture the 4-11-26 Blastware ACH session including event deletion. Usage: python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998

  • ACH server: key-based state tracking — ach_state.json now stores downloaded_keys: [hex_strings] and max_downloaded_key: hex_string per unit instead of event_count: N. This correctly handles the standard workflow where events are deleted from the device after upload — a count-based approach would see count=0 on the next call-home and silently skip new events.

  • ACH server: --clear-after-download flag — after a successful download (at least one new event saved), erases all events from the device using delete_all_events(). Mirrors the standard Blastware ACH workflow. On success, downloaded_keys and max_downloaded_key are reset to empty so the next session starts fresh.

  • ACH server: post-erase key-reuse detection — after an external erase (Blastware or manual), device keys restart from 0x01110000, colliding with previously downloaded keys. On each browse walk, if max(device_keys) < max_downloaded_key (device counter rolled back), all device keys are treated as new regardless of seen_keys. This also catches erases performed by Blastware between our sessions.

Protocol / Documentation

  • SUB 0xA3 / SUB 0xA2 — erase-all sequence confirmed (✅ 4-11-26 MITM capture): Both frames use token=0xFE at params[7] and are standard build_bw_frame requests (not write-format). Response SUBs follow the standard formula: 0x5C and 0x5D. The intermediate 0x1C + 0x06 reads between them are required by Blastware.

  • SUB 0x06 — event storage range read confirmed (✅ 4-11-26 MITM capture): Two-step read, data offset = 0x24 (36 bytes). The last 8 bytes of the response contain the first and last stored event keys (4 bytes each). After a successful erase, both keys read as 01110000 (device-empty state).

  • Event key counter resets to 0x01110000 after erase — confirmed by observing key 01110000 on the device immediately after the MITM erase session.

v0.8.0 — 2026-04-07

Added

  • Write pipeline end-to-end — push_config_raw(event_index_data, compliance_data, trigger_data, waveform_data) on MiniMateClient orchestrates the full 68→73 | 71×3→72 | 82→83 | 69→74→72 write sequence.

  • build_bw_write_frame(sub, data, *, offset, params) in framing.py — dedicated frame builder for write commands (SUBs 0x68–0x83). Doubles only the BW_CMD byte; all other bytes including offset, params, data, and checksum are written raw. Uses the large-frame DLE-aware checksum (sum(b for b in payload[2:] if b != 0x10) + 0x10) & 0xFF).

  • MiniMateProtocol write methods — write_event_index(), write_compliance(), write_trigger_config(), write_waveform_data(), write_confirm(), start_monitoring(), stop_monitoring().

  • AchSession inbound server (bridges/ach_server.py) — accepts call-home TCP connections, runs the full handshake + device-info + event-download sequence, saves device_info.json + events.json per session.

Protocol / Documentation

  • Write frame format confirmed (✅ 3-11-26 BW TX capture, all 11 frames): only BW_CMD byte 0x10 is doubled; all other bytes sent raw. Standard build_bw_frame DLE-stuffing is incorrect for write commands.
  • Write ack responses confirmed as 17-byte zero-data S3 frames.
  • Monitoring SUBs 0x96/0x97 confirmed from 4-8-26 capture.
  • SESSION_RESET signal (41 03) required before POLL for monitoring units.
  • SUB 0x1C monitoring flag at section[1]: 0x00 = idle, 0x10 = monitoring. Confirmed by byte-diff of all 144 data frames in 4-8-26/2ndtry capture.

v0.7.0 — 2026-04-03

Added

  • Raw ADC waveform decode — _decode_a5_waveform(frames_data, event) in client.py. Parses the complete set of SUB 5A A5 response frames into per-channel time-series:
    • Reads the STRT record from A5[0] (bytes 7+): extracts total_samples (BE uint16 at +8), pretrig_samples (BE uint16 at +16), and rectime_seconds (uint8 at +18) into event.total_samples / pretrig_samples / rectime_seconds.
    • Skips the 6-byte preamble (00 00 ff ff ff ff) that follows the 21-byte STRT header; waveform data begins at strt_pos + 27.
    • Strips the 8-byte per-frame counter header from A5[1–6, 8] before appending waveform bytes.
    • Skips A5[7] (metadata-only) and A5[9] (terminator).
    • Cross-frame alignment correction: accumulates running_offset % 8 across all frames and discards (8 − align) % 8 leading bytes per frame to re-align to a T/V/L/M boundary. Required because individual frame waveform payloads are not always multiples of 8 bytes.
    • Decodes as 4-channel interleaved signed 16-bit LE at 8 bytes per sample-set: bytes 0–1 = Tran, 2–3 = Vert, 4–5 = Long, 6–7 = Mic.
    • Stores result in event.raw_samples = {"Tran": [...], "Vert": [...], "Long": [...], "Mic": [...]}.
  • download_waveform(event) public method on MiniMateClient. Issues a full SUB 5A stream with stop_after_metadata=False, then calls _decode_a5_waveform() to populate event.raw_samples and event.total_samples / pretrig_samples / rectime_seconds. Previously only metadata frames were fetched during get_events(); raw waveform data is now available on demand.
  • Event model new fields (models.py): total_samples, pretrig_samples, rectime_seconds (from STRT record), and _waveform_key (4-byte key stored during get_events() for later use by download_waveform()).

Protocol / Documentation

  • SUB 5A A5[0] STRT record layout confirmed (✅ 2026-04-03, 4-2-26 blast capture):
    • STRT header is 21 bytes: b"STRT" + length fields + total_samples (BE uint16 at +8) + pretrig_samples (BE uint16 at +16) + rectime_seconds (uint8 at +18).
    • Followed by 6-byte preamble: 00 00 ff ff ff ff. Waveform begins at strt_pos + 27.
    • Confirmed: 4-2-26 blast → total_samples=9306, pretrig_samples=298, rectime_seconds=70.
  • Blast/waveform mode A5 format confirmed (✅ 2026-04-03, 4-2-26 blast capture): 4-channel interleaved int16 LE at 8 bytes per sample-set; cross-frame alignment correction required. 948 of 9306 total sample-sets captured via stop_after_metadata=True (10 frames).
  • Noise/histogram mode A5 format — endianness corrected (✅ 2026-04-03, 3-31-26 capture): 32-byte block samples are signed 16-bit little-endian (previously documented as BE). 0a 00 → LE int16 = 10 (correct noise floor); BE would give 2560 (wrong).
  • Protocol reference §7.6 rewritten — split into §7.6.1 (Blast/Waveform mode) and §7.6.2 (Noise/Histogram mode), each with confirmed field layouts and open questions noted.

v0.6.0 — 2026-04-02

Added

  • True event-time metadata via SUB 5A bulk waveform stream — get_events() now issues a SUB 5A request after each SUB 0C download, reads the A5 response frames, and extracts the Client:, User Name:, and Seis Loc: fields as they existed at the moment the event was recorded. Previously these fields were backfilled from the current compliance config (SUB 1A), which reflects today's setup, not the setup active when the event triggered.
    • build_5a_frame(offset_word, raw_params) in framing.py — reproduces Blastware's exact wire format for SUB 5A requests: raw (non-DLE-stuffed) offset_hi, DLE-stuffed params, and a DLE-aware checksum where 10 XX pairs count only XX.
    • bulk_waveform_params() returns 11 bytes (extra trailing 0x00 confirmed from 1-2-26 BW wire capture).
    • read_bulk_waveform_stream(key4, *, stop_after_metadata=True, max_chunks=32) in protocol.py — loops sending chunk requests (counter increments 0x0400 per chunk), stops early when b"Project:" is found, then sends a termination frame.
    • _decode_a5_metadata_into(frames_data, event) in client.py — needle-searches A5 frame data for Project:, Client:, User Name:, Seis Loc:, Extended Notes and overwrites event.project_info.
  • get_events() sequence extended — now 1E → 0A → 0C → 5A → 1F per event.

Fixed

  • Compliance config (SUB 1A) channel block missing — orphaned self._send(build_bw_frame(SUB_COMPLIANCE, 0x2A, _DATA_PARAMS)) before the B/C/D receive loop had no corresponding recv_one(), shifting all subsequent receives one step behind and leaving frame D's channel-block data (trigger_level_geo, alarm_level_geo, max_range_geo) unread. Removed the orphaned send. Total config bytes received now correctly ~2126 (was ~1071).
  • Compliance config anchor search range — _decode_compliance_config_into() searched cfg[40:100] for the sample-rate/record-time anchor. With the orphaned-send bug fixed the 44-byte padding it had been adding is gone, and the anchor now appears at cfg[11]. Search widened to cfg[0:150] to be robust to future layout shifts.
  • Removed byte-content deduplication from read_compliance_config() — was masking the real receive-ordering bug.

Protocol / Documentation

  • SUB 5A frame format confirmed — offset_hi byte (0x10) must be sent raw (not DLE-stuffed); checksum is DLE-aware (only the second byte of a 10 XX pair is summed). Standard build_bw_frame DLE-stuffs 0x10 incorrectly for 5A — a dedicated build_5a_frame is required.
  • Event-time metadata source confirmed — Client:, User Name:, and Seis Loc: strings are present in A5 frame 7 of the bulk waveform stream (SUB 5A), not in the 210-byte SUB 0C waveform record. They reflect the compliance setup as it was when the event was stored on the device.

v0.5.0 — 2026-03-31

Added

  • Console tab in seismo_lab.py — direct device connection without the bridge subprocess.
    • Serial and TCP transport selectable via radio buttons.
    • Four one-click commands: POLL, Serial #, Full Config, Event Index.
    • Colour-coded scrolling output: TX (blue), RX raw hex (teal), parsed/decoded (green), errors (red).
    • Save Log and Send to Analyzer buttons; logs auto-saved to bridges/captures/console_<ts>.log.
    • Queue/after(100) pattern — no UI blocking or performance impact.
  • minimateplus package — clean Python client library for the MiniMate Plus S3 protocol.
    • SerialTransport and TcpTransport (for Sierra Wireless RV50/RV55 cellular modems).
    • MiniMateProtocol — DLE frame parser/builder, two-step paged reads, checksum validation.
    • MiniMateClient — high-level client: connect(), get_serial(), get_config(), get_events().
  • TCP/cellular transport (TcpTransport) — connect to field units via Sierra Wireless RV50/RV55 modems over cellular.
    • read_until_idle(idle_gap=1.5s) to handle modem data-forwarding buffer delay.
    • Confirmed working end-to-end: TCP → RV50/RV55 → RS-232 → MiniMate Plus.
  • bridges/tcp_serial_bridge.py — local TCP-to-serial bridge for bench testing TcpTransport without a cellular modem.
  • SFM REST server (sfm/server.py) — FastAPI server with device info, event list, and event record endpoints over both serial and TCP.

Fixed

  • protocol.py startup() was using a hardcoded POLL_RECV_TIMEOUT = 10.0 constant, ignoring the configurable self._recv_timeout. Fixed to use self._recv_timeout throughout.
  • sfm/server.py now retries once on ProtocolError for TCP connections to handle cold-boot timing on first connect.

Protocol / Documentation

  • Sierra Wireless RV50/RV55 modem config — confirmed required ACEmanager settings: Quiet Mode = Enable, Data Forwarding Timeout = 1, TCP Connect Response Delay = 0. Quiet Mode disabled causes modem to inject RING\r\nCONNECT\r\n onto the serial line, breaking the S3 handshake.
  • Calibration year confirmed at SUB FE (Full Config) destuffed payload offset 0x56–0x57 (uint16 BE). 0x07E7 = 2023, 0x07E9 = 2025.
  • "Operating System" boot string — 16-byte UART boot message captured on cold-start before unit enters DLE-framed mode. Parser handles correctly by scanning for DLE+STX.
  • RV50/RV55 sends RING/CONNECT over TCP to the calling client even with Quiet Mode enabled — this is normal behaviour, parser discards it.

v0.4.0 — 2026-03-12

Added

  • seismo_lab.py — combined Bridge + Analyzer GUI. Single window with two tabs; bridge start auto-wires live mode in the Analyzer.
  • frame_db.py — SQLite frame database. Captures accumulate over time; Query DB tab searches across all sessions.
  • bridges/s3-bridge/proxy.py — bridge proxy module.
  • Large BW→S3 write frame checksum algorithm confirmed and implemented (SUM8 of payload [2:-1] skipping 0x10 bytes, plus constant 0x10, mod 256).
  • SUB A4 identified as composite container frame with embedded inner frames; _extract_a4_inner_frames() and _diff_a4_payloads() reduce diff noise from 2300 → 17 meaningful entries.

Fixed

  • BAD CHK false positives on BW POLL frames — BW frame terminator 03 41 was being included in the de-stuffed payload. Fixed to strip correctly.
  • Aux Trigger read location confirmed at SUB FE offset 0x0109.

v0.3.0 — 2026-03-09

Added

  • Record time confirmed at SUB E5 page2 offset +0x28 as float32 BE.
  • Trigger Sample Width confirmed at BW→S3 write frame SUB 0x82, destuffed payload offset [22].
  • Mode-gating documented: several settings only appear on the wire when the appropriate mode is active.

Fixed

  • 0x082A mystery resolved — fixed-size E5 payload length (2090 bytes), not a record-time field.

v0.2.0 — 2026-03-01

Added

  • Channel config float layout fully confirmed: trigger level, alarm level, and unit string per channel (IEEE 754 BE floats).
  • Blastware .set file format decoded — little-endian binary struct mirroring the wire payload.
  • Operator manual (716U0101 Rev 15) added as cross-reference source.

v0.1.0 — 2026-02-26

Added

  • Initial s3_bridge.py serial bridge — transparent RS-232 tap between Blastware and MiniMate Plus.
  • s3_parser.py — deterministic DLE state machine frame extractor.
  • s3_analyzer.py — session parser, frame differ, Claude export.
  • gui_bridge.py and gui_analyzer.py — Tkinter GUIs.
  • DLE framing confirmed: DLE+STX / DLE+ETX, 0x41 = ACK (not STX), DLE stuffing rule.
  • Response SUB rule confirmed: response_SUB = 0xFF - request_SUB.
  • Year 0x07CB = 1995 confirmed as MiniMate factory RTC default.
  • Full write command family documented (SUBs 68–83).
Reference in New Issue View Git Blame Copy Permalink