Commit Graph
100 Commits
Author SHA1 Message Date
claude 3e7de848bc fix: update unique constraints in events and monitor_log tables to use timestamp and serial number. Can't use event keys because minimates resuse them after clearing memory. 2026-04-16 21:22:04 +00:00
claude 72a4209cfd fix: sfm_webapp.html remove display: flex from base class, now shows active tab 2026-04-16 21:22:04 +00:00
claude 2b5574511e feat: add waveform viewer endpoint and enhance UI with new tabs for history, units, monitor log, and sessions 2026-04-16 21:22:04 +00:00
claude ce2c859f11 fix: update event count retrieval logic in AchSession and MiniMateClient 2026-04-16 21:22:04 +00:00
claude 7f322f9ff9 feat: add option to restart monitoring after event download in AchSession 2026-04-16 21:22:04 +00:00
claude c474db4f69 build: update build backend to setuptools.build_meta 2026-04-16 21:22:04 +00:00
claude 2765ee6ea7 build: add pyproject.toml for editable install 2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 ef88240796 docs: update README to v0.12.0
Rewrites the v0.6.0 README to reflect current project state:
ACH server, SQLite DB, SFM REST API with caching, monitor/erase, updated roadmap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 5591d345d9 feat: v0.12.0 — live device cache (_LiveCache) in sfm/server.py
Ports the intelligent-caching branch concept to a plain Python in-memory
implementation — no SQLAlchemy, no extra DB table, no new dependencies.

_LiveCache (threading.Lock + dicts) caches:
  - device info: indefinite, invalidated by POST /device/config
  - events: keyed by (conn_key, device_event_count); count-probe fast path
    (~2s poll+count_events) avoids full downloads when nothing is new
  - monitor status: 30-second TTL, invalidated by monitor start/stop
  - waveforms: permanent per (conn_key, event_index)

All four cached endpoints accept ?force=true to bypass the cache.
Removes sfm/cache.py (SQLAlchemy experiment, now superseded).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 7883a31aa7 v0.11.0 — SQLite persistence layer (SeismoDb)
sfm/database.py (new)
- SeismoDb class: three tables keyed by unit serial number
  - ach_sessions: one row per ACH call-home
  - events: one row per triggered event, deduped by (serial, waveform_key)
  - monitor_log: one row per monitoring interval, deduped by (serial, waveform_key)
- WAL mode, per-request connections, silent dedup via UNIQUE constraint
- Query helpers: query_events(), query_monitor_log(), get_sessions(), query_units()
- false_trigger flag on events for future review UI / report filtering

bridges/ach_server.py
- Import SeismoDb; create shared instance at startup pointed at
  bridges/captures/seismo_relay.db
- After each call-home: insert_events() + insert_monitor_log() + insert_ach_session()
- DB failures logged as warnings, never abort the session

sfm/server.py
- Import SeismoDb; lazy singleton via _get_db()
- New DB read endpoints: GET /db/units, /db/events, /db/monitor_log, /db/sessions
- PATCH /db/events/{id}/false_trigger for manual review flagging

CLAUDE.md / CHANGELOG.md
- Document DB schema, SFM DB endpoints, architecture decision (unit-keyed only)
- Version bump to v0.11.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:19:47 +00:00
claudeandClaude Sonnet 4.6 b241da970d v0.10.0 — monitor log entry support (SUB 0x0A partial records)
Add full decode pipeline for 0x2C partial records from the device's event
list, representing continuous monitoring intervals where no threshold was
crossed.  These records appear interleaved with full triggered events in the
browse walk and were previously ignored.

minimateplus/models.py
- Add MonitorLogEntry dataclass: key, start_time, stop_time, serial,
  geo_threshold_ips, raw_header, duration_seconds property

minimateplus/protocol.py
- read_waveform_header() now returns (data_rsp.data, length) — full payload
  including the record-type byte at position 0 — instead of the sliced header.
  Callers that need the old slice use raw_data[11:11+length] as before.

minimateplus/client.py
- Add _decode_0a_partial_header(): auto-detects 9-byte (sub_code=0x10) vs
  10-byte (sub_code=0x03) timestamp format, handles 1-byte inter-timestamp
  gap, extracts serial via BE anchor and geo threshold via Geo: anchor.
- Add get_monitor_log_entries(skip_keys=None): browse walk (1E → 0A → 1F),
  decodes partial records, skips full records and already-seen keys.

minimateplus/__init__.py
- Export MonitorLogEntry

bridges/ach_server.py
- After get_events(), call get_monitor_log_entries(skip_keys=seen_keys) and
  save new entries to monitor_log.json in the session directory.
- Add _monitor_log_entry_to_dict() helper.
- Include monitor log keys in downloaded_keys for state persistence.

CLAUDE.md / CHANGELOG.md
- Document 0x2C partial record layout (timestamp format, ASCII metadata
  region, 1-byte gap edge case) confirmed from 4-11-26 MITM capture.
- Version bump to v0.10.0; update What's next.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 6acb419ebd docs: update protocol reference with v0.9.0 erase-all protocol
Changelog section:
- 5 new entries (2026-04-11): erase-all confirmation, SUB 0x06 purpose
  resolved, §7.11 added, §14.6 ACH session lifecycle marked IMPLEMENTED

§5.1 Request Commands:
- SUB 0x06 description updated: "EVENT STORAGE RANGE READ" (not "CHANNEL
  CONFIG READ"), token=0xFE, last 8 bytes = first/last stored event keys
- SUB 0xA3 added: ERASE ALL BEGIN — standard build_bw_frame, token=0xFE, ack 0x5C
- SUB 0xA2 added: ERASE ALL CONFIRM — standard build_bw_frame, token=0xFE, ack 0x5D

§5.2 Response SUBs:
- 0x06→0xF9 marked CONFIRMED 2026-04-11
- 0xA3→0x5C and 0xA2→0x5D added with CONFIRMED status

§7.11 (new section): Erase-All Protocol
- Full wire sequence (6 request/response pairs)
- SUB 0x06 storage range payload layout (36 bytes, last 8 = first/last key)
- Post-erase key counter reset: device restarts from 0x01110000
- Implementation notes pointing to client.py and ach_server.py

§14.6 ACH Session Lifecycle:
- Removed "Future" label — fully implemented in bridges/ach_server.py
- Added step 6 (optional erase), step 8 (DCD/DTR auto-resume)
- Documents ach_server.py flags and ach_state.json schema
- Notes RV55 DCD/DTR issue as known open problem

Open Questions table:
- SUB 0x06 purpose RESOLVED
- Erase-all sequence RESOLVED
- ACH server RESOLVED
- Sensor Check byte: still open, added as formal question
- RV55 DCD/DTR: added as new open question

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 f6a0846bab docs: update CHANGELOG and CLAUDE.md for v0.9.0
CHANGELOG.md:
- New v0.9.0 section covering erase-all protocol, browse helpers,
  delete_all_events(), ach_mitm.py, and ACH server overhaul
- Back-filled v0.8.0 section (write pipeline, monitoring, ACH server)
  that was missing from the previous release notes

CLAUDE.md:
- Bump version to v0.9.0
- Add erase-all protocol section with full wire sequence, SUB 0x06
  storage range response layout, and post-erase key counter reset notes
- Document ACH server state format (ach_state.json v0.9.0 schema with
  downloaded_keys + max_downloaded_key)
- Add RV55 DCD/DTR issue to What's next

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 3d9db8b662 feat: add ach_mitm.py — transparent TCP MITM proxy for ACH session capture
Listens for inbound unit connections, connects upstream to a real Blastware
ACH server, and forwards bytes bidirectionally while saving both directions to
raw_bw_<ts>.bin and raw_s3_<ts>.bin in the existing capture format.

Used to capture the 4-11-26 Blastware ACH session that confirmed the erase-all
protocol (SUBs 0xA3/0x1C/0x06/0xA2) and the event deletion wire sequence.

Usage:
  python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998
  Point the unit's call-home destination at this machine:9998.
  Point this proxy's --bw-host/port at the upstream Blastware ACH server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 c7e7d177e6 feat: overhaul ACH server with key-based state, erase support, and reset detection
State format (ach_state.json):
- Replace event_count with downloaded_keys (set of hex strings) + max_downloaded_key
- Key-based tracking correctly handles delete-then-re-record: after device erase the
  count drops to 0, but new events have new (or recycled) keys

Browse pre-check:
- list_event_keys() walk before get_events() to bail early when nothing is new
- get_events() called with skip_waveform_for_keys= for already-seen keys, so repeat
  call-homes only download waveforms for genuinely new events

--clear-after-download flag:
- After saving new events, calls client.delete_all_events() (0xA3→0x1C→0x06→0xA2)
- On success: resets downloaded_keys=[] and max_downloaded_key="00000000" so the
  next session starts fresh (device counter resets to 0x01110000 after erase)

Post-erase key-reuse detection:
- Device counter resets to 0x01110000 after any erase; new events reuse old keys
- If max(device_keys) < max_downloaded_key, the device was wiped externally
  (Blastware, manual) — seen_keys is discarded and all device keys treated as new

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 a3b8d10fa8 feat: add erase-all protocol and browse helpers to protocol/client layer
protocol.py:
- SUB_ERASE_ALL_BEGIN = 0xA3, SUB_ERASE_ALL_CONFIRM = 0xA2 (confirmed 4-11-26 MITM)
- SUB_CHANNEL_CONFIG (0x06) data length = 0x24 (36 bytes) in DATA_LENGTHS
- begin_erase_all()              — single frame, token=0xFE, response 0x5C
- confirm_erase_all()            — single frame, token=0xFE, response 0x5D
- read_event_storage_range()     — two-step read (probe+data), token=0xFE
  Response last 8 bytes = first/last stored event key; both 0x01110000 after erase

client.py:
- list_event_keys()              — browse-mode 1E→0A→1F walk, no waveform download;
  returns list of hex key strings; used as fast pre-check before get_events()
- get_events(skip_waveform_for_keys=set())
  — for already-seen keys: only 0A+1F(browse), skips 1E-arm/0C/POLL×3/5A entirely
- delete_all_events()            — orchestrates the confirmed erase sequence:
  0xA3 → 0x1C → 0x06 → 0xA2; logs first/last key from storage range response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 4921b0489a fix: correct Event and PeakValues field names in ach_server serialization
Event model uses peak_values (not peaks) and project_info (not direct fields).
PeakValues fields are tran/vert/long/micl/peak_vector_sum (not transverse etc).
ProjectInfo fields accessed via ev.project_info.project etc.

Also fix ev.timestamp serialization: use str() instead of .isoformat() since
Timestamp is a custom dataclass, not datetime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 8688d815a0 fix: remove non-existent DeviceInfo fields from ach_server log and dict
calibration_date, aux_trigger, setup_name etc. don't exist directly on
DeviceInfo — they live in DeviceInfo.compliance_config (ComplianceConfig).
_device_info_to_dict now accesses them via cc = d.compliance_config.
Log line updated to show serial/firmware/model/event_count instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 9b50ec9133 fix: make Ctrl-C work on Windows by setting accept() timeout
socket.accept() on Windows blocks indefinitely and ignores KeyboardInterrupt.
Setting a 1-second timeout on the server socket causes the accept loop to wake
up every second and re-check, so Ctrl-C is handled within ~1 second.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 cba8b1b401 feat: defer session dir creation and add --allow-ip allowlist
- Session directory and log file are now created ONLY after startup() succeeds.
  Internet scanners and dropped connections no longer litter the output folder.
  Raw bytes are buffered in memory until startup succeeds, then flushed to disk.

- Add --allow-ip IP flag (repeatable) to allowlist specific source IPs.
  Connections from un-listed IPs are rejected immediately (socket closed, no log).
  If no --allow-ip flags are given, all IPs are still accepted (original behavior).
  Usage: --allow-ip 63.43.212.232 --allow-ip 152.1.2.3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 41a14ca468 fix: correct event count field offset and eliminate count_events() walk
_decode_event_count: read uint16 BE at offset 10 (confirmed 2026-04-10 from
live BE11529 event index — data[10:12]=0x0006=6, matches device LCD).
Previous uint32 at offset 3 always returned 1 regardless of event count.

ach_server.py: use device_info.event_count (already fetched during connect())
instead of calling count_events() separately. This saves 2*N round-trips and
avoids the 1F linked-list walk which was overcounting on some devices.
count_events() kept as fallback when connect() is skipped (--events-only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 1bfc6e4258 fix: replace Unicode chars in log messages, fix DeviceInfo.serial, UTF-8 file log
- Replace all Unicode arrows/checkmarks (->  [OK]  [FAIL]) in ach_server.py
  and client.py log calls — Windows cp1252 console can't encode them
- Fix DeviceInfo attribute: serial_number -> serial
- Fix _device_info_to_dict key: serial_number -> serial
- Demote count_events 1E/1F per-key log lines from WARNING to DEBUG
  (they were flooding the console on devices with many stored events)
- FileHandler now opens with encoding='utf-8' so session log files
  can hold any characters without codec errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claude 574d40027f feat: enhance logging messages in ach_server.py and add experiments.py for protocol minimization 2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 0358acb51d feat: add high-water mark state tracking to ach_server + fix monitoring flag
ach_server.py:
- Add ach_state.json per-unit state tracking (keyed by serial number)
- count_events() before any download; skip session if no new events since last call-home
- Download only events beyond the previous high-water mark (all_events[last_count:])
- --max-events N safety cap for first-run units with many stored events
- state_path and max_events wired through AchSession constructor and serve()

client.py (_decode_monitor_status):
- Revert monitoring flag to section[1] == 0x10 (was incorrectly changed to section[6])
- Fix battery/memory offsets to section[-10:-8], [-8:-4], [-4:] (no trailing checksum byte)
- Both confirmed by full byte diff of all 144 0xE3 data frames in 4-8-26/2ndtry capture

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 cf7d838bf4 feat: add SocketTransport and ach_server.py inbound ACH server
minimateplus/transport.py:
- Add SocketTransport(TcpTransport) — wraps an already-accepted inbound
  socket; connect() is a no-op; everything else inherited from TcpTransport.
  Enables the ACH server to reuse all existing protocol/client code without
  any changes.

bridges/ach_server.py:
- Minimal inbound ACH server — listens on port 12345, accepts call-home
  connections from MiniMate Plus units, runs the full BW protocol:
  startup handshake → get_device_info → get_events(full_waveform=True)
- Saves device_info.json + events.json + raw_rx_<ts>.bin + session log
  per connection to bridges/captures/ach_inbound_<ts>/
- raw_rx.bin is byte-compatible with existing Analyzer tooling
- Taps transport.read() to capture raw S3 bytes alongside parsed output
- Each connection runs in its own daemon thread
- Clearly distinguishes push vs pull protocol in the startup log

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 5e44cdc668 feat: add splitter mode to ach_bridge.py (--mirror HOST:PORT)
Adds a production-safe headphone-splitter mode:
- Device bytes tee'd to both --upstream (primary/prod) and --mirror (new server)
- Only primary server responses are returned to the device
- Mirror connect/write failures are non-fatal and logged; prod is unaffected
- New raw_mirror_<ts>.bin capture file alongside raw_client/raw_server

Three modes: standalone (capture only), bridge (one upstream), splitter (two).
Default listen port changed to 12345 to match project ACH setup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 37d32077a4 feat: add ACH TCP bridge, serial tap tool, and Serial Watch tab
- bridges/ach_bridge.py: transparent TCP bridge that MITMs the MiniMate Plus
  call-home connection — forwards to real ACH server while logging all frames
  to raw_client/raw_server .bin files compatible with parse_capture.py;
  standalone capture mode for lab use without a real server

- bridges/serial_watch.py: RS-232 serial monitor with live S3 frame parsing;
  taps the line between MiniMate and modem (RV50/RV55); captures raw bytes,
  .log and .jsonl; --ack-ok mode auto-replies to AT commands; fixed fatal
  indentation bug in the original that silently prevented any data capture

- seismo_lab.py: new "Serial Watch" fourth tab (SerialWatchPanel) wrapping
  serial_watch.py functionality; COM port picker with refresh, baud config,
  ack-ok toggle, colour-coded live frame log (teal frames / yellow ctrl /
  blue AT), raw .bin capture auto-fed into Analyzer tab on stop

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claude 990cb8850e fix: correct monitoring flag and battery/memory offsets in _decode_monitor_status
section[6] is the monitoring flag (was wrongly section[1] — section[1] is always
0x00 in both states). Battery and memory fields use relative-from-end offsets
(section[-11:-9], section[-9:-5], section[-5:-1]) instead of absolute positions,
which broke when the payload grew by 3 bytes in monitoring mode.

Confirmed from full byte diff of 142 0xE3 frames in 4-8-26/2ndtry capture.
SFM start_monitoring now polls /device/monitor/status every 5s for up to 60s
instead of a fixed 25s delay (unit runs ~40s on-device sensor check before
confirming monitoring state).

Also corrects stale 1C→6E response anomaly claim in protocol reference — no
exceptions to the 0xFF−SUB rule are known.
2026-04-08 23:41:11 -04:00
claude dda5683572 fix: improve monitoring functionality with session-reset signal and payload adjustments 2026-04-08 18:29:51 -04:00
claude 16e072698b feat: Implement poll() method for efficient device communication and update monitoring status retrieval 2026-04-08 16:33:21 -04:00
claude c8c57e950c fix: replace helper in server.py with correct name. 2026-04-08 16:16:47 -04:00
claude a41e7a9e1a feat: Add monitoring functionality to MiniMate protocol and web interface
- Introduced new SUBs for monitoring status, start, and stop commands in protocol.py.
- Implemented read_monitor_status, start_monitoring, and stop_monitoring methods in MiniMateProtocol class.
- Added new API endpoints for monitoring status retrieval and control in server.py.
- Enhanced the web application with a monitoring panel, including battery and memory status display.
- Created a new Python script to parse SUB 0x1C response frames for monitoring status.
- Documented the monitoring status response format and field locations in markdown and text files.
2026-04-08 14:34:42 -04:00
claude 8545daac04 fix: show mic as dbL (not psi) 2026-04-07 19:49:06 -04:00
claude 1a9dcc04b4 feat: add webapp 2026-04-07 19:33:29 -04:00
claude a7ab6eaf7c feat: add config API endpoint and JSON schema draft 2026-04-07 17:26:24 -04:00
claude 7005ae766d feat: implement set_project_info functionality and add POC test script 2026-04-07 02:49:17 -04:00
claude bcc044655a feat: updates to 0.8.0 - initial write functions 2026-04-07 02:09:29 -04:00
claudeandClaude Sonnet 4.6 c2ab94f20c docs: mark fi==9 decoder skip as fixed in CLAUDE.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 23:34:43 -04:00
claudeandClaude Sonnet 4.6 b5828de534 fix: remove hardcoded fi==9 skip in _decode_a5_waveform
Frame index 9 was assumed to be the device terminator based on the
9-frame original blast capture. For streams with >9 frames (current
device produces 35), fi==9 is live waveform data — the skip was
dropping ~133 sample-sets per event.

Terminator detection is handled upstream via page_key==0x0000 in
read_bulk_waveform_stream, so no index-based skip is needed here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 23:34:33 -04:00
claudeandClaude Sonnet 4.6 9bef430451 docs: document 5A end-of-stream signal, chunk timing, fi==9 bug, ADC conversion
Adds §7.8.4 to protocol reference and corresponding CLAUDE.md sections:

- End-of-stream: device sends exactly 1 raw byte after last chunk; handled
  via TimeoutError + bytes_fed>0 check → graceful break to termination
- Chunk timing: ~1s per chunk, 35 chunks for a 9,306-sample event, safe
  timeout is 10s (not default 120s)
- fi==9 decoder bug: hardcoded skip drops ~133 sample-sets per event;
  noted as known issue pending fix
- ADC conversion: counts × (range/32767) → physical units (in/s for geo)

Changelog entries added for all four items (2026-04-06).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:28:26 -04:00
claudeandClaude Sonnet 4.6 781d21f132 perf: reduce 5A chunk timeout to 10s and stop iteration at requested event index
Two improvements to eliminate the ~2-min-per-event wait and unnecessary
full-event-list download when only one event is requested:

1. protocol.py: pass timeout=10.0 to _recv_one in the 5A chunk loop.
   Device responds within ~1s per chunk; 10s gives a safe 10x buffer.
   End-of-stream detection (raw_bytes=1) now fires in 10s instead of 120s,
   cutting ~110s of dead wait per event.

2. client.py: add stop_after_index parameter to get_events(). When set,
   iteration stops immediately after the target event is collected — no
   further 0A/1E/0C/5A/1F cycles for events the caller doesn't need.

3. server.py: pass stop_after_index=index to both /device/event/{idx}
   and /device/event/{idx}/waveform endpoints so a single-event request
   only downloads that one event.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 19:02:01 -04:00
claudeandClaude Sonnet 4.6 e3a5c6f07d 5A: treat partial end-of-stream byte as graceful stream termination
When the device finishes streaming waveform data, it sends a single
partial byte (raw_bytes=1) rather than a complete A5 frame, then goes
silent for the full 120s timeout.

Observed: 35 chunks succeed, chunk 36 times out with raw_bytes=1 —
identical for both events in the test run.

Fix: on TimeoutError, if bytes_fed > 0 and we already have collected
frames, treat it as natural end-of-stream and break to the termination
step rather than propagating the exception.  True transport failures
(raw_bytes=0, no prior frames) still raise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 17:41:29 -04:00
claudeandClaude Sonnet 4.6 1397f8486f diag: per-chunk WARNING logging in 5A bulk stream loop
Wraps each recv_one call in a try/except TimeoutError, logging:
- On timeout: chunk_num, counter, raw bytes_fed (distinguishes "device
  silent" from "device sent unparseable bytes")
- On success: chunk_num, page_key, data_len, contains_Project flag

Parser is explicitly reset before each chunk recv so bytes_fed is
accurate per-chunk rather than cumulative. Helps identify exactly which
chunk fails and whether the device is responding at all.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 17:32:49 -04:00
claude 5b3e8af1e3 fix: remove special case chunk counter, all chunks use chunk_num * 0x0400 2026-04-06 17:03:09 -04:00
claude ad1c9e48b0 fix: update MiniMateClient and protocol to ensure correct handling of 1F calls and improve event download sequence 2026-04-06 15:58:03 -04:00
claude 227c481022 fix: update MiniMateClient to correctly handle 1F calls for event iteration and 5A arming 2026-04-06 14:37:36 -04:00
claude 33de4239f4 fix: update MiniMateClient to use browse=False for 1F in download mode to ensure correct token handling for 5A 2026-04-06 13:39:11 -04:00
claude d0d5a18d5c fix: update event handling in MiniMateClient and protocol to ensure correct sequence for waveform downloads 2026-04-06 13:19:51 -04:00
claude 41090a9346 fix: enhance logging for event download process in MiniMateClient 2026-04-06 12:57:24 -04:00
claude d87e02fab2 fix: update event handling in MiniMateClient and protocol to address 5A timeout issue 2026-04-06 12:41:27 -04:00
claude 57e7225a62 fix: correct DLE-stuffing handling in build_5a_frame function and update parameter description 2026-04-06 03:02:36 -04:00
claude 5d43acd827 fix: clarify bulk counter step handling and improve comments for waveform chunk processing 2026-04-06 01:35:09 -04:00
claude dfa09d2a4f fix: clarify event handling in waveform viewer 2026-04-06 00:00:06 -04:00
claude ecb1147216 fix: update Peak Vector Sum offset calculation and clarify event_count handling in device info 2026-04-05 02:48:58 -04:00
claude 1c570b083a fix: update timestamp decoding for Waveform and Continuous records in models and client 2026-04-04 00:09:55 -04:00
claude 2286d2ccf8 fix: enhance event record handling to include MonitorLog sub_code and adjust timestamp parsing logic 2026-04-03 18:58:46 -04:00
claude 755050b347 fix: ensure proper waveform context by always calling 0A before 0C and use all-zero params for event advancement 2026-04-03 18:48:04 -04:00
claude 6adf8b6078 Fix: advance_event(browse=True) now sends token_params(0) — all zeros — matching the confirmed BW browse sequence. 2026-04-03 18:15:42 -04:00
claude 4fb1bbfe35 fix: fix token position in params and enhance event iteration logic for MiniMateClient 2026-04-03 17:30:47 -04:00
claude 3effa1aab5 feat: add detailed logging for event counting process in MiniMateClient 2026-04-03 16:47:06 -04:00
claude 95f2becf21 feat: update event iteration logic to use null sentinel for end-of-events detection 2026-04-03 16:29:10 -04:00
claude 2cb95cd45e feat: implement reliable event counting via 1E/1F chain and update device info 2026-04-03 16:02:10 -04:00
claude 7cd8fda5e8 feat: add logging for raw event index bytes and decoded count in event count decoder 2026-04-03 15:28:51 -04:00
claude f495b91d8a feat: enhance waveform viewer with record type handling and improved empty state messaging 2026-04-03 15:22:26 -04:00
claude e4730376ad feat: enhance waveform viewer with unit info display and event selection functionality 2026-04-03 15:08:57 -04:00
claude 23e4febba6 feat: add CORS middleware to allow cross-origin requests for waveform viewer 2026-04-03 14:50:43 -04:00
claude 8941dd0aef feat: add waveform_viewer page 2026-04-03 14:26:47 -04:00
claude dfb974d658 feat: add endpoint to download full raw ADC waveform for a single event 2026-04-03 13:54:54 -04:00
claude 790e442a7a feat: implement raw ADC waveform decoding and download functionality
- Added `_decode_a5_waveform()` to parse SUB 5A frames into per-channel time-series data.
- Introduced `download_waveform(event)` method in `MiniMateClient` to fetch full waveform data.
- Updated `Event` model to include new fields: `total_samples`, `pretrig_samples`, `rectime_seconds`, and `_waveform_key`.
- Enhanced documentation in `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect new features and confirmed protocol details.
2026-04-03 13:53:09 -04:00
claude 5d0f0855f2 doc: update to .0.6.0 with full working event read loop 2026-04-02 17:30:33 -04:00
claude 0f5aa7a3fc fix: adjust anchor search range in compliance config decoder for improved data handling 2026-04-02 17:22:56 -04:00
claude 3b04d4683b fix: update compliance config read logic to handle buffered responses on slow links 2026-04-02 17:18:48 -04:00
claude 0363425d83 fix: add addition recv after loop to make up for modem TCP/IP lag 2026-04-02 17:13:12 -04:00
claude 66967e036c fix: match BW's 5A frame probe to parse event-time metadata. 2026-04-02 16:57:12 -04:00
claudeandClaude Sonnet 4.6 9bf20803c2 feat: fetch event-time metadata from SUB 5A bulk waveform stream
Add read_bulk_waveform_stream() to MiniMateProtocol and wire it into
get_events() so each event gets authoritative client/operator/sensor_location
from the A5 frames recorded at event-time, not the current compliance config.

- framing.py: bulk_waveform_params() and bulk_waveform_term_params() helpers
  (probe/chunk params and termination params for SUB 5A, confirmed from
  1-2-26 BW TX capture)
- protocol.py: read_bulk_waveform_stream(key4, stop_after_metadata=True) —
  probe + chunk loop (counter += 0x0400), early stop when b"Project:" found
  (A5[7] of 9), then sends termination at offset=0x005A
- client.py: _decode_a5_metadata_into() needle-searches concatenated A5 frame
  data for Project/Client/User Name/Seis Loc/Extended Notes; get_events() now
  calls SUB 5A after each SUB 0C, overwriting project_info with event-time values

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 16:08:21 -04:00
claude 9b1ed1f3a8 client: update channel block layout documentation and improve validation checks for 'Tran' label 2026-04-02 15:31:09 -04:00
claudeandClaude Sonnet 4.6 501b5080e9 server: backfill event.project_info fields from compliance config
The 210-byte waveform record only stores "Project:" — client, operator,
sensor_location, and notes are device-level settings in SUB 1A, not
per-event fields. Backfill those into each event's project_info after
download, same pattern as the sample_rate backfill.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:33:33 -04:00
claudeandClaude Sonnet 4.6 5948c833bd client: lower Tran search floor from 1000 to 44 in channel block extraction
The channel block is in frame C data (cfg[44:1071]) not deep in a
hypothetical frame D section.  The offset-1000 assumption was wrong —
searching from 44 lets us find it while unit string validation still
prevents false positives.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:22:00 -04:00
claudeandClaude Sonnet 4.6 c4a5da893c client: add WARNING-level diagnostics for Tran channel block search
Temporary: log tran_pos, surrounding bytes, and exact unit string check
results at WARNING level so we can see why trigger/alarm/max_range are
still null even with the full 2126-byte cfg.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:18:27 -04:00
claudeandClaude Sonnet 4.6 638e60532c protocol: tighten compliance config dedup to exact byte content
Old key was (page_key, chunk_len) which would incorrectly drop a second
config section that has the same length as the first (e.g. current-config
vs event-time-config when settings haven't changed).

New key is the full chunk bytes — only truly byte-identical chunks are
dropped.  Different data that happens to share page_key and length now
comes through correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 03:36:25 -04:00
claudeandClaude Sonnet 4.6 6eecd0c1d1 client/models/server: wire event_count from SUB 08 event index into connect()
- DeviceInfo.event_count: Optional[int] = None  (new field in models.py)
- connect() now calls proto.read_event_index() after compliance config and
  stores the decoded count in device_info.event_count
- _serialise_device_info() exposes event_count in /device/info and /device/events
  JSON responses

event_count is decoded from uint32 BE at offset +3 of the 88-byte F7 payload
(🔶 inferred — needs live device confirmation against a multi-event device).
Any ProtocolError from the index read is caught and logged; event_count stays
None rather than failing the whole connect().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 02:00:37 -04:00
claudeandClaude Sonnet 4.6 870a10365e server: fill ev.sample_rate from compliance config for /device/events
sample_rate is a device-level setting stored in the compliance config,
not per-event in the waveform record.  After downloading events, backfill
ev.sample_rate from info.compliance_config.sample_rate for any event
that didn't get it from the waveform record decode path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 02:00:00 -04:00
claudeandClaude Sonnet 4.6 b2d10fd689 client: wire trigger_level_geo, alarm_level_geo, max_range_geo from channel block
The channel block is only present in the full ~2126-byte cfg (when frame D
delivers correctly rather than duplicating frame B's page).  Layout per §7.6:
  [00 00][max_range f32][00 00][trigger f32]["in.\0"][alarm f32]["/s\0\0"][00 01][label]

Relative offsets from the "Tran" label position (label-24/label-18/label-10)
are validated by checking the unit strings "in.\0" at label-14 and "/s\0\0"
at label-6 before reading the floats.  Guard against "Tran2" false-match.

When frame D duplicates, cfg is ~1071 bytes and tran_pos search returns a hit
without the unit string sentinels — we log the miss and leave fields None.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 01:59:56 -04:00
claudeandClaude Sonnet 4.6 ce44852383 protocol: add read_event_index() for SUB 08/F7
Two-step probe+fetch for SUB 08 (EVENT_INDEX), returning the raw 88-byte
(0x58) index block.  SUB_EVENT_INDEX and DATA_LENGTHS[0x08]=0x58 were
already registered — this just wires the method that calls them.

Docstring notes the partially-decoded layout (event count at +3 as uint32 BE,
timestamps at +7) pending live device confirmation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 01:59:48 -04:00
claude 6a42facf02 docs: update protocol reference for SUB 1A compliance config read
New/corrected sections:

§7.6.1 Record Time — corrected: anchor+10 supersedes the +0x28 absolute
offset.  Added 3 s, 5 s, 8 s confirmations alongside existing 7/10/13.
Warning added: do NOT use fixed offset for BE11529.

§7.6.2 (NEW) SUB 1A Multi-Frame Read Protocol — 4-frame A/B/C/D
sequence documented (reverse-engineered from raw_bw capture).  E5
page_key field explained.  BE11529 duplicate-page behaviour and
(page_key, chunk_len) dedup strategy documented.

§7.6.3 (NEW) Sample Rate and DLE Jitter — Normal/Fast/Faster = 1024/
2048/4096 Sa/s confirmed.  Root cause of ±1 byte cfg jitter explained:
4096 = 0x1000 → `10 10 00` in raw frame → `10 00` after DLE unstuffing
= 1 byte shorter than 04 00/08 00.  Anchor search requirement explained.

Changelog — 5 new entries covering the 4-frame sequence, duplicate-page
detection, record_time anchor correction, sample_rate confirmation, and
_pending_frames / reset_parser=False implementation notes.

Quick Reference — Record Time and Sample Rate rows updated with correct
locations, types, and confirmed values.

Open Questions — SUB 1A item updated to "substantially resolved".
Record time item updated. Sample rate added as resolved.
2026-04-01 16:54:51 -04:00
claude 4b703811d9 client: remove _cfg_diagnostic now that all compliance fields are confirmed
record_time (float32_BE) and sample_rate (uint16_BE) both validated
against live device across normal / fast / faster modes and multiple
record time settings.  Diagnostic scaffolding no longer needed.
2026-04-01 16:49:37 -04:00
claude ea4475c9ad client: use anchor-relative offsets for record_time and sample_rate
Fixed a 1-byte offset jitter that produced garbage values when the
device was set to "faster" (4096 Sa/s) mode.

Root cause: 4096 = 0x1000, so the sample_rate bytes in the raw S3
frame are `10 10 00` (DLE-escaped).  After DLE unstuffing → `10 00`
(2 bytes vs 3 for 1024/2048), making frame C 1 byte shorter and
shifting all subsequent field offsets by -1.

Fix: locate the stable 10-byte anchor `01 2c 00 00 be 80 00 00 00 00`
(max-record-limit constant + first two alarm-level floats) and read:
  sample_rate  = uint16_BE at anchor - 2
  record_time  = float32_BE at anchor + 10

Offline-validated against all five logged hex dumps (1071 and 1070
byte cfg, record times 3/5/8 s, sample rates 1024 and 4096):
  all five: correct values with anchor approach.
2026-04-01 16:45:40 -04:00
claude df51fe0668 client: wire record_time (cfg[64] f32_BE) and sample_rate (cfg[52] u16_BE)
Both offsets identified from _cfg_diagnostic scan on BE11529:
  cfg[64] float32_BE = 3.0  → record_time in seconds
  cfg[52] uint16_BE  = 1024 → sample_rate in Sa/s

Values are plausible but NOT yet validated by changing device settings
and re-reading.  Marked as ✅ candidate in docstring — confirm and
remove the ⚠️ note once a device-side change is observed here.
2026-04-01 15:40:35 -04:00
claude 114cbb4679 protocol: skip duplicate E5 chunks in read_compliance_config
BE11529 sometimes returns frame D with page_key=0x0000 (44 bytes),
identical to the frame B response, inflating cfg to ~1115 bytes and
mis-aligning all field offsets.  Track (page_key, chunk_size) pairs
and drop any repeat before appending to the running config buffer.
2026-04-01 15:39:56 -04:00
claudeandClaude Sonnet 4.6 58a5f15ed5 client: fix setup_name; add diagnostic scan for record_time/sample_rate
- setup_name was broken: _find_string_after(b"Standard Recording Setup")
  returned what comes AFTER the string (i.e. "Project:"), not the name
  itself.  Fixed by searching for the first long (>=8 char) ASCII string
  in cfg[40:250] with _find_first_string().

- record_time offset 0x28 was wrong (that location holds "(L)", a unit
  label string).  Disabled for now to avoid returning garbage; correct
  offset will be determined from _cfg_diagnostic() output.

- Added _cfg_diagnostic(): logs all strings and all plausible float32/uint16
  values across the full cfg so record_time and sample_rate offsets can be
  pinpointed from a single device run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 15:18:41 -04:00
claudeandClaude Sonnet 4.6 eee1e36a1b protocol: send 3-step data requests for full compliance config (SUB 1A)
Reverse-engineered the full Blastware 4-frame sequence for SUB 1A:
  A (probe):      offset=0x0000, params[7]=0x64
  B (data req 1): offset=0x0400, params[2]=0x00, params[7]=0x64  → bytes 0..1023
  C (data req 2): offset=0x0400, params[2]=0x04, params[7]=0x64  → bytes 1024..2047
  D (data req 3): offset=0x002A, params[2]=0x08, params[7]=0x64  → bytes 2048..2089

We were only sending A+D and getting 44 bytes (the last chunk).
Now sends B, C, D in sequence; each E5 response has 11-byte echo header
stripped, and chunks are concatenated.  Devices that return all data in
one frame (BE18189 style) are handled — timeouts on B/C are skipped
gracefully and data from D still accumulates.

Total expected: 0x0400 + 0x0400 + 0x002A = 0x082A = 2090 bytes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 15:08:50 -04:00
claudeandClaude Sonnet 4.6 a5069c302d protocol: accumulate multiple E5 frames for chunked compliance config
BE11529 sends the compliance config (SUB 1A / E5 response) as a stream
of small frames (~44 bytes each) rather than one large frame like BE18189.

Changes:
- read_compliance_config(): loop calling _recv_one() until 0x082A bytes
  accumulated or 2s inter-frame gap; first frame strips 11-byte echo header,
  subsequent frames logged in full for structure analysis
- _recv_one(): add reset_parser=False option to preserve parser state and
  _pending_frames buffer between consecutive reads from one device response;
  also stash any extra frames parsed in a single TCP chunk so they are not lost

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 14:54:16 -04:00
claude 6e0b83efa5 fix: update compliance config reading to handle variable payload sizes and improve logging 2026-04-01 14:31:03 -04:00
claude d4b1b834a7 fix: refine SUB 1A compliance config read parameters and logging 2026-04-01 14:20:12 -04:00
claude 824322597a fix: validate offset size in build_bw_frame and adjust payload construction 2026-04-01 13:28:00 -04:00
claudeandClaude Haiku 4.5 7e501620fc fix: skip trigger/alarm extraction pending offset confirmation
The heuristic offsets for trigger/alarm levels were causing struct unpack
errors. These fields require detailed field mapping from actual E5 captures
to determine exact byte positions relative to channel labels.

For now, skip extraction and leave trigger_level_geo/alarm_level_geo as None.
This prevents the '500 Device error: bytes must be in range(0, 256)' error.

Once we capture an E5 response and map the exact float positions, we can
re-enable this section with correct offsets.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-01 13:07:04 -04:00
claudeandClaude Sonnet 4.6 32b9d3050c feat: implement SUB 1A (compliance config) read
Adds full support for reading device compliance configuration (2090-byte E5
response) containing record time, trigger/alarm levels, and project strings.

protocol.py:
- Implement read_compliance_config() two-step read (SUB 1A → E5)
- Fixed length 0x082A (2090 bytes)

models.py:
- Add ComplianceConfig dataclass with fields: record_time, sample_rate,
  trigger_level_geo, alarm_level_geo, max_range_geo, project strings
- Add compliance_config field to DeviceInfo

client.py:
- Implement _decode_compliance_config_into() to extract:
  * Record time float at offset +0x28 ✅
  * Trigger/alarm levels per-channel (heuristic parsing) 🔶
  * Project/setup strings from E5 payload
  * Placeholder for sample_rate (location TBD ❓)
- Update connect() to read SUB 1A after SUB 01, cache in device_info
- Add ComplianceConfig to imports

sfm/server.py:
- Add _serialise_compliance_config() JSON encoder
- Include compliance_config in /device/info response
- Updated _serialise_device_info() to output compliance config

Both record_time (at fixed offset 0x28) and project strings are ✅ CONFIRMED
from protocol reference §7.6. Trigger/alarm extraction uses heuristics
pending more detailed field mapping from captured data.

Sample rate remains undiscovered in the E5 payload — likely in the
mystery flags at offset +0x12 or requires a "fast mode" capture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 12:08:43 -04:00
claude a8187eccd0 chore: add /example-events/ to gitignore 2026-04-01 01:20:30 -04:00
claudeandClaude Sonnet 4.6 4944974f6e feat: decode waveform record timestamp, record type, and Peak Vector Sum
Confirmed 2026-04-01 against Blastware event report for BE11529 thump
event ("00:28:12 April 1, 2026", PVS 3.906 in/s).

models.py:
- Timestamp.from_waveform_record(): decode 9-byte format from 0C record
  bytes[0-8]: [day][sub_code][month][year:2BE][?][hour][min][sec]
- Timestamp: add hour/minute/second optional fields; __str__ includes
  time when available
- PeakValues: add peak_vector_sum field (confirmed fixed offset 87)

client.py:
- _decode_waveform_record_into: add timestamp decode from bytes[0:9]
- _extract_record_type: decode byte[1] (sub_code), not ASCII string
  search; 0x10 → "Waveform", histogram TBD
- _extract_peak_floats: add PVS from offset 87 (IEEE 754 BE float32)
  = √(T²+V²+L²) at max instantaneous vector moment

sfm/server.py:
- _serialise_timestamp: add hour/minute/second/day fields to JSON
- _serialise_peak_values: add peak_vector_sum to JSON

docs: update §7.7.5 and §8 with confirmed 9-byte timestamp layout,
PVS field, and byte[1] record type encoding; update command table;
close resolved open questions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 00:53:34 -04:00
claude f74992f4e5 fix: serial decode offset, PPV label scan, debug mode for waveform records
- _decode_serial_number: read from data[11:] not data[:8] — was returning
  the LENGTH_ECHO byte (0x0A = '\n') instead of the serial string
- _extract_peak_floats: search for channel label strings ("Tran" etc) and
  read float at label+6; old step-4 aligned scan was reading trigger levels
  instead of PPV values
- get_events: add debug=False param; stashes raw 210-byte record on
  Event._raw_record when True for field-layout inspection
- server /device/events: add ?debug=true query param; includes
  raw_record_hex + raw_record_len in response when set
- models: add Event._raw_record optional bytes field
2026-03-31 23:46:07 -04:00