Commit Graph

108 Commits

Author SHA1 Message Date
claude b66cc9d075 fix(blastware_file): update TERM detection logic and strip duplicate header blocks for accurate file writing 2026-05-04 14:28:11 -04:00
claude 45e61fbcaf big refactor of waveform protocol. 2026-05-03 01:20:21 -04:00
claude d758825c67 fix(protocol): correct continuous-mode record header classification for accurate timestamp extraction 2026-05-01 20:28:55 -04:00
claude 0fbb39c21a Big event bugfix. see details:
## v0.13.0 — 2026-05-01

### Fixed

- **SUB 5A bulk waveform stream — over-read bug for events ≥ 2 sec.**
  `read_bulk_waveform_stream` was walking the chunk counter past the actual
  end of the event, picking up post-event circular-buffer garbage that
  corrupted reconstructed Blastware files for any waveform > ~1 sec.  The
  loop now extracts the event's `end_offset` from the STRT record at
  `data[23:27]` of the probe response and stops the chunk walk when the next
  counter would step past it.  Verified against three BW MITM captures
  (4-27-26 + 5-1-26): 2-sec event drops from 37 over-read chunks to 7
  bounded chunks; 3-sec drops to 9; non-zero-start "event 2" drops to 9.

### Added

- `framing.bulk_waveform_term_v2(key4, end_offset, last_chunk_counter)` —
  computes the corrected SUB 5A TERM frame's `(offset_word, params)` per the
  formula confirmed across all 3 BW captures.  Not yet wired into
  `read_bulk_waveform_stream` (the legacy TERM is still used to preserve the
  existing `blastware_file.write_blastware_file` frame-structure expectations);
  available for the next iteration that switches to BW's 0x0200 chunk step.
- `framing.parse_strt_end_offset(a5_data)` — extracts the event-end pointer
  from the STRT record in an A5 response payload.
2026-05-01 18:37:34 -04:00
Claude 625b0a4dfc feat(seismo_lab): add Download tab that captures wire bytes during event download
Adds a new CapturingTransport wrapper in minimateplus.transport that mirrors
every TX/RX byte to two raw .bin files using the same on-wire format as
bridges/ach_mitm.py, so the resulting captures are byte-for-byte compatible
with the existing Blastware MITM captures and load directly in the Analyzer.

A new "Download" tab in seismo_lab.py lets the user connect to a device over
TCP or serial and run connect / list-keys / download-events while the wrapper
saves raw_bw_<ts>.bin (our TX) and raw_s3_<ts>.bin (device TX) into a
seismo_dl_<ts>[_<label>]/ session directory. On completion, the panel hands
both files to the Analyzer and switches tabs, mirroring the UX of the
existing Bridge capture flow.
2026-05-01 00:12:02 +00:00
claude a7585cb5e0 fix(blastware_file, server): implement logic to skip extra chunks after metadata for accurate file writing 2026-04-26 16:32:32 -04:00
claude ae30a02898 fix(blastware_file, server): enhance logging and correct chunk handling for accurate data processing 2026-04-26 16:03:07 -04:00
claude 2f084ed105 fix(protocol): update chunk counter formula to use max(key4[2:4], 0x0400) for accurate data streaming 2026-04-26 01:28:47 -04:00
claude 7976b544ed fix(blastware_file): never skip A5 frames based on classification at fi>0
Frame 0 is always the probe; frames 1+ are always data (waveform ADC
chunks, compliance config, compliance continuation).  Gating on
classify_frame() at fi>0 produces false positives: ADC binary data
can coincidentally contain b"STRT\xff\xfe", causing frames 1 and 5
to be silently dropped from the body (confirmed from live capture on
event key=01110000).  Remove all type-based filtering; include every
frame unconditionally with the standard index-based skip amounts.
2026-04-26 00:59:36 -04:00
claude 0415af19b4 fix(blastware_file): remove seen_metadata flag and adjust frame processing logic 2026-04-24 20:21:03 -04:00
claude 35c3f4f945 fix(protocol): correct A5 frame classification and chunk counter formula 2026-04-24 17:25:29 -04:00
claude 43c8158493 feat(blastware_file): classify A5 frames, only write waveform frames to body
Add classify_frame() which categorises each A5 frame by content:
  terminator    — page_key == 0x0000
  probe_or_strt — contains b"STRT"
  metadata      — contains compliance-config ASCII markers
                  (Project:, Client:, Standard Recording Setup, …)
  waveform      — binary-heavy (< 20% printable ASCII), i.e. raw ADC data
  unknown       — fallback

Update write_blastware_file() body loop: frame 0 (probe) is still
always processed; frames 1+ are only included when classify_frame
returns "waveform".  Metadata frames (compliance config block with
Project:/Client:/etc.) and any stray STRT-bearing frames are skipped
with a warning/debug log.  Terminator frame handling is unchanged.

Adds temporary print() diagnostics so each frame's classification is
visible in the server log to aid debugging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 15:48:37 -04:00
claude 242666f358 fix(protocol): correct chunk counter formula for accurate data streaming 2026-04-24 12:52:02 -04:00
claude 03540fdc00 fix: raise max_chunks to 128 for metadata-only 5A download
For 2-second events at 1024 sps the "Project:" metadata frame appears
beyond chunk 32 (the old default cap), causing the safety limit to be
hit and ~34 KB of waveform data to be downloaded instead of stopping
at the metadata frame.  Raising max_chunks to 128 ensures
stop_after_metadata=True can locate the metadata frame for record
times up to ~4 seconds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 02:19:27 -04:00
claude ab2c11e9a9 fix(protocol): refine extra chunk fetching logic for accurate termination response 2026-04-23 20:30:07 -04:00
claude fa887b85d9 fix(protocol): update extra chunk fetching logic to stop at silence detection 2026-04-23 18:28:14 -04:00
claude ecd980d345 fix(protocol): enhance extra chunk fetching logic to ensure footer detection 2026-04-23 18:22:27 -04:00
claude aa2b02535b fix(protocol): add record_time based chunk scaling for longer event record times 2026-04-23 17:33:16 -04:00
claude 2a2031c3a9 fix(protocol): fetch additional chunk after metadata to ensure valid termination response 2026-04-23 17:08:36 -04:00
claude ec6362cb8e fix(protocol): include terminator in waveform stream downloads 2026-04-23 12:45:59 -04:00
claude 3eeafd24aa fix(protocol): improve terminator frame detection in write_blastware_file.
fix: rename .n00 to just blastware file (.n00 was false positive)
2026-04-23 01:33:44 -04:00
claude 6dcca4da79 feat(protocol): fully decode Blastware filename encoding and update related documentation 2026-04-22 23:43:31 -04:00
claude c47e3a3af0 feat(protocol): update Blastware file format documentation and encoding details 2026-04-22 19:16:05 -04:00
claude dfbc9f29c5 feat: first try at building waveform binary files. 2026-04-21 22:57:53 -04:00
claude b3dcfe7239 fix(client): correct recording_mode anchor position in compliance config encoding 2026-04-21 01:17:45 -04:00
claude 9b5cdfd857 feat(logging): add detailed logging for anchor position in compliance config encoding/decoding 2026-04-21 00:23:15 -04:00
claude 7129aae279 fix(client): update compliance data size handling (less strict now) 2026-04-21 00:09:30 -04:00
claude 3fb24e1895 feat(call-home): Implement Auto Call Home configuration management
- Added `CallHomeConfig` model to represent the Auto Call Home settings.
- Introduced methods in `MiniMateClient` for reading (`get_call_home_config`) and writing (`set_call_home_config`) the call home configuration.
- Updated `MiniMateProtocol` with new commands for call home operations (SUB 0x2C for read, SUB 0x7E for write, and SUB 0x7F for confirm).
- Created API endpoints for retrieving and updating call home settings in the server.
- Enhanced the web interface with a new "Call Home" tab for user interaction with call home settings.
- Implemented JavaScript functions for reading and writing call home configurations from the web app.
2026-04-20 18:23:48 -04:00
claude b6ffdcfa87 feat: implement geophone sensitivity and recording mode settings in compliance config 2026-04-20 17:03:58 -04:00
claude eec6c3dc6a feat: add histogram_interval setting and update UI with new field. 2026-04-20 16:25:56 -04:00
claude e04114fd6c feat: mapped record_mode protocol 2026-04-20 15:49:31 -04:00
claude aa28495a43 fix: rename max_geo_range to ADC scale, and make it so its not user configurable.
fix: change max_geo_range_enum to geo_range with two options (normal and sensitive)
2026-04-19 18:15:23 -04:00
claude b23cf4bb50 fix: max_geo_range correctly identified as ADC Scale factor number. 2026-04-17 19:43:45 -04:00
claude 27db663579 fix: update event count retrieval logic in AchSession and MiniMateClient 2026-04-13 18:46:23 -04:00
claude ef2c38e7db v0.10.0 — monitor log entry support (SUB 0x0A partial records)
Add full decode pipeline for 0x2C partial records from the device's event
list, representing continuous monitoring intervals where no threshold was
crossed.  These records appear interleaved with full triggered events in the
browse walk and were previously ignored.

minimateplus/models.py
- Add MonitorLogEntry dataclass: key, start_time, stop_time, serial,
  geo_threshold_ips, raw_header, duration_seconds property

minimateplus/protocol.py
- read_waveform_header() now returns (data_rsp.data, length) — full payload
  including the record-type byte at position 0 — instead of the sliced header.
  Callers that need the old slice use raw_data[11:11+length] as before.

minimateplus/client.py
- Add _decode_0a_partial_header(): auto-detects 9-byte (sub_code=0x10) vs
  10-byte (sub_code=0x03) timestamp format, handles 1-byte inter-timestamp
  gap, extracts serial via BE anchor and geo threshold via Geo: anchor.
- Add get_monitor_log_entries(skip_keys=None): browse walk (1E → 0A → 1F),
  decodes partial records, skips full records and already-seen keys.

minimateplus/__init__.py
- Export MonitorLogEntry

bridges/ach_server.py
- After get_events(), call get_monitor_log_entries(skip_keys=seen_keys) and
  save new entries to monitor_log.json in the session directory.
- Add _monitor_log_entry_to_dict() helper.
- Include monitor log keys in downloaded_keys for state persistence.

CLAUDE.md / CHANGELOG.md
- Document 0x2C partial record layout (timestamp format, ASCII metadata
  region, 1-byte gap edge case) confirmed from 4-11-26 MITM capture.
- Version bump to v0.10.0; update What's next.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 02:59:40 -04:00
claude e712d68505 feat: add erase-all protocol and browse helpers to protocol/client layer
protocol.py:
- SUB_ERASE_ALL_BEGIN = 0xA3, SUB_ERASE_ALL_CONFIRM = 0xA2 (confirmed 4-11-26 MITM)
- SUB_CHANNEL_CONFIG (0x06) data length = 0x24 (36 bytes) in DATA_LENGTHS
- begin_erase_all()              — single frame, token=0xFE, response 0x5C
- confirm_erase_all()            — single frame, token=0xFE, response 0x5D
- read_event_storage_range()     — two-step read (probe+data), token=0xFE
  Response last 8 bytes = first/last stored event key; both 0x01110000 after erase

client.py:
- list_event_keys()              — browse-mode 1E→0A→1F walk, no waveform download;
  returns list of hex key strings; used as fast pre-check before get_events()
- get_events(skip_waveform_for_keys=set())
  — for already-seen keys: only 0A+1F(browse), skips 1E-arm/0C/POLL×3/5A entirely
- delete_all_events()            — orchestrates the confirmed erase sequence:
  0xA3 → 0x1C → 0x06 → 0xA2; logs first/last key from storage range response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:14:37 -04:00
claude 3d2ebfc057 fix: correct event count field offset and eliminate count_events() walk
_decode_event_count: read uint16 BE at offset 10 (confirmed 2026-04-10 from
live BE11529 event index — data[10:12]=0x0006=6, matches device LCD).
Previous uint32 at offset 3 always returned 1 regardless of event count.

ach_server.py: use device_info.event_count (already fetched during connect())
instead of calling count_events() separately. This saves 2*N round-trips and
avoids the 1F linked-list walk which was overcounting on some devices.
count_events() kept as fallback when connect() is skipped (--events-only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:10:49 -04:00
claude 9d9c14af79 fix: replace Unicode chars in log messages, fix DeviceInfo.serial, UTF-8 file log
- Replace all Unicode arrows/checkmarks (->  [OK]  [FAIL]) in ach_server.py
  and client.py log calls — Windows cp1252 console can't encode them
- Fix DeviceInfo attribute: serial_number -> serial
- Fix _device_info_to_dict key: serial_number -> serial
- Demote count_events 1E/1F per-key log lines from WARNING to DEBUG
  (they were flooding the console on devices with many stored events)
- FileHandler now opens with encoding='utf-8' so session log files
  can hold any characters without codec errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:06:27 -04:00
claude 0baf343bf5 feat: add high-water mark state tracking to ach_server + fix monitoring flag
ach_server.py:
- Add ach_state.json per-unit state tracking (keyed by serial number)
- count_events() before any download; skip session if no new events since last call-home
- Download only events beyond the previous high-water mark (all_events[last_count:])
- --max-events N safety cap for first-run units with many stored events
- state_path and max_events wired through AchSession constructor and serve()

client.py (_decode_monitor_status):
- Revert monitoring flag to section[1] == 0x10 (was incorrectly changed to section[6])
- Fix battery/memory offsets to section[-10:-8], [-8:-4], [-4:] (no trailing checksum byte)
- Both confirmed by full byte diff of all 144 0xE3 data frames in 4-8-26/2ndtry capture

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:38:44 -04:00
claude 05421764a5 feat: add SocketTransport and ach_server.py inbound ACH server
minimateplus/transport.py:
- Add SocketTransport(TcpTransport) — wraps an already-accepted inbound
  socket; connect() is a no-op; everything else inherited from TcpTransport.
  Enables the ACH server to reuse all existing protocol/client code without
  any changes.

bridges/ach_server.py:
- Minimal inbound ACH server — listens on port 12345, accepts call-home
  connections from MiniMate Plus units, runs the full BW protocol:
  startup handshake → get_device_info → get_events(full_waveform=True)
- Saves device_info.json + events.json + raw_rx_<ts>.bin + session log
  per connection to bridges/captures/ach_inbound_<ts>/
- raw_rx.bin is byte-compatible with existing Analyzer tooling
- Taps transport.read() to capture raw S3 bytes alongside parsed output
- Each connection runs in its own daemon thread
- Clearly distinguishes push vs pull protocol in the startup log

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:34:27 -04:00
claude 990cb8850e fix: correct monitoring flag and battery/memory offsets in _decode_monitor_status
section[6] is the monitoring flag (was wrongly section[1] — section[1] is always
0x00 in both states). Battery and memory fields use relative-from-end offsets
(section[-11:-9], section[-9:-5], section[-5:-1]) instead of absolute positions,
which broke when the payload grew by 3 bytes in monitoring mode.

Confirmed from full byte diff of 142 0xE3 frames in 4-8-26/2ndtry capture.
SFM start_monitoring now polls /device/monitor/status every 5s for up to 60s
instead of a fixed 25s delay (unit runs ~40s on-device sensor check before
confirming monitoring state).

Also corrects stale 1C→6E response anomaly claim in protocol reference — no
exceptions to the 0xFF−SUB rule are known.
2026-04-08 23:41:11 -04:00
claude dda5683572 fix: improve monitoring functionality with session-reset signal and payload adjustments 2026-04-08 18:29:51 -04:00
claude 16e072698b feat: Implement poll() method for efficient device communication and update monitoring status retrieval 2026-04-08 16:33:21 -04:00
claude a41e7a9e1a feat: Add monitoring functionality to MiniMate protocol and web interface
- Introduced new SUBs for monitoring status, start, and stop commands in protocol.py.
- Implemented read_monitor_status, start_monitoring, and stop_monitoring methods in MiniMateProtocol class.
- Added new API endpoints for monitoring status retrieval and control in server.py.
- Enhanced the web application with a monitoring panel, including battery and memory status display.
- Created a new Python script to parse SUB 0x1C response frames for monitoring status.
- Documented the monitoring status response format and field locations in markdown and text files.
2026-04-08 14:34:42 -04:00
claude a7ab6eaf7c feat: add config API endpoint and JSON schema draft 2026-04-07 17:26:24 -04:00
claude 7005ae766d feat: implement set_project_info functionality and add POC test script 2026-04-07 02:49:17 -04:00
claude bcc044655a feat: updates to 0.8.0 - initial write functions 2026-04-07 02:09:29 -04:00
claude b5828de534 fix: remove hardcoded fi==9 skip in _decode_a5_waveform
Frame index 9 was assumed to be the device terminator based on the
9-frame original blast capture. For streams with >9 frames (current
device produces 35), fi==9 is live waveform data — the skip was
dropping ~133 sample-sets per event.

Terminator detection is handled upstream via page_key==0x0000 in
read_bulk_waveform_stream, so no index-based skip is needed here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 23:34:33 -04:00
claude 781d21f132 perf: reduce 5A chunk timeout to 10s and stop iteration at requested event index
Two improvements to eliminate the ~2-min-per-event wait and unnecessary
full-event-list download when only one event is requested:

1. protocol.py: pass timeout=10.0 to _recv_one in the 5A chunk loop.
   Device responds within ~1s per chunk; 10s gives a safe 10x buffer.
   End-of-stream detection (raw_bytes=1) now fires in 10s instead of 120s,
   cutting ~110s of dead wait per event.

2. client.py: add stop_after_index parameter to get_events(). When set,
   iteration stops immediately after the target event is collected — no
   further 0A/1E/0C/5A/1F cycles for events the caller doesn't need.

3. server.py: pass stop_after_index=index to both /device/event/{idx}
   and /device/event/{idx}/waveform endpoints so a single-event request
   only downloads that one event.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 19:02:01 -04:00
claude e3a5c6f07d 5A: treat partial end-of-stream byte as graceful stream termination
When the device finishes streaming waveform data, it sends a single
partial byte (raw_bytes=1) rather than a complete A5 frame, then goes
silent for the full 120s timeout.

Observed: 35 chunks succeed, chunk 36 times out with raw_bytes=1 —
identical for both events in the test run.

Fix: on TimeoutError, if bytes_fed > 0 and we already have collected
frames, treat it as natural end-of-stream and break to the termination
step rather than propagating the exception.  True transport failures
(raw_bytes=0, no prior frames) still raise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 17:41:29 -04:00