Commit Graph
100 Commits
Author SHA1 Message Date
claude 738b39f3cb Manually Merged seismo lab persistent connection branch into the new direct download branch, creating a new branch called seismo-lab-new 2026-05-01 15:13:50 -04:00
claude 5cd5652560 Merge branch 'seismo-lab' of https://github.com/serversdwn/seismo-relay into seismo-lab 2026-04-26 18:16:52 -04:00
claude a7585cb5e0 fix(blastware_file, server): implement logic to skip extra chunks after metadata for accurate file writing 2026-04-26 16:32:32 -04:00
claude ae30a02898 fix(blastware_file, server): enhance logging and correct chunk handling for accurate data processing 2026-04-26 16:03:07 -04:00
claude 2f084ed105 fix(protocol): update chunk counter formula to use max(key4[2:4], 0x0400) for accurate data streaming 2026-04-26 01:28:47 -04:00
claude 7976b544ed fix(blastware_file): never skip A5 frames based on classification at fi>0
Frame 0 is always the probe; frames 1+ are always data (waveform ADC
chunks, compliance config, compliance continuation).  Gating on
classify_frame() at fi>0 produces false positives: ADC binary data
can coincidentally contain b"STRT\xff\xfe", causing frames 1 and 5
to be silently dropped from the body (confirmed from live capture on
event key=01110000).  Remove all type-based filtering; include every
frame unconditionally with the standard index-based skip amounts.
2026-04-26 00:59:36 -04:00
claude 0415af19b4 fix(blastware_file): remove seen_metadata flag and adjust frame processing logic 2026-04-24 20:21:03 -04:00
claude 35c3f4f945 fix(protocol): correct A5 frame classification and chunk counter formula 2026-04-24 17:25:29 -04:00
claudeandClaude Sonnet 4.6 43c8158493 feat(blastware_file): classify A5 frames, only write waveform frames to body
Add classify_frame() which categorises each A5 frame by content:
  terminator    — page_key == 0x0000
  probe_or_strt — contains b"STRT"
  metadata      — contains compliance-config ASCII markers
                  (Project:, Client:, Standard Recording Setup, …)
  waveform      — binary-heavy (< 20% printable ASCII), i.e. raw ADC data
  unknown       — fallback

Update write_blastware_file() body loop: frame 0 (probe) is still
always processed; frames 1+ are only included when classify_frame
returns "waveform".  Metadata frames (compliance config block with
Project:/Client:/etc.) and any stray STRT-bearing frames are skipped
with a warning/debug log.  Terminator frame handling is unchanged.

Adds temporary print() diagnostics so each frame's classification is
visible in the server log to aid debugging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 15:48:37 -04:00
claude 242666f358 fix(protocol): correct chunk counter formula for accurate data streaming 2026-04-24 12:52:02 -04:00
claudeandClaude Sonnet 4.6 03540fdc00 fix: raise max_chunks to 128 for metadata-only 5A download
For 2-second events at 1024 sps the "Project:" metadata frame appears
beyond chunk 32 (the old default cap), causing the safety limit to be
hit and ~34 KB of waveform data to be downloaded instead of stopping
at the metadata frame.  Raising max_chunks to 128 ensures
stop_after_metadata=True can locate the metadata frame for record
times up to ~4 seconds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 02:19:27 -04:00
claude f83fd880c0 fix(protocol): update device_event_blastware_file to include extra chunk for accurate data retrieval 2026-04-24 00:35:34 -04:00
claude ab2c11e9a9 fix(protocol): refine extra chunk fetching logic for accurate termination response 2026-04-23 20:30:07 -04:00
claude fa887b85d9 fix(protocol): update extra chunk fetching logic to stop at silence detection 2026-04-23 18:28:14 -04:00
claude ecd980d345 fix(protocol): enhance extra chunk fetching logic to ensure footer detection 2026-04-23 18:22:27 -04:00
claude bc9f16e503 fix(protocol): adjust extra_chunks calculation to use integer conversion of record_time 2026-04-23 17:39:28 -04:00
claude aa2b02535b fix(protocol): add record_time based chunk scaling for longer event record times 2026-04-23 17:33:16 -04:00
claude 2a2031c3a9 fix(protocol): fetch additional chunk after metadata to ensure valid termination response 2026-04-23 17:08:36 -04:00
claude 9e7e0bce2a fix(protocol): adjust full_waveform setting for event downloads to end when it should. 2026-04-23 16:43:59 -04:00
claude 5e2f3bf2a1 fix(protocol): enable full_waveform for continuous mode. 2026-04-23 16:24:39 -04:00
claude 39ebd4bdaa fix(protocol): revert endpoint back to stop_after_metadata=True 2026-04-23 15:11:56 -04:00
claude 84c87d0b57 fix(protocol): adjust waveform download to use full_waveform for accurate event streaming 2026-04-23 13:02:55 -04:00
claude ec6362cb8e fix(protocol): include terminator in waveform stream downloads 2026-04-23 12:45:59 -04:00
claude 3eeafd24aa fix(protocol): improve terminator frame detection in write_blastware_file.
fix: rename .n00 to just blastware file (.n00 was false positive)
2026-04-23 01:33:44 -04:00
claude 8cb8b86192 fix(server): add error logging for device event handling 2026-04-22 23:48:59 -04:00
claude 6dcca4da79 feat(protocol): fully decode Blastware filename encoding and update related documentation 2026-04-22 23:43:31 -04:00
claude c47e3a3af0 feat(protocol): update Blastware file format documentation and encoding details 2026-04-22 19:16:05 -04:00
claude dfbc9f29c5 feat: first try at building waveform binary files. 2026-04-21 22:57:53 -04:00
claude 4331215e23 feat(protocol): enhance raw capture functionality and documentation updates
- Update `s3_bridge.py` to default raw capture file paths to "auto" for timestamped naming.
- Modify `gui_bridge.py` to pre-check raw capture options and streamline path handling.
- Extend `ach_server.py` to save both incoming and outgoing raw bytes for analysis.
- Revise `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect changes in recording mode handling and compliance data encoding.
2026-04-21 16:07:24 -04:00
claude b3dcfe7239 fix(client): correct recording_mode anchor position in compliance config encoding 2026-04-21 01:17:45 -04:00
claude 9b5cdfd857 feat(logging): add detailed logging for anchor position in compliance config encoding/decoding 2026-04-21 00:23:15 -04:00
claude 7129aae279 fix(client): update compliance data size handling (less strict now) 2026-04-21 00:09:30 -04:00
claude 2186bc238b fix: call home settings tab display 2026-04-20 21:15:16 -04:00
claude 3fb24e1895 feat(call-home): Implement Auto Call Home configuration management
- Added `CallHomeConfig` model to represent the Auto Call Home settings.
- Introduced methods in `MiniMateClient` for reading (`get_call_home_config`) and writing (`set_call_home_config`) the call home configuration.
- Updated `MiniMateProtocol` with new commands for call home operations (SUB 0x2C for read, SUB 0x7E for write, and SUB 0x7F for confirm).
- Created API endpoints for retrieving and updating call home settings in the server.
- Enhanced the web interface with a new "Call Home" tab for user interaction with call home settings.
- Implemented JavaScript functions for reading and writing call home configurations from the web app.
2026-04-20 18:23:48 -04:00
claude 7bdd7c92f2 Merge branch 'protocol-exp' of https://gitea.serversdown.net/serversdown/seismo-relay into protocol-exp 2026-04-20 17:04:00 -04:00
claude b6ffdcfa87 feat: implement geophone sensitivity and recording mode settings in compliance config 2026-04-20 17:03:58 -04:00
claude eec6c3dc6a feat: add histogram_interval setting and update UI with new field. 2026-04-20 16:25:56 -04:00
claude 702e06873e fix: add recording_mode option in html 2026-04-20 15:56:52 -04:00
claude 94767f5a9d feat: add recording_mode to config editor in sfm webapp 2026-04-20 15:54:08 -04:00
claude e04114fd6c feat: mapped record_mode protocol 2026-04-20 15:49:31 -04:00
claude f10c5c1b86 feat: add persistent bridge and streamlined capture pipeline to seismo_lab.py 2026-04-20 15:09:55 -04:00
claude aa28495a43 fix: rename max_geo_range to ADC scale, and make it so its not user configurable.
fix: change max_geo_range_enum to geo_range with two options (normal and sensitive)
2026-04-19 18:15:23 -04:00
claude b23cf4bb50 fix: max_geo_range correctly identified as ADC Scale factor number. 2026-04-17 19:43:45 -04:00
claude b8ed237363 docs: update to 0.12.1 2026-04-16 18:31:20 -04:00
claude 5866ecdb3e docs: update protocol doc to reflect unkown status of max_range_geo. 2026-04-16 18:17:16 -04:00
claude 71bcf71cf7 fix: convert raw psi 32 float into db(L). 2026-04-16 21:22:04 +00:00
claude 3e7de848bc fix: update unique constraints in events and monitor_log tables to use timestamp and serial number. Can't use event keys because minimates resuse them after clearing memory. 2026-04-16 21:22:04 +00:00
claude 72a4209cfd fix: sfm_webapp.html remove display: flex from base class, now shows active tab 2026-04-16 21:22:04 +00:00
claude 2b5574511e feat: add waveform viewer endpoint and enhance UI with new tabs for history, units, monitor log, and sessions 2026-04-16 21:22:04 +00:00
claude ce2c859f11 fix: update event count retrieval logic in AchSession and MiniMateClient 2026-04-16 21:22:04 +00:00
claude 7f322f9ff9 feat: add option to restart monitoring after event download in AchSession 2026-04-16 21:22:04 +00:00
claude c474db4f69 build: update build backend to setuptools.build_meta 2026-04-16 21:22:04 +00:00
claude 2765ee6ea7 build: add pyproject.toml for editable install 2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 ef88240796 docs: update README to v0.12.0
Rewrites the v0.6.0 README to reflect current project state:
ACH server, SQLite DB, SFM REST API with caching, monitor/erase, updated roadmap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 5591d345d9 feat: v0.12.0 — live device cache (_LiveCache) in sfm/server.py
Ports the intelligent-caching branch concept to a plain Python in-memory
implementation — no SQLAlchemy, no extra DB table, no new dependencies.

_LiveCache (threading.Lock + dicts) caches:
  - device info: indefinite, invalidated by POST /device/config
  - events: keyed by (conn_key, device_event_count); count-probe fast path
    (~2s poll+count_events) avoids full downloads when nothing is new
  - monitor status: 30-second TTL, invalidated by monitor start/stop
  - waveforms: permanent per (conn_key, event_index)

All four cached endpoints accept ?force=true to bypass the cache.
Removes sfm/cache.py (SQLAlchemy experiment, now superseded).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:22:04 +00:00
claudeandClaude Sonnet 4.6 7883a31aa7 v0.11.0 — SQLite persistence layer (SeismoDb)
sfm/database.py (new)
- SeismoDb class: three tables keyed by unit serial number
  - ach_sessions: one row per ACH call-home
  - events: one row per triggered event, deduped by (serial, waveform_key)
  - monitor_log: one row per monitoring interval, deduped by (serial, waveform_key)
- WAL mode, per-request connections, silent dedup via UNIQUE constraint
- Query helpers: query_events(), query_monitor_log(), get_sessions(), query_units()
- false_trigger flag on events for future review UI / report filtering

bridges/ach_server.py
- Import SeismoDb; create shared instance at startup pointed at
  bridges/captures/seismo_relay.db
- After each call-home: insert_events() + insert_monitor_log() + insert_ach_session()
- DB failures logged as warnings, never abort the session

sfm/server.py
- Import SeismoDb; lazy singleton via _get_db()
- New DB read endpoints: GET /db/units, /db/events, /db/monitor_log, /db/sessions
- PATCH /db/events/{id}/false_trigger for manual review flagging

CLAUDE.md / CHANGELOG.md
- Document DB schema, SFM DB endpoints, architecture decision (unit-keyed only)
- Version bump to v0.11.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:19:47 +00:00
claudeandClaude Sonnet 4.6 b241da970d v0.10.0 — monitor log entry support (SUB 0x0A partial records)
Add full decode pipeline for 0x2C partial records from the device's event
list, representing continuous monitoring intervals where no threshold was
crossed.  These records appear interleaved with full triggered events in the
browse walk and were previously ignored.

minimateplus/models.py
- Add MonitorLogEntry dataclass: key, start_time, stop_time, serial,
  geo_threshold_ips, raw_header, duration_seconds property

minimateplus/protocol.py
- read_waveform_header() now returns (data_rsp.data, length) — full payload
  including the record-type byte at position 0 — instead of the sliced header.
  Callers that need the old slice use raw_data[11:11+length] as before.

minimateplus/client.py
- Add _decode_0a_partial_header(): auto-detects 9-byte (sub_code=0x10) vs
  10-byte (sub_code=0x03) timestamp format, handles 1-byte inter-timestamp
  gap, extracts serial via BE anchor and geo threshold via Geo: anchor.
- Add get_monitor_log_entries(skip_keys=None): browse walk (1E → 0A → 1F),
  decodes partial records, skips full records and already-seen keys.

minimateplus/__init__.py
- Export MonitorLogEntry

bridges/ach_server.py
- After get_events(), call get_monitor_log_entries(skip_keys=seen_keys) and
  save new entries to monitor_log.json in the session directory.
- Add _monitor_log_entry_to_dict() helper.
- Include monitor log keys in downloaded_keys for state persistence.

CLAUDE.md / CHANGELOG.md
- Document 0x2C partial record layout (timestamp format, ASCII metadata
  region, 1-byte gap edge case) confirmed from 4-11-26 MITM capture.
- Version bump to v0.10.0; update What's next.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 6acb419ebd docs: update protocol reference with v0.9.0 erase-all protocol
Changelog section:
- 5 new entries (2026-04-11): erase-all confirmation, SUB 0x06 purpose
  resolved, §7.11 added, §14.6 ACH session lifecycle marked IMPLEMENTED

§5.1 Request Commands:
- SUB 0x06 description updated: "EVENT STORAGE RANGE READ" (not "CHANNEL
  CONFIG READ"), token=0xFE, last 8 bytes = first/last stored event keys
- SUB 0xA3 added: ERASE ALL BEGIN — standard build_bw_frame, token=0xFE, ack 0x5C
- SUB 0xA2 added: ERASE ALL CONFIRM — standard build_bw_frame, token=0xFE, ack 0x5D

§5.2 Response SUBs:
- 0x06→0xF9 marked CONFIRMED 2026-04-11
- 0xA3→0x5C and 0xA2→0x5D added with CONFIRMED status

§7.11 (new section): Erase-All Protocol
- Full wire sequence (6 request/response pairs)
- SUB 0x06 storage range payload layout (36 bytes, last 8 = first/last key)
- Post-erase key counter reset: device restarts from 0x01110000
- Implementation notes pointing to client.py and ach_server.py

§14.6 ACH Session Lifecycle:
- Removed "Future" label — fully implemented in bridges/ach_server.py
- Added step 6 (optional erase), step 8 (DCD/DTR auto-resume)
- Documents ach_server.py flags and ach_state.json schema
- Notes RV55 DCD/DTR issue as known open problem

Open Questions table:
- SUB 0x06 purpose RESOLVED
- Erase-all sequence RESOLVED
- ACH server RESOLVED
- Sensor Check byte: still open, added as formal question
- RV55 DCD/DTR: added as new open question

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 f6a0846bab docs: update CHANGELOG and CLAUDE.md for v0.9.0
CHANGELOG.md:
- New v0.9.0 section covering erase-all protocol, browse helpers,
  delete_all_events(), ach_mitm.py, and ACH server overhaul
- Back-filled v0.8.0 section (write pipeline, monitoring, ACH server)
  that was missing from the previous release notes

CLAUDE.md:
- Bump version to v0.9.0
- Add erase-all protocol section with full wire sequence, SUB 0x06
  storage range response layout, and post-erase key counter reset notes
- Document ACH server state format (ach_state.json v0.9.0 schema with
  downloaded_keys + max_downloaded_key)
- Add RV55 DCD/DTR issue to What's next

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 3d9db8b662 feat: add ach_mitm.py — transparent TCP MITM proxy for ACH session capture
Listens for inbound unit connections, connects upstream to a real Blastware
ACH server, and forwards bytes bidirectionally while saving both directions to
raw_bw_<ts>.bin and raw_s3_<ts>.bin in the existing capture format.

Used to capture the 4-11-26 Blastware ACH session that confirmed the erase-all
protocol (SUBs 0xA3/0x1C/0x06/0xA2) and the event deletion wire sequence.

Usage:
  python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998
  Point the unit's call-home destination at this machine:9998.
  Point this proxy's --bw-host/port at the upstream Blastware ACH server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 c7e7d177e6 feat: overhaul ACH server with key-based state, erase support, and reset detection
State format (ach_state.json):
- Replace event_count with downloaded_keys (set of hex strings) + max_downloaded_key
- Key-based tracking correctly handles delete-then-re-record: after device erase the
  count drops to 0, but new events have new (or recycled) keys

Browse pre-check:
- list_event_keys() walk before get_events() to bail early when nothing is new
- get_events() called with skip_waveform_for_keys= for already-seen keys, so repeat
  call-homes only download waveforms for genuinely new events

--clear-after-download flag:
- After saving new events, calls client.delete_all_events() (0xA3→0x1C→0x06→0xA2)
- On success: resets downloaded_keys=[] and max_downloaded_key="00000000" so the
  next session starts fresh (device counter resets to 0x01110000 after erase)

Post-erase key-reuse detection:
- Device counter resets to 0x01110000 after any erase; new events reuse old keys
- If max(device_keys) < max_downloaded_key, the device was wiped externally
  (Blastware, manual) — seen_keys is discarded and all device keys treated as new

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 a3b8d10fa8 feat: add erase-all protocol and browse helpers to protocol/client layer
protocol.py:
- SUB_ERASE_ALL_BEGIN = 0xA3, SUB_ERASE_ALL_CONFIRM = 0xA2 (confirmed 4-11-26 MITM)
- SUB_CHANNEL_CONFIG (0x06) data length = 0x24 (36 bytes) in DATA_LENGTHS
- begin_erase_all()              — single frame, token=0xFE, response 0x5C
- confirm_erase_all()            — single frame, token=0xFE, response 0x5D
- read_event_storage_range()     — two-step read (probe+data), token=0xFE
  Response last 8 bytes = first/last stored event key; both 0x01110000 after erase

client.py:
- list_event_keys()              — browse-mode 1E→0A→1F walk, no waveform download;
  returns list of hex key strings; used as fast pre-check before get_events()
- get_events(skip_waveform_for_keys=set())
  — for already-seen keys: only 0A+1F(browse), skips 1E-arm/0C/POLL×3/5A entirely
- delete_all_events()            — orchestrates the confirmed erase sequence:
  0xA3 → 0x1C → 0x06 → 0xA2; logs first/last key from storage range response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 4921b0489a fix: correct Event and PeakValues field names in ach_server serialization
Event model uses peak_values (not peaks) and project_info (not direct fields).
PeakValues fields are tran/vert/long/micl/peak_vector_sum (not transverse etc).
ProjectInfo fields accessed via ev.project_info.project etc.

Also fix ev.timestamp serialization: use str() instead of .isoformat() since
Timestamp is a custom dataclass, not datetime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 8688d815a0 fix: remove non-existent DeviceInfo fields from ach_server log and dict
calibration_date, aux_trigger, setup_name etc. don't exist directly on
DeviceInfo — they live in DeviceInfo.compliance_config (ComplianceConfig).
_device_info_to_dict now accesses them via cc = d.compliance_config.
Log line updated to show serial/firmware/model/event_count instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 9b50ec9133 fix: make Ctrl-C work on Windows by setting accept() timeout
socket.accept() on Windows blocks indefinitely and ignores KeyboardInterrupt.
Setting a 1-second timeout on the server socket causes the accept loop to wake
up every second and re-check, so Ctrl-C is handled within ~1 second.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 cba8b1b401 feat: defer session dir creation and add --allow-ip allowlist
- Session directory and log file are now created ONLY after startup() succeeds.
  Internet scanners and dropped connections no longer litter the output folder.
  Raw bytes are buffered in memory until startup succeeds, then flushed to disk.

- Add --allow-ip IP flag (repeatable) to allowlist specific source IPs.
  Connections from un-listed IPs are rejected immediately (socket closed, no log).
  If no --allow-ip flags are given, all IPs are still accepted (original behavior).
  Usage: --allow-ip 63.43.212.232 --allow-ip 152.1.2.3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 41a14ca468 fix: correct event count field offset and eliminate count_events() walk
_decode_event_count: read uint16 BE at offset 10 (confirmed 2026-04-10 from
live BE11529 event index — data[10:12]=0x0006=6, matches device LCD).
Previous uint32 at offset 3 always returned 1 regardless of event count.

ach_server.py: use device_info.event_count (already fetched during connect())
instead of calling count_events() separately. This saves 2*N round-trips and
avoids the 1F linked-list walk which was overcounting on some devices.
count_events() kept as fallback when connect() is skipped (--events-only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 1bfc6e4258 fix: replace Unicode chars in log messages, fix DeviceInfo.serial, UTF-8 file log
- Replace all Unicode arrows/checkmarks (->  [OK]  [FAIL]) in ach_server.py
  and client.py log calls — Windows cp1252 console can't encode them
- Fix DeviceInfo attribute: serial_number -> serial
- Fix _device_info_to_dict key: serial_number -> serial
- Demote count_events 1E/1F per-key log lines from WARNING to DEBUG
  (they were flooding the console on devices with many stored events)
- FileHandler now opens with encoding='utf-8' so session log files
  can hold any characters without codec errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claude 574d40027f feat: enhance logging messages in ach_server.py and add experiments.py for protocol minimization 2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 0358acb51d feat: add high-water mark state tracking to ach_server + fix monitoring flag
ach_server.py:
- Add ach_state.json per-unit state tracking (keyed by serial number)
- count_events() before any download; skip session if no new events since last call-home
- Download only events beyond the previous high-water mark (all_events[last_count:])
- --max-events N safety cap for first-run units with many stored events
- state_path and max_events wired through AchSession constructor and serve()

client.py (_decode_monitor_status):
- Revert monitoring flag to section[1] == 0x10 (was incorrectly changed to section[6])
- Fix battery/memory offsets to section[-10:-8], [-8:-4], [-4:] (no trailing checksum byte)
- Both confirmed by full byte diff of all 144 0xE3 data frames in 4-8-26/2ndtry capture

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 cf7d838bf4 feat: add SocketTransport and ach_server.py inbound ACH server
minimateplus/transport.py:
- Add SocketTransport(TcpTransport) — wraps an already-accepted inbound
  socket; connect() is a no-op; everything else inherited from TcpTransport.
  Enables the ACH server to reuse all existing protocol/client code without
  any changes.

bridges/ach_server.py:
- Minimal inbound ACH server — listens on port 12345, accepts call-home
  connections from MiniMate Plus units, runs the full BW protocol:
  startup handshake → get_device_info → get_events(full_waveform=True)
- Saves device_info.json + events.json + raw_rx_<ts>.bin + session log
  per connection to bridges/captures/ach_inbound_<ts>/
- raw_rx.bin is byte-compatible with existing Analyzer tooling
- Taps transport.read() to capture raw S3 bytes alongside parsed output
- Each connection runs in its own daemon thread
- Clearly distinguishes push vs pull protocol in the startup log

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 5e44cdc668 feat: add splitter mode to ach_bridge.py (--mirror HOST:PORT)
Adds a production-safe headphone-splitter mode:
- Device bytes tee'd to both --upstream (primary/prod) and --mirror (new server)
- Only primary server responses are returned to the device
- Mirror connect/write failures are non-fatal and logged; prod is unaffected
- New raw_mirror_<ts>.bin capture file alongside raw_client/raw_server

Three modes: standalone (capture only), bridge (one upstream), splitter (two).
Default listen port changed to 12345 to match project ACH setup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claudeandClaude Sonnet 4.6 37d32077a4 feat: add ACH TCP bridge, serial tap tool, and Serial Watch tab
- bridges/ach_bridge.py: transparent TCP bridge that MITMs the MiniMate Plus
  call-home connection — forwards to real ACH server while logging all frames
  to raw_client/raw_server .bin files compatible with parse_capture.py;
  standalone capture mode for lab use without a real server

- bridges/serial_watch.py: RS-232 serial monitor with live S3 frame parsing;
  taps the line between MiniMate and modem (RV50/RV55); captures raw bytes,
  .log and .jsonl; --ack-ok mode auto-replies to AT commands; fixed fatal
  indentation bug in the original that silently prevented any data capture

- seismo_lab.py: new "Serial Watch" fourth tab (SerialWatchPanel) wrapping
  serial_watch.py functionality; COM port picker with refresh, baud config,
  ack-ok toggle, colour-coded live frame log (teal frames / yellow ctrl /
  blue AT), raw .bin capture auto-fed into Analyzer tab on stop

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claude b384ba66d1 fix: convert raw psi 32 float into db(L). 2026-04-14 01:13:21 -04:00
claude 27d9823cc1 fix: update unique constraints in events and monitor_log tables to use timestamp and serial number. Can't use event keys because minimates resuse them after clearing memory. 2026-04-13 22:45:58 -04:00
claude 70c9528611 fix: sfm_webapp.html remove display: flex from base class, now shows active tab 2026-04-13 22:40:40 -04:00
claude e8bef1ac7c feat: add waveform viewer endpoint and enhance UI with new tabs for history, units, monitor log, and sessions 2026-04-13 22:34:28 -04:00
claude 27db663579 fix: update event count retrieval logic in AchSession and MiniMateClient 2026-04-13 18:46:23 -04:00
claude e5ea17388a feat: add option to restart monitoring after event download in AchSession 2026-04-13 18:23:27 -04:00
claude 4ec2f33308 build: update build backend to setuptools.build_meta 2026-04-13 17:56:15 -04:00
claude 6282eacf8b build: add pyproject.toml for editable install 2026-04-13 17:34:58 -04:00
claudeandClaude Sonnet 4.6 034b3f044d docs: update README to v0.12.0
Rewrites the v0.6.0 README to reflect current project state:
ACH server, SQLite DB, SFM REST API with caching, monitor/erase, updated roadmap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 16:12:07 -04:00
claudeandClaude Sonnet 4.6 48d7e94c02 feat: v0.12.0 — live device cache (_LiveCache) in sfm/server.py
Ports the intelligent-caching branch concept to a plain Python in-memory
implementation — no SQLAlchemy, no extra DB table, no new dependencies.

_LiveCache (threading.Lock + dicts) caches:
  - device info: indefinite, invalidated by POST /device/config
  - events: keyed by (conn_key, device_event_count); count-probe fast path
    (~2s poll+count_events) avoids full downloads when nothing is new
  - monitor status: 30-second TTL, invalidated by monitor start/stop
  - waveforms: permanent per (conn_key, event_index)

All four cached endpoints accept ?force=true to bypass the cache.
Removes sfm/cache.py (SQLAlchemy experiment, now superseded).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 15:57:02 -04:00
claudeandClaude Sonnet 4.6 03d224ccc3 v0.11.0 — SQLite persistence layer (SeismoDb)
sfm/database.py (new)
- SeismoDb class: three tables keyed by unit serial number
  - ach_sessions: one row per ACH call-home
  - events: one row per triggered event, deduped by (serial, waveform_key)
  - monitor_log: one row per monitoring interval, deduped by (serial, waveform_key)
- WAL mode, per-request connections, silent dedup via UNIQUE constraint
- Query helpers: query_events(), query_monitor_log(), get_sessions(), query_units()
- false_trigger flag on events for future review UI / report filtering

bridges/ach_server.py
- Import SeismoDb; create shared instance at startup pointed at
  bridges/captures/seismo_relay.db
- After each call-home: insert_events() + insert_monitor_log() + insert_ach_session()
- DB failures logged as warnings, never abort the session

sfm/server.py
- Import SeismoDb; lazy singleton via _get_db()
- New DB read endpoints: GET /db/units, /db/events, /db/monitor_log, /db/sessions
- PATCH /db/events/{id}/false_trigger for manual review flagging

CLAUDE.md / CHANGELOG.md
- Document DB schema, SFM DB endpoints, architecture decision (unit-keyed only)
- Version bump to v0.11.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 00:45:38 -04:00
claudeandClaude Sonnet 4.6 ef2c38e7db v0.10.0 — monitor log entry support (SUB 0x0A partial records)
Add full decode pipeline for 0x2C partial records from the device's event
list, representing continuous monitoring intervals where no threshold was
crossed.  These records appear interleaved with full triggered events in the
browse walk and were previously ignored.

minimateplus/models.py
- Add MonitorLogEntry dataclass: key, start_time, stop_time, serial,
  geo_threshold_ips, raw_header, duration_seconds property

minimateplus/protocol.py
- read_waveform_header() now returns (data_rsp.data, length) — full payload
  including the record-type byte at position 0 — instead of the sliced header.
  Callers that need the old slice use raw_data[11:11+length] as before.

minimateplus/client.py
- Add _decode_0a_partial_header(): auto-detects 9-byte (sub_code=0x10) vs
  10-byte (sub_code=0x03) timestamp format, handles 1-byte inter-timestamp
  gap, extracts serial via BE anchor and geo threshold via Geo: anchor.
- Add get_monitor_log_entries(skip_keys=None): browse walk (1E → 0A → 1F),
  decodes partial records, skips full records and already-seen keys.

minimateplus/__init__.py
- Export MonitorLogEntry

bridges/ach_server.py
- After get_events(), call get_monitor_log_entries(skip_keys=seen_keys) and
  save new entries to monitor_log.json in the session directory.
- Add _monitor_log_entry_to_dict() helper.
- Include monitor log keys in downloaded_keys for state persistence.

CLAUDE.md / CHANGELOG.md
- Document 0x2C partial record layout (timestamp format, ASCII metadata
  region, 1-byte gap edge case) confirmed from 4-11-26 MITM capture.
- Version bump to v0.10.0; update What's next.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 02:59:40 -04:00
claudeandClaude Sonnet 4.6 b9a8e50b3c docs: update protocol reference with v0.9.0 erase-all protocol
Changelog section:
- 5 new entries (2026-04-11): erase-all confirmation, SUB 0x06 purpose
  resolved, §7.11 added, §14.6 ACH session lifecycle marked IMPLEMENTED

§5.1 Request Commands:
- SUB 0x06 description updated: "EVENT STORAGE RANGE READ" (not "CHANNEL
  CONFIG READ"), token=0xFE, last 8 bytes = first/last stored event keys
- SUB 0xA3 added: ERASE ALL BEGIN — standard build_bw_frame, token=0xFE, ack 0x5C
- SUB 0xA2 added: ERASE ALL CONFIRM — standard build_bw_frame, token=0xFE, ack 0x5D

§5.2 Response SUBs:
- 0x06→0xF9 marked CONFIRMED 2026-04-11
- 0xA3→0x5C and 0xA2→0x5D added with CONFIRMED status

§7.11 (new section): Erase-All Protocol
- Full wire sequence (6 request/response pairs)
- SUB 0x06 storage range payload layout (36 bytes, last 8 = first/last key)
- Post-erase key counter reset: device restarts from 0x01110000
- Implementation notes pointing to client.py and ach_server.py

§14.6 ACH Session Lifecycle:
- Removed "Future" label — fully implemented in bridges/ach_server.py
- Added step 6 (optional erase), step 8 (DCD/DTR auto-resume)
- Documents ach_server.py flags and ach_state.json schema
- Notes RV55 DCD/DTR issue as known open problem

Open Questions table:
- SUB 0x06 purpose RESOLVED
- Erase-all sequence RESOLVED
- ACH server RESOLVED
- Sensor Check byte: still open, added as formal question
- RV55 DCD/DTR: added as new open question

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:20:43 -04:00
claudeandClaude Sonnet 4.6 77d9c17680 docs: update CHANGELOG and CLAUDE.md for v0.9.0
CHANGELOG.md:
- New v0.9.0 section covering erase-all protocol, browse helpers,
  delete_all_events(), ach_mitm.py, and ACH server overhaul
- Back-filled v0.8.0 section (write pipeline, monitoring, ACH server)
  that was missing from the previous release notes

CLAUDE.md:
- Bump version to v0.9.0
- Add erase-all protocol section with full wire sequence, SUB 0x06
  storage range response layout, and post-erase key counter reset notes
- Document ACH server state format (ach_state.json v0.9.0 schema with
  downloaded_keys + max_downloaded_key)
- Add RV55 DCD/DTR issue to What's next

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:15:11 -04:00
claudeandClaude Sonnet 4.6 8a1bd34551 feat: add ach_mitm.py — transparent TCP MITM proxy for ACH session capture
Listens for inbound unit connections, connects upstream to a real Blastware
ACH server, and forwards bytes bidirectionally while saving both directions to
raw_bw_<ts>.bin and raw_s3_<ts>.bin in the existing capture format.

Used to capture the 4-11-26 Blastware ACH session that confirmed the erase-all
protocol (SUBs 0xA3/0x1C/0x06/0xA2) and the event deletion wire sequence.

Usage:
  python bridges/ach_mitm.py --bw-host 127.0.0.1 --bw-port 9999 --listen-port 9998
  Point the unit's call-home destination at this machine:9998.
  Point this proxy's --bw-host/port at the upstream Blastware ACH server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:15:02 -04:00
claudeandClaude Sonnet 4.6 09788b931a feat: overhaul ACH server with key-based state, erase support, and reset detection
State format (ach_state.json):
- Replace event_count with downloaded_keys (set of hex strings) + max_downloaded_key
- Key-based tracking correctly handles delete-then-re-record: after device erase the
  count drops to 0, but new events have new (or recycled) keys

Browse pre-check:
- list_event_keys() walk before get_events() to bail early when nothing is new
- get_events() called with skip_waveform_for_keys= for already-seen keys, so repeat
  call-homes only download waveforms for genuinely new events

--clear-after-download flag:
- After saving new events, calls client.delete_all_events() (0xA3→0x1C→0x06→0xA2)
- On success: resets downloaded_keys=[] and max_downloaded_key="00000000" so the
  next session starts fresh (device counter resets to 0x01110000 after erase)

Post-erase key-reuse detection:
- Device counter resets to 0x01110000 after any erase; new events reuse old keys
- If max(device_keys) < max_downloaded_key, the device was wiped externally
  (Blastware, manual) — seen_keys is discarded and all device keys treated as new

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:14:50 -04:00
claudeandClaude Sonnet 4.6 e712d68505 feat: add erase-all protocol and browse helpers to protocol/client layer
protocol.py:
- SUB_ERASE_ALL_BEGIN = 0xA3, SUB_ERASE_ALL_CONFIRM = 0xA2 (confirmed 4-11-26 MITM)
- SUB_CHANNEL_CONFIG (0x06) data length = 0x24 (36 bytes) in DATA_LENGTHS
- begin_erase_all()              — single frame, token=0xFE, response 0x5C
- confirm_erase_all()            — single frame, token=0xFE, response 0x5D
- read_event_storage_range()     — two-step read (probe+data), token=0xFE
  Response last 8 bytes = first/last stored event key; both 0x01110000 after erase

client.py:
- list_event_keys()              — browse-mode 1E→0A→1F walk, no waveform download;
  returns list of hex key strings; used as fast pre-check before get_events()
- get_events(skip_waveform_for_keys=set())
  — for already-seen keys: only 0A+1F(browse), skips 1E-arm/0C/POLL×3/5A entirely
- delete_all_events()            — orchestrates the confirmed erase sequence:
  0xA3 → 0x1C → 0x06 → 0xA2; logs first/last key from storage range response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 01:14:37 -04:00
claudeandClaude Sonnet 4.6 8f5da918b5 fix: correct Event and PeakValues field names in ach_server serialization
Event model uses peak_values (not peaks) and project_info (not direct fields).
PeakValues fields are tran/vert/long/micl/peak_vector_sum (not transverse etc).
ProjectInfo fields accessed via ev.project_info.project etc.

Also fix ev.timestamp serialization: use str() instead of .isoformat() since
Timestamp is a custom dataclass, not datetime.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 02:09:57 -04:00
claudeandClaude Sonnet 4.6 a03c77af09 fix: remove non-existent DeviceInfo fields from ach_server log and dict
calibration_date, aux_trigger, setup_name etc. don't exist directly on
DeviceInfo — they live in DeviceInfo.compliance_config (ComplianceConfig).
_device_info_to_dict now accesses them via cc = d.compliance_config.
Log line updated to show serial/firmware/model/event_count instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:43:02 -04:00
claudeandClaude Sonnet 4.6 87fa9c954f fix: make Ctrl-C work on Windows by setting accept() timeout
socket.accept() on Windows blocks indefinitely and ignores KeyboardInterrupt.
Setting a 1-second timeout on the server socket causes the accept loop to wake
up every second and re-check, so Ctrl-C is handled within ~1 second.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:19:36 -04:00
claudeandClaude Sonnet 4.6 3f7b5c07b5 feat: defer session dir creation and add --allow-ip allowlist
- Session directory and log file are now created ONLY after startup() succeeds.
  Internet scanners and dropped connections no longer litter the output folder.
  Raw bytes are buffered in memory until startup succeeds, then flushed to disk.

- Add --allow-ip IP flag (repeatable) to allowlist specific source IPs.
  Connections from un-listed IPs are rejected immediately (socket closed, no log).
  If no --allow-ip flags are given, all IPs are still accepted (original behavior).
  Usage: --allow-ip 63.43.212.232 --allow-ip 152.1.2.3

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:17:30 -04:00
claudeandClaude Sonnet 4.6 3d2ebfc057 fix: correct event count field offset and eliminate count_events() walk
_decode_event_count: read uint16 BE at offset 10 (confirmed 2026-04-10 from
live BE11529 event index — data[10:12]=0x0006=6, matches device LCD).
Previous uint32 at offset 3 always returned 1 regardless of event count.

ach_server.py: use device_info.event_count (already fetched during connect())
instead of calling count_events() separately. This saves 2*N round-trips and
avoids the 1F linked-list walk which was overcounting on some devices.
count_events() kept as fallback when connect() is skipped (--events-only).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:10:49 -04:00
claudeandClaude Sonnet 4.6 9d9c14af79 fix: replace Unicode chars in log messages, fix DeviceInfo.serial, UTF-8 file log
- Replace all Unicode arrows/checkmarks (->  [OK]  [FAIL]) in ach_server.py
  and client.py log calls — Windows cp1252 console can't encode them
- Fix DeviceInfo attribute: serial_number -> serial
- Fix _device_info_to_dict key: serial_number -> serial
- Demote count_events 1E/1F per-key log lines from WARNING to DEBUG
  (they were flooding the console on devices with many stored events)
- FileHandler now opens with encoding='utf-8' so session log files
  can hold any characters without codec errors

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:06:27 -04:00
claude ab14328c8b feat: enhance logging messages in ach_server.py and add experiments.py for protocol minimization 2026-04-10 00:58:54 -04:00
claudeandClaude Sonnet 4.6 0baf343bf5 feat: add high-water mark state tracking to ach_server + fix monitoring flag
ach_server.py:
- Add ach_state.json per-unit state tracking (keyed by serial number)
- count_events() before any download; skip session if no new events since last call-home
- Download only events beyond the previous high-water mark (all_events[last_count:])
- --max-events N safety cap for first-run units with many stored events
- state_path and max_events wired through AchSession constructor and serve()

client.py (_decode_monitor_status):
- Revert monitoring flag to section[1] == 0x10 (was incorrectly changed to section[6])
- Fix battery/memory offsets to section[-10:-8], [-8:-4], [-4:] (no trailing checksum byte)
- Both confirmed by full byte diff of all 144 0xE3 data frames in 4-8-26/2ndtry capture

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:38:44 -04:00
claudeandClaude Sonnet 4.6 05421764a5 feat: add SocketTransport and ach_server.py inbound ACH server
minimateplus/transport.py:
- Add SocketTransport(TcpTransport) — wraps an already-accepted inbound
  socket; connect() is a no-op; everything else inherited from TcpTransport.
  Enables the ACH server to reuse all existing protocol/client code without
  any changes.

bridges/ach_server.py:
- Minimal inbound ACH server — listens on port 12345, accepts call-home
  connections from MiniMate Plus units, runs the full BW protocol:
  startup handshake → get_device_info → get_events(full_waveform=True)
- Saves device_info.json + events.json + raw_rx_<ts>.bin + session log
  per connection to bridges/captures/ach_inbound_<ts>/
- raw_rx.bin is byte-compatible with existing Analyzer tooling
- Taps transport.read() to capture raw S3 bytes alongside parsed output
- Each connection runs in its own daemon thread
- Clearly distinguishes push vs pull protocol in the startup log

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:34:27 -04:00
claudeandClaude Sonnet 4.6 74233d7e31 feat: add splitter mode to ach_bridge.py (--mirror HOST:PORT)
Adds a production-safe headphone-splitter mode:
- Device bytes tee'd to both --upstream (primary/prod) and --mirror (new server)
- Only primary server responses are returned to the device
- Mirror connect/write failures are non-fatal and logged; prod is unaffected
- New raw_mirror_<ts>.bin capture file alongside raw_client/raw_server

Three modes: standalone (capture only), bridge (one upstream), splitter (two).
Default listen port changed to 12345 to match project ACH setup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:17:57 -04:00