Commit Graph

51 Commits

Author SHA1 Message Date
claude 45e61fbcaf big refactor of waveform protocol. 2026-05-03 01:20:21 -04:00
claude a7585cb5e0 fix(blastware_file, server): implement logic to skip extra chunks after metadata for accurate file writing 2026-04-26 16:32:32 -04:00
claude ae30a02898 fix(blastware_file, server): enhance logging and correct chunk handling for accurate data processing 2026-04-26 16:03:07 -04:00
claude f83fd880c0 fix(protocol): update device_event_blastware_file to include extra chunk for accurate data retrieval 2026-04-24 00:35:34 -04:00
claude ab2c11e9a9 fix(protocol): refine extra chunk fetching logic for accurate termination response 2026-04-23 20:30:07 -04:00
claude fa887b85d9 fix(protocol): update extra chunk fetching logic to stop at silence detection 2026-04-23 18:28:14 -04:00
claude ecd980d345 fix(protocol): enhance extra chunk fetching logic to ensure footer detection 2026-04-23 18:22:27 -04:00
claude bc9f16e503 fix(protocol): adjust extra_chunks calculation to use integer conversion of record_time 2026-04-23 17:39:28 -04:00
claude aa2b02535b fix(protocol): add record_time based chunk scaling for longer event record times 2026-04-23 17:33:16 -04:00
claude 9e7e0bce2a fix(protocol): adjust full_waveform setting for event downloads to end when it should. 2026-04-23 16:43:59 -04:00
claude 5e2f3bf2a1 fix(protocol): enable full_waveform for continuous mode. 2026-04-23 16:24:39 -04:00
claude 39ebd4bdaa fix(protocol): revert endpoint back to stop_after_metadata=True 2026-04-23 15:11:56 -04:00
claude 84c87d0b57 fix(protocol): adjust waveform download to use full_waveform for accurate event streaming 2026-04-23 13:02:55 -04:00
claude 3eeafd24aa fix(protocol): improve terminator frame detection in write_blastware_file.
fix: rename .n00 to just blastware file (.n00 was false positive)
2026-04-23 01:33:44 -04:00
claude 8cb8b86192 fix(server): add error logging for device event handling 2026-04-22 23:48:59 -04:00
claude 6dcca4da79 feat(protocol): fully decode Blastware filename encoding and update related documentation 2026-04-22 23:43:31 -04:00
claude c47e3a3af0 feat(protocol): update Blastware file format documentation and encoding details 2026-04-22 19:16:05 -04:00
claude dfbc9f29c5 feat: first try at building waveform binary files. 2026-04-21 22:57:53 -04:00
claude 3fb24e1895 feat(call-home): Implement Auto Call Home configuration management
- Added `CallHomeConfig` model to represent the Auto Call Home settings.
- Introduced methods in `MiniMateClient` for reading (`get_call_home_config`) and writing (`set_call_home_config`) the call home configuration.
- Updated `MiniMateProtocol` with new commands for call home operations (SUB 0x2C for read, SUB 0x7E for write, and SUB 0x7F for confirm).
- Created API endpoints for retrieving and updating call home settings in the server.
- Enhanced the web interface with a new "Call Home" tab for user interaction with call home settings.
- Implemented JavaScript functions for reading and writing call home configurations from the web app.
2026-04-20 18:23:48 -04:00
claude b6ffdcfa87 feat: implement geophone sensitivity and recording mode settings in compliance config 2026-04-20 17:03:58 -04:00
claude eec6c3dc6a feat: add histogram_interval setting and update UI with new field. 2026-04-20 16:25:56 -04:00
claude 94767f5a9d feat: add recording_mode to config editor in sfm webapp 2026-04-20 15:54:08 -04:00
claude aa28495a43 fix: rename max_geo_range to ADC scale, and make it so its not user configurable.
fix: change max_geo_range_enum to geo_range with two options (normal and sensitive)
2026-04-19 18:15:23 -04:00
claude b23cf4bb50 fix: max_geo_range correctly identified as ADC Scale factor number. 2026-04-17 19:43:45 -04:00
claude 2b5574511e feat: add waveform viewer endpoint and enhance UI with new tabs for history, units, monitor log, and sessions 2026-04-16 21:22:04 +00:00
claude 5591d345d9 feat: v0.12.0 — live device cache (_LiveCache) in sfm/server.py
Ports the intelligent-caching branch concept to a plain Python in-memory
implementation — no SQLAlchemy, no extra DB table, no new dependencies.

_LiveCache (threading.Lock + dicts) caches:
  - device info: indefinite, invalidated by POST /device/config
  - events: keyed by (conn_key, device_event_count); count-probe fast path
    (~2s poll+count_events) avoids full downloads when nothing is new
  - monitor status: 30-second TTL, invalidated by monitor start/stop
  - waveforms: permanent per (conn_key, event_index)

All four cached endpoints accept ?force=true to bypass the cache.
Removes sfm/cache.py (SQLAlchemy experiment, now superseded).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:22:04 +00:00
claude 7883a31aa7 v0.11.0 — SQLite persistence layer (SeismoDb)
sfm/database.py (new)
- SeismoDb class: three tables keyed by unit serial number
  - ach_sessions: one row per ACH call-home
  - events: one row per triggered event, deduped by (serial, waveform_key)
  - monitor_log: one row per monitoring interval, deduped by (serial, waveform_key)
- WAL mode, per-request connections, silent dedup via UNIQUE constraint
- Query helpers: query_events(), query_monitor_log(), get_sessions(), query_units()
- false_trigger flag on events for future review UI / report filtering

bridges/ach_server.py
- Import SeismoDb; create shared instance at startup pointed at
  bridges/captures/seismo_relay.db
- After each call-home: insert_events() + insert_monitor_log() + insert_ach_session()
- DB failures logged as warnings, never abort the session

sfm/server.py
- Import SeismoDb; lazy singleton via _get_db()
- New DB read endpoints: GET /db/units, /db/events, /db/monitor_log, /db/sessions
- PATCH /db/events/{id}/false_trigger for manual review flagging

CLAUDE.md / CHANGELOG.md
- Document DB schema, SFM DB endpoints, architecture decision (unit-keyed only)
- Version bump to v0.11.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:19:47 +00:00
serversdown 2db565ff9c Add intelligent caching layer for SFM device data
Introduces sfm/cache.py — a SQLite-backed cache (via SQLAlchemy) that
sits between the SFM REST endpoints and the device, eliminating redundant
cellular downloads for data that doesn't change.

Cache behaviour by data type:
- Device info / compliance config: cached until a config write occurs;
  POST /device/config now calls mark_config_dirty() to force a fresh read
  on the next /device/info call.
- Event headers + peak values: cached permanently (append-only). On
  subsequent calls to /device/events, the server does a fast count_events()
  (~2s) instead of a full download (~10-30s); only new events are fetched
  from the device and merged into the cache.
- Full waveforms (raw ADC samples): cached permanently — immutable once
  recorded. Repeated requests for the same waveform return instantly with
  zero device contact.
- Monitor status (battery, memory, is_monitoring): 30-second TTL; auto-
  invalidated on start/stop monitoring commands.

All endpoints gain a ?force=true param to bypass the cache when needed.
New endpoints: GET /cache/stats, DELETE /cache/device.
Adds requirements.txt listing fastapi, uvicorn, sqlalchemy, pyserial.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 07:14:51 +00:00
claude 16e072698b feat: Implement poll() method for efficient device communication and update monitoring status retrieval 2026-04-08 16:33:21 -04:00
claude c8c57e950c fix: replace helper in server.py with correct name. 2026-04-08 16:16:47 -04:00
claude a41e7a9e1a feat: Add monitoring functionality to MiniMate protocol and web interface
- Introduced new SUBs for monitoring status, start, and stop commands in protocol.py.
- Implemented read_monitor_status, start_monitoring, and stop_monitoring methods in MiniMateProtocol class.
- Added new API endpoints for monitoring status retrieval and control in server.py.
- Enhanced the web application with a monitoring panel, including battery and memory status display.
- Created a new Python script to parse SUB 0x1C response frames for monitoring status.
- Documented the monitoring status response format and field locations in markdown and text files.
2026-04-08 14:34:42 -04:00
claude 1a9dcc04b4 feat: add webapp 2026-04-07 19:33:29 -04:00
claude a7ab6eaf7c feat: add config API endpoint and JSON schema draft 2026-04-07 17:26:24 -04:00
claude 7005ae766d feat: implement set_project_info functionality and add POC test script 2026-04-07 02:49:17 -04:00
claude 781d21f132 perf: reduce 5A chunk timeout to 10s and stop iteration at requested event index
Two improvements to eliminate the ~2-min-per-event wait and unnecessary
full-event-list download when only one event is requested:

1. protocol.py: pass timeout=10.0 to _recv_one in the 5A chunk loop.
   Device responds within ~1s per chunk; 10s gives a safe 10x buffer.
   End-of-stream detection (raw_bytes=1) now fires in 10s instead of 120s,
   cutting ~110s of dead wait per event.

2. client.py: add stop_after_index parameter to get_events(). When set,
   iteration stops immediately after the target event is collected — no
   further 0A/1E/0C/5A/1F cycles for events the caller doesn't need.

3. server.py: pass stop_after_index=index to both /device/event/{idx}
   and /device/event/{idx}/waveform endpoints so a single-event request
   only downloads that one event.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 19:02:01 -04:00
claude ecb1147216 fix: update Peak Vector Sum offset calculation and clarify event_count handling in device info 2026-04-05 02:48:58 -04:00
claude 2cb95cd45e feat: implement reliable event counting via 1E/1F chain and update device info 2026-04-03 16:02:10 -04:00
claude f495b91d8a feat: enhance waveform viewer with record type handling and improved empty state messaging 2026-04-03 15:22:26 -04:00
claude e4730376ad feat: enhance waveform viewer with unit info display and event selection functionality 2026-04-03 15:08:57 -04:00
claude 23e4febba6 feat: add CORS middleware to allow cross-origin requests for waveform viewer 2026-04-03 14:50:43 -04:00
claude dfb974d658 feat: add endpoint to download full raw ADC waveform for a single event 2026-04-03 13:54:54 -04:00
claude 501b5080e9 server: backfill event.project_info fields from compliance config
The 210-byte waveform record only stores "Project:" — client, operator,
sensor_location, and notes are device-level settings in SUB 1A, not
per-event fields. Backfill those into each event's project_info after
download, same pattern as the sample_rate backfill.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 14:33:33 -04:00
claude 6eecd0c1d1 client/models/server: wire event_count from SUB 08 event index into connect()
- DeviceInfo.event_count: Optional[int] = None  (new field in models.py)
- connect() now calls proto.read_event_index() after compliance config and
  stores the decoded count in device_info.event_count
- _serialise_device_info() exposes event_count in /device/info and /device/events
  JSON responses

event_count is decoded from uint32 BE at offset +3 of the 88-byte F7 payload
(🔶 inferred — needs live device confirmation against a multi-event device).
Any ProtocolError from the index read is caught and logged; event_count stays
None rather than failing the whole connect().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 02:00:37 -04:00
claude 870a10365e server: fill ev.sample_rate from compliance config for /device/events
sample_rate is a device-level setting stored in the compliance config,
not per-event in the waveform record.  After downloading events, backfill
ev.sample_rate from info.compliance_config.sample_rate for any event
that didn't get it from the waveform record decode path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 02:00:00 -04:00
claude 32b9d3050c feat: implement SUB 1A (compliance config) read
Adds full support for reading device compliance configuration (2090-byte E5
response) containing record time, trigger/alarm levels, and project strings.

protocol.py:
- Implement read_compliance_config() two-step read (SUB 1A → E5)
- Fixed length 0x082A (2090 bytes)

models.py:
- Add ComplianceConfig dataclass with fields: record_time, sample_rate,
  trigger_level_geo, alarm_level_geo, max_range_geo, project strings
- Add compliance_config field to DeviceInfo

client.py:
- Implement _decode_compliance_config_into() to extract:
  * Record time float at offset +0x28 
  * Trigger/alarm levels per-channel (heuristic parsing) 🔶
  * Project/setup strings from E5 payload
  * Placeholder for sample_rate (location TBD )
- Update connect() to read SUB 1A after SUB 01, cache in device_info
- Add ComplianceConfig to imports

sfm/server.py:
- Add _serialise_compliance_config() JSON encoder
- Include compliance_config in /device/info response
- Updated _serialise_device_info() to output compliance config

Both record_time (at fixed offset 0x28) and project strings are  CONFIRMED
from protocol reference §7.6. Trigger/alarm extraction uses heuristics
pending more detailed field mapping from captured data.

Sample rate remains undiscovered in the E5 payload — likely in the
mystery flags at offset +0x12 or requires a "fast mode" capture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 12:08:43 -04:00
claude 4944974f6e feat: decode waveform record timestamp, record type, and Peak Vector Sum
Confirmed 2026-04-01 against Blastware event report for BE11529 thump
event ("00:28:12 April 1, 2026", PVS 3.906 in/s).

models.py:
- Timestamp.from_waveform_record(): decode 9-byte format from 0C record
  bytes[0-8]: [day][sub_code][month][year:2BE][?][hour][min][sec]
- Timestamp: add hour/minute/second optional fields; __str__ includes
  time when available
- PeakValues: add peak_vector_sum field (confirmed fixed offset 87)

client.py:
- _decode_waveform_record_into: add timestamp decode from bytes[0:9]
- _extract_record_type: decode byte[1] (sub_code), not ASCII string
  search; 0x10 → "Waveform", histogram TBD
- _extract_peak_floats: add PVS from offset 87 (IEEE 754 BE float32)
  = √(T²+V²+L²) at max instantaneous vector moment

sfm/server.py:
- _serialise_timestamp: add hour/minute/second/day fields to JSON
- _serialise_peak_values: add peak_vector_sum to JSON

docs: update §7.7.5 and §8 with confirmed 9-byte timestamp layout,
PVS field, and byte[1] record type encoding; update command table;
close resolved open questions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 00:53:34 -04:00
claude f74992f4e5 fix: serial decode offset, PPV label scan, debug mode for waveform records
- _decode_serial_number: read from data[11:] not data[:8] — was returning
  the LENGTH_ECHO byte (0x0A = '\n') instead of the serial string
- _extract_peak_floats: search for channel label strings ("Tran" etc) and
  read float at label+6; old step-4 aligned scan was reading trigger levels
  instead of PPV values
- get_events: add debug=False param; stashes raw 210-byte record on
  Event._raw_record when True for field-layout inspection
- server /device/events: add ?debug=true query param; includes
  raw_record_hex + raw_record_len in response when set
- models: add Event._raw_record optional bytes field
2026-03-31 23:46:07 -04:00
claude 8074bf0fee bump timeout to 30s to deal with modem slowness. 2026-03-31 12:12:36 -04:00
claude de02f9cccf Handle cold-boot timeout for TCP connections
- bridges/tcp_serial_bridge.py: increase default boot_delay 2s → 8s to
  cover MiniMate Plus cold-start time (unit wakes from RS-232 line
  assertion but takes 5-10s to be ready for POLL_PROBE).
- sfm/server.py: add _run_with_retry() — on TCP connections only, retries
  once on ProtocolError. Serial timeouts are not retried (usually a real
  fault). Confirmed behaviour: unit wakes purely from RS-232 line voltage,
  no software wake-up frame needed.
2026-03-31 12:02:52 -04:00
claude 51d1aa917a Add TCP/modem transport (Sierra Wireless RV55/RX55 field units)
- minimateplus/transport.py: add TcpTransport — stdlib socket-based transport
  with same interface as SerialTransport. Overrides read_until_idle() with
  idle_gap=1.5s to absorb the modem's 1-second serial data forwarding buffer.
- minimateplus/client.py: make `port` param optional (default "") so
  MiniMateClient works cleanly when a pre-built transport is injected.
- minimateplus/__init__.py: export SerialTransport and TcpTransport.
- sfm/server.py: add `host` / `tcp_port` query params to all device endpoints.
  New _build_client() helper selects TCP or serial transport automatically.
  OSError (connection refused, timeout) now returns HTTP 502.
- docs/instantel_protocol_reference.md: add changelog entry and full §14
  (TCP/Modem Transport) documenting confirmed transparent passthrough, no ENQ
  on connect, modem forwarding delay, call-up vs ACH modes, and hardware note
  deprecating Raven X in favour of RV55/RX55.

Usage: GET /device/info?host=<modem_ip>&tcp_port=12345
2026-03-31 00:44:50 -04:00