Commit Graph

61 Commits

Author SHA1 Message Date
serversdown beca5de06e docs: clean up and verify s3 protocol docs 2026-05-20 17:55:02 +00:00
serversdown d85df4c886 Merge pull request 'merge full s3 codec decoded' (#23) from codec-re into main
Reviewed-on: #23
2026-05-20 13:45:32 -04:00
Claude 0466bb4f44 codec: crack wide-NN blocks (1X NN / 2X NN); loud events now fully decode
When NN exceeds 0xFC, the codec extends to 12-bit NN by using the
low nibble of the TYPE byte as the high nibble of NN:

    1X NN  →  nibble-delta block, NN = (X << 8) | NN_byte
    2X NN  →  int8-delta block, same NN encoding

Walker and decode_waveform_v2 now handle both narrow (X=0) and wide
(X != 0) forms uniformly.

Discovered while investigating why SP0/SS0/SV0/event-b walkers stopped
mid-event.  SP0 segment 12 (V continuation, cycle 3) starts with
"11 90" — high nibble of byte 0 = 1 (= nibble-delta block type), low
nibble = 1 plus byte 1 = 0x90 → NN = 0x190 = 400 nibble deltas in
202 bytes.  Walker was rejecting "11" as a non-tag.

Sample count went from 47,364 to 72,972 verified byte-exact:

  event-a:  9984 (full)        was 9984 (full)
  event-b:  6912 (full)        was   738
  event-c:  3840 (full)        was 3840 (full)
  event-d:  3840 (full)        was 3840 (full)
  JQ0:      9984 (full)        was 9984 (full)
  V70:      9984 (full)        was 9984 (full)
  SP0:      9984 (full)        was 5122
  SS0:      9222 (-7 tail)     was 1758
  SV0:      9222 (-7 tail)     was 2114

7 of 9 fixtures now decode end-to-end across all 3 geo channels.
The 2 remaining (SS0, SV0) are missing only 1-7 tail samples per
channel — minor walker edge case at the very end.

74 tests pass (was 71).
2026-05-20 17:28:54 +00:00
Claude 85f4bcfe86 codec: wire decode_waveform_v2 into production; add MicL dB helper
Replaces the broken legacy int16 LE decoder in client.py with the
verified multi-channel codec.  Three changes:

1. blastware_file.extract_body_bytes(a5_frames) — new helper that
   factors out the body-reconstruction logic from write_blastware_file
   so both writers (BW binary) and decoders (sample arrays) can use
   the same canonical bytes.

2. waveform_codec.decode_a5_frames(a5_frames) — production entry point.
   Returns the raw_samples dict consumers expect (Tran/Vert/Long as
   int16 ADC counts; MicL as native ADC counts).  Internally:
     A5 frames → extract_body_bytes → decode_waveform_v2
                → decoded_to_adc_counts (geos ×16; mic pass-through)

3. waveform_codec.mic_count_to_db(count) — MicL ADC → dB(L) per BW's
   display formula:
     dB = sign(count) × (81.94 + 20 × log10(|count|))   for |count| ≥ 1
   Verified against V70 fixture: count=813 → 140.14 dB (BW PSPL 140.1).

client.py:_decode_a5_waveform is reduced to a thin wrapper that calls
decode_a5_frames and populates event.raw_samples.  Original implementation
preserved as _decode_a5_waveform_LEGACY (dead code; reference only).

Also fixed a tail-end bug in decode_waveform_v2 where trailer-section
"40 02" markers (containing ASCII serial bytes, NOT real segment headers)
were being mis-interpreted, producing 2 spurious samples per channel at
the end of each event.  Added bytes [12:14] == "02 00" validation to
reject non-header markers.

7 new pytest tests cover the new helpers and dB conversion.  Total:
71 passing (up from 64).

Known limitation (carried over from before): the walker still stops
mid-event on the loudest fixtures (SP0/SS0/SV0/event-b) at some
mid-segment edge cases not yet characterized.  Every sample reached
is decoded correctly; the walker just doesn't reach all of them.
Loud events still yield 5,000–15,000 byte-exact samples each.
2026-05-20 17:28:54 +00:00
Claude 2ff2762eec codec-re: 30 NN block CRACKED — codec fully decoded
User intuition (16-bit) + 12-bit packing hypothesis + the int16 ADC
range constraint led to the final piece.

30 NN block format (CONFIRMED across all 14 blocks in the fixture
bundle):

  NN 12-bit signed deltas packed as NN/4 groups of 6 bytes each.
  Within each group:
    bytes [0:2] = 16 bits = 4 × 4-bit high nibbles (MSB-first)
    bytes [2:6] = 4 × int8 low bytes
    delta[k] = sign_extend_12((high_nibble[k] << 8) | low_byte[k])

  Block length = NN × 1.5 + 2 bytes (tag included).  Earlier walker
  used NN × 4 which is only correct in the TRAILER section.

Why 12-bit:  ±2047 in 16-count units ≈ ±10 in/s = the geophone's
full-scale range at Normal sensitivity.  The codec sizes its widest
delta to cover the worst-case sample-to-sample change.

Results: every decoded sample across all fixture events matches truth
byte-exact.  ZERO divergences.

  event-a:  9984 samples (full event, all 3 geos)
  event-c:  3840 (full event)
  event-d:  3840 (full event)
  JQ0:      9984 (full event)
  V70:      9984 (full event)
  SP0:      5122 (walker stops early on edge cases)
  SS0:      1758
  SV0:      2114
  event-b:   738

  TOTAL: 47,364 ADC samples verified, zero errors.

Three full 3-sec events decode end-to-end across all three geo
channels.  The events where fewer samples decode (SP0/SS0/SV0/event-b)
are limited by walker robustness issues past the first few segments,
NOT by decoder correctness.

64 tests pass (up from 55).  Files: minimateplus/waveform_codec.py
(new 30 NN decode + corrected walker length), tests/test_waveform_codec.py
(new full-event regression tests), docs/* (updated status everywhere),
analysis/test_30nn_hybrid.py (new — the analysis script that confirmed
the format).
2026-05-20 17:28:54 +00:00
Claude ce5dc640ba codec-re: quiet bundle decodes FULLY (17k samples, zero errors)
User asked the right question: do events without 30 NN blocks decode
fully?  Answer: YES.

  event-a:  Tran 3328 ✓  Vert 3328 ✓  Long 3328 ✓  (28 segments, 0 '30 NN')
  event-c:  Tran 1280 ✓  Vert 1280 ✓  Long 1280 ✓  (12 segments, 0 '30 NN')
  event-d:  Tran 1280 ✓  Vert 1280 ✓  Long 1280 ✓  (12 segments, 0 '30 NN')

17,664 ADC samples decoded byte-exact against BW's ASCII export.
Zero divergences across event-a, event-c, event-d.

This means the codec is FULLY SOLVED for any event without 30 NN
blocks.  The remaining gap is the 30 NN block format only — used for
high-amplitude regions where deltas exceed int8 range.  For quiet
events (or quiet stretches of loud events), the decoder is complete.

9 new regression tests bring the total to 55, all passing.

Files: tests/test_waveform_codec.py + docs/waveform_codec_re_status.md
+ new analysis/verify_quiet_bundle.py.
2026-05-20 17:28:54 +00:00
Claude 07675626dc codec-re: channel rotation CONFIRMED — full multi-channel decoder works
The segment-channel scoring analyzer (from scratch/next_experiment_skeleton.py)
ran and immediately confirmed the rotation hypothesis:

  SP0 seg 0: best fit Vert  508/508  ✓
  SP0 seg 1: best fit Long  508/508  ✓
  SP0 seg 3: best fit Tran  508/508  ✓  (Tran continuation)
  SP0 seg 5: best fit Long  508/508  ✓
  SP0 seg 9: best fit Long  508/508  ✓
  V70 seg 0: best fit Vert  508/508  ✓
  V70 seg 1: best fit Long  508/508  ✓

Channels rotate Tran → Vert → Long → MicL per 40 02 segment header.

Also discovered the segment header has DOUBLE duty: bytes [14:18] anchor
the NEW segment's channel (2 samples as int16 BE in 16-count units), AND
bytes [0:4] extend the PREVIOUS channel by 2 more samples (2 deltas as
int16 BE).  This is the same "2 anchors + delta stream" structure as the
body preamble for Tran.

decode_waveform_v2 now returns full per-channel sample dicts.
Byte-exact verified ranges:
  V70: Tran 512, Vert 512, Long 512   (all first segments)
  JQ0: Tran 512, Vert 258
  SP0: Long 1536 (all 3 L segments)

Still open: the 30 NN block format (high-amplitude packed deltas) —
appears mid-segment when single-byte deltas can't carry the magnitude.

6 new tests bring the count to 46.  All passing.
2026-05-20 17:28:54 +00:00
Claude f68ee9f0f9 docs: clean up waveform-codec doc layers per review
Three "truth layers" had drifted apart between commits.  Fixed:

1. waveform_codec.py docstring rewritten from the 2026-05-08
   "structural framing only" state to the 2026-05-11 "Tran segment 0
   solved + segment-header partially decoded" state.  Killed stale
   "~80 sample-sets per segment" language (real segments are
   flash-page-byte-sized, not sample-count-sized; observed first-segment
   sizes are 42-510 samples depending on signal).  Killed stale
   "preamble is 7 or 9 bytes" language (always 7).

2. docs/instantel_protocol_reference.md §7.6.1: added a clear
   "CURRENT STATUS" box at the top with a status table.  Replaced the
   stale "~80 sample-sets" line with the verified per-event segment
   sizes.  Merged two redundant segment-header field-table sections.

3. docs/waveform_codec_re_status.md (NEW): clean working-status doc.
   Solved / not solved / hypothesis / next experiment / fixtures /
   tests.  The protocol reference remains the historical Rosetta
   Stone; this new file is the current-truth working note that
   shouldn't accumulate fossil layers.

4. CLAUDE.md §"Waveform body codec": prominent warning box at top —
   "DO NOT TRUST decoded sample arrays yet."  BW binary passthrough
   is the only sample-bearing output to trust until the decoder
   lands.  Added a "Next experiment" subsection pointing the next
   pass at the segment-channel scoring analyzer.

40 tests still pass.
2026-05-20 17:28:54 +00:00
Claude a0c9a482c7 codec-re: 00 NN is RLE; full Tran segment-0 decode (4 of 5 events)
User uploaded a Vert-heavy event (JQ0) and a Mic-heavy event (V70).
Those two were exactly what was needed to crack the next piece:

- 00 NN block = run-length-encoded zero deltas in the current channel.
  Append NN copies of the current cumulative value (no change).
- find_data_start now recognizes 00 NN as a valid first tag (some events
  begin with a leading 00 NN RLE block).
- decode_tran_initial now decodes the FULL segment 0 (not just the first
  data block).

Results across 5 fixture events:
  - M529LL1A.SP0 (loud-all-channels)  : 510 / 510  ✓
  - M529LL1L.JQ0 (Vert-heavy)         : 510 / 510  ✓
  - M529LL1L.V70 (Mic-heavy)          : 510 / 510  ✓
  - M529LL1A.SV0 (loud-from-start)    :  58 /  58  ✓
  - M529LL1A.SS0 (loud-from-start)    :  42 / 502  (stops at first 30 04)

The 30 04 block (only seen in loud-from-start events) hasn't been
decoded yet — likely a channel-switch marker for the high-amplitude
regime.

Also discovered: segment header (40 02) payload bytes [0:2] = T_delta
at first sample of new segment, [6:8] = byte length to next segment.
Multi-segment Tran decoding still diverges after sample 512 because
the per-segment channel ordering after the header is unknown.

Tests: 40 pass (up from 36).

Files:
- minimateplus/waveform_codec.py: find_data_start fix, RLE handling,
  full segment-0 decode in decode_tran_initial
- tests/test_waveform_codec.py: synthetic RLE test, full segment 0
  tests for JQ0 and V70
- tests/fixtures/5-11-26/: M529LL1L.JQ0, M529LL1L.V70 + TXT exports
- docs/instantel_protocol_reference.md §7.6.1: RLE + segment-header docs
2026-05-20 17:28:54 +00:00
Claude 6ac126e05c codec-re: crack Tran channel codec with high-amplitude May 11 bundle
User uploaded 3 high-amplitude events (PPV 6-7 in/s — shook the geophone
hard) to decode-re/5-11-26/.  These cracked the Tran codec:

- Preamble bytes [3:5] and [5:7] = Tran[0] and Tran[1] as int16 BE
  in 16-count units (LSB = 0.005 in/s).  Confirmed across all 7
  fixtures.
- First data block carries Tran deltas from sample 2 onward:
  * 10 NN block: NN/2 bytes of payload, each byte = two 4-bit signed
    nibble deltas (high nibble first)
  * 20 NN block: NN int8 signed deltas

Verified 22+42+46 = 110 Tran samples across SP0/SS0/SV0 with 0 errors
against BW's ASCII export.

Why the earlier 96-combination brute force failed: the quiet 5-8
events all had T[0] = T[1] ≈ 0 so the preamble's per-channel encoding
was undetectable.  Loud events made the encoding obvious.

What's solved:
- minimateplus.waveform_codec.decode_tran_initial: returns first
  N Tran samples in 16-count units for any body.
- Walker length formula for in-data 30 NN blocks (NN*2 instead of NN*4).
- Walker now handles bodies that start with 20 NN (in addition to 10 NN).

What's still open:
- Tran past the first data block (multi-block channel switching).
- Vert / Long / MicL channel encodings.
- Walker correctness past offset ~427 in event-b.

Tests: 36 pass.  decode_waveform_v2 still returns None — the full
multi-channel decoder is not wired up.  decode_tran_initial is the
new verified entry point.

Files: minimateplus/waveform_codec.py, tests/test_waveform_codec.py
(adds 5-11-26 fixtures + decode_tran_initial tests), and
docs/instantel_protocol_reference.md §7.6.1 (Tran codec spec).
2026-05-20 17:28:54 +00:00
Claude d3f77d1d96 codec-re: solve waveform body block framing; per-byte sample mapping still open
Decoded the structural framing of the Blastware waveform body — the bytes
between the 21-byte STRT record and the 26-byte file footer.  The body is
a sequence of tagged variable-length blocks, NOT raw int16 LE.  Five tag
types (10/20/00/30/40 NN) and their lengths are now confirmed against the
4-event May 2026 fixture bundle.  Body splits cleanly into ~16 segments
(for a 1280-sample event) separated by 40 02 segment headers carrying a
monotonically incrementing uint32 LE counter at bytes [8:12].

What's done:
- minimateplus/waveform_codec.py — block walker, segment splitter, segment
  header parser.  decode_waveform_v2 is a stub returning None until the
  byte-to-sample mapping is solved; client.py is unchanged.
- tests/test_waveform_codec.py — 31 tests covering block detection, lengths,
  contiguous-walk, segment splitting, segment-header parsing, and counter
  monotonicity.  All pass.
- tests/fixtures/decode-re-5-8-26/ — bundled fixtures (4 events, BW binary
  + Blastware ASCII export each).
- docs/instantel_protocol_reference.md §7.6.1 — replaced retraction box
  with the verified structural decoding plus an explicit list of what's
  still open.

What's still open: the per-byte mapping inside 10 NN / 20 NN blocks.  96
channel-permutation × nibble-order × sign-convention combinations were
brute-force tested; none match BW's ASCII export to within ±1 ADC count.
The codec is more elaborate than uniform 4-bit deltas — likely a hybrid
variable-bit-width scheme with segment-anchor resync points.  Next
recommended step: capture an event with a known calibration tone to pin
down magnitude scaling.

Walker also bails out partway through event-b (open issue documented in
both the module and the protocol reference).
2026-05-20 17:28:54 +00:00
Claude 8316a1bbd8 docs(protocol): accuracy sweep across the protocol reference
Three-pass audit of docs/instantel_protocol_reference.md against
CLAUDE.md and the minimateplus/ implementation. Closes long-standing
discrepancies that had accumulated as the protocol understanding
evolved month over month.

Major corrections:
- §2/§3: S3 frames terminate on bare ETX, not DLE+ETX; payload
  byte[1] is flags / byte[2] is SUB (was wrongly DLE/ADDR).
- §4.2: probe responses do not carry data length; DATA_LENGTH
  is a per-SUB hardcoded constant.
- §5.1: dropped stale duplicate "SUB 1C = TRIGGER CONFIG READ"
  row; SUB 0A lengths corrected from 0x30/0x26 to 0x46/0x2C.
- §5.3: added the missing write-frame mechanics (BW_CMD-only
  doubling, DLE-aware checksum, offset = data[1]+2, ack format,
  SUB 71 chunk parameters).
- §7.6.x: switched compliance-anchor convention from the unstable
  10-byte form to the canonical 6-byte `\xbe\x80\x00\x00\x00\x00`;
  recording_mode confirmed at anchor−8 in both read and write
  (the prior anchor−3/−4 split caused anchor drift on write).
  Sample_rate at anchor−6, histogram_interval at anchor−4 (now ),
  record_time at anchor+6. Geo_range row added at channel_label+33.
- §7.5b/§8: added the 10-byte sub_code=0x03 continuous-mode
  timestamp variant; peak vector sum location corrected from
  fixed offset 87 to label-relative tran_pos−12.
- §7.7.2: SUB 1E/1F token byte at params[7], not params[6].
- §7.7.3: SUB 0A length disambiguation rewritten.
- §7.8.4/§7.8.7: fi==9 skip marked FIXED; metadata-page TODO
  replaced with current decoder state.
- §11: POLL example wire bytes corrected; SUB 5A row added to
  checksum table.
- §13/§14: device-under-test updated to BE11529/S338.17; TCP
  Idle Timeout consistency fix (0→2 min); Data Forwarding
  Timeout units clarified.
- §15 (renumbered from second §14): open-question entries
  already resolved in CLAUDE.md closed out.
- Appendix D: extension taxonomy rewritten — extensions encode
  a timestamp (AB0T scheme), not recording mode.

Navigation note added to §7 acknowledging the organic-growth
duplicate section numbers (§7.5/§7.5b, §7.6, §7.7, §7.8, §7.9)
and pointing readers to the canonical sections for each topic.

https://claude.ai/code/session_019tWZybD94YUsBaEGhnM5A2
2026-05-20 15:41:42 +00:00
serversdown ecc935482b seismo-relay v0.19.0 — device-family separation + micromate/ package
Tighten the Series III / Series IV boundary so UI and storage dispatch
on a clean signal instead of sniffing filenames or applying magnitude
heuristics.

Phase 1 — events.device_family column ("series3" | "series4"):
  self-applying migration with filename-based backfill of existing rows
  (1,132 backfilled on prod 2026-05-20); plumbed through every import
  path (BW endpoint, IDF endpoint, ACH server, BW CLI, sidecar
  backfill); UPSERT preserves via COALESCE; UI dispatches on it.

Phase 2 — extract micromate/ package alongside minimateplus/:
  native IdfEvent / IdfReport / IdfPeaks / IdfProjectInfo /
  IdfSensorCheck (mic in dB(L), not pseudo-psi); moved
  idf_ascii_report.py from sfm/ to micromate/; refactored
  save_imported_idf to use IdfEvent and bridge to minimateplus.Event at
  the SQL-insert boundary; idf_file.py stub for the future binary codec.

Phase 3 prep — docs/idf_protocol_reference.md captures the two
observed Thor binary header signatures (1,012 newer-firmware files vs
2 old files whose layout is byte-for-byte BW-STRT-compatible), file-size
hints suggesting int8 sample encoding, open questions in dependency
order, and a concrete first-session plan for cracking the codec.

Also rolled in the v0.18.1 hotfixes that motivated this work:
  - idf_ascii_report parser now handles "<0.005 in/s" (below-threshold)
    and "N/A" markers without leaving raw strings in numeric DB columns.
  - sfm_webapp.html: defensive _ppvFmt / mic formatter so future
    data-shape drift can't kill the whole events table render.

All 1,014 example-data sidecars round-trip through the new package.
See CHANGELOG.md for full notes.
2026-05-20 15:19:49 +00:00
serversdown 1fff8179d6 Add runbook for recovering wedged units and new scripts for device management
- Created a comprehensive runbook (`wedged_unit_recovery.md`) detailing the recovery process for units stuck in a call-home loop, including symptoms, recovery steps, and explanations of the failure mode.
- Added `blind_stop.sh` script to send stop-monitoring commands in a tight loop for unresponsive devices.
- Introduced `rescue_device.sh` script to disable Auto Call Home and erase events from a busy device.
- Implemented `slow_drip.sh` script to send stop-monitoring frames at a slow rate to prevent UART overrun.
- Developed `spam_stop.sh` script to rapidly send stop-monitoring commands to a device.
- Created `watch_unit.sh` script for passive monitoring of device reachability, logging results over time.
2026-05-17 07:58:13 +00:00
serversdown 8aea46b8a0 doc(fix): retracts raw int16 LE sample set assumptions. 2026-05-08 19:26:25 +00:00
claude c914a15e12 docs: update for v0.14.3 - Full continuous waveform download successful! 2026-05-05 20:37:52 -04:00
claude 7444738883 debug(protocol): event-N probe is now at counter = start_offset instead of start_offset + 0x46 2026-05-05 16:46:35 -04:00
claude 1ef55521b1 Fix: Removed duplicates from merge botch. Stable version of seismo_lab.py 2026-05-01 17:34:41 -04:00
claude 2f084ed105 fix(protocol): update chunk counter formula to use max(key4[2:4], 0x0400) for accurate data streaming 2026-04-26 01:28:47 -04:00
claude 35c3f4f945 fix(protocol): correct A5 frame classification and chunk counter formula 2026-04-24 17:25:29 -04:00
claude aa2b02535b fix(protocol): add record_time based chunk scaling for longer event record times 2026-04-23 17:33:16 -04:00
claude 6dcca4da79 feat(protocol): fully decode Blastware filename encoding and update related documentation 2026-04-22 23:43:31 -04:00
claude c47e3a3af0 feat(protocol): update Blastware file format documentation and encoding details 2026-04-22 19:16:05 -04:00
claude dfbc9f29c5 feat: first try at building waveform binary files. 2026-04-21 22:57:53 -04:00
claude 4331215e23 feat(protocol): enhance raw capture functionality and documentation updates
- Update `s3_bridge.py` to default raw capture file paths to "auto" for timestamped naming.
- Modify `gui_bridge.py` to pre-check raw capture options and streamline path handling.
- Extend `ach_server.py` to save both incoming and outgoing raw bytes for analysis.
- Revise `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect changes in recording mode handling and compliance data encoding.
2026-04-21 16:07:24 -04:00
claude 3fb24e1895 feat(call-home): Implement Auto Call Home configuration management
- Added `CallHomeConfig` model to represent the Auto Call Home settings.
- Introduced methods in `MiniMateClient` for reading (`get_call_home_config`) and writing (`set_call_home_config`) the call home configuration.
- Updated `MiniMateProtocol` with new commands for call home operations (SUB 0x2C for read, SUB 0x7E for write, and SUB 0x7F for confirm).
- Created API endpoints for retrieving and updating call home settings in the server.
- Enhanced the web interface with a new "Call Home" tab for user interaction with call home settings.
- Implemented JavaScript functions for reading and writing call home configurations from the web app.
2026-04-20 18:23:48 -04:00
claude b6ffdcfa87 feat: implement geophone sensitivity and recording mode settings in compliance config 2026-04-20 17:03:58 -04:00
claude e04114fd6c feat: mapped record_mode protocol 2026-04-20 15:49:31 -04:00
claude b23cf4bb50 fix: max_geo_range correctly identified as ADC Scale factor number. 2026-04-17 19:43:45 -04:00
claude 5866ecdb3e docs: update protocol doc to reflect unkown status of max_range_geo. 2026-04-16 18:17:16 -04:00
claude 6acb419ebd docs: update protocol reference with v0.9.0 erase-all protocol
Changelog section:
- 5 new entries (2026-04-11): erase-all confirmation, SUB 0x06 purpose
  resolved, §7.11 added, §14.6 ACH session lifecycle marked IMPLEMENTED

§5.1 Request Commands:
- SUB 0x06 description updated: "EVENT STORAGE RANGE READ" (not "CHANNEL
  CONFIG READ"), token=0xFE, last 8 bytes = first/last stored event keys
- SUB 0xA3 added: ERASE ALL BEGIN — standard build_bw_frame, token=0xFE, ack 0x5C
- SUB 0xA2 added: ERASE ALL CONFIRM — standard build_bw_frame, token=0xFE, ack 0x5D

§5.2 Response SUBs:
- 0x06→0xF9 marked CONFIRMED 2026-04-11
- 0xA3→0x5C and 0xA2→0x5D added with CONFIRMED status

§7.11 (new section): Erase-All Protocol
- Full wire sequence (6 request/response pairs)
- SUB 0x06 storage range payload layout (36 bytes, last 8 = first/last key)
- Post-erase key counter reset: device restarts from 0x01110000
- Implementation notes pointing to client.py and ach_server.py

§14.6 ACH Session Lifecycle:
- Removed "Future" label — fully implemented in bridges/ach_server.py
- Added step 6 (optional erase), step 8 (DCD/DTR auto-resume)
- Documents ach_server.py flags and ach_state.json schema
- Notes RV55 DCD/DTR issue as known open problem

Open Questions table:
- SUB 0x06 purpose RESOLVED
- Erase-all sequence RESOLVED
- ACH server RESOLVED
- Sensor Check byte: still open, added as formal question
- RV55 DCD/DTR: added as new open question

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 21:14:58 +00:00
claude 574d40027f feat: enhance logging messages in ach_server.py and add experiments.py for protocol minimization 2026-04-16 21:14:58 +00:00
claude 990cb8850e fix: correct monitoring flag and battery/memory offsets in _decode_monitor_status
section[6] is the monitoring flag (was wrongly section[1] — section[1] is always
0x00 in both states). Battery and memory fields use relative-from-end offsets
(section[-11:-9], section[-9:-5], section[-5:-1]) instead of absolute positions,
which broke when the payload grew by 3 bytes in monitoring mode.

Confirmed from full byte diff of 142 0xE3 frames in 4-8-26/2ndtry capture.
SFM start_monitoring now polls /device/monitor/status every 5s for up to 60s
instead of a fixed 25s delay (unit runs ~40s on-device sensor check before
confirming monitoring state).

Also corrects stale 1C→6E response anomaly claim in protocol reference — no
exceptions to the 0xFF−SUB rule are known.
2026-04-08 23:41:11 -04:00
claude a41e7a9e1a feat: Add monitoring functionality to MiniMate protocol and web interface
- Introduced new SUBs for monitoring status, start, and stop commands in protocol.py.
- Implemented read_monitor_status, start_monitoring, and stop_monitoring methods in MiniMateProtocol class.
- Added new API endpoints for monitoring status retrieval and control in server.py.
- Enhanced the web application with a monitoring panel, including battery and memory status display.
- Created a new Python script to parse SUB 0x1C response frames for monitoring status.
- Documented the monitoring status response format and field locations in markdown and text files.
2026-04-08 14:34:42 -04:00
claude a7ab6eaf7c feat: add config API endpoint and JSON schema draft 2026-04-07 17:26:24 -04:00
claude 9bef430451 docs: document 5A end-of-stream signal, chunk timing, fi==9 bug, ADC conversion
Adds §7.8.4 to protocol reference and corresponding CLAUDE.md sections:

- End-of-stream: device sends exactly 1 raw byte after last chunk; handled
  via TimeoutError + bytes_fed>0 check → graceful break to termination
- Chunk timing: ~1s per chunk, 35 chunks for a 9,306-sample event, safe
  timeout is 10s (not default 120s)
- fi==9 decoder bug: hardcoded skip drops ~133 sample-sets per event;
  noted as known issue pending fix
- ADC conversion: counts × (range/32767) → physical units (in/s for geo)

Changelog entries added for all four items (2026-04-06).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:28:26 -04:00
claude 5b3e8af1e3 fix: remove special case chunk counter, all chunks use chunk_num * 0x0400 2026-04-06 17:03:09 -04:00
claude ad1c9e48b0 fix: update MiniMateClient and protocol to ensure correct handling of 1F calls and improve event download sequence 2026-04-06 15:58:03 -04:00
claude 790e442a7a feat: implement raw ADC waveform decoding and download functionality
- Added `_decode_a5_waveform()` to parse SUB 5A frames into per-channel time-series data.
- Introduced `download_waveform(event)` method in `MiniMateClient` to fetch full waveform data.
- Updated `Event` model to include new fields: `total_samples`, `pretrig_samples`, `rectime_seconds`, and `_waveform_key`.
- Enhanced documentation in `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect new features and confirmed protocol details.
2026-04-03 13:53:09 -04:00
claude 5d0f0855f2 doc: update to .0.6.0 with full working event read loop 2026-04-02 17:30:33 -04:00
claude 6a42facf02 docs: update protocol reference for SUB 1A compliance config read
New/corrected sections:

§7.6.1 Record Time — corrected: anchor+10 supersedes the +0x28 absolute
offset.  Added 3 s, 5 s, 8 s confirmations alongside existing 7/10/13.
Warning added: do NOT use fixed offset for BE11529.

§7.6.2 (NEW) SUB 1A Multi-Frame Read Protocol — 4-frame A/B/C/D
sequence documented (reverse-engineered from raw_bw capture).  E5
page_key field explained.  BE11529 duplicate-page behaviour and
(page_key, chunk_len) dedup strategy documented.

§7.6.3 (NEW) Sample Rate and DLE Jitter — Normal/Fast/Faster = 1024/
2048/4096 Sa/s confirmed.  Root cause of ±1 byte cfg jitter explained:
4096 = 0x1000 → `10 10 00` in raw frame → `10 00` after DLE unstuffing
= 1 byte shorter than 04 00/08 00.  Anchor search requirement explained.

Changelog — 5 new entries covering the 4-frame sequence, duplicate-page
detection, record_time anchor correction, sample_rate confirmation, and
_pending_frames / reset_parser=False implementation notes.

Quick Reference — Record Time and Sample Rate rows updated with correct
locations, types, and confirmed values.

Open Questions — SUB 1A item updated to "substantially resolved".
Record time item updated. Sample rate added as resolved.
2026-04-01 16:54:51 -04:00
claude 4944974f6e feat: decode waveform record timestamp, record type, and Peak Vector Sum
Confirmed 2026-04-01 against Blastware event report for BE11529 thump
event ("00:28:12 April 1, 2026", PVS 3.906 in/s).

models.py:
- Timestamp.from_waveform_record(): decode 9-byte format from 0C record
  bytes[0-8]: [day][sub_code][month][year:2BE][?][hour][min][sec]
- Timestamp: add hour/minute/second optional fields; __str__ includes
  time when available
- PeakValues: add peak_vector_sum field (confirmed fixed offset 87)

client.py:
- _decode_waveform_record_into: add timestamp decode from bytes[0:9]
- _extract_record_type: decode byte[1] (sub_code), not ASCII string
  search; 0x10 → "Waveform", histogram TBD
- _extract_peak_floats: add PVS from offset 87 (IEEE 754 BE float32)
  = √(T²+V²+L²) at max instantaneous vector moment

sfm/server.py:
- _serialise_timestamp: add hour/minute/second/day fields to JSON
- _serialise_peak_values: add peak_vector_sum to JSON

docs: update §7.7.5 and §8 with confirmed 9-byte timestamp layout,
PVS field, and byte[1] record type encoding; update command table;
close resolved open questions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 00:53:34 -04:00
claude 9f52745bb4 feat: add full event download pipeline 2026-03-31 20:48:03 -04:00
claude 1078576023 Add Console tab to seismo_lab + document RV50/RV55 modem config
seismo_lab.py:
- Add ConsolePanel — third tab for direct device connections over serial
  or TCP without the bridge subprocess
- Commands: POLL, Serial #, Full Config, Event Index (open/close per cmd)
- Colour-coded output: TX blue, RX raw teal, parsed green, errors red
- Save Log and Send to Analyzer buttons; auto-saves to bridges/captures/
- Queue/after(100) pattern — no performance impact
- Add SCRIPT_DIR to sys.path so minimateplus imports work from GUI

docs/instantel_protocol_reference.md:
- Confirm calibration year field at SUB FE payload offset 0x56–0x57
  (uint16 BE): 0x07E7=2023 (BE18189), 0x07E9=2025 (BE11529)
- Document full Sierra Wireless RV50/RV55 required ACEmanager settings
  (Quiet Mode, Data Forwarding Timeout, TCP Connect Response Delay, etc.)
- Correct §14.2: RV50/RV55 sends RING/CONNECT over TCP to caller even
  with Quiet Mode on; parser handles by scanning for DLE+STX
- Confirm "Operating System" boot string capture via cold-start Console
- Resolve open question: 0x07E7 field = calibration year

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 17:19:31 -04:00
claude 51d1aa917a Add TCP/modem transport (Sierra Wireless RV55/RX55 field units)
- minimateplus/transport.py: add TcpTransport — stdlib socket-based transport
  with same interface as SerialTransport. Overrides read_until_idle() with
  idle_gap=1.5s to absorb the modem's 1-second serial data forwarding buffer.
- minimateplus/client.py: make `port` param optional (default "") so
  MiniMateClient works cleanly when a pre-built transport is injected.
- minimateplus/__init__.py: export SerialTransport and TcpTransport.
- sfm/server.py: add `host` / `tcp_port` query params to all device endpoints.
  New _build_client() helper selects TCP or serial transport automatically.
  OSError (connection refused, timeout) now returns HTTP 502.
- docs/instantel_protocol_reference.md: add changelog entry and full §14
  (TCP/Modem Transport) documenting confirmed transparent passthrough, no ENQ
  on connect, modem forwarding delay, call-up vs ACH modes, and hardware note
  deprecating Raven X in favour of RV55/RX55.

Usage: GET /device/info?host=<modem_ip>&tcp_port=12345
2026-03-31 00:44:50 -04:00
claude f8f590b19b sfm first build 2026-03-30 23:23:29 -04:00
claude 58a35a3afd feat: add large BW→S3 write frame checksum validation in s3_parser 2026-03-13 17:53:11 -04:00
claude 99d66453fe feat: enhance session completeness tracking in s3_analyzer and seismo_lab 2026-03-11 18:48:42 -04:00
claude faa869d03b doc: protocol ref updated to v0.20 2026-03-09 19:02:53 -04:00
claude fa9873cf4a doc: §2, §10, Appendix C | **MILESTONE — Link-layer grammar formally confirmed.** 2026-03-04 17:42:15 -05:00