Two improvements to eliminate the ~2-min-per-event wait and unnecessary
full-event-list download when only one event is requested:
1. protocol.py: pass timeout=10.0 to _recv_one in the 5A chunk loop.
Device responds within ~1s per chunk; 10s gives a safe 10x buffer.
End-of-stream detection (raw_bytes=1) now fires in 10s instead of 120s,
cutting ~110s of dead wait per event.
2. client.py: add stop_after_index parameter to get_events(). When set,
iteration stops immediately after the target event is collected — no
further 0A/1E/0C/5A/1F cycles for events the caller doesn't need.
3. server.py: pass stop_after_index=index to both /device/event/{idx}
and /device/event/{idx}/waveform endpoints so a single-event request
only downloads that one event.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the device finishes streaming waveform data, it sends a single
partial byte (raw_bytes=1) rather than a complete A5 frame, then goes
silent for the full 120s timeout.
Observed: 35 chunks succeed, chunk 36 times out with raw_bytes=1 —
identical for both events in the test run.
Fix: on TimeoutError, if bytes_fed > 0 and we already have collected
frames, treat it as natural end-of-stream and break to the termination
step rather than propagating the exception. True transport failures
(raw_bytes=0, no prior frames) still raise.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wraps each recv_one call in a try/except TimeoutError, logging:
- On timeout: chunk_num, counter, raw bytes_fed (distinguishes "device
silent" from "device sent unparseable bytes")
- On success: chunk_num, page_key, data_len, contains_Project flag
Parser is explicitly reset before each chunk recv so bytes_fed is
accurate per-chunk rather than cumulative. Helps identify exactly which
chunk fails and whether the device is responding at all.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added `_decode_a5_waveform()` to parse SUB 5A frames into per-channel time-series data.
- Introduced `download_waveform(event)` method in `MiniMateClient` to fetch full waveform data.
- Updated `Event` model to include new fields: `total_samples`, `pretrig_samples`, `rectime_seconds`, and `_waveform_key`.
- Enhanced documentation in `CHANGELOG.md` and `instantel_protocol_reference.md` to reflect new features and confirmed protocol details.
Add read_bulk_waveform_stream() to MiniMateProtocol and wire it into
get_events() so each event gets authoritative client/operator/sensor_location
from the A5 frames recorded at event-time, not the current compliance config.
- framing.py: bulk_waveform_params() and bulk_waveform_term_params() helpers
(probe/chunk params and termination params for SUB 5A, confirmed from
1-2-26 BW TX capture)
- protocol.py: read_bulk_waveform_stream(key4, stop_after_metadata=True) —
probe + chunk loop (counter += 0x0400), early stop when b"Project:" found
(A5[7] of 9), then sends termination at offset=0x005A
- client.py: _decode_a5_metadata_into() needle-searches concatenated A5 frame
data for Project/Client/User Name/Seis Loc/Extended Notes; get_events() now
calls SUB 5A after each SUB 0C, overwriting project_info with event-time values
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The channel block is in frame C data (cfg[44:1071]) not deep in a
hypothetical frame D section. The offset-1000 assumption was wrong —
searching from 44 lets us find it while unit string validation still
prevents false positives.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Temporary: log tran_pos, surrounding bytes, and exact unit string check
results at WARNING level so we can see why trigger/alarm/max_range are
still null even with the full 2126-byte cfg.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old key was (page_key, chunk_len) which would incorrectly drop a second
config section that has the same length as the first (e.g. current-config
vs event-time-config when settings haven't changed).
New key is the full chunk bytes — only truly byte-identical chunks are
dropped. Different data that happens to share page_key and length now
comes through correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DeviceInfo.event_count: Optional[int] = None (new field in models.py)
- connect() now calls proto.read_event_index() after compliance config and
stores the decoded count in device_info.event_count
- _serialise_device_info() exposes event_count in /device/info and /device/events
JSON responses
event_count is decoded from uint32 BE at offset +3 of the 88-byte F7 payload
(🔶 inferred — needs live device confirmation against a multi-event device).
Any ProtocolError from the index read is caught and logged; event_count stays
None rather than failing the whole connect().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The channel block is only present in the full ~2126-byte cfg (when frame D
delivers correctly rather than duplicating frame B's page). Layout per §7.6:
[00 00][max_range f32][00 00][trigger f32]["in.\0"][alarm f32]["/s\0\0"][00 01][label]
Relative offsets from the "Tran" label position (label-24/label-18/label-10)
are validated by checking the unit strings "in.\0" at label-14 and "/s\0\0"
at label-6 before reading the floats. Guard against "Tran2" false-match.
When frame D duplicates, cfg is ~1071 bytes and tran_pos search returns a hit
without the unit string sentinels — we log the miss and leave fields None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two-step probe+fetch for SUB 08 (EVENT_INDEX), returning the raw 88-byte
(0x58) index block. SUB_EVENT_INDEX and DATA_LENGTHS[0x08]=0x58 were
already registered — this just wires the method that calls them.
Docstring notes the partially-decoded layout (event count at +3 as uint32 BE,
timestamps at +7) pending live device confirmation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
record_time (float32_BE) and sample_rate (uint16_BE) both validated
against live device across normal / fast / faster modes and multiple
record time settings. Diagnostic scaffolding no longer needed.
Fixed a 1-byte offset jitter that produced garbage values when the
device was set to "faster" (4096 Sa/s) mode.
Root cause: 4096 = 0x1000, so the sample_rate bytes in the raw S3
frame are `10 10 00` (DLE-escaped). After DLE unstuffing → `10 00`
(2 bytes vs 3 for 1024/2048), making frame C 1 byte shorter and
shifting all subsequent field offsets by -1.
Fix: locate the stable 10-byte anchor `01 2c 00 00 be 80 00 00 00 00`
(max-record-limit constant + first two alarm-level floats) and read:
sample_rate = uint16_BE at anchor - 2
record_time = float32_BE at anchor + 10
Offline-validated against all five logged hex dumps (1071 and 1070
byte cfg, record times 3/5/8 s, sample rates 1024 and 4096):
all five: correct values with anchor approach.
Both offsets identified from _cfg_diagnostic scan on BE11529:
cfg[64] float32_BE = 3.0 → record_time in seconds
cfg[52] uint16_BE = 1024 → sample_rate in Sa/s
Values are plausible but NOT yet validated by changing device settings
and re-reading. Marked as ✅ candidate in docstring — confirm and
remove the ⚠️ note once a device-side change is observed here.
BE11529 sometimes returns frame D with page_key=0x0000 (44 bytes),
identical to the frame B response, inflating cfg to ~1115 bytes and
mis-aligning all field offsets. Track (page_key, chunk_size) pairs
and drop any repeat before appending to the running config buffer.
- setup_name was broken: _find_string_after(b"Standard Recording Setup")
returned what comes AFTER the string (i.e. "Project:"), not the name
itself. Fixed by searching for the first long (>=8 char) ASCII string
in cfg[40:250] with _find_first_string().
- record_time offset 0x28 was wrong (that location holds "(L)", a unit
label string). Disabled for now to avoid returning garbage; correct
offset will be determined from _cfg_diagnostic() output.
- Added _cfg_diagnostic(): logs all strings and all plausible float32/uint16
values across the full cfg so record_time and sample_rate offsets can be
pinpointed from a single device run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reverse-engineered the full Blastware 4-frame sequence for SUB 1A:
A (probe): offset=0x0000, params[7]=0x64
B (data req 1): offset=0x0400, params[2]=0x00, params[7]=0x64 → bytes 0..1023
C (data req 2): offset=0x0400, params[2]=0x04, params[7]=0x64 → bytes 1024..2047
D (data req 3): offset=0x002A, params[2]=0x08, params[7]=0x64 → bytes 2048..2089
We were only sending A+D and getting 44 bytes (the last chunk).
Now sends B, C, D in sequence; each E5 response has 11-byte echo header
stripped, and chunks are concatenated. Devices that return all data in
one frame (BE18189 style) are handled — timeouts on B/C are skipped
gracefully and data from D still accumulates.
Total expected: 0x0400 + 0x0400 + 0x002A = 0x082A = 2090 bytes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BE11529 sends the compliance config (SUB 1A / E5 response) as a stream
of small frames (~44 bytes each) rather than one large frame like BE18189.
Changes:
- read_compliance_config(): loop calling _recv_one() until 0x082A bytes
accumulated or 2s inter-frame gap; first frame strips 11-byte echo header,
subsequent frames logged in full for structure analysis
- _recv_one(): add reset_parser=False option to preserve parser state and
_pending_frames buffer between consecutive reads from one device response;
also stash any extra frames parsed in a single TCP chunk so they are not lost
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The heuristic offsets for trigger/alarm levels were causing struct unpack
errors. These fields require detailed field mapping from actual E5 captures
to determine exact byte positions relative to channel labels.
For now, skip extraction and leave trigger_level_geo/alarm_level_geo as None.
This prevents the '500 Device error: bytes must be in range(0, 256)' error.
Once we capture an E5 response and map the exact float positions, we can
re-enable this section with correct offsets.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
claude