perf(micromate): 16 KB per request, not 1024 -- 14x fewer round trips
The chunk-size ceiling, measured on UM20147 with every size checked byte-for-byte against a known-good download: 1024 / 2048 / 4096 / 8192 / 16384 -> served in full 32768 / 65535 -> SILENTLY CLAMPED to 16384 The clamp is the important part: a 32,768 B request returns 16,384 B of perfectly good data and no error. Nothing in the response says it was truncated; the length is the only signal. THAT DICTATES HOW THE LOOP MUST BE WRITTEN. read_event_file() now tracks its offset by BYTES RECEIVED rather than striding by chunk index. A fixed stride would either fail on a clamp or skip the bytes it never collected; tracking what actually arrived makes a clamp cost one extra request, and makes the loop self-correcting against any short response -- precisely the failure mode that has bitten the Series III side repeatedly. Consequence of that change, recorded because it reverses an earlier decision: a short chunk is NO LONGER AN ERROR. It used to raise, on the principle that a silently short event is this codebase's recurring bug. But the device returns short legitimately, and the real protection is the offset arithmetic plus the final total-length check -- which still raises on a genuinely truncated event. The test was rewritten rather than deleted, and says why. CHUNK_SIZE = 16384. For UM20147's events: 4,796 B goes 5 requests -> 1, 30,230 B goes 30 -> 2, and 72,560 B goes 71 -> 5. Over cellular at ~0.65 s per round trip that is ~46 s -> ~3.2 s on the large one. Measured on one unit over USB, so THOR_CHUNK_SIZE = 1024 stays available and the replay tests pin it -- reproducing THOR's exact traffic is one argument away, and client.download_event()/get_event() take chunk_size for a link where large responses are not surviving. Full suite: 472 passed, 16 pre-existing failures unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
+11
-4
@@ -474,16 +474,23 @@ class MicromateClient:
|
||||
ref.peak_vector_sum_ips = struct.unpack(
|
||||
">f", c[tran - _REC_PVS_BACK: tran - _REC_PVS_BACK + 4])[0]
|
||||
|
||||
def download_event(self, ref: MicromateEventRef) -> bytes:
|
||||
def download_event(self, ref: MicromateEventRef, *,
|
||||
chunk_size: Optional[int] = None) -> bytes:
|
||||
"""The raw `.IDFW`/`.IDFH` bytes, exactly as THOR would have stored them.
|
||||
|
||||
Feeds `micromate.idf_file.read_idf_file()` and `/db/import/idf_file`
|
||||
unchanged — no new codec work is needed for a directly downloaded event.
|
||||
|
||||
`chunk_size` defaults to the measured device ceiling (16,384 B), which is
|
||||
16x THOR's 1024 and therefore ~14x fewer round trips on a large event.
|
||||
Pass `THOR_CHUNK_SIZE` to reproduce THOR's wire traffic exactly, or a
|
||||
smaller value on a link where big responses are not surviving.
|
||||
"""
|
||||
return self._proto.read_event_file(ref.key, ref.size)
|
||||
kw = {} if chunk_size is None else {"chunk_size": chunk_size}
|
||||
return self._proto.read_event_file(ref.key, ref.size, **kw)
|
||||
|
||||
def get_event(self, ref: MicromateEventRef, *, verify: bool = True,
|
||||
tolerance: float = 0.01):
|
||||
tolerance: float = 0.01, chunk_size: Optional[int] = None):
|
||||
"""Download and decode one event.
|
||||
|
||||
Returns the codec's `IdfReadResult`. Needs `ref.record_type`, since
|
||||
@@ -510,7 +517,7 @@ class MicromateClient:
|
||||
f"event {ref.key_hex}: record_type is unknown, so the codec "
|
||||
f"cannot be dispatched. Use list_events(with_records=True)."
|
||||
)
|
||||
blob = self.download_event(ref)
|
||||
blob = self.download_event(ref, chunk_size=chunk_size)
|
||||
|
||||
import tempfile
|
||||
from .idf_file import read_idf_file
|
||||
|
||||
Reference in New Issue
Block a user