perf(micromate): 16 KB per request, not 1024 -- 14x fewer round trips
The chunk-size ceiling, measured on UM20147 with every size checked byte-for-byte against a known-good download: 1024 / 2048 / 4096 / 8192 / 16384 -> served in full 32768 / 65535 -> SILENTLY CLAMPED to 16384 The clamp is the important part: a 32,768 B request returns 16,384 B of perfectly good data and no error. Nothing in the response says it was truncated; the length is the only signal. THAT DICTATES HOW THE LOOP MUST BE WRITTEN. read_event_file() now tracks its offset by BYTES RECEIVED rather than striding by chunk index. A fixed stride would either fail on a clamp or skip the bytes it never collected; tracking what actually arrived makes a clamp cost one extra request, and makes the loop self-correcting against any short response -- precisely the failure mode that has bitten the Series III side repeatedly. Consequence of that change, recorded because it reverses an earlier decision: a short chunk is NO LONGER AN ERROR. It used to raise, on the principle that a silently short event is this codebase's recurring bug. But the device returns short legitimately, and the real protection is the offset arithmetic plus the final total-length check -- which still raises on a genuinely truncated event. The test was rewritten rather than deleted, and says why. CHUNK_SIZE = 16384. For UM20147's events: 4,796 B goes 5 requests -> 1, 30,230 B goes 30 -> 2, and 72,560 B goes 71 -> 5. Over cellular at ~0.65 s per round trip that is ~46 s -> ~3.2 s on the large one. Measured on one unit over USB, so THOR_CHUNK_SIZE = 1024 stays available and the replay tests pin it -- reproducing THOR's exact traffic is one argument away, and client.download_event()/get_event() take chunk_size for a link where large responses are not surviving. Full suite: 472 passed, 16 pre-existing failures unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
This commit is contained in:
@@ -732,14 +732,44 @@ THOR's chunk size. The offset field is a **uint16**, so the structural ceiling i
|
||||
**65,535 bytes per request** — which would make that same event **2 requests,
|
||||
~1.3 seconds**.
|
||||
|
||||
⚠ The *device's* ceiling is not yet known, only that it is at least 4,380. It may
|
||||
be bounded by an internal buffer well below 65,535. `scratch/mm_stream_probe.py`
|
||||
walks ascending sizes and checks each against a known-good chunk-loop download,
|
||||
so a pass means byte-identical output rather than a plausible length.
|
||||
#### ✅ The ceiling is 16,384 bytes, and over it the device CLAMPS SILENTLY
|
||||
|
||||
⚠ `micromate/protocol.py` still uses **1024** — THOR's value, the one with
|
||||
captures behind it. Raising it is a one-constant change once the ceiling is
|
||||
measured, and it should not be raised on inference.
|
||||
Measured on UM20147, each size checked byte-for-byte against a known-good
|
||||
download:
|
||||
|
||||
| requested | served | |
|
||||
|---|---|---|
|
||||
| 1,024 | 1,024 | ✅ |
|
||||
| 2,048 | 2,048 | ✅ |
|
||||
| 4,096 | 4,096 | ✅ |
|
||||
| 8,192 | 8,192 | ✅ |
|
||||
| **16,384** | **16,384** | ✅ |
|
||||
| 32,768 | **16,384** | ⚠ clamped |
|
||||
| 65,535 | **16,384** | ⚠ clamped |
|
||||
|
||||
⚠ **The clamp is silent and the bytes are correct.** A 32,768-byte request
|
||||
returns 16,384 bytes of perfectly good data and no error. There is nothing in the
|
||||
response to say it was truncated — the length is the only signal.
|
||||
|
||||
**That is what dictates how the download loop must be written.** A loop striding
|
||||
by a fixed chunk size would either fail on a clamp or, worse, skip the bytes it
|
||||
never collected. `read_event_file()` therefore tracks its offset by **bytes
|
||||
received**, which makes a clamp cost one extra request rather than corrupting the
|
||||
file — and makes the loop self-correcting against any short response, which is
|
||||
precisely the failure mode that has bitten the Series III side repeatedly.
|
||||
|
||||
`CHUNK_SIZE` is now **16,384**. For UM20147's events:
|
||||
|
||||
| event | THOR's 1024 | 16,384 | cellular |
|
||||
|---|---|---|---|
|
||||
| 4,796 B | 5 requests | **1** | ~3.3 s → ~0.7 s |
|
||||
| 30,230 B | 30 | **2** | ~20 s → ~1.3 s |
|
||||
| 72,560 B | 71 | **5** | ~46 s → ~3.2 s |
|
||||
|
||||
⚠ **Measured on one unit, over USB.** `THOR_CHUNK_SIZE = 1024` remains available
|
||||
and the replay tests pin it, so reproducing THOR's exact traffic is still one
|
||||
argument away. A unit that clamps lower than 16,384 costs extra requests, not a
|
||||
failure.
|
||||
|
||||
**Implement THOR's chunked form.** It is verified byte-exact across six events
|
||||
and five distinct sizes, and it is what the firmware runs every day.
|
||||
|
||||
Reference in New Issue
Block a user