docs: record the 5A page-boundary bug, and assess SFM as a tool
Two things Brian asked for after the BE12599 work. The known bug: the 5A walk discards the key's page byte, so once a unit has recorded more than 64 KB since its last erase, an event spanning the boundary reads an end_offset behind its own start. The chunk loop then fetches nothing and TERM packs a negative offset_word, which is the 500. Reproduced on BE12599. It hid this long because every capture the walk was verified against came from a freshly-erased BE11529 — all three confirmed TERM examples sit inside page 0x11. Prod is unaffected; it ingests complete files and never runs this walk. The status doc exists because "is SFM reliable?" has three different answers depending on which tier is meant. The codec library and the data side are production — verified per-sample at scale, carrying Terra-View daily. The device side is emergency-grade: it works, but it is synchronous, unauthenticated, and thinly tested. The lab is research artifacts. Most confusion comes from answering for the wrong tier. It covers all three of what Brian asked for: maturity per capability, an operator-facing "what to use when" (the cheap probes are cheap and the event walk is not), the known-issues table, and the gap analysis. That gap is mostly auth, async and guardrails — not protocol work. The protocol is the finished part. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
This commit is contained in:
@@ -61,6 +61,24 @@ Read this first when picking the project back up.
|
||||
4th-decimal tick and are Thor's own rounding — no single linear LSB can
|
||||
reproduce every printed value (the constraints are infeasible by 7e-5
|
||||
relative), so do NOT retune `_GEO_LSB_IPS`.
|
||||
- **⚠ KNOWN BUG — the 5A walk breaks once a unit's buffer crosses 64 KB.**
|
||||
`parse_strt_end_offset()` returns only `(end_key[2] << 8) | end_key[3]`,
|
||||
discarding the key's page byte. An event starting at `0x0111F2A2` and ending
|
||||
at `0x0112_1010` therefore reads `end_offset = 0x1010` — *behind* its own
|
||||
start. The chunk loop then exits before fetching anything and TERM computes
|
||||
a negative `offset_word`, which `struct.pack(">H", ...)` rejects: the
|
||||
`/device/events` walk 500s. Reproduced on BE12599 (2026-09-19), which had
|
||||
78 KB stored and had rolled into page `0x12`.
|
||||
**Why it hid so long:** every 5A capture the walk was verified against came
|
||||
from a freshly-erased BE11529 — all three confirmed TERM examples in
|
||||
`framing.py` (`0x1ABE`, `0x21F2`, `0x417E`) sit inside page `0x11`. Prod is
|
||||
unaffected: it ingests complete files via BW ACH, never this walk.
|
||||
**Fixing it has two layers** — the arithmetic (`if end < start: end +=
|
||||
0x10000`) stops the crash and bounds the loop correctly; carrying the page
|
||||
byte through the chunk requests (`params[1]` 0x11 -> 0x12, counter rolling
|
||||
over) needs a BW capture of a spanning event first. Do not ship layer one
|
||||
alone without a loud truncation warning — a silently short event is the
|
||||
failure mode this codec has been bitten by repeatedly.
|
||||
- **Open, not blocking:** 14 sensitive-range files show an exact 8x
|
||||
(= 10.0/1.25) units discrepancy; `scripts/backfill_sidecars.py --force` also
|
||||
inserts DB rows for store files that have none (one-time per store) and the
|
||||
|
||||
Reference in New Issue
Block a user