Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  serversdownandClaude Opus 5
|
922c304734 | ||
|
serversdownandClaude Opus 5
|
5f81fb0217 |
+68
-3
@@ -46,6 +46,50 @@ All notable changes to seismo-relay are documented here.
|
||||
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
|
||||
files via the watcher path and never runs this walk.
|
||||
|
||||
- **The Micromate (Series IV) live wire protocol, reverse-engineered end to
|
||||
end** — `docs/micromate_protocol_reference.md`. Worked out against a bench
|
||||
UM12947 over USB and a recording relay, with THOR driving every write so that
|
||||
no command has ever been originated against a unit by this project. **A
|
||||
Micromate answers Series III command frames**, with three framing differences:
|
||||
responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
|
||||
or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
|
||||
**uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
|
||||
`SUB 0x1A` by 47x. Read path, event chain, and `SUB 0x5A` streaming the
|
||||
`.IDFW` file verbatim are all confirmed.
|
||||
- **Series IV setup management, fully mapped.** `0x41` reads the active setup
|
||||
name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
|
||||
write it back. **Setups are read-modify-write** — the written block is the
|
||||
read block, 91% byte-identical at a fixed 11-byte shift. `0xDA` **creates**
|
||||
files rather than only overwriting, confirmed on the unit's own screen, and an
|
||||
overwrite is protocol-identical to a create: no handshake, no warning, and no
|
||||
protection even over the *active* setup of a monitoring unit.
|
||||
- **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
|
||||
records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
|
||||
half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
|
||||
name. Verified entry-for-entry against the operator's own THOR screen.
|
||||
- **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
|
||||
`0x8D`/`0x8E` write. This **retracts** an earlier conclusion in the same
|
||||
document that no such command existed; that was inferred from absent firmware
|
||||
strings and was wrong.
|
||||
- **Monitoring control and per-event delete.** `0x96`/`0x97` start and stop as
|
||||
on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
|
||||
for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
|
||||
constant. Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
|
||||
which is safer than Series III's erase-everything. `SUB 0x1C` also carries the
|
||||
device clock.
|
||||
- **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
|
||||
identically as "disconnected": refused, connect timeout (the silent-drop
|
||||
signature of a trusted-IP whitelist), **connected but no reply** (the modem
|
||||
answered and the unit did not), and replied. Each verdict names what to try
|
||||
next. `--slots N` tests single-session modem behaviour.
|
||||
- **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
|
||||
decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
|
||||
`onewaydev`) driven by a control file. No pyserial; stdlib `termios` only.
|
||||
- **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
|
||||
— a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
|
||||
all, since it scans for `DLE+STX`), byte-exact capture recovery from a
|
||||
`socat -x` relay log, and a serial-port stand-in that answers as a unit.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Connecting to a unit no longer walks its event chain.** `/device/events`
|
||||
@@ -58,11 +102,32 @@ All notable changes to seismo-relay are documented here.
|
||||
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
|
||||
event it has stored.
|
||||
|
||||
- **Recorded what THOR actually does on the wire**, measured rather than assumed.
|
||||
A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
|
||||
MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
|
||||
`MONITOR_STATUS` check at 60 s. The **status interval is honoured; the
|
||||
connection interval is not** — it sets `(status / connection) - 1` checks per
|
||||
cycle, so equal values yield *zero* cheap checks and every connection becomes
|
||||
the expensive one.
|
||||
- **Two THOR defects reproduced with timestamps.** After a connection drops
|
||||
mid-download it retries **once**, stops polling entirely and **never resumes**,
|
||||
while displaying `Connected` for as long as it is left alone — and `Idle` for a
|
||||
unit that is actively recording. Separately, THOR's own log shows a
|
||||
**subscription leak**: one logical event dispatched to a growing number of
|
||||
handlers, **1 to 12 over ten hours** of uptime, consistent with the field
|
||||
report that only a restart recovers it.
|
||||
- **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
|
||||
either firmware line. A PL2303 cable (Benfei) leaves a unit with no working
|
||||
modem port; an FTDI cable (Sabrent) works. Both are in circulation and
|
||||
indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
|
||||
|
||||
### Migration
|
||||
|
||||
**None.** Frontend and documentation only — no codec, waveform-store or DB
|
||||
change, no schema change, and no `TOOL_VERSION` bump. The webapp is served
|
||||
from the image, so the change appears after the next `sfm` rebuild.
|
||||
**None.** Frontend, documentation and bench tooling only — no codec,
|
||||
waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump. The
|
||||
webapp is served from the image, so its changes appear after the next `sfm`
|
||||
rebuild. The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only;
|
||||
nothing under `sfm/`, `minimateplus/` or `micromate/` was touched.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -10,38 +10,10 @@ pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
|
||||
|
||||
---
|
||||
|
||||
## Where things stand (updated 2026-09-26)
|
||||
## Where things stand (updated 2026-08-28)
|
||||
|
||||
Read this first when picking the project back up.
|
||||
|
||||
- **The Series-4 LIVE wire protocol is reverse-engineered end to end
|
||||
(2026-09-25).** `docs/micromate_protocol_reference.md` is the Series-4
|
||||
Rosetta Stone, sibling to `instantel_protocol_reference.md`. **A Micromate
|
||||
answers Series III command frames** — three framing differences: responses
|
||||
have **no leading `DLE`** (a bare `STX`), `payload[1]` is `0xC5` (Blastware
|
||||
firmware) or `0x03` (Thor firmware) rather than `0x10`, and the data length
|
||||
is a **uint16 BE at `payload[8:10]`** (as a byte it under-reads `SUB 0x1A`
|
||||
by 47x). Read path, event chain, setups, scheduler, monitoring control and
|
||||
per-event delete are all mapped; **the inbound call-home session is the only
|
||||
protocol unknown left.**
|
||||
⚠ **No command has ever been originated against a unit by this project.**
|
||||
Every write was performed by THOR while we recorded. That line is worth
|
||||
keeping.
|
||||
⚠ `micromate/` still has **no live client** — it is codec-only. The
|
||||
`minimateplus/` stack (transport/framing/protocol/client) has no Series-4
|
||||
counterpart yet. `minimateplus.transport` is protocol-agnostic and reusable.
|
||||
- **Bench tooling for device diagnosis (2026-09-25).** `bridges/mm_probe.py`
|
||||
distinguishes the four faults THOR reports identically as "disconnected"
|
||||
(refused / connect timeout / **connected but no reply** / replied) and names
|
||||
what to try next. `bridges/mm_link.py` is a stand-in for a cellular modem
|
||||
with a decoded log and fault injection. `scratch/mm_frame_parse.py` exists
|
||||
because **`S3FrameParser` cannot see Micromate responses at all** — it scans
|
||||
for `DLE+STX`, which never appears in Series-4 traffic.
|
||||
- **A Micromate's USB-A host port drives FTDI and CDC-ACM only** — no Prolific,
|
||||
in either firmware line. TMI buys both Sabrent (FTDI) and Benfei (PL2303)
|
||||
cables and they are indistinguishable by eye. A PL2303 cable leaves a unit
|
||||
with **no working modem port at all**; identify by `lsusb` VID, `0403` vs
|
||||
`067b`. This accounted for a unit that could not be deployed.
|
||||
- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
|
||||
archive decodes **14,338 / 14,338** paired files exactly against their
|
||||
preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
|
||||
@@ -120,9 +92,6 @@ Read this first when picking the project back up.
|
||||
**v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
|
||||
fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
|
||||
files are archive-only and were never ingested).
|
||||
✅ **The v0.30.0 Series-4 backfill HAS been run on prod (2026-09-25).** Every
|
||||
stored Series-4 geophone value was ~3.3% low until then; that is corrected and
|
||||
the job does not need repeating.
|
||||
- **The "offset" hardware fault has its own journal** --
|
||||
`docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
|
||||
**persistent** — it stays until the geophone is serviced. Detect it with
|
||||
@@ -134,18 +103,7 @@ Read this first when picking the project back up.
|
||||
`SUB 0x0E` (unimplemented), which may carry those very numbers.
|
||||
|
||||
|
||||
When new information about a protocol is discovered, record it in the matching
|
||||
reference **in addition to** this document:
|
||||
|
||||
| series | document |
|
||||
|---|---|
|
||||
| Series III (MiniMate Plus / BlastMate) | `docs/instantel_protocol_reference.md` |
|
||||
| **Series IV (Micromate / THOR)** | **`docs/micromate_protocol_reference.md`** |
|
||||
| Thor IDF file format | `docs/idf_protocol_reference.md` |
|
||||
|
||||
Both protocol references carry retractions in place rather than deleting what
|
||||
turned out to be wrong — that convention has already saved re-deriving the same
|
||||
mistakes twice, so keep it.
|
||||
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
|
||||
|
||||
---
|
||||
|
||||
@@ -318,15 +276,8 @@ minimateplus/ ← Python client library (primary focus)
|
||||
|
||||
sfm/server.py ← FastAPI REST server exposing device data over HTTP
|
||||
seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs)
|
||||
bridges/
|
||||
mm_probe.py ← name the fault behind a dead unit (4 verdicts, read-only)
|
||||
mm_link.py ← bench stand-in for a cellular modem, with fault injection
|
||||
ach_mitm.py ← TCP relay for recording a Series-3 ACH session
|
||||
|
||||
docs/
|
||||
instantel_protocol_reference.md ← Series III protocol spec ("the Rosetta Stone")
|
||||
micromate_protocol_reference.md ← Series IV protocol spec + THOR's measured behaviour
|
||||
idf_protocol_reference.md ← Thor IDF file format
|
||||
instantel_protocol_reference.md ← reverse-engineered protocol spec ("the Rosetta Stone")
|
||||
CHANGELOG.md ← version history
|
||||
```
|
||||
|
||||
|
||||
+10
-23
@@ -134,30 +134,17 @@ def probe(host: str, port: int, timeout: float) -> int:
|
||||
step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
|
||||
print("\nverdict: the MODEM answered but the unit did not.")
|
||||
print(" TCP is fine end to end — something accepted the connection.")
|
||||
print(" What is missing is the serial side. Two quite different")
|
||||
print(" causes produce this, and they are NOT distinguishable from")
|
||||
print(" here:")
|
||||
print("\n 1. SOMEONE ELSE HOLDS THE SESSION. These modems bridge ONE")
|
||||
print(" TCP session to serial at a time. A second connection is")
|
||||
print(" accepted and then simply not forwarded. Confirmed 2026-09-26:")
|
||||
print(" with THOR connected this probe saw exactly this; the moment")
|
||||
print(" THOR disconnected the same probe returned the serial number.")
|
||||
print(" ** Check whether THOR (or anything else) has the unit first. **")
|
||||
print("\n 2. The serial path is genuinely broken — a stale session the")
|
||||
print(" modem never released, a cable the unit cannot enumerate, or")
|
||||
print(" a unit that is off.")
|
||||
print(" What is missing is the serial side. Most likely the modem is")
|
||||
print(" not forwarding to its serial port, which is what a wedged")
|
||||
print(" transparent-TCP session looks like: the slot is held by a")
|
||||
print(" connection that never closed.")
|
||||
print("\n Try, in order:")
|
||||
print(" 1. Disconnect any other client and re-probe. If it answers,")
|
||||
print(" it was contention, not a fault.")
|
||||
print(" 2. The cable's chipset. A Micromate drives FTDI and CDC-ACM")
|
||||
print(" only — a Prolific PL2303 gives it no serial port at all.")
|
||||
print(" lsusb: FTDI is 0403, Prolific 067b.")
|
||||
print(" 3. Power-cycle the UNIT with the cable attached (hold power")
|
||||
print(" 5 s, through the two-stage prompt). Its USB host rescans")
|
||||
print(" on cold boot; it may not on hot-swap.")
|
||||
print(" 4. AirLink OS -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot indefinitely. 2 minutes is the")
|
||||
print(" value this project standardised on.")
|
||||
print(" 1. ACEmanager -> TCP Idle Timeout. If 0/disabled, a stale")
|
||||
print(" session holds the slot forever. 2 minutes is the value")
|
||||
print(" this project standardised on.")
|
||||
print(" 2. Reboot the modem. If that fixes it, the modem was")
|
||||
print(" holding state and the timeout is the permanent fix.")
|
||||
print(" 3. Check the unit's own screen — serial cable, power.")
|
||||
sock.close()
|
||||
return 5
|
||||
|
||||
|
||||
@@ -1,291 +0,0 @@
|
||||
# Spec — a live client for Series IV (Micromate)
|
||||
|
||||
Drafted 2026-09-26, ahead of implementation. The protocol work is finished; this
|
||||
is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
|
||||
run.
|
||||
|
||||
**Read that document first.** Everything here assumes it, and every constant
|
||||
below is sourced from it rather than restated with justification.
|
||||
|
||||
---
|
||||
|
||||
## Goal and scope
|
||||
|
||||
`micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
|
||||
writers. There is no way to talk to a unit. This adds the live half, mirroring
|
||||
`minimateplus/`.
|
||||
|
||||
**In scope, first pass:**
|
||||
|
||||
- connect over TCP (a field modem) or serial/USB (a bench unit)
|
||||
- identify a unit, read its state, clock, memory and setups
|
||||
- walk the event chain and download events
|
||||
- return `Event` objects the existing codec already understands
|
||||
|
||||
**Explicitly out of scope, first pass:**
|
||||
|
||||
- ⚠ **Any write.** Setups, schedules, call-home config, monitoring start/stop,
|
||||
and per-event delete are all mapped, and none of them will be implemented
|
||||
here. **No command has ever been originated against a unit by this project**
|
||||
— every write observed was performed by THOR while we recorded. Keeping that
|
||||
true through the read client is deliberate: it means the first thing we ever
|
||||
send to a customer's instrument is a decision someone made on purpose, not a
|
||||
side effect of a client that happened to grow a method.
|
||||
- the inbound call-home session — still the one protocol unknown
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
micromate/
|
||||
framing.py NEW frame building, response parsing, checksum
|
||||
protocol.py NEW one method per wire command, returns raw payloads
|
||||
client.py NEW high-level API, returns models
|
||||
idf_file.py (existing — decodes what 0x5A returns, unchanged)
|
||||
models.py (existing — extend, do not fork)
|
||||
```
|
||||
|
||||
**Transport is reused, not rewritten.** `minimateplus/transport.py` is
|
||||
byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
|
||||
`TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
|
||||
habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller. Import it.
|
||||
|
||||
⚠ Do **not** import `minimateplus.framing`. The two framings differ in ways
|
||||
that look small and are not, and a shared module would accumulate `if series ==`
|
||||
branches until neither case is readable.
|
||||
|
||||
---
|
||||
|
||||
## `micromate/framing.py`
|
||||
|
||||
### Requests
|
||||
|
||||
Series IV accepts Series III request frames unmodified. The simplest correct
|
||||
implementation re-exports the builder rather than duplicating it:
|
||||
|
||||
```python
|
||||
from minimateplus.framing import build_bw_frame # requests are identical
|
||||
```
|
||||
|
||||
⚠ One open question, flagged in the protocol reference and **not** settled:
|
||||
whether `0x10` bytes inside request *params* need stuffing. No probe we sent
|
||||
carried one. Until it is settled, assert on it rather than guessing:
|
||||
|
||||
```python
|
||||
def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
|
||||
if 0x10 in params:
|
||||
raise NotImplementedError(
|
||||
"params containing 0x10 — stuffing rule unconfirmed; see "
|
||||
"micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'"
|
||||
)
|
||||
return build_bw_frame(sub, offset, params)
|
||||
```
|
||||
|
||||
That turns an unknown into a loud failure instead of a corrupt frame.
|
||||
|
||||
### Responses — where Series III's parser cannot follow
|
||||
|
||||
| | Series III | Micromate |
|
||||
|---|---|---|
|
||||
| frame start | `DLE STX` | **bare `STX`** |
|
||||
| `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
|
||||
| destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |
|
||||
|
||||
The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
|
||||
it scans for `DLE+STX`, which never appears.
|
||||
|
||||
The third is a genuine **simplification** — no inner-frame carve-out. Validated
|
||||
by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
|
||||
four candidate destuffing rules were tried, and only this one makes all frames
|
||||
validate.
|
||||
|
||||
### Checksum
|
||||
|
||||
```python
|
||||
def checksum(payload: bytes) -> int:
|
||||
return sum(b for b in payload if b != 0x10) & 0xFF
|
||||
```
|
||||
|
||||
The DLE-aware variant, same as Series III's `5A` and write frames — not the plain
|
||||
SUM8 of ordinary Series III reads.
|
||||
|
||||
### ⚠ The SUB byte can be escaped
|
||||
|
||||
When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
|
||||
Reading it positionally without destuffing reports `0x10`. This bit once
|
||||
already — `SUB 0x02` was logged as `SUB_10` for an afternoon. Destuff first,
|
||||
then index.
|
||||
|
||||
### Response shape
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class MicromateFrame:
|
||||
sub: int # response SUB; request = 0xFF - sub
|
||||
flags: int # 0xC5 Blastware line, 0x03 Thor line
|
||||
page_hi: int
|
||||
page_lo: int
|
||||
data: bytes # payload[5:], checksum stripped
|
||||
checksum_valid: bool
|
||||
|
||||
@property
|
||||
def firmware_line(self) -> str: # "blastware" | "thor" | "unknown"
|
||||
@property
|
||||
def declared_length(self) -> int: # uint16 BE at data[3:5] (= payload[8:10])
|
||||
```
|
||||
|
||||
⚠ **`declared_length` is a uint16 BE.** Read as a single byte it under-reads
|
||||
`SUB 0x1A` by 47x — 44 against a true 2092. This is the single most expensive
|
||||
mistake available in this protocol and it has already been made once.
|
||||
|
||||
`MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
|
||||
accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
|
||||
what distinguishes "no bytes at all" from "bytes but no complete frame" on a
|
||||
timeout, and that distinction earned its keep during the Series III work).
|
||||
|
||||
---
|
||||
|
||||
## `micromate/protocol.py`
|
||||
|
||||
One method per command, returning raw payload bytes. No interpretation — that
|
||||
belongs in `client.py`.
|
||||
|
||||
**Reads use `offset = 0xFFFF`** and return the whole block in one response;
|
||||
Series III's two-step probe/data dance is unnecessary. `POLL` is the exception,
|
||||
taking its data length. Per-command offsets, all observed:
|
||||
|
||||
| command | SUB | rsp | offset | returns |
|
||||
|---|---|---|---|---|
|
||||
| poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
|
||||
| serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
|
||||
| device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
|
||||
| state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
|
||||
| monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
|
||||
| storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
|
||||
| active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
|
||||
| first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
|
||||
| next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
|
||||
| compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
|
||||
| call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
|
||||
| arm event | `0x93` | `0x6C` | — | before every event |
|
||||
| first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
|
||||
| next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
|
||||
| event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks |
|
||||
| event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record |
|
||||
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** |
|
||||
|
||||
⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
|
||||
Parse **forward** from `declared_length`, never backward from the end — Series
|
||||
III reads battery and memory from the end of that block, and doing so on a BD
|
||||
unit yields a battery voltage of **577.92 V**.
|
||||
|
||||
⚠ **Test the monitoring flag for non-zero**, never against a constant. It has
|
||||
read both `0x0E` and `0x0C` while monitoring.
|
||||
|
||||
### `0x5A` — simpler than Series III, deliberately
|
||||
|
||||
No arming ritual, no chunk loop, no `STRT` end-offset parsing, no `TERM` frame.
|
||||
One request returns the whole event:
|
||||
|
||||
```python
|
||||
offset_word = 0x1000 + 2 * ceil(size / 512) # size from the chain walk
|
||||
```
|
||||
|
||||
The payload **is** the `.IDFW` file, byte for byte — so it feeds
|
||||
`micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.
|
||||
|
||||
⚠ Do not port the Series III `5A` walk. Its address arithmetic caused a 5x
|
||||
over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
|
||||
III side. None of that applies here.
|
||||
|
||||
---
|
||||
|
||||
## `micromate/client.py`
|
||||
|
||||
```python
|
||||
class MicromateClient:
|
||||
def __init__(self, transport: BaseTransport): ...
|
||||
def open(self) / close(self) / is_open(self)
|
||||
|
||||
# identity and state
|
||||
def connect(self) -> DeviceInfo # poll → serial → device info → state
|
||||
def get_state(self) -> UnitState # monitoring?, clock, battery, memory
|
||||
def get_active_setup(self) -> str
|
||||
def list_setups(self) -> list[str] # 0x3F → 0x40… until empty
|
||||
|
||||
# events
|
||||
def list_events(self) -> list[EventRef] # 0x93 → 0x1E → 0x1F… (key + size)
|
||||
def download_event(self, ref) -> bytes # raw .IDFW/.IDFH
|
||||
def get_event(self, ref) -> Event # download + decode via idf_file
|
||||
```
|
||||
|
||||
`connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
|
||||
⚠ but note the reference records that **whether the unit requires it is
|
||||
untested**. Do it because it is known-good, not because it is known-necessary,
|
||||
and say so in the docstring.
|
||||
|
||||
`list_events()` returns the key *and* the size, because `download_event()` needs
|
||||
the size to compute its offset word.
|
||||
|
||||
---
|
||||
|
||||
## Tests
|
||||
|
||||
**Offline, from captured bytes — no hardware.** This is the part worth doing
|
||||
first, because it can be fully verified tonight's-captures-style before any unit
|
||||
is involved.
|
||||
|
||||
```
|
||||
tests/test_micromate_framing.py
|
||||
```
|
||||
|
||||
⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
|
||||
not depend on files being present. **Embed the frames as hex constants** — they
|
||||
are 16–68 bytes each, and a handful covers every case:
|
||||
|
||||
| case | why |
|
||||
|---|---|
|
||||
| POLL probe reply, 19 B | shortest valid frame |
|
||||
| POLL data reply, 68 B | contains a literal `0x10` — only the DLE-aware checksum matches |
|
||||
| `0x1A` response, 2108 B | exercises `declared_length` as a true uint16 (`0x082C`) |
|
||||
| a Thor-line reply, `flags = 0x03` | `0x03` is ETX; proves destuffing before framing |
|
||||
| a frame whose SUB is `0x02` | arrives as `10 02`; proves destuff-then-index |
|
||||
| a truncated frame | parser must return nothing, not a bad frame |
|
||||
| a corrupted checksum | `checksum_valid == False`, frame still returned |
|
||||
|
||||
Then a round-trip assertion: feed a whole captured session through the parser and
|
||||
assert the frame count and every SUB, against `scratch/mm_frame_parse.py`'s
|
||||
output — which is already known good, having parsed 24, 38 and 40-frame sessions
|
||||
with zero bad checksums.
|
||||
|
||||
**Live, second:** against the bench unit on mint-mac via `mm_link.py`.
|
||||
`connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
|
||||
then `download_event()` and assert the bytes decode and match a
|
||||
`/db/import/idf_file` ingest of the same event.
|
||||
|
||||
---
|
||||
|
||||
## Order of work
|
||||
|
||||
1. `framing.py` + its tests — offline, verifiable immediately
|
||||
2. `protocol.py` — reads only, one method per row of the table above
|
||||
3. `client.py` — `connect()`, `get_state()`, `list_setups()`
|
||||
4. the event chain and `download_event()`
|
||||
5. decode end-to-end and compare against a store event
|
||||
|
||||
Steps 1–2 need no hardware at all.
|
||||
|
||||
---
|
||||
|
||||
## Open questions to settle while implementing
|
||||
|
||||
- **Request param stuffing** — raise `NotImplementedError` rather than guess.
|
||||
- **Is THOR's preamble required?** Try one command cold and find out; it is a
|
||||
two-minute test with the bench unit and it removes a ritual if unnecessary.
|
||||
- **`Event` model fit** — Series IV carries fields Series III lacks (setup file
|
||||
name, `LMic`/`SMic` channels). Extend `micromate/models.py`; do not fork the
|
||||
shared `Event`.
|
||||
- **Which `0x0C` fields to trust.** The peak float there runs 2–5% above
|
||||
`max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
|
||||
established. The reference marks it do-not-rely-on — prefer decoded samples.
|
||||
@@ -2119,94 +2119,20 @@ FTDI is VID `0403`, Prolific is `067b`. Note also that counterfeit FTDI chips
|
||||
are common in cheap cables — they carry FTDI's VID but may not behave like one,
|
||||
and an embedded host with a single driver is far less forgiving than Linux.
|
||||
|
||||
### ✅ This accounts for the 2026-09-22 failure
|
||||
### Does this explain the 2026-09-22 field outage?
|
||||
|
||||
An earlier draft of this section said the timeline did not fit, because the unit
|
||||
was described as *working at first and then degrading* — which a Prolific cable
|
||||
cannot do, since it never enumerates at all.
|
||||
⚠ **Not on the timeline as reported.** A Prolific cable does not fail
|
||||
*gradually* — it never enumerates, so the modem port never works at all. The
|
||||
unit was described as working at first and degrading.
|
||||
|
||||
**The timeline was the thing that needed correcting, not the finding.** From the
|
||||
operator, 2026-09-25:
|
||||
There is one story where it fits: if the initial success was over a **different
|
||||
path** — the USB **PC** port, or a bench test before deployment — that would work
|
||||
regardless of which serial cable was attached. The modem path would then have
|
||||
been broken from the moment it was deployed, and "it worked and then stopped"
|
||||
would be a recollection conflating two connection types days after the fact.
|
||||
|
||||
1. Initial setup — **one** modem, **one** cable. Worked: configs sent, status
|
||||
read, no trouble.
|
||||
2. Two Micromates then had to be deployed, so a **second cable** was fetched to
|
||||
run both at once.
|
||||
3. **Only one of the two ever worked.** The other would not talk over its modem
|
||||
at any point.
|
||||
4. That unit was never deployed — a **MiniMate Plus was put out in its place**,
|
||||
and the Micromate came back to the bench.
|
||||
5. The cable on that bench unit is a **Benfei (Prolific PL2303)**.
|
||||
|
||||
So nothing degraded. The "worked at first" was the original single-cable setup;
|
||||
the failure began when the pair was split across two cables and one of them was
|
||||
the wrong chipset. **One working and one not, set up side by side, is the
|
||||
signature of a mixed cable supply** — not of a unit fault.
|
||||
|
||||
✅ **The cable travelled home with the unit** (confirmed 2026-09-25). So the
|
||||
Benfei on the bench is physically the cable from the failed office setup — not a
|
||||
substitute picked up later. There is no inference left in the chain:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| the cable from the failed unit | **is** a Benfei / Prolific PL2303 |
|
||||
| the Micromate's USB host | has **no** Prolific driver, in either firmware line |
|
||||
| a laptop on that same cable + modem | round-trips perfectly |
|
||||
| the Micromate on it | never answered |
|
||||
|
||||
The FTDI cable arriving 2026-09-26 is now a confirmation rather than a test.
|
||||
|
||||
**Corollary worth knowing.** The standing workaround — *"just deploy a Series III
|
||||
instead"* — works partly because a **MiniMate Plus has a DB-9 port directly on the
|
||||
unit.** No USB-to-serial adapter anywhere in the path, so there is no chipset to
|
||||
get wrong. That workaround has been quietly routing around this exact failure
|
||||
mode.
|
||||
|
||||
### ✅ RESOLVED (2026-09-26) — and it took a cold boot as well as the cable
|
||||
|
||||
Swapping to a genuine FTDI cable (`lsusb`: `0403:6001`, FT232) **did not work on
|
||||
its own.** A probe immediately after the swap returned the same *connected, no
|
||||
reply*.
|
||||
|
||||
**A cold boot of the unit was also required.** Holding the power button for five
|
||||
seconds — through the two-stage prompt that disconnects the internal battery —
|
||||
and powering back up **with the cable already attached**, brought it straight up.
|
||||
THOR at the office connected the moment it finished booting, and an independent
|
||||
probe returned `UM12947, idle` over cellular.
|
||||
|
||||
⚠ **The USB host does not rescan on hot-swap.** The Micromate enumerates its
|
||||
USB-A host port at boot; once it has failed to identify a device there it does
|
||||
not appear to try again. Changing that cable is a power-cycle operation, and
|
||||
anyone swapping one in the field who probes immediately will conclude the new
|
||||
cable is faulty too.
|
||||
|
||||
So the full chain for a Micromate on a modem is:
|
||||
|
||||
1. an **FTDI** or CDC-ACM cable — a Prolific PL2303 gives the unit no serial port
|
||||
2. a **cold boot** with that cable attached
|
||||
|
||||
### ✅ These modems bridge ONE session at a time — confirmed
|
||||
|
||||
Long suspected, never demonstrated. Demonstrated now, by accident:
|
||||
|
||||
| | probe result |
|
||||
|---|---|
|
||||
| THOR connected to the unit | `TCP connect ok` … **`NO REPLY`** |
|
||||
| THOR disconnected, nothing else changed | `reply 68 B`, **`UM12947`**, idle |
|
||||
|
||||
The PAD **accepts** a second TCP connection and then does not forward it. It
|
||||
does not refuse, and it does not close — it simply never bridges.
|
||||
|
||||
⚠ **This means "connected but no reply" has two causes that are indistinguishable
|
||||
from the client side:** a genuinely broken serial path, and *somebody else
|
||||
already has the unit*. `bridges/mm_probe.py` originally reported only the first,
|
||||
which would send a diagnosis in exactly the wrong direction; its verdict now
|
||||
names both and tells you to eliminate contention first.
|
||||
|
||||
**For SFM this is a design constraint, not a footnote.** Our receiver and THOR
|
||||
cannot both hold a unit, and during any migration both will exist. "Another
|
||||
client holds this unit" needs to be a distinct, visible state — not folded into
|
||||
a failure, and certainly not into a green tick.
|
||||
Plausible, unverified, and recorded as such. Identifying that unit's cable would
|
||||
settle it.
|
||||
|
||||
### How this was isolated — the method is reusable
|
||||
|
||||
|
||||
Reference in New Issue
Block a user