Author SHA1 Message Date
serversdownandClaude Opus 5 a7631a8179 docs(series4): RESOLVED -- FTDI cable AND a cold boot; modems bridge one session
The cable swap alone did not fix it.  A probe immediately after fitting a genuine
FTDI cable (lsusb 0403:6001, FT232) returned the same "connected, no reply".

A COLD BOOT of the unit was also required -- power button held five seconds
through the two-stage battery-disconnect prompt, powered back up with the cable
already attached.  THOR connected the moment it finished booting, and an
independent probe returned UM12947, idle, over cellular.

So the Micromate's USB host does not rescan on hot-swap: it enumerates that port
at boot and, having once failed to identify a device, does not try again.
Changing the cable is a power-cycle operation -- and anyone swapping one in the
field who probes immediately will conclude the new cable is faulty too.  Full
chain: an FTDI or CDC-ACM cable, AND a cold boot with it attached.

SEPARATELY CONFIRMED, by accident: these modems bridge ONE TCP session to serial
at a time.  With THOR connected, the probe got "TCP ok, NO REPLY"; with THOR
disconnected and nothing else changed, the same probe returned the serial number.
The PAD accepts a second connection and then never forwards it -- it does not
refuse and does not close.

That makes "connected but no reply" ambiguous between a broken serial path and
somebody else already holding the unit, and mm_probe reported only the former --
which would push a diagnosis in exactly the wrong direction, on the very fault we
spent two days chasing.  Its verdict now names both causes, tells you to
eliminate contention first, and includes the cable-chipset and cold-boot steps.

Recorded as an SFM design constraint: our receiver and THOR cannot both hold a
unit, and during any migration both will exist.  "Another client holds this unit"
has to be a distinct visible state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-26 18:11:16 -04:00
serversdownandClaude Opus 5 8a1d1c3f17 docs(series4): spec the live client, so tomorrow is implementation not design
Turns docs/micromate_protocol_reference.md into a build plan for the missing half
of micromate/ -- it is codec-only today with no way to talk to a unit.

Layout mirrors minimateplus/: framing, protocol, client.  Transport is REUSED --
minimateplus/transport.py is byte-level and protocol-agnostic, and already
handles the RV50/RV55 habit of emitting RING/CONNECT to a caller.  But
minimateplus/framing is deliberately NOT shared: the two framings differ in ways
that look small and are not, and a shared module would accumulate `if series ==`
branches until neither case is readable.

Records the three things that make S3FrameParser useless on Series IV (bare STX,
0xC5/0x03 flags, uniform 10 XX -> XX destuffing with no inner-frame carve-out),
and the traps that have already cost time: the declared length is a uint16 BE and
reading it as a byte under-reads SUB 0x1A by 47x; SUB 0x1C is four bytes longer
on the Thor line so parse forward not backward, or a BD unit reports 577.92 V;
the monitoring flag must be tested non-zero; and the SUB byte itself can arrive
DLE-escaped, so destuff before indexing.

Scope is READ-ONLY, stated with the reasoning: no command has ever been
originated against a unit by this project, and keeping that true through the read
client means the first thing we ever send to a customer's instrument is a
deliberate decision rather than a side effect of a client that grew a method.

Tests are specified to embed frames as hex constants rather than read fixtures,
since bridges/captures/ and tests/fixtures/ are both gitignored -- with a table
of which cases matter and why, and a round-trip assertion against
scratch/mm_frame_parse.py, which is already known good across three sessions at
zero bad checksums.

Steps 1-2 need no hardware.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-26 01:59:35 -04:00
serversdownandClaude Opus 5 27b0b25f15 docs(claude): refresh "Where things stand" -- Series-4 live protocol and tooling
The orientation block was dated 2026-08-28 and predated every finding from the
Series-4 live-protocol work, which is exactly the content a fresh session needs
and cannot discover on its own.

Adds, at the top where it will actually be read:

  * The Series-4 live wire protocol is mapped end to end, with the three framing
    differences that break a Series III parser (no leading DLE, 0xC5/0x03 flags,
    uint16 length at payload[8:10]) and the note that the inbound call-home
    session is the only protocol unknown left.
  * That NO command has ever been originated against a unit by this project --
    every write was performed by THOR while we recorded.  Worth stating plainly
    so the next session does not casually break it.
  * That micromate/ is still codec-only with no live client, and that
    minimateplus.transport is protocol-agnostic and reusable when one is built.
  * The bench tooling, including why mm_frame_parse.py has to exist at all:
    S3FrameParser scans for DLE+STX and therefore cannot see Micromate
    responses.
  * The FTDI/CDC-ACM-only USB host constraint and the Sabrent/Benfei cable trap,
    since a PL2303 cable leaves a unit with no working modem port and the cables
    are indistinguishable by eye.

Corrects a stale operational claim: the v0.30.0 Series-4 backfill HAS now been
run on prod, so the ~3.3%-low geophone values are fixed and the job does not need
repeating.

Also generalises the "update the protocol reference" instruction from the single
Series III document to a table of all three, and adds bridges/ plus the two newer
docs to the project layout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-26 00:09:55 -04:00
serversdownandClaude Opus 5 5fa181fb79 docs(series4): confirmed -- the cable came home with the failed unit
Removes the last caveat.  The Benfei (Prolific PL2303) on the bench is physically
the cable from the failed office setup, not a substitute picked up later.

The chain has no inference left: that cable is a PL2303; the Micromate USB host
has no Prolific driver in either firmware line; a laptop on the same cable and
modem round-trips perfectly; the Micromate on it never answered.  The FTDI cable
arriving 2026-09-26 is now a confirmation rather than a test.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 23:27:39 -04:00
serversdownandClaude Opus 5 0659543a15 docs(series4): the 2026-09-22 failure is accounted for -- the timeline was wrong
An earlier draft said the Prolific-cable finding did not fit the timeline, since
the unit was described as working and then degrading, which a cable that never
enumerates cannot do.  The timeline was what needed correcting.

From the operator: the initial setup was ONE modem and ONE cable, and it worked.
Two Micromates then had to be deployed, so a second cable was fetched to run both
at once -- and only one of the two ever worked.  The other never talked over its
modem at any point, was never deployed, and a MiniMate Plus went out in its place.
That Micromate came back to the bench, and the cable on it is a Benfei (Prolific
PL2303).

So nothing degraded.  The "worked at first" was the single-cable setup before the
split.  One unit working and one not, configured side by side, is the signature of
a mixed cable supply rather than a unit fault.

Still worth confirming whether that cable travelled home with the unit or was
picked up at the bench; if it travelled with it there is no inference left.  The
FTDI cable arriving 2026-09-26 settles it either way -- same unit, same modem, one
variable.

Records a corollary: "just deploy a Series III instead" works partly because a
MiniMate Plus has a DB-9 port directly on the unit, with no USB-to-serial adapter
in the path and therefore no chipset to get wrong.  That workaround has been
routing around this exact failure mode.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 23:27:21 -04:00
5 changed files with 454 additions and 92 deletions
+3 -68
View File
@@ -46,50 +46,6 @@ All notable changes to seismo-relay are documented here.
500s. Reproduced on BE12599. Production is unaffected: it ingests complete
files via the watcher path and never runs this walk.
- **The Micromate (Series IV) live wire protocol, reverse-engineered end to
end** — `docs/micromate_protocol_reference.md`. Worked out against a bench
UM12947 over USB and a recording relay, with THOR driving every write so that
no command has ever been originated against a unit by this project. **A
Micromate answers Series III command frames**, with three framing differences:
responses carry no leading `DLE`, `payload[1]` is `0xC5` (Blastware firmware)
or `0x03` (Thor firmware) rather than `0x10`, and the data length is a
**uint16 BE at `payload[8:10]`** — read as a single byte it under-reads
`SUB 0x1A` by 47x. Read path, event chain, and `SUB 0x5A` streaming the
`.IDFW` file verbatim are all confirmed.
- **Series IV setup management, fully mapped.** `0x41` reads the active setup
name, `0x1A` its config block, `0xDA` names the target `.MMB`, `0x71`/`0x72`
write it back. **Setups are read-modify-write** — the written block is the
read block, 91% byte-identical at a fixed 11-byte shift. `0xDA` **creates**
files rather than only overwriting, confirmed on the unit's own screen, and an
overwrite is protocol-identical to a create: no handshake, no warning, and no
protection even over the *active* setup of a monitoring unit.
- **The scheduler file decoded** — `\system\schedule\schedule.dat`, 260-byte
records carrying an action bitmask (2 start, 4 stop, 8 self-check, 16 ACH), a
half-hour slot (48/day), day-of-week (0 = Sunday) and a length-prefixed setup
name. Verified entry-for-entry against the operator's own THOR screen.
- **A generic file transfer addressed by full path** — `0x94`/`0x48` read,
`0x8D`/`0x8E` write. This **retracts** an earlier conclusion in the same
document that no such command existed; that was inferred from absent firmware
strings and was wrong.
- **Monitoring control and per-event delete.** `0x96`/`0x97` start and stop as
on Series III, but the monitoring flag at `SUB 0x1C` `data[12]` must be tested
for **non-zero** (observed as both `0x0E` and `0x0C`) rather than compared to a
constant. Deletion is **per-event** — `0xA8` with the event key, then `0xAA` —
which is safer than Series III's erase-everything. `SUB 0x1C` also carries the
device clock.
- **`bridges/mm_probe.py`** — distinguishes the four faults THOR reports
identically as "disconnected": refused, connect timeout (the silent-drop
signature of a trusted-IP whitelist), **connected but no reply** (the modem
answered and the unit did not), and replied. Each verdict names what to try
next. `--slots N` tests single-session modem behaviour.
- **`bridges/mm_link.py`** — a bench stand-in for a cellular modem, with a
decoded timestamped log and fault injection (`blackhole`, `drop`, `delay`,
`onewaydev`) driven by a control file. No pyserial; stdlib `termios` only.
- **`scratch/mm_frame_parse.py`**, **`socat_log_split.py`** and **`fake_unit.py`**
— a Micromate-aware frame parser (`S3FrameParser` cannot see these responses at
all, since it scans for `DLE+STX`), byte-exact capture recovery from a
`socat -x` relay log, and a serial-port stand-in that answers as a unit.
### Changed
- **Connecting to a unit no longer walks its event chain.** `/device/events`
@@ -102,32 +58,11 @@ All notable changes to seismo-relay are documented here.
toolbar. Knowing whether a unit's ACH is on no longer requires reading every
event it has stored.
- **Recorded what THOR actually does on the wire**, measured rather than assumed.
A "status check" is **eleven commands, ~2.2 KB including TCP setup** — 18.8
MB/day per unit at a 10 s cadence, against ~0.2 MB/day for a `POLL` +
`MONITOR_STATUS` check at 60 s. The **status interval is honoured; the
connection interval is not** — it sets `(status / connection) - 1` checks per
cycle, so equal values yield *zero* cheap checks and every connection becomes
the expensive one.
- **Two THOR defects reproduced with timestamps.** After a connection drops
mid-download it retries **once**, stops polling entirely and **never resumes**,
while displaying `Connected` for as long as it is left alone — and `Idle` for a
unit that is actively recording. Separately, THOR's own log shows a
**subscription leak**: one logical event dispatched to a growing number of
handlers, **1 to 12 over ten hours** of uptime, consistent with the field
report that only a restart recovers it.
- **The Micromate's USB host supports FTDI and CDC-ACM only** — no Prolific, in
either firmware line. A PL2303 cable (Benfei) leaves a unit with no working
modem port; an FTDI cable (Sabrent) works. Both are in circulation and
indistinguishable by eye — identify by `lsusb` VID, `0403` against `067b`.
### Migration
**None.** Frontend, documentation and bench tooling only — no codec,
waveform-store or DB change, no schema change, and no `TOOL_VERSION` bump. The
webapp is served from the image, so its changes appear after the next `sfm`
rebuild. The Series-4 work adds `docs/`, `bridges/` and `scratch/` files only;
nothing under `sfm/`, `minimateplus/` or `micromate/` was touched.
**None.** Frontend and documentation only — no codec, waveform-store or DB
change, no schema change, and no `TOOL_VERSION` bump. The webapp is served
from the image, so the change appears after the next `sfm` rebuild.
---
+52 -3
View File
@@ -10,10 +10,38 @@ pair — lives in `../terra-view/docs/tmi-stack.md`, which is also loaded as
---
## Where things stand (updated 2026-08-28)
## Where things stand (updated 2026-09-26)
Read this first when picking the project back up.
- **The Series-4 LIVE wire protocol is reverse-engineered end to end
(2026-09-25).** `docs/micromate_protocol_reference.md` is the Series-4
Rosetta Stone, sibling to `instantel_protocol_reference.md`. **A Micromate
answers Series III command frames** — three framing differences: responses
have **no leading `DLE`** (a bare `STX`), `payload[1]` is `0xC5` (Blastware
firmware) or `0x03` (Thor firmware) rather than `0x10`, and the data length
is a **uint16 BE at `payload[8:10]`** (as a byte it under-reads `SUB 0x1A`
by 47x). Read path, event chain, setups, scheduler, monitoring control and
per-event delete are all mapped; **the inbound call-home session is the only
protocol unknown left.**
⚠ **No command has ever been originated against a unit by this project.**
Every write was performed by THOR while we recorded. That line is worth
keeping.
⚠ `micromate/` still has **no live client** — it is codec-only. The
`minimateplus/` stack (transport/framing/protocol/client) has no Series-4
counterpart yet. `minimateplus.transport` is protocol-agnostic and reusable.
- **Bench tooling for device diagnosis (2026-09-25).** `bridges/mm_probe.py`
distinguishes the four faults THOR reports identically as "disconnected"
(refused / connect timeout / **connected but no reply** / replied) and names
what to try next. `bridges/mm_link.py` is a stand-in for a cellular modem
with a decoded log and fault injection. `scratch/mm_frame_parse.py` exists
because **`S3FrameParser` cannot see Micromate responses at all** — it scans
for `DLE+STX`, which never appears in Series-4 traffic.
- **A Micromate's USB-A host port drives FTDI and CDC-ACM only** — no Prolific,
in either firmware line. TMI buys both Sabrent (FTDI) and Benfei (PL2303)
cables and they are indistinguishable by eye. A PL2303 cable leaves a unit
with **no working modem port at all**; identify by `lsusb` VID, `0403` vs
`067b`. This accounted for a unit that could not be deployed.
- **Series-3 decode is verified per-sample at scale (v0.27.0).** The full DL2
archive decodes **14,338 / 14,338** paired files exactly against their
preserved Blastware ASCII exports — 1,249 waveform + 13,089 histogram, 45
@@ -92,6 +120,9 @@ Read this first when picking the project back up.
**v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block
fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered
files are archive-only and were never ingested).
✅ **The v0.30.0 Series-4 backfill HAS been run on prod (2026-09-25).** Every
stored Series-4 geophone value was ~3.3% low until then; that is corrected and
the job does not need repeating.
- **The "offset" hardware fault has its own journal** --
`docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is
**persistent** — it stays until the geophone is serviced. Detect it with
@@ -103,7 +134,18 @@ Read this first when picking the project back up.
`SUB 0x0E` (unimplemented), which may carry those very numbers.
When new information about the protocol is discovered, please update the instantel_protocol_reference.md with the findings in addition to this document
When new information about a protocol is discovered, record it in the matching
reference **in addition to** this document:
| series | document |
|---|---|
| Series III (MiniMate Plus / BlastMate) | `docs/instantel_protocol_reference.md` |
| **Series IV (Micromate / THOR)** | **`docs/micromate_protocol_reference.md`** |
| Thor IDF file format | `docs/idf_protocol_reference.md` |
Both protocol references carry retractions in place rather than deleting what
turned out to be wrong — that convention has already saved re-deriving the same
mistakes twice, so keep it.
---
@@ -276,8 +318,15 @@ minimateplus/ ← Python client library (primary focus)
sfm/server.py ← FastAPI REST server exposing device data over HTTP
seismo_lab.py ← Tkinter GUI (Bridge + Analyzer + Console tabs)
bridges/
mm_probe.py ← name the fault behind a dead unit (4 verdicts, read-only)
mm_link.py ← bench stand-in for a cellular modem, with fault injection
ach_mitm.py ← TCP relay for recording a Series-3 ACH session
docs/
instantel_protocol_reference.md ← reverse-engineered protocol spec ("the Rosetta Stone")
instantel_protocol_reference.md ← Series III protocol spec ("the Rosetta Stone")
micromate_protocol_reference.md ← Series IV protocol spec + THOR's measured behaviour
idf_protocol_reference.md ← Thor IDF file format
CHANGELOG.md ← version history
```
+23 -10
View File
@@ -134,17 +134,30 @@ def probe(host: str, port: int, timeout: float) -> int:
step(2, "POLL", f"NO REPLY in {timeout:.1f} s")
print("\nverdict: the MODEM answered but the unit did not.")
print(" TCP is fine end to end — something accepted the connection.")
print(" What is missing is the serial side. Most likely the modem is")
print(" not forwarding to its serial port, which is what a wedged")
print(" transparent-TCP session looks like: the slot is held by a")
print(" connection that never closed.")
print(" What is missing is the serial side. Two quite different")
print(" causes produce this, and they are NOT distinguishable from")
print(" here:")
print("\n 1. SOMEONE ELSE HOLDS THE SESSION. These modems bridge ONE")
print(" TCP session to serial at a time. A second connection is")
print(" accepted and then simply not forwarded. Confirmed 2026-09-26:")
print(" with THOR connected this probe saw exactly this; the moment")
print(" THOR disconnected the same probe returned the serial number.")
print(" ** Check whether THOR (or anything else) has the unit first. **")
print("\n 2. The serial path is genuinely broken — a stale session the")
print(" modem never released, a cable the unit cannot enumerate, or")
print(" a unit that is off.")
print("\n Try, in order:")
print(" 1. ACEmanager -> TCP Idle Timeout. If 0/disabled, a stale")
print(" session holds the slot forever. 2 minutes is the value")
print(" this project standardised on.")
print(" 2. Reboot the modem. If that fixes it, the modem was")
print(" holding state and the timeout is the permanent fix.")
print(" 3. Check the unit's own screen — serial cable, power.")
print(" 1. Disconnect any other client and re-probe. If it answers,")
print(" it was contention, not a fault.")
print(" 2. The cable's chipset. A Micromate drives FTDI and CDC-ACM")
print(" only — a Prolific PL2303 gives it no serial port at all.")
print(" lsusb: FTDI is 0403, Prolific 067b.")
print(" 3. Power-cycle the UNIT with the cable attached (hold power")
print(" 5 s, through the two-stage prompt). Its USB host rescans")
print(" on cold boot; it may not on hot-swap.")
print(" 4. AirLink OS -> TCP Idle Timeout. If 0/disabled, a stale")
print(" session holds the slot indefinitely. 2 minutes is the")
print(" value this project standardised on.")
sock.close()
return 5
+291
View File
@@ -0,0 +1,291 @@
# Spec — a live client for Series IV (Micromate)
Drafted 2026-09-26, ahead of implementation. The protocol work is finished; this
is the plan for turning `docs/micromate_protocol_reference.md` into code SFM can
run.
**Read that document first.** Everything here assumes it, and every constant
below is sourced from it rather than restated with justification.
---
## Goal and scope
`micromate/` is codec-only today — `idf_file.py`, `models.py`, the report
writers. There is no way to talk to a unit. This adds the live half, mirroring
`minimateplus/`.
**In scope, first pass:**
- connect over TCP (a field modem) or serial/USB (a bench unit)
- identify a unit, read its state, clock, memory and setups
- walk the event chain and download events
- return `Event` objects the existing codec already understands
**Explicitly out of scope, first pass:**
- ⚠ **Any write.** Setups, schedules, call-home config, monitoring start/stop,
and per-event delete are all mapped, and none of them will be implemented
here. **No command has ever been originated against a unit by this project**
— every write observed was performed by THOR while we recorded. Keeping that
true through the read client is deliberate: it means the first thing we ever
send to a customer's instrument is a decision someone made on purpose, not a
side effect of a client that happened to grow a method.
- the inbound call-home session — still the one protocol unknown
---
## Layout
```
micromate/
framing.py NEW frame building, response parsing, checksum
protocol.py NEW one method per wire command, returns raw payloads
client.py NEW high-level API, returns models
idf_file.py (existing — decodes what 0x5A returns, unchanged)
models.py (existing — extend, do not fork)
```
**Transport is reused, not rewritten.** `minimateplus/transport.py` is
byte-level and protocol-agnostic — `BaseTransport`, `SerialTransport`,
`TcpTransport`, plus `read_until_idle()` which already handles the RV50/RV55
habit of emitting `\r\nRING\r\n\r\nCONNECT\r\n` to a caller. Import it.
⚠ Do **not** import `minimateplus.framing`. The two framings differ in ways
that look small and are not, and a shared module would accumulate `if series ==`
branches until neither case is readable.
---
## `micromate/framing.py`
### Requests
Series IV accepts Series III request frames unmodified. The simplest correct
implementation re-exports the builder rather than duplicating it:
```python
from minimateplus.framing import build_bw_frame # requests are identical
```
⚠ One open question, flagged in the protocol reference and **not** settled:
whether `0x10` bytes inside request *params* need stuffing. No probe we sent
carried one. Until it is settled, assert on it rather than guessing:
```python
def build_request(sub: int, offset: int = 0, params: bytes = bytes(10)) -> bytes:
if 0x10 in params:
raise NotImplementedError(
"params containing 0x10 — stuffing rule unconfirmed; see "
"micromate_protocol_reference.md, 'Untested and unsafe-until-agreed'"
)
return build_bw_frame(sub, offset, params)
```
That turns an unknown into a loud failure instead of a corrupt frame.
### Responses — where Series III's parser cannot follow
| | Series III | Micromate |
|---|---|---|
| frame start | `DLE STX` | **bare `STX`** |
| `payload[1]` | `0x10` | `0xC5` (Blastware fw) / `0x03` (Thor fw) |
| destuffing | `DLE+ETX` kept as literal inner-frame data | **`10 XX` → `XX`, uniformly** |
The first row is why `S3FrameParser` returns nothing at all on Series IV traffic:
it scans for `DLE+STX`, which never appears.
The third is a genuine **simplification** — no inner-frame carve-out. Validated
by checksum across every capture in `bridges/captures/9-24-26 - micromate2/`:
four candidate destuffing rules were tried, and only this one makes all frames
validate.
### Checksum
```python
def checksum(payload: bytes) -> int:
return sum(b for b in payload if b != 0x10) & 0xFF
```
The DLE-aware variant, same as Series III's `5A` and write frames — not the plain
SUM8 of ordinary Series III reads.
### ⚠ The SUB byte can be escaped
When a SUB's value is `0x02`, `0x03`, `0x04` or `0x10` it arrives as `10 XX`.
Reading it positionally without destuffing reports `0x10`. This bit once
already — `SUB 0x02` was logged as `SUB_10` for an afternoon. Destuff first,
then index.
### Response shape
```python
@dataclass
class MicromateFrame:
sub: int # response SUB; request = 0xFF - sub
flags: int # 0xC5 Blastware line, 0x03 Thor line
page_hi: int
page_lo: int
data: bytes # payload[5:], checksum stripped
checksum_valid: bool
@property
def firmware_line(self) -> str: # "blastware" | "thor" | "unknown"
@property
def declared_length(self) -> int: # uint16 BE at data[3:5] (= payload[8:10])
```
⚠ **`declared_length` is a uint16 BE.** Read as a single byte it under-reads
`SUB 0x1A` by 47x — 44 against a true 2092. This is the single most expensive
mistake available in this protocol and it has already been made once.
`MicromateFrameParser` mirrors `S3FrameParser`: `feed(bytes) -> list[frame]`,
accumulates in `.frames`, `reset()`, and keeps the `bytes_fed` counter (it is
what distinguishes "no bytes at all" from "bytes but no complete frame" on a
timeout, and that distinction earned its keep during the Series III work).
---
## `micromate/protocol.py`
One method per command, returning raw payload bytes. No interpretation — that
belongs in `client.py`.
**Reads use `offset = 0xFFFF`** and return the whole block in one response;
Series III's two-step probe/data dance is unnecessary. `POLL` is the exception,
taking its data length. Per-command offsets, all observed:
| command | SUB | rsp | offset | returns |
|---|---|---|---|---|
| poll | `0x5B` | `0xA4` | `0x0030` | device string, model |
| serial | `0x15` | `0xEA` | `0x000A` | `UM12947` |
| device info | `0x01` | `0xFE` | `0xFFFF` | firmware, calibration |
| state | `0x49` | `0xB6` | `0xFFFF` | `data[11]`: non-zero = monitoring |
| monitor status | `0x1C` | `0xE3` | `0xFFFF` | flag, **device clock**, battery, memory |
| storage range | `0x06` | `0xF9` | `0xFFFF` | event storage extent |
| active setup name | `0x41` | `0xBE` | `0xFFFF` | `TEST1.mmb` |
| first setup | `0x3F` | `0xC0` | `0xFFFF` | setup-list walk head |
| next setup | `0x40` | `0xBF` | `0xFFFF` | …until an empty name |
| compliance config | `0x1A` | `0xE5` | `0xFFFF` | ~2103 B setup block |
| call-home config | `0x2C` | `0xD3` | `0xFFFF` | 137 B |
| arm event | `0x93` | `0x6C` | — | before every event |
| first event | `0x1E` | `0xE1` | `0xFFFF` | key + size |
| next event | `0x1F` | `0xE0` | `0xFFFF` | key + size |
| event record | `0x0C` | `0xF3` | `0xFFFF` | 210 B — project, location, peaks |
| event header | `0x0A` | `0xF5` | `0xFFFF` | 30 B list record |
| bulk download | `0x5A` | `0xA5` | computed | **the `.IDFW` verbatim** |
⚠ **`SUB 0x1C` is 4 bytes longer on the Thor firmware line** (`0x30` vs `0x2C`).
Parse **forward** from `declared_length`, never backward from the end — Series
III reads battery and memory from the end of that block, and doing so on a BD
unit yields a battery voltage of **577.92 V**.
⚠ **Test the monitoring flag for non-zero**, never against a constant. It has
read both `0x0E` and `0x0C` while monitoring.
### `0x5A` — simpler than Series III, deliberately
No arming ritual, no chunk loop, no `STRT` end-offset parsing, no `TERM` frame.
One request returns the whole event:
```python
offset_word = 0x1000 + 2 * ceil(size / 512) # size from the chain walk
```
The payload **is** the `.IDFW` file, byte for byte — so it feeds
`micromate.idf_file.read_idf_file()` and `/db/import/idf_file` unchanged.
⚠ Do not port the Series III `5A` walk. Its address arithmetic caused a 5x
over-read and a `> 64 KB` page-boundary bug that is *still open* on the Series
III side. None of that applies here.
---
## `micromate/client.py`
```python
class MicromateClient:
def __init__(self, transport: BaseTransport): ...
def open(self) / close(self) / is_open(self)
# identity and state
def connect(self) -> DeviceInfo # poll → serial → device info → state
def get_state(self) -> UnitState # monitoring?, clock, battery, memory
def get_active_setup(self) -> str
def list_setups(self) -> list[str] # 0x3F → 0x40… until empty
# events
def list_events(self) -> list[EventRef] # 0x93 → 0x1E → 0x1F… (key + size)
def download_event(self, ref) -> bytes # raw .IDFW/.IDFH
def get_event(self, ref) -> Event # download + decode via idf_file
```
`connect()` should mirror THOR's preamble (`POLL → SERIAL → 0x49 → POLL`) —
⚠ but note the reference records that **whether the unit requires it is
untested**. Do it because it is known-good, not because it is known-necessary,
and say so in the docstring.
`list_events()` returns the key *and* the size, because `download_event()` needs
the size to compute its offset word.
---
## Tests
**Offline, from captured bytes — no hardware.** This is the part worth doing
first, because it can be fully verified tonight's-captures-style before any unit
is involved.
```
tests/test_micromate_framing.py
```
⚠ `bridges/captures/` and `tests/fixtures/` are both gitignored, so tests must
not depend on files being present. **Embed the frames as hex constants** — they
are 16–68 bytes each, and a handful covers every case:
| case | why |
|---|---|
| POLL probe reply, 19 B | shortest valid frame |
| POLL data reply, 68 B | contains a literal `0x10` — only the DLE-aware checksum matches |
| `0x1A` response, 2108 B | exercises `declared_length` as a true uint16 (`0x082C`) |
| a Thor-line reply, `flags = 0x03` | `0x03` is ETX; proves destuffing before framing |
| a frame whose SUB is `0x02` | arrives as `10 02`; proves destuff-then-index |
| a truncated frame | parser must return nothing, not a bad frame |
| a corrupted checksum | `checksum_valid == False`, frame still returned |
Then a round-trip assertion: feed a whole captured session through the parser and
assert the frame count and every SUB, against `scratch/mm_frame_parse.py`'s
output — which is already known good, having parsed 24, 38 and 40-frame sessions
with zero bad checksums.
**Live, second:** against the bench unit on mint-mac via `mm_link.py`.
`connect()`, `list_setups()` (should return the 23 known names), `list_events()`,
then `download_event()` and assert the bytes decode and match a
`/db/import/idf_file` ingest of the same event.
---
## Order of work
1. `framing.py` + its tests — offline, verifiable immediately
2. `protocol.py` — reads only, one method per row of the table above
3. `client.py` — `connect()`, `get_state()`, `list_setups()`
4. the event chain and `download_event()`
5. decode end-to-end and compare against a store event
Steps 1–2 need no hardware at all.
---
## Open questions to settle while implementing
- **Request param stuffing** — raise `NotImplementedError` rather than guess.
- **Is THOR's preamble required?** Try one command cold and find out; it is a
two-minute test with the bench unit and it removes a ritual if unnecessary.
- **`Event` model fit** — Series IV carries fields Series III lacks (setup file
name, `LMic`/`SMic` channels). Extend `micromate/models.py`; do not fork the
shared `Event`.
- **Which `0x0C` fields to trust.** The peak float there runs 2–5% above
`max(T,V,L)` and is **not** the vector sum; its offset was inferred, not
established. The reference marks it do-not-rely-on — prefer decoded samples.
+85 -11
View File
@@ -2119,20 +2119,94 @@ FTDI is VID `0403`, Prolific is `067b`. Note also that counterfeit FTDI chips
are common in cheap cables — they carry FTDI's VID but may not behave like one,
and an embedded host with a single driver is far less forgiving than Linux.
### Does this explain the 2026-09-22 field outage?
### ✅ This accounts for the 2026-09-22 failure
⚠ **Not on the timeline as reported.** A Prolific cable does not fail
*gradually* — it never enumerates, so the modem port never works at all. The
unit was described as working at first and degrading.
An earlier draft of this section said the timeline did not fit, because the unit
was described as *working at first and then degrading* — which a Prolific cable
cannot do, since it never enumerates at all.
There is one story where it fits: if the initial success was over a **different
path** — the USB **PC** port, or a bench test before deployment — that would work
regardless of which serial cable was attached. The modem path would then have
been broken from the moment it was deployed, and "it worked and then stopped"
would be a recollection conflating two connection types days after the fact.
**The timeline was the thing that needed correcting, not the finding.** From the
operator, 2026-09-25:
Plausible, unverified, and recorded as such. Identifying that unit's cable would
settle it.
1. Initial setup — **one** modem, **one** cable. Worked: configs sent, status
read, no trouble.
2. Two Micromates then had to be deployed, so a **second cable** was fetched to
run both at once.
3. **Only one of the two ever worked.** The other would not talk over its modem
at any point.
4. That unit was never deployed — a **MiniMate Plus was put out in its place**,
and the Micromate came back to the bench.
5. The cable on that bench unit is a **Benfei (Prolific PL2303)**.
So nothing degraded. The "worked at first" was the original single-cable setup;
the failure began when the pair was split across two cables and one of them was
the wrong chipset. **One working and one not, set up side by side, is the
signature of a mixed cable supply** — not of a unit fault.
✅ **The cable travelled home with the unit** (confirmed 2026-09-25). So the
Benfei on the bench is physically the cable from the failed office setup — not a
substitute picked up later. There is no inference left in the chain:
| | |
|---|---|
| the cable from the failed unit | **is** a Benfei / Prolific PL2303 |
| the Micromate's USB host | has **no** Prolific driver, in either firmware line |
| a laptop on that same cable + modem | round-trips perfectly |
| the Micromate on it | never answered |
The FTDI cable arriving 2026-09-26 is now a confirmation rather than a test.
**Corollary worth knowing.** The standing workaround — *"just deploy a Series III
instead"* — works partly because a **MiniMate Plus has a DB-9 port directly on the
unit.** No USB-to-serial adapter anywhere in the path, so there is no chipset to
get wrong. That workaround has been quietly routing around this exact failure
mode.
### ✅ RESOLVED (2026-09-26) — and it took a cold boot as well as the cable
Swapping to a genuine FTDI cable (`lsusb`: `0403:6001`, FT232) **did not work on
its own.** A probe immediately after the swap returned the same *connected, no
reply*.
**A cold boot of the unit was also required.** Holding the power button for five
seconds — through the two-stage prompt that disconnects the internal battery —
and powering back up **with the cable already attached**, brought it straight up.
THOR at the office connected the moment it finished booting, and an independent
probe returned `UM12947, idle` over cellular.
⚠ **The USB host does not rescan on hot-swap.** The Micromate enumerates its
USB-A host port at boot; once it has failed to identify a device there it does
not appear to try again. Changing that cable is a power-cycle operation, and
anyone swapping one in the field who probes immediately will conclude the new
cable is faulty too.
So the full chain for a Micromate on a modem is:
1. an **FTDI** or CDC-ACM cable — a Prolific PL2303 gives the unit no serial port
2. a **cold boot** with that cable attached
### ✅ These modems bridge ONE session at a time — confirmed
Long suspected, never demonstrated. Demonstrated now, by accident:
| | probe result |
|---|---|
| THOR connected to the unit | `TCP connect ok` … **`NO REPLY`** |
| THOR disconnected, nothing else changed | `reply 68 B`, **`UM12947`**, idle |
The PAD **accepts** a second TCP connection and then does not forward it. It
does not refuse, and it does not close — it simply never bridges.
⚠ **This means "connected but no reply" has two causes that are indistinguishable
from the client side:** a genuinely broken serial path, and *somebody else
already has the unit*. `bridges/mm_probe.py` originally reported only the first,
which would send a diagnosis in exactly the wrong direction; its verdict now
names both and tells you to eliminate contention first.
**For SFM this is a design constraint, not a footnote.** Our receiver and THOR
cannot both hold a unit, and during any migration both will exist. "Another
client holds this unit" needs to be a distinct, visible state — not folded into
a failure, and certainly not into a green tick.
### How this was isolated — the method is reusable