701af47170174554f268b3aa61b16288abe79456
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  serversdownandClaude Opus 5
|
701af47170 |
docs(series4): generate IDF filenames rather than detecting record type
Closes the record-type gap flagged earlier, and corrects the premise behind it.
Series III does NOT detect record type from file content --
event_file_io.derive_record_type_from_filename() reads the last character of
the extension (M529LKIQ.G10H -> H -> Histogram). Nothing in the codebase infers
record type from content, for either family.
Nor is there an obvious type field to find in an IDF: the first 64 bytes of a
histogram and a waveform are byte-identical, and they diverge at ~0x0947 into
wholly different structures rather than differing by a flag.
The answer is the Series III pattern -- generate the name. Series III has
blastware_filename(); Series IV needs the same, and its convention is far
simpler:
<serial>_<YYYYMMDDHHMMSS>.IDF{W,H} e.g. UM12947_20260923163319.IDFW
against Series III's <letter><serial3><base-36 stem><AB0T ext>.
All three inputs are already available on a direct download: serial and
timestamp from extract_binary_metadata(), and type from the chain walk (SUB
0x0A returns 0x1E for a histogram, 0x00 for a waveform). Verified on all five
bench events -- generated names match real production-store filenames byte for
byte, so a directly downloaded event can be filed under exactly the name Thor
would have given it and /db/import/idf_file needs no change.
The type still comes from the protocol rather than the payload, so a
downloader must carry it out of the chain walk; losing it means losing the
ability to name the file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
|
||
|
serversdownandClaude Opus 5
|
02ed22f561 |
docs(series4): firmware static analysis, and all five bench events decoded
Solo session while the bench was unattended. Read-only throughout.
Architecture: ColdFire/68K, big-endian, Freescale MQX RTOS -- not ARM as the
vector table first suggested. The tell is 4E 5E 4E 75 4E 56 (UNLK A6 / RTS /
LINK A6) throughout both images, plus an MQX_OK assertion.
CB vs BD: a byte diff is useless (68% of bytes differ -- separately linked
builds, everything relocated). A string-set diff is position-independent and
shows 17,128 strings shared, with almost every "unique" string being the same
message at a different source line:
CB: MONITOR[3268]: STATUS_BATTERY_LOW
BD: MONITOR[3258]: STATUS_BATTERY_LOW
Consistently 10 lines apart across five different MONITOR messages, so one
~10-line block differs in the monitor module and essentially nothing else. The
only functional string unique to either build is CITIZEN (a printer brand) in
BD. This corroborates the bench A/B from the other direction: the split is a
tiny code delta, not two protocol stacks.
The SUB dispatch is a 68K switch jump table, so byte-pattern hunting will not
isolate the write opcodes -- that needs a disassembler.
Call-home config field names recovered from the firmware's own debug dump:
Enable, DialString, Retries, SessionTimeout, WaitForConnection, WarmupTime,
PowerSave -- seven fields for the 126-byte SUB 0x2C block. SessionTimeout and
PowerSave have no Series III equivalent, and Series III's scheduled-time fields
are absent, consistent with scheduling moving into the THOR-downloaded
scheduler. AT+CSQ is present, so the firmware speaks AT to the modem directly.
All five bench events downloaded and decoded over USB: each arrived at exactly
its declared size, every channel equal length, timestamps sequential.
Two gaps recorded:
- No content-based record-type discriminator. read_idf_file() dispatches on the
.IDFH/.IDFW filename suffix, which does not exist over the wire, and the
first 64 bytes of a histogram and a waveform are byte-identical. The protocol
supplies one instead: SUB 0x0A returns 0x1E for a histogram and 0x00 for a
waveform, so the type must be carried from the chain walk.
- The 0x0C peak float runs 2-5% above max(Tran,Vert,Long) and is not the vector
sum either. Its offset was inferred from a byte marker rather than
established, so it may not be the peak at all. Marked do-not-rely-on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
|
||
|
serversdownandClaude Opus 5
|
23cdbef737 |
docs(series4): setups are files; and the length field is a uint16
Two findings and one correction. CORRECTION: the probe response's data length is a uint16 BE at payload[8:10], not a single byte at payload[9] as an earlier draft claimed. That reading is right only while the high byte is zero. For SUB 0x1A the real length is 0x082C = 2092; read as a byte it gives 44, a 47x under-read. Setups are FILES, not a config block. Series III has one compliance config you overwrite; Series IV keeps named .MMB setup files on an on-device filesystem with a current-selection pointer -- csetup.MMB, factory.MMB, and callhome.MMB for the call-home config. Names up to 20 chars. Filesystem primitives exist internally (NS_ReadFile_internal / NS_WriteFile_internal / NS_SeekFile_internal) but no generic file-transfer command is exposed on the wire, so setups are unlikely to be pushed as raw .MMB blobs over the protocol. SUB 0x1A reads the whole active setup in 2,092 bytes -- structurally close to Series III's ~2,126-byte compliance block -- carrying the setup FILE NAME, all four title note/value pairs (Location, Client, Company, General Notes), the sensor location, and per-channel labels with units. Note LMic and SMic (linear and sound-level microphone variants) which Series III does not have. That is the read half of setup management, so a setup can in principle be round-tripped. The write half has not been attempted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
71f19c90d1 |
docs(series4): SUB 5A streams the .IDFW file verbatim -- read path complete
The complete read path now works with no Instantel software in the loop. Three divergences from Series III, all simplifications: - No arming sequence. Series III ignores a 5A probe unless preceded by 1E / 0A / 1E(0xFE) / 0C / 1F(0xFE) / POLL x3. The Micromate answers a bare 5A request with nothing before it. - The offset word is a LENGTH, not a position: 0x1000 + 2*pages, where pages = ceil(event_size / 512), and event_size comes from the chain walk. ONE request returns the entire event -- no chunk loop, no STRT end-offset parsing, no TERM frame. Over-requesting is safe; the device caps at the real size. - Params are the Series III probe form: [0x00][key4][6 x 0x00]. The payload is the .IDFW file byte for byte. It begins 00 12 01 00 00 00 "Instantel\0" -- _THOR_PREFIX + _INSTANTEL_TAG from micromate/idf_file.py -- and the first 32 bytes are identical to a production .IDFW from the store. Responses are DLE-stuffed, so destuff before locating the file (11,781 raw -> 11,049 destuffed for an 11,032-byte event). End-to-end: event 055d4a82 downloaded over USB and fed straight to read_idf_file() yields serial UM12947, timestamp 2026-09-23 16:33:19, and 3072 samples on all four channels. Cross-check: the 0C record reports a stored Vert peak of 1.3720 for this event; the decoded samples give 1.3706 -- two unrelated paths agreeing to 0.1%. Consequence: no new codec work is needed. The bytes off the wire are the same bytes thor-watcher forwards today, so /db/import/idf_file ingests a directly downloaded event unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
45007e12d8 |
docs(series4): the firmware images are unencrypted and self-documenting
Both MICROMATE(CB).BIN and MICROMATE(BD).BIN are plain code and data --
entropy 6.08 bits/byte, big-endian vector table at 0x4010_30xx, ~16,700
extractable strings including the developers' own debug printf formats with
function names intact.
This answers, from strings alone, questions I had scoped as needing a live
modem capture.
The call-home state machine, verbatim:
ACH_NOT_STARTED -> ACH_IDLE -> ACH_INITIALIZING -> ACH_CONNECTING
-> ACH_CONNECTED -> ACH_TRANSFER_DATA -> ACH_RETRY / ACH_QUITTING
And with it:
- Retry limit is three ("three attempts and it's over").
- ExpectedCommunicationsDetected() gates the session: if the host does not say
something the unit recognises, the call is cancelled and rescheduled after
TimeBetweenRetries. A homebrew receiver must satisfy this check or units
retry forever -- exactly the BE12599 failure mode.
- The unit stops monitoring to call home and restarts after
(Send CMD_STOP_MONITOR / CMD_START_MONITOR), so monitoring state around a
call is the device's own doing.
- Calls are not re-entrant.
- CMD_CALLHOME_CONNECTION_CONFIRMED exists as a state distinct from
CONNECTION_COMPLETE, implying a handshake the host must complete before data
flows.
Event delivery, inferred not confirmed: "All Events Uploaded" plus
"Mark/Unmark File" / "Delete Marked Events" / CMD_PURGE_EVENT_FLASH suggest
events are marked as transferred rather than deleted on send, with purging a
separate explicit act. If so, a receiver that fails to mark would see the same
events re-offered every call. Needs a live capture or disassembly to confirm.
The firmware also embeds its own HTML user manual, documenting modem mode
(Generic vs USB to PC), the modem baud options (9600-230400, confirming 115200
is a setting not a fixed rate), modem relay/warmup, record modes, and a
scheduler downloaded from THOR that pairs with CMD_CALLHOME_SET_SCHEDULE.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
|
||
|
serversdownandClaude Opus 5
|
38ad58d4a4 |
docs(series4): A/B the two firmware lines -- the protocol is the same
UM12947 (11.0CB, Blastware line) and UM20147 (11.0BD, Thor line) each given the identical read-only sweep on the bench. Both answer Series III command frames: all ten read SUBs, correct response-SUB rule, valid DLE-aware checksums, working two-step probe/data reads. The firmware line does not change the wire protocol. One protocol stack can drive the whole fleet regardless of build, which downgrades "standardise the fleet on one firmware" from a prerequisite to an optional convenience. Two differences do exist: 1. Response payload[1] (flags) is 0xC5 on the Blastware line and 0x03 on the Thor line, constant across all ten SUBs on both units -- so the build is detectable from any response without reading device info. Two units, one each, so this is a strong hypothesis rather than a proven encoding. Note 0x03 is ETX, so it arrives DLE-escaped as 10 03 on Thor-line units. A parser that does not destuff will mis-locate every field by one byte on half the fleet. 2. SUB 0x1C (monitor status) is 4 bytes longer on the Thor line, 0x30 vs 0x2C, with four extra trailing bytes (0f a0 00 00, purpose unknown). That second one breaks relative-to-end parsing: Series III reads battery and memory from the end of the 0x1C block, and those offsets yield a battery reading of 577.92 V on UM20147. Parse forward from the declared length, not backward from the end. With the shift applied, UM20147 reads 3.81 V and 15,000,000 bytes total/free. Also noted: ID string is MM/ISEE/S/IO on the Blastware unit and MM/ISEE/S on the Thor one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
492b6683a4 |
docs(series4): fleet firmware audit, and retract the Thor-compatibility claim
Physical audit of all nine Micromates: 4 on the Blastware line (11.0CB), 2 on the Thor line (11.0BD), 3 pre-split (11.0AK x2, 10.90GC). The Blastware line is already the plurality, which makes "standardise on Blastware" less disruptive than it first looked. Cross-checked against a store-derived audit (firmware is recorded in every .sfm.json as extensions.idf_report.version): 7 of 9 agree. The two that differ, UM6047 and UM14133, are the most recently deployed and were reflashed after their last stored event -- so the store reconstructs firmware history without touching a unit, but lags reality by one deployment. RETRACTION: an earlier draft suggested UM12947's trouble with Thor was explained by its Blastware firmware. Not supported. Ped Bridge runs UM11402 (11.0BD) and UM11719 (11.0CB) side by side from the same deploy date and both call Thor fine -- UM11719 has 331 Thor-collected events while on 11.0CB. A Blastware-line unit does feed Thor, so the CB/BD split is not "which host can collect from it", and UM12947's problem remains unexplained. What is actually established is narrower: a 11.0CB unit answers Series III command frames. Whether a 11.0BD unit does is untested -- and UM20147 (11.0BD) is on the bench, so that is one A/B away. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
73eaa0a6ac |
docs(series4): the event chain, walked end to end
Five events on the bench unit (4 waveform + 1 histogram). The Series III browse walk -- 1E, then 0A/0C per key, then 1F to advance -- works unmodified, and the null sentinel terminated correctly after exactly 5. Findings: - Event keys are a sequential counter (055d4a81..85), NOT flash-buffer addresses. Series III key arithmetic does not carry over; its 5A chunk walk assumes addresses and must not be ported blindly. - The 4 bytes after the key in 1E/1F are the event's SIZE in bytes, where Series III puts an offset to the next key. 4,076 for the histogram and 8.7-13.4 KB for the waveforms, matching real .IDFH/.IDFW file sizes. - SUB 0x0C returns a 210-byte (0xD2) waveform record -- the same length as Series III -- carrying the event key, date/time, the title note "Location", the PROJECT STRING, the serial, channel labels Tran/Vert/Long/Mic and float32 peaks. That last point closes the biggest open question for the call-home receiver: the job identity strings that today arrive only via Thor's .txt sidecar, and which no amount of sample decoding can reconstruct, are readable over the wire. Direct-to-SFM events need not arrive with blank metadata. - SUB 0x0A returns len 0x1E for the histogram and 0x00 for every waveform. The histogram payload holds two timestamps plus a "Vert: 0.300 in/s" trigger string -- structurally the Series III monitor-log partial record. So 0A describes interval records and 0C describes triggered events; Series III's 0x46-vs-0x2C length discriminator does not apply. - DLE stuffing in responses is now confirmed (previously marked untested): the 0C timestamp contains 10 10, which destuffs to one 0x10 and yields a clock reading of 16:33 on 23 Sep 2026 -- matching when the events were recorded. Read-only throughout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
b9c52442a7 |
docs(series4): the Series III behaviour is firmware-conditional
The bench unit reports 11.0CB -- Instantel's *Blastware* firmware line. It almost certainly answers Series III commands because it is in Blastware mode, not because the Micromate natively speaks Series III. Instantel ships two lines: 11.0CB (Blastware) and 11.0BD (THOR, Vision, Vision II). That also explains the two-ACH-server problem as designed behaviour rather than misconfiguration. Corpus firmware audit: 932 event files from 11.0AK, 83 from 10.90GC. UM12947 itself produced 10.90GC files in production last year and reports 11.0CB now, so units get reflashed and firmware is not stable per-unit over time. Records the resulting strategic fork (standardise on the Blastware line vs reverse-engineer the Thor line) with the four unknowns that decide it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |
||
|
serversdownandClaude Opus 5
|
095834183e |
docs(series4): open the Micromate live-protocol reference
First bench session against a Micromate over USB. The headline: the unit answers Series III command frames unmodified. An untouched Series III POLL (SUB 0x5B), built by build_bw_frame with no changes, completed a full two-step probe/data cycle. Ten Series III read commands were then tried and all ten answered, every one obeying the response_SUB = 0xFF - request_SUB rule. Confirmed this session: - Transport is a plain USB CDC-ACM port (2504:0300, "MICROMATE COM PORT"). No vendor driver, no Thor, no Windows box needed. Baud is ignored over USB (identical responses at 38400 and 115200). - Device never speaks first -- 20 s idle listen produced nothing. - Responses are Series III framing MINUS the leading DLE: bare [STX][payload][chk][ETX]. This alone means Blastware can never find a frame boundary in Micromate traffic, since its parser scans for DLE+STX. - Response flags byte is 0xC5, not Series III's 0x10. - Checksum is the DLE-aware variant (SUM8 excluding 0x10 bytes) -- the same one Series III uses for 5A and write frames, not the plain SUM8 of its ordinary reads. Disambiguated by the POLL data frame, which contains a 0x10. - The probe response carries the data length at payload[9]. Four of four known Series III lengths match; call-home config differs (0x7E vs 0x7C). - Series III monitor-status field offsets apply unchanged: battery 3.81 V (Thor's own reports say 3.8), memory 15,000,000 total and free, date 23 Sep 2026. - SUB 0x2C carries the string "RADIO RING" -- the same string seen in the RV50 ALEOS debug during the BE12599 incident. That block holds the modem dial/answer strings and is the most relevant command to the call-home goal. Read commands only. Nothing that writes, erases, or changes monitoring state has been sent to a unit; those are listed as unsafe-until-agreed. Caveat recorded in the doc: one unit, over USB, with zero events stored, so the event-walk commands (0x08, 0x1E, 0x0A, 0x06) could only be probed, not exercised. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL |