Commit Graph
509 Commits
Author SHA1 Message Date
serversdownandClaude Opus 5 45f2997a5b feat(bridges): mm_link -- a bench "modem" with a readable log and fault injection
THOR gives almost no visibility into a connection: a refresh button, two poll
intervals, and no way to see whether a check succeeded, timed out, or was never
attempted.  When a unit "won't stay connected" there is nothing to look at.  This
sits where the cellular modem would and answers that directly.

Over socat -x it adds the two things that were missing:

  * A READABLE LOG.  Frames are decoded and timestamped as they pass --
    "THOR->unit  POLL  (21 B)" rather than hex -- so THOR's polling cadence, and
    its silences, are visible.  Raw .bin pairs are still written alongside and
    load straight into scratch/mm_frame_parse.py.
  * FAULT INJECTION, via a control file read on the fly:
        pass       normal relay
        blackhole  TCP stays up, bytes are swallowed
        drop       close the connection abruptly
        delay:N    forward N seconds late, both directions
        onewaydev  THOR->unit passes, unit->THOR is swallowed

`blackhole` is the point of the exercise.  It reproduces the classic cellular
failure -- socket open at both ends, nothing crossing -- which a real cell link
will not do on cue.  THOR was observed last night holding one TCP connection for
17 minutes (00:30 to 00:47), so if the link dies silently the OS will not tell it
for roughly the default keepalive, ~2 hours.  That is a candidate explanation for
"refresh does nothing and only a restart helps", and this makes it testable
rather than speculative.

No pyserial: the port is driven through stdlib termios.  The bench hosts are
whatever is to hand and requiring a pip install on someone else's machine is a
poor trade for ~30 lines.  Deployed and verified on mint-mac (Python 3.12, no
third-party modules) against UM12947 -- a POLL round-trips and decodes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 11:29:44 -04:00
serversdownandClaude Opus 5 96a8831472 docs(series4): pin down what the ACH config's volatile field is NOT
Followed up the write[118:120] field flagged in the previous commit.  A third
sample plus a brute-force sweep rules out most of the obvious explanations.

Samples: 43 23 at 00:50 (idle), still 43 23 at 01:16 twenty-six minutes later,
then 2e 5e after a config write.  Thor echoes back whatever it last read --
including a value that no longer matches the config it is sending -- and the
write is accepted regardless.

Ruled out:
  * a clock or timer  -- identical across 26 minutes of idle; only a write moved it
  * a counter         -- it decreased, 17187 -> 11870
  * computed by Thor  -- Thor demonstrably sends a stale value
  * a standard CRC16  -- swept all 65,536 polynomials x init {0x0000,0xFFFF} x all
                         four reflection combinations over four candidate regions.
                         No match.  Recorded so the sweep is not repeated.

It behaves like a unit-computed hash: a one-byte input change scattered the output
(XOR 0x6D7D) where a sum would move by 1.  But two samples cannot separate that
from a nonce regenerated per write.

Operationally it does not matter, which is the point: the unit does not validate
the field on input, so read-modify-write with the rest of the block echoed
verbatim is provably safe.  Never synthesise or zero it.

Notes what would resolve it -- several config writes with times recorded, cheap to
collect during any future ACH capture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 01:56:21 -04:00
serversdownandClaude Opus 5 8e37803d40 docs(series4): event download, PER-EVENT delete, and the ACH config write
Three captures with operator ground truth (Thor screenshots of the event list and
the weekly schedule).  All Thor-originated.

DAY AND SCHEDULE TYPE, both settled by a weekly schedule.  Thor's screen showed
"Start Monitoring 8:00 AM every day, alternating TEST1/test2, Repeat Weekly
disabled".  The file is 7 x 260 + 4 = 1824 bytes and every record matches row for
row:

  Day = 0 Sunday .. 6 Saturday

and [0] on record 0 read 4, against 2 and 3 in the daily schedules, so it carries
schedule TYPE as well as Repeat:  2 daily, 3 daily+repeat, 4 weekly,
5 weekly+repeat (predicted, unobserved).  Bit 0 is Repeat; 2 and 4 are the bases.
Same bitmask style as [6].

In daily schedules Day reads 3 everywhere and is presumably ignored -- inference,
and the value 3 is unexplained.

EVENT DOWNLOAD.  SUB 0x93 -> 0x6C arms each event before 1E/1F, with empty params
and an all-zero ack -- the Series IV analogue of Series III's 1E(token=0xFE), and
simpler.  Event keys are a plain sequential counter (055D4A81..86 for six events)
at data[11:15], with the event size at data[17:19].  SUB 0x0A walks the list as
30-byte timestamped records; the dates match Thor's event list exactly.

DELETE IS PER-EVENT -- and this is the last piece a homebrew ACH receiver was
missing:

    0xA8  params[0:4] = <event key>  -> ack 0x57
    0xAA  params = zeros             -> ack 0x55

The operator deleted the top row of Thor's list (the newest event) and 0xA8
carried 055D4A86, the highest key from the walk.  Confirmed end to end.

Strictly safer than Series III, which can only erase everything: a receiver can
delete exactly what it has confirmed it stored.  Different opcodes -- do not reach
for 0xA3/0xA2.  Noted that SUB 0x06 read identically before and after, so it is
not a way to confirm a deletion landed.

ACH CONFIG.  0x2C / 0x7E / 0x7F with acks 0xD3 / 0x81 / 0x80 -- identical to
Series III.  126-byte write payload, offset 0x007E; the 0x2C read returns the same
bytes behind an 11-byte prefix.  The enable flag is write[5]: 0x05 enabled, 0x04
disabled.  Bit 0 is the flag, bit 2 set in both states -- do NOT test for
0x01/0x00 as Series III does.  Dial string at write[6:] ("RADIO RING").

Flagged: write[118:120] changed on its own between the two sessions (43 23 ->
2e 5e) with nothing touched, and Thor writes back whatever it read.  Round-trip
that field, never synthesise it.  Beyond the enable byte and dial string the field
map is NOT established -- only one setting was varied, and Series III's offsets
are a hypothesis, not a transfer.

Every command on the unsafe list is now observed.  None has been originated by
us, which is the line that still matters.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 01:26:42 -04:00
serversdownandClaude Opus 5 f839541d08 docs(series4): CORRECT the schedule record -- [6] is the Action bitmask, not [0]
A five-entry schedule (start/stop/self-check/start/ACH) overturns the two
previous readings of this record, and the operator supplied a Thor screenshot of
the schedule as ground truth.

[6] is the Action, and the values are powers of two:

    2 = Start Monitoring    4 = Stop Monitoring
    8 = Self Check         16 = Auto Call Home

Bits 1-4 of a bitmask; bit 0 (value 1) is unobserved -- a natural home for the
setup-less DUTYCYCLE_START_MONITOR, but that is a guess.

Two retractions:

  * [6] was recorded as "the one unidentified field" and predicted to be the
    Repeat flag.  It is the Action.
  * [0] was labelled Action, then "Action with repeat folded in".  Both wrong.
    [0] is non-zero only on record 0 -- records 0 and 3 here are the SAME action
    with different [0] values.  It is a schedule-level field carried in the first
    record, holding Repeat: 3 on, 2 off, matching "Repeat Daily: Disabled" on the
    Thor screen.

The earlier repeat capture was consistent with both readings because it had one
start entry and moved one byte.  A single-variable test is not always enough; it
took four distinct actions to separate the fields.

SUB 0x47 is confirmed as the scheduler enable.  Previously recorded as "genuinely
undetermined" whether it sets or reads -- Thor's notification pane timestamps it:
schedule write completes 00:51:29, "successfully SENT" 00:51:31, the 0x47 pair at
00:51:31 and 00:51:33, "successfully ENABLED" 00:51:35.  Nothing else sits between
the two notifications.  params[7] in {1,3} is still open and is more likely a
selector than a value, since a lone params[7]=3 also appears at session start.
It must be DLE-escaped -- a bare 0x03 truncates the frame.

SECOND RETRACTION: setups ARE written as raw .MMB files.  This document twice
said they are not.  Thor used both paths in one session, choosing by whether the
setup is active:

    TEST1.mmb (active)     0xDA -> 0x68/0x73 -> 0x82/0x83 -> 0x71/0x72
    test2.mmb (not active) 0x8D \system\setups\test2.mmb -> 0x8E (2192 B)

The .MMB file is nearly the compliance block -- 1968/2086 bytes equal (94.3%) at
a 4-byte shift, 102 bytes longer, name at offset 38 vs 42.  Same structure,
different framing.  Writing a setup as a file is the cleaner path for SFM: two
frames, no 0xDA/0x68/0x82 ritual, and it does not disturb the active setup.

Also: file writes are chunked.  The schedule's 1,304 bytes went as 1,024 + 280
with each offset = that chunk's length, but the 2,192-byte setup went in one
frame, so 1,024 is not a hard ceiling.  Rule unexplained, recorded as observed.

Capture provenance: the seismo_lab bins were empty (capture not stopped), and the
session was recovered from the socat relay log again -- 38 frames each way, 0 bad
checksums.  That fallback has now saved two captures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 00:55:54 -04:00
serversdownandClaude Opus 5 cc1388df65 feat(scratch): recover captures from the socat relay log -- validated byte-exact
The bench relay runs socat with -x, which hex-dumps every forwarded byte in both
directions.  That makes its log a complete second copy of every capture taken
through it, independent of whether seismo_lab was recording.

On 2026-09-25 a capture's .bin files never left the Windows machine and the
session was rebuilt from the relay log instead.  When the real bins turned up
afterwards, the reconstruction was byte-for-byte IDENTICAL in both directions
(3,595 and 4,004 bytes) -- verified again through the committed script, not just
the ad-hoc version used at the time.  So this is a validated fallback, not a lossy
approximation.

Adds scratch/socat_log_split.py, with --from-line/--to-line for picking one
session out of a log that spans several (split on the "accepting connection"
markers, or the frame walk runs sessions together).

Also documents the -x flag and the fallback in the session-provenance section, so
the next person runs the relay in a way that keeps the safety net.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 00:37:10 -04:00
serversdownandClaude Opus 5 98cbdad489 docs(series4): Repeat is folded into Action -- and [6] is not the repeat flag
A capture that changed ONLY the Repeat Daily checkbox (pull schedule, disable
repeat, push schedule) moved exactly one byte in the entire session:

    record @0   Action  0x03 -> 0x02

Record 2 unchanged, [6] unchanged, and 0xDA / 0x68 / 0x82 / 0x71 / 0x94 / 0x8D all
byte-identical.

Two corrections to the previous commit:

  * [6] is NOT the repeat flag.  That was the field I predicted this capture would
    isolate; it stayed 2 and 16.  Still unidentified.
  * The label "Action" on [0] was too simple -- it carries repeat behaviour too.

Leading hypothesis: 2 and 3 are the two setup-bearing start actions the firmware
names, with repeat selecting between them --
    0 = DUTYCYCLE_CALLHOME
    2 = DUTYCYCLE_START_MONITOR_WITH_SETUP                 (repeat off)
    3 = DUTYCYCLE_START_MONITOR_WITH_SETUP_STOP_COMPLETE   (repeat on)

Supported independently by the THOR manual, which says a repeating schedule
hitting a Start Monitoring event while already monitoring will "stop the current
monitoring session, run any Auto Call Home actions, load the compliance setup and
continue monitoring" -- exactly what _STOP_COMPLETE should mean.  A repeating
start must terminate the in-progress session; a one-shot start need not.  The
firmware name, the manual's behaviour and the single moved byte all agree.

Explicitly NOT claiming the enum ordering: the action names were recovered with
`strings | sort`, so source order is lost.  Do not infer 1 = START_MONITOR just
because it falls between the two known values.  Three of six codes observed.

Also noted: Thor re-pushed the whole 2,090-byte config for a one-byte schedule
change -- a third instance of the schedule<->config coupling.

Capture provenance: the seismo_lab bins did not reach the dev box, but the socat
relay on mint-mac keeps its own timestamped -x log, and the session was
reconstructed from it byte-for-byte (25 frames each way, 0 bad checksums).  That
backup log is worth keeping in the loop -- it has now saved a capture once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-25 00:35:09 -04:00
serversdownandClaude Opus 5 4a1cccf3f5 docs(series4): the schedule file is DECODED -- two records, five confirmations
The operator supplied the ground truth: entry 1 is "start monitoring TEST1" at
07:30, entry 2 is "Auto Call Home" at 19:30.  That decodes the file completely.

The body is two 260-byte records plus four zero bytes = 524 exactly, and every
non-zero byte falls inside them:

  [0]     Action       (0 = Auto Call Home, 3 = start monitoring with setup)
  [1:4]   padding
  [4]     1/2h         half-hour slot, 0-47
  [5]     Day
  [6]     ??           the one unidentified field
  [7]     name length
  [8:260] setup name, null-padded

  record @  0: Action=3  1/2h=15 -> 07:30  Day=3  [6]=2   namelen=9  "TEST1.mmb"
  record @260: Action=0  1/2h=39 -> 19:30  Day=3  [6]=16  namelen=0  (no setup)

Five independent confirmations, no fitting:
  1. Slots 15 and 39 match the stated 07:30 and 19:30 on a 30-minute grid, and
     39-15 = 24 slots = exactly 12 hours.
  2. The length byte reads 9 for TEST1.mmb, 40 for the long name in the read
     capture, and 0 for the Auto Call Home entry.
  3. Auto Call Home carries NO setup name -- direct proof that a schedule entry
     can exist with no setup attached, which is what the earlier
     DUTYCYCLE_START_MONITOR finding predicted from the firmware side.
  4. The 260-byte stride lands record 2's 1/2h exactly at [264].
  5. 2 x 260 + 4 = 524, the whole body, nothing left over.

CORRECTION: `27 03 10` at [264] was recorded in the previous commit as a possible
trailer.  It is record 2's 1/2h, Day and [6] fields.  I had assumed the file held
one record and read the second one as padding -- the non-zero bytes were sitting
there the whole time.

Still open: [6] (2 on the start entry, 16 on the ACH entry -- a Repeat or Day/Week
capture would isolate it), the four remaining action codes, and whether the file
can hold unused record slots.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 21:40:05 -04:00
serversdownandClaude Opus 5 508448e2dd docs(series4): the schedule record format, named by the firmware itself
A debug printf in the scheduler names the record's fields outright:

  SCHEDULER : _ReadRecord(%d) -> %s (Action=%u, 1/2h=%u, Day=%u, Setup="%s") [WDAY=%d]

They fit the captured record, and the name-length byte anchors the alignment --
it reads 40 for the 40-character name pulled off the unit and 9 for TEST1.mmb
written back, same position, both directions:

  [0]     Action = 3
  [1:4]   zero (padding, or Action is a uint32)
  [4]     1/2h = 15   -> 30-minute resolution, 48 slots/day
  [5]     Day = 3
  [6]     unidentified (WDAY?)
  [7]     name length -- 0x28=40 read, 0x09=9 written  <-- confirms the layout
  [8:]    setup name
  [264]   trailer 27 03 10

Slot 15 would be 07:30 counted from midnight; flagged unconfirmed because the
schedule's actual time was not recorded with the capture.

Six duty-cycle actions, not the five previously recorded -- there is also
DUTYCYCLE_START_MONITOR_WITH_SETUP_STOP_COMPLETE.  THOR exposes four.  Two start
variants it never offers, one needing no setup file.

Strengthened the setup-less-action finding and ruled out an alternative
explanation I had not considered: the Micromate has a separate Timer Mode
(MODE_TIMER, Monitor Once Only, under Special Setup), so START_MONITOR could have
belonged to that path.  It does not -- it is a case in _PSA(), the scheduler's own
dispatcher for _ReadRecord's Action field, and `_PSA() send ->> CMD_DUTYCYCLE_
START_MONITOR` shows it is live code sending a real message, not a dead case.

Also: SysPref.bMonitorScheduler places the scheduler enable in system
preferences, which confirms from the other side why the 0x71 block was
byte-identical when the scheduler was switched on -- the flag was never going to
be in the compliance config.  It also suggests 0x47 is a SysPref get/set rather
than anything scheduler-specific, which would explain its params[7] selector and
its response shape matching 0x48's page-0 descriptor.  Still a hypothesis.

Names the one capture that would settle the rest: a schedule with TWO entries at
different times with different actions.  That yields the record stride, the action
code values, and the time encoding at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 20:29:51 -04:00
serversdownandClaude Opus 5 3f58402085 docs(series4): the ACH session, from the THOR manual -- and I was asking the wrong question
Brian uploaded the Instantel manuals (gitignored, manuals/).  The THOR Operator
Manual Rev 08 settles the thing this document called the blocker for a homebrew
receiver.

I had written that what gates a receiver is "how it learns an event was accepted
so it stops re-sending it."  There is no such mechanism to find, because the unit
does not track it.  Per THOR manual 6.2.2.2 an ACH session is a list of
SERVER-chosen actions -- Copy events, Copy monitor log, Delete events and Logs
from Unit, Set Date/Time -- and "Only applies to events not previously
downloaded" is computer-side bookkeeping.  The manual's own warning proves copy
and delete are decoupled: enable delete but disable copy and "the events and logs
will be deleted without being uploaded."

That is exactly the model our Series III ACH server already implements
(ach_state.json high-water mark, erase as a deliberate separate step).  No new
mechanism is needed for Series IV.  A receiver needs: accept, identify, walk the
events (already solved), keep our own high-water mark, optionally erase.  The
ERASE OPCODES are now the only genuinely missing piece and stay on the unsafe
list.

Other things the manual settles:

  * The session is server-driven, matching the firmware state machine.  Scheduled
    and event-triggered ACH differ: with Monitoring While Calling Home enabled, an
    event-triggered session will NOT delete events or sync time.  So a receiver
    that relies on erase to avoid re-reading would silently never erase on those
    units -- the high-water mark has to be primary, erase an optimisation.
  * Session Time Out is unit-side only, which places it in callhome.MMB -- another
    reason to read that file with 0x94.
  * Units are routed by serial number with wildcards, so the serial is presented
    early enough for a server to dispatch on it.
  * THOR requires Idle for ACH setup too, confirming the greyed-out send is
    deliberate policy rather than a device refusal.
  * THOR exposes four schedule actions; the firmware has five.  6.3.2 step 8
    ("A Unit Setup must exist") is THOR's own requirement, while the same section
    says the unit "will execute any actions in a schedule using its current
    settings" -- the two pull opposite ways, consistent with START_MONITOR
    existing and THOR never emitting it.
  * Schedule fields to look for when the entry is decoded: action, setup name,
    time, day-or-week, day selection, repeat.  The captured entry has seven bytes
    before the name, the right order of magnitude for that list.

Flagged: the manual's filter example contradicts its own table (it has UM* and MP*
backwards).  The table is right.

Marked throughout as vendor documentation rather than observed bytes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 20:02:30 -04:00
serversdownandClaude Opus 5 8c2dacf035 docs(series4): the schedule has a setup-less start action -- Thor never uses it
The schedule<->config coupling looked like it might be a workaround for the unit
crashing on a missing setup.  The firmware says otherwise: there are two distinct
start-monitoring actions in the scheduler's duty-cycle dispatch, and only one
involves a setup file.

    _PSA() case DUTYCYCLE_START_MONITOR
    _PSA() case DUTYCYCLE_START_MONITOR_WITH_SETUP

Full action set: START_MONITOR, START_MONITOR_WITH_SETUP, STOP_MONITOR,
CALLHOME, SELF_CHECK, plus ON/OFF/NEXT for scheduler state.

So the coupling is not a crash workaround -- Thor picks the more demanding of two
available actions every time.  SFM can emit START_MONITOR and skip the config.

On whether a missing setup would actually break the unit: the firmware suggests
graceful degradation (`Setup File Not Found`, and `Invalid parameters reset to
factory default - please review setup`, a deliberate fallback).  Untested, and
recorded as untested.

Hypothesis, flagged as such: the schedule entry's leading byte may be the action
code -- the one captured entry reads `03 00 00 00 0f 03 02 [namelen][name]` and
03 would fit START_MONITOR_WITH_SETUP.  One entry, nothing to diff, unverified.

Names the capture that would settle it, and which is worth more than the 0x47
disable/enable test: a schedule entry with a setup-less action (stop monitoring,
or call home).  It would confirm or kill the action-code hypothesis and prove
from the other direction that a schedule needs no config push.

Also noted: DUTYCYCLE_CALLHOME -> CMD_SCHEDULE_CALL_HOME means a scheduled
call-home can make a unit dial out on demand -- the one remaining lever on the
unsolved call-home direction.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 19:49:59 -04:00
serversdownandClaude Opus 5 ebcc55e2ec docs(series4): the schedule<->config coupling is Thor's, not the protocol's
In Thor, a schedule entry that starts monitoring forces you to attach a setup,
and sending the schedule pushes that setup too, overwriting anything with the
same name.  The protocol requires none of it.

Evidence from the scheduler capture:

  * The two writes are separate operations, not one transaction.  The config
    write ends at frame 18, Thor sends a fresh POLL preamble, and only then opens
    the schedule at frame 20.  Different commands, different paths:
        config    0xDA -> 0x68/0x73 -> 0x82/0x83 -> 0x71/0x72
        schedule  0x8D -> 0x8E
  * The schedule stores a length-prefixed NAME, not a config blob.  It is a
    reference, and a reference does not require rewriting its referent.
  * The config Thor pushed was already on the unit unchanged -- its 2,090-byte
    0x71 payload is byte-identical to the previous capture's, zero differences.
    Thor spent a whole block write re-sending a setup the device already had.

So SFM can, with today's protocol: enumerate setups with 0x3F/0x40, write ONLY
the schedule when the referenced setup already exists, and push a config only
when it is genuinely missing or deliberately edited.  Common case drops from
524 + 2090 bytes to 524, and the write disappears entirely.

It also removes a real hazard.  Because the reference is by name, and a same-name
write overwrites silently with an indistinguishable ack, Thor's pattern means
scheduling something can quietly rewrite a setup that other schedules or the
operator's own work depend on.  Validating the reference instead of rewriting the
referent avoids the class of problem.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 19:46:09 -04:00
serversdownandClaude Opus 5 bd876b6752 docs(series4): separate Thor's conventions from the protocol's requirements
SFM is not meant to reimplement Thor.  Thor is the only available teacher of the
wire protocol, but almost nothing about how it sequences its work has been shown
to be required by the device, and this document was starting to blur the two --
it said "a client should mirror it" where the honest claim is "Thor does this and
we have not checked whether the unit cares."

Adds a table separating the two, with required / not-required / unknown marked
honestly, and corrects the two places that gave Thor-copying advice.

The consequential unknowns, all testable:

  * Thor's POLL -> 0x15 -> 0x49 -> POLL preamble before EVERY operation.
    Plausibly required (Series III needed POLL x3 before 5A) but Thor sends it
    before trivial reads too.
  * 0x68 and 0x82 appear in every setup push carrying near-zero payloads that
    changed nothing in either capture.  If optional, our setup write is 3 frames
    instead of 7 with less to get wrong.  Worth settling BEFORE building the
    writer.
  * Whether a narrower write than the full 2,090-byte block is accepted.

One place Thor's shortcut is probably worse than the alternative: it reads with
offset=0xFFFF and skips the probe, but the probe works and reports the length
rather than making us trust a fixed one.

Two reliability problems to design against, both observed rather than assumed:
a zero ack does not mean a write applied (no failing write has ever been seen),
and nothing warns before clobbering a monitoring unit's active setup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 19:44:05 -04:00
serversdownandClaude Opus 5 c5eed6fa46 docs(series4): RETRACT "no file transfer" -- 0x94/0x48/0x8D/0x8E read and write by path
The scheduler capture caught a generic file transfer in the open, and it
invalidates a claim made earlier today.

RETRACTION.  The "Setups are FILES" section concluded "no generic file-transfer
command is exposed on the wire", reasoning from the absence of firmware strings.
Wrong.  The commands exist, they carry a full filesystem path in plain ASCII,
and they were the first thing Thor did when asked for the schedule:

    0x94 <path>  -> 0x6B   open for read
    0x48         -> 0xB7   read next page, until an all-zero response = EOF
    0x8D <path>  -> 0x72   open for write
    0x8E <body>  -> 0x71   write the body

Path is unpadded with offset = its exact length; 0x94 and 0x8D sent byte-
identical payloads for "\system\schedule\schedule.dat".  The 0x48 read is paged
with the page number in the response header at payload[3:5].

The lesson: absence of a firmware string is not absence of a command.  Dispatch
is a 68K jump table and these carry no strings.  The setups half of the original
claim survives -- setups go via 0xDA plus the config block, not via this.

Why it matters beyond the scheduler: callhome.MMB is a file too, and call-home
is the last unsolved goal.  Reading it may be a matter of pointing 0x94 at the
right path.  Recorded as a lead -- no path but schedule.dat has been tried.

The schedule file: an entry carries a length-prefixed SETUP FILE NAME (0x28=40
for the name read off the unit, 0x09=9 for TEST1.mmb written back -- confirmed
both directions).  So a schedule entry says "at this time, load this setup",
which is how the help text's "change the record mode" works, and it couples the
scheduler to the setup list.  Entry internals are NOT decoded and are recorded
as observed bytes only -- one entry, no variation to diff.

SUB 0x47 is the scheduler enable, probably: two bare frames differing only in
params[7] (0x01, 0x03).  Whether it sets or reads is genuinely undetermined --
both returned the same value and there is no disabled reading to compare.
Flagged do-not-implement until a disable-then-enable capture settles it.

A prediction made before the capture -- that the Scheduler On/Off switch would
show up as a byte in the 0x71 block, since the unit's help text lists it beside
Record Mode -- did not hold.  0x71, 0x68 and 0x82 are byte-identical to the
previous capture.  Noted that the test is weak (the operator re-sent the same
config), but enabling the scheduler required no config write either way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 19:26:39 -04:00
serversdownandClaude Opus 5 5400f1bef7 docs(series4): monitoring control, the setup-list walk, and the device clock
Thor started monitoring, listed the unit's setups and stopped monitoring while
seismo_lab recorded.  40 requests, 40 responses, every checksum valid.  As
before, Thor did all of it -- we have still never originated any of these.

Confirmed identical to Series III:

  * SUB 0x96 start monitoring -> ack 0x69
  * SUB 0x97 stop monitoring  -> ack 0x68

Both bare frames, no params, no data.  These were on the unsafe-until-agreed
list as entirely unobserved; they are now observed but still never sent by us.
Erase (0xA3/0xA2) is now the only genuinely untouched destructive path.

NOT identical to Series III, and worth not reusing constants for:

  * The monitoring flag is SUB 0x1C data[12] = 0x0E monitoring / 0x00 idle.
    Series III uses 0x10.
  * SUB 0x49 -> 0xB6 is a second, cheaper monitoring indicator at data[11]
    (0x02 monitoring / 0x00 idle) in a 21-byte response rather than 60.  Thor
    puts it in its preamble before every operation, so it is the routine check.

New this capture:

  * SUB 0x1C carries the DEVICE CLOCK at data[13:21] -- day, month, year (u16
    BE), hour, minute, second.  Verified against the capture's own wall time.
    Nothing else read so far reports the unit's time.  data[17] remains
    unidentified (32 monitoring, 100 idle) -- not claimed as anything.
  * Memory total is exactly 15,000,000 bytes; free dropped 4,096 bytes across a
    ~70s monitoring session, so free memory is not stable to compare against.
  * SUB 0x3F/0x40 walk the setup-file list, the same first/next shape as
    Series III's 1E/1F event walk.  0x3F -> 0xC0 first, 0x40 -> 0xBF next,
    terminating on an empty name.  23 setups on this unit.
  * Setup records carry ONLY the name -- 11-byte header, null-terminated name,
    zero padding.  There is no active-setup flag; the header is byte-identical
    on every record including the terminator.  The active setup is identified
    solely by SUB 0x41, which uses the same record format.  The asterisk on the
    unit's screen is UI decoration, not a field.
  * TEST1.mmb, created over the wire earlier today, appears in the list and is
    what 0x41 reports as active -- a written setup becomes a real enumerable
    file.

Also: Thor greys out send-to-unit while a unit is monitoring, and transmits
nothing (this capture contains no 0xDA or 0x71).  That is Thor policy, not a
device refusal -- nothing suggests the Micromate would reject it, and a push to
the active setup overwrites silently.  Thor is guarding the footgun the protocol
leaves open, and any client we write should do the same.  Checking 0x49 data[11]
first makes that cheap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 19:18:58 -04:00
serversdownandClaude Opus 5 b5e34ce8ae docs(series4): overwrite is protocol-identical to create -- no handshake
The firmware carries `Overwrite File`, `MFS FILE EXISTS` and `Cannot be
Overwritten`, which suggested the wire path might negotiate an overwrite.  It
does not.  Those strings belong to the on-device Save screen (CSaveSetupFile),
not the protocol.

A second Thor push to TEST1.mmb -- a name that now existed, and which SUB 0x41
confirmed was the ACTIVE setup -- produced an identical sequence:

  * same 12 SUBs in the same order, same offset fields
  * 0xDA / 0x68 / 0x82 data byte-identical
  * 0x71 differs in exactly 18 bytes = the one edited note string
  * all seven write acks identical and still all-zero
  * no dialog on Thor

Verified on the unit: the edited General Notes string is present in the setup on
the device.  The write applied silently and in place, and being the active setup
bought it no protection.

Two consequences recorded:

  * A writer needs no exists-check and no overwrite negotiation.
  * We have never seen this protocol report a FAILED write -- acks are all-zero
    across create and overwrite alike.  Do not treat a zero ack as proof a write
    applied; read back with 0x41 + 0x1A and compare.  And a remote push to a
    monitoring unit's active setup changes what it is recording with, unprompted
    -- gating that belongs in SFM, because the device will not do it.

Still untested: overwriting a non-active setup, and factory.MMB.  Neither
blocks a writer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 18:56:20 -04:00
serversdownandClaude Opus 5 d33e2d85be docs(series4): 0xDA creates setup files -- confirmed on the device
TEST1.mmb did not exist on UM12947 before the push.  After it, the setup is
present in the unit's own setup list and selected as active -- verified on the
Micromate's screen, not inferred from the ack.

This was the last open question about whether Series IV setup management is
reachable without Thor.  It is: 0x41 read name, 0x1A read block, 0xDA name the
target, 0x71 -> 0x72 write it back.  No file-transfer primitive is needed and
the target file does not have to exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 18:48:17 -04:00
serversdownandClaude Opus 5 c8d972f685 docs(series4): the setup-write path, observed end to end
Thor pushed a setup named TEST1.mmb to UM12947 while seismo_lab's TCP bridge
recorded both directions.  We still have not originated a write frame -- the
wire format is now known, our encoder is not written.

Topology worth reusing: socat shares /dev/ttyACM0 on TCP from mint-mac,
seismo_lab relays Thor to it.  Thor is pointed at 127.0.0.1 as if the unit were
a field modem.  No modem, no SIM, production Thor box untouched.

The sequence is Series III's, plus one command:

    Thor:  5B | 41 | 08 | 2E | 1A | DA | 68->73 | 82->83 | 71->72
    unit:  A4 | BE | F7 | D1 | E5 | 25 | 97  8C | 7D  7C | 8E  8D

All 12 device responses checksum-validate and every write is acked.  Every
write response SUB matches the Series III table exactly.

New:
  * SUB 0xDA names the target .MMB file -- 256 bytes, filename null-padded,
    nothing else.  This is why no generic file-transfer command exists: Thor
    names the file, then writes the ordinary config block into it.
  * SUB 0x41 reads the active setup's filename; SUB 0x2E reads trigger config.
  * Reads are single-step -- Thor asks offset=0xFFFF and skips the probe.
  * 0x71 writes the whole 2090-byte block in ONE frame, not Series III's three
    chunks.  0x69/0x74 are absent.

Write-frame destuffing is `10 XX` -> `XX` uniformly, including `10 03`.  Chosen
by checksum, not assumption: of four candidate rules, only this one makes all
four data-carrying write frames validate.  0x71's data holds 4 literal 0x03
bytes escaped as `10 03`, so escaping is mandatory for any writer.

The write body IS the read body -- 0x71 and the 0xE5 response align at a fixed
11-byte shift with 1902/2090 bytes equal (91.0%).  Setups are read-modify-write.
The 12 differing regions are fully mapped: setup name, four 64-byte
[label:22][value:42] note entries, sensor location, and the three geo trigger
levels (0.3 -> 0.5 in/s) on a 48-byte channel stride.

Independent confirmation of the geo LSB: each channel block carries float32BE
3.10308 at label+24.  3.10308/10000 = 0.000310308 = _GEO_LSB_IPS to 8 figures,
and 10.0/3.10308*10000 = 32226.046 = the 32226.05 full scale.  That value was
derived statistically from 991,415 rounding constraints in v0.30.0; the unit
reports it directly.  It is exactly half Series III's 6.206053, so the ADC runs
10,000 counts per volt.  Do NOT retune _GEO_LSB_IPS -- this corroborates it.

The `offset` field is NOT a single length formula: two frames are len, two are
len+2, and Series III's data[1]+2 reproduces neither.  Recorded as observed
constants the device accepted; pinning the rule needs a capture with
differently-sized payloads.  This doc has been wrong once by inferring a length
field -- not inferring this one.

Also adds scratch/mm_frame_parse.py, because S3FrameParser cannot see Micromate
responses at all (it scans for DLE+STX; Micromate responses start at a bare
STX).  That is why the first pass at this capture looked like 12 unanswered
requests.  24/24 frames parse with 0 bad checksums.

Stale claims corrected: the "write half is not yet attempted" note, the
"empty unit" limitation (5 events since 2026-09-23), and the unsafe-until-agreed
list, which now distinguishes observed from exercised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-24 18:35:58 -04:00
serversdownandClaude Opus 5 701af47170 docs(series4): generate IDF filenames rather than detecting record type
Closes the record-type gap flagged earlier, and corrects the premise behind it.

Series III does NOT detect record type from file content --
event_file_io.derive_record_type_from_filename() reads the last character of
the extension (M529LKIQ.G10H -> H -> Histogram). Nothing in the codebase infers
record type from content, for either family.

Nor is there an obvious type field to find in an IDF: the first 64 bytes of a
histogram and a waveform are byte-identical, and they diverge at ~0x0947 into
wholly different structures rather than differing by a flag.

The answer is the Series III pattern -- generate the name. Series III has
blastware_filename(); Series IV needs the same, and its convention is far
simpler:

    <serial>_<YYYYMMDDHHMMSS>.IDF{W,H}     e.g. UM12947_20260923163319.IDFW

against Series III's <letter><serial3><base-36 stem><AB0T ext>.

All three inputs are already available on a direct download: serial and
timestamp from extract_binary_metadata(), and type from the chain walk (SUB
0x0A returns 0x1E for a histogram, 0x00 for a waveform). Verified on all five
bench events -- generated names match real production-store filenames byte for
byte, so a directly downloaded event can be filed under exactly the name Thor
would have given it and /db/import/idf_file needs no change.

The type still comes from the protocol rather than the payload, so a
downloader must carry it out of the chain walk; losing it means losing the
ability to name the file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 19:50:45 -04:00
serversdownandClaude Opus 5 02ed22f561 docs(series4): firmware static analysis, and all five bench events decoded
Solo session while the bench was unattended. Read-only throughout.

Architecture: ColdFire/68K, big-endian, Freescale MQX RTOS -- not ARM as the
vector table first suggested. The tell is 4E 5E 4E 75 4E 56 (UNLK A6 / RTS /
LINK A6) throughout both images, plus an MQX_OK assertion.

CB vs BD: a byte diff is useless (68% of bytes differ -- separately linked
builds, everything relocated). A string-set diff is position-independent and
shows 17,128 strings shared, with almost every "unique" string being the same
message at a different source line:

    CB:  MONITOR[3268]: STATUS_BATTERY_LOW
    BD:  MONITOR[3258]: STATUS_BATTERY_LOW

Consistently 10 lines apart across five different MONITOR messages, so one
~10-line block differs in the monitor module and essentially nothing else. The
only functional string unique to either build is CITIZEN (a printer brand) in
BD. This corroborates the bench A/B from the other direction: the split is a
tiny code delta, not two protocol stacks.

The SUB dispatch is a 68K switch jump table, so byte-pattern hunting will not
isolate the write opcodes -- that needs a disassembler.

Call-home config field names recovered from the firmware's own debug dump:
Enable, DialString, Retries, SessionTimeout, WaitForConnection, WarmupTime,
PowerSave -- seven fields for the 126-byte SUB 0x2C block. SessionTimeout and
PowerSave have no Series III equivalent, and Series III's scheduled-time fields
are absent, consistent with scheduling moving into the THOR-downloaded
scheduler. AT+CSQ is present, so the firmware speaks AT to the modem directly.

All five bench events downloaded and decoded over USB: each arrived at exactly
its declared size, every channel equal length, timestamps sequential.

Two gaps recorded:

- No content-based record-type discriminator. read_idf_file() dispatches on the
  .IDFH/.IDFW filename suffix, which does not exist over the wire, and the
  first 64 bytes of a histogram and a waveform are byte-identical. The protocol
  supplies one instead: SUB 0x0A returns 0x1E for a histogram and 0x00 for a
  waveform, so the type must be carried from the chain walk.
- The 0x0C peak float runs 2-5% above max(Tran,Vert,Long) and is not the vector
  sum either. Its offset was inferred from a byte marker rather than
  established, so it may not be the peak at all. Marked do-not-rely-on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 19:14:26 -04:00
serversdownandClaude Opus 5 23cdbef737 docs(series4): setups are files; and the length field is a uint16
Two findings and one correction.

CORRECTION: the probe response's data length is a uint16 BE at payload[8:10],
not a single byte at payload[9] as an earlier draft claimed. That reading is
right only while the high byte is zero. For SUB 0x1A the real length is 0x082C
= 2092; read as a byte it gives 44, a 47x under-read.

Setups are FILES, not a config block. Series III has one compliance config you
overwrite; Series IV keeps named .MMB setup files on an on-device filesystem
with a current-selection pointer -- csetup.MMB, factory.MMB, and callhome.MMB
for the call-home config. Names up to 20 chars. Filesystem primitives exist
internally (NS_ReadFile_internal / NS_WriteFile_internal / NS_SeekFile_internal)
but no generic file-transfer command is exposed on the wire, so setups are
unlikely to be pushed as raw .MMB blobs over the protocol.

SUB 0x1A reads the whole active setup in 2,092 bytes -- structurally close to
Series III's ~2,126-byte compliance block -- carrying the setup FILE NAME, all
four title note/value pairs (Location, Client, Company, General Notes), the
sensor location, and per-channel labels with units. Note LMic and SMic
(linear and sound-level microphone variants) which Series III does not have.

That is the read half of setup management, so a setup can in principle be
round-tripped. The write half has not been attempted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 18:55:20 -04:00
serversdownandClaude Opus 5 71f19c90d1 docs(series4): SUB 5A streams the .IDFW file verbatim -- read path complete
The complete read path now works with no Instantel software in the loop.

Three divergences from Series III, all simplifications:

- No arming sequence. Series III ignores a 5A probe unless preceded by
  1E / 0A / 1E(0xFE) / 0C / 1F(0xFE) / POLL x3. The Micromate answers a bare
  5A request with nothing before it.
- The offset word is a LENGTH, not a position: 0x1000 + 2*pages, where
  pages = ceil(event_size / 512), and event_size comes from the chain walk.
  ONE request returns the entire event -- no chunk loop, no STRT end-offset
  parsing, no TERM frame. Over-requesting is safe; the device caps at the
  real size.
- Params are the Series III probe form: [0x00][key4][6 x 0x00].

The payload is the .IDFW file byte for byte. It begins 00 12 01 00 00 00
"Instantel\0" -- _THOR_PREFIX + _INSTANTEL_TAG from micromate/idf_file.py --
and the first 32 bytes are identical to a production .IDFW from the store.
Responses are DLE-stuffed, so destuff before locating the file (11,781 raw ->
11,049 destuffed for an 11,032-byte event).

End-to-end: event 055d4a82 downloaded over USB and fed straight to
read_idf_file() yields serial UM12947, timestamp 2026-09-23 16:33:19, and
3072 samples on all four channels. Cross-check: the 0C record reports a
stored Vert peak of 1.3720 for this event; the decoded samples give 1.3706 --
two unrelated paths agreeing to 0.1%.

Consequence: no new codec work is needed. The bytes off the wire are the same
bytes thor-watcher forwards today, so /db/import/idf_file ingests a directly
downloaded event unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 18:40:54 -04:00
serversdownandClaude Opus 5 45007e12d8 docs(series4): the firmware images are unencrypted and self-documenting
Both MICROMATE(CB).BIN and MICROMATE(BD).BIN are plain code and data --
entropy 6.08 bits/byte, big-endian vector table at 0x4010_30xx, ~16,700
extractable strings including the developers' own debug printf formats with
function names intact.

This answers, from strings alone, questions I had scoped as needing a live
modem capture.

The call-home state machine, verbatim:
  ACH_NOT_STARTED -> ACH_IDLE -> ACH_INITIALIZING -> ACH_CONNECTING
  -> ACH_CONNECTED -> ACH_TRANSFER_DATA -> ACH_RETRY / ACH_QUITTING

And with it:

- Retry limit is three ("three attempts and it's over").
- ExpectedCommunicationsDetected() gates the session: if the host does not say
  something the unit recognises, the call is cancelled and rescheduled after
  TimeBetweenRetries. A homebrew receiver must satisfy this check or units
  retry forever -- exactly the BE12599 failure mode.
- The unit stops monitoring to call home and restarts after
  (Send CMD_STOP_MONITOR / CMD_START_MONITOR), so monitoring state around a
  call is the device's own doing.
- Calls are not re-entrant.
- CMD_CALLHOME_CONNECTION_CONFIRMED exists as a state distinct from
  CONNECTION_COMPLETE, implying a handshake the host must complete before data
  flows.

Event delivery, inferred not confirmed: "All Events Uploaded" plus
"Mark/Unmark File" / "Delete Marked Events" / CMD_PURGE_EVENT_FLASH suggest
events are marked as transferred rather than deleted on send, with purging a
separate explicit act. If so, a receiver that fails to mark would see the same
events re-offered every call. Needs a live capture or disassembly to confirm.

The firmware also embeds its own HTML user manual, documenting modem mode
(Generic vs USB to PC), the modem baud options (9600-230400, confirming 115200
is a setting not a fixed rate), modem relay/warmup, record modes, and a
scheduler downloaded from THOR that pairs with CMD_CALLHOME_SET_SCHEDULE.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 18:28:40 -04:00
serversdownandClaude Opus 5 38ad58d4a4 docs(series4): A/B the two firmware lines -- the protocol is the same
UM12947 (11.0CB, Blastware line) and UM20147 (11.0BD, Thor line) each given
the identical read-only sweep on the bench. Both answer Series III command
frames: all ten read SUBs, correct response-SUB rule, valid DLE-aware
checksums, working two-step probe/data reads.

The firmware line does not change the wire protocol. One protocol stack can
drive the whole fleet regardless of build, which downgrades "standardise the
fleet on one firmware" from a prerequisite to an optional convenience.

Two differences do exist:

1. Response payload[1] (flags) is 0xC5 on the Blastware line and 0x03 on the
   Thor line, constant across all ten SUBs on both units -- so the build is
   detectable from any response without reading device info. Two units, one
   each, so this is a strong hypothesis rather than a proven encoding.

   Note 0x03 is ETX, so it arrives DLE-escaped as 10 03 on Thor-line units. A
   parser that does not destuff will mis-locate every field by one byte on
   half the fleet.

2. SUB 0x1C (monitor status) is 4 bytes longer on the Thor line, 0x30 vs
   0x2C, with four extra trailing bytes (0f a0 00 00, purpose unknown).

That second one breaks relative-to-end parsing: Series III reads battery and
memory from the end of the 0x1C block, and those offsets yield a battery
reading of 577.92 V on UM20147. Parse forward from the declared length, not
backward from the end. With the shift applied, UM20147 reads 3.81 V and
15,000,000 bytes total/free.

Also noted: ID string is MM/ISEE/S/IO on the Blastware unit and MM/ISEE/S on
the Thor one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 18:20:11 -04:00
serversdownandClaude Opus 5 492b6683a4 docs(series4): fleet firmware audit, and retract the Thor-compatibility claim
Physical audit of all nine Micromates: 4 on the Blastware line (11.0CB), 2 on
the Thor line (11.0BD), 3 pre-split (11.0AK x2, 10.90GC). The Blastware line
is already the plurality, which makes "standardise on Blastware" less
disruptive than it first looked.

Cross-checked against a store-derived audit (firmware is recorded in every
.sfm.json as extensions.idf_report.version): 7 of 9 agree. The two that differ,
UM6047 and UM14133, are the most recently deployed and were reflashed after
their last stored event -- so the store reconstructs firmware history without
touching a unit, but lags reality by one deployment.

RETRACTION: an earlier draft suggested UM12947's trouble with Thor was
explained by its Blastware firmware. Not supported. Ped Bridge runs UM11402
(11.0BD) and UM11719 (11.0CB) side by side from the same deploy date and both
call Thor fine -- UM11719 has 331 Thor-collected events while on 11.0CB. A
Blastware-line unit does feed Thor, so the CB/BD split is not "which host can
collect from it", and UM12947's problem remains unexplained.

What is actually established is narrower: a 11.0CB unit answers Series III
command frames. Whether a 11.0BD unit does is untested -- and UM20147 (11.0BD)
is on the bench, so that is one A/B away.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 17:08:47 -04:00
serversdownandClaude Opus 5 73eaa0a6ac docs(series4): the event chain, walked end to end
Five events on the bench unit (4 waveform + 1 histogram). The Series III
browse walk -- 1E, then 0A/0C per key, then 1F to advance -- works unmodified,
and the null sentinel terminated correctly after exactly 5.

Findings:

- Event keys are a sequential counter (055d4a81..85), NOT flash-buffer
  addresses. Series III key arithmetic does not carry over; its 5A chunk walk
  assumes addresses and must not be ported blindly.
- The 4 bytes after the key in 1E/1F are the event's SIZE in bytes, where
  Series III puts an offset to the next key. 4,076 for the histogram and
  8.7-13.4 KB for the waveforms, matching real .IDFH/.IDFW file sizes.
- SUB 0x0C returns a 210-byte (0xD2) waveform record -- the same length as
  Series III -- carrying the event key, date/time, the title note "Location",
  the PROJECT STRING, the serial, channel labels Tran/Vert/Long/Mic and
  float32 peaks.

That last point closes the biggest open question for the call-home receiver:
the job identity strings that today arrive only via Thor's .txt sidecar, and
which no amount of sample decoding can reconstruct, are readable over the
wire. Direct-to-SFM events need not arrive with blank metadata.

- SUB 0x0A returns len 0x1E for the histogram and 0x00 for every waveform. The
  histogram payload holds two timestamps plus a "Vert: 0.300 in/s" trigger
  string -- structurally the Series III monitor-log partial record. So 0A
  describes interval records and 0C describes triggered events; Series III's
  0x46-vs-0x2C length discriminator does not apply.
- DLE stuffing in responses is now confirmed (previously marked untested): the
  0C timestamp contains 10 10, which destuffs to one 0x10 and yields a clock
  reading of 16:33 on 23 Sep 2026 -- matching when the events were recorded.

Read-only throughout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 16:39:11 -04:00
serversdownandClaude Opus 5 b9c52442a7 docs(series4): the Series III behaviour is firmware-conditional
The bench unit reports 11.0CB -- Instantel's *Blastware* firmware line. It
almost certainly answers Series III commands because it is in Blastware mode,
not because the Micromate natively speaks Series III. Instantel ships two
lines: 11.0CB (Blastware) and 11.0BD (THOR, Vision, Vision II).

That also explains the two-ACH-server problem as designed behaviour rather
than misconfiguration.

Corpus firmware audit: 932 event files from 11.0AK, 83 from 10.90GC. UM12947
itself produced 10.90GC files in production last year and reports 11.0CB now,
so units get reflashed and firmware is not stable per-unit over time.

Records the resulting strategic fork (standardise on the Blastware line vs
reverse-engineer the Thor line) with the four unknowns that decide it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 16:04:42 -04:00
serversdownandClaude Opus 5 095834183e docs(series4): open the Micromate live-protocol reference
First bench session against a Micromate over USB. The headline: the unit
answers Series III command frames unmodified.

An untouched Series III POLL (SUB 0x5B), built by build_bw_frame with no
changes, completed a full two-step probe/data cycle. Ten Series III read
commands were then tried and all ten answered, every one obeying the
response_SUB = 0xFF - request_SUB rule.

Confirmed this session:

- Transport is a plain USB CDC-ACM port (2504:0300, "MICROMATE COM PORT").
  No vendor driver, no Thor, no Windows box needed. Baud is ignored over USB
  (identical responses at 38400 and 115200).
- Device never speaks first -- 20 s idle listen produced nothing.
- Responses are Series III framing MINUS the leading DLE: bare
  [STX][payload][chk][ETX]. This alone means Blastware can never find a frame
  boundary in Micromate traffic, since its parser scans for DLE+STX.
- Response flags byte is 0xC5, not Series III's 0x10.
- Checksum is the DLE-aware variant (SUM8 excluding 0x10 bytes) -- the same
  one Series III uses for 5A and write frames, not the plain SUM8 of its
  ordinary reads. Disambiguated by the POLL data frame, which contains a 0x10.
- The probe response carries the data length at payload[9]. Four of four
  known Series III lengths match; call-home config differs (0x7E vs 0x7C).
- Series III monitor-status field offsets apply unchanged: battery 3.81 V
  (Thor's own reports say 3.8), memory 15,000,000 total and free, date
  23 Sep 2026.
- SUB 0x2C carries the string "RADIO RING" -- the same string seen in the
  RV50 ALEOS debug during the BE12599 incident. That block holds the modem
  dial/answer strings and is the most relevant command to the call-home goal.

Read commands only. Nothing that writes, erases, or changes monitoring state
has been sent to a unit; those are listed as unsafe-until-agreed.

Caveat recorded in the doc: one unit, over USB, with zero events stored, so
the event-walk commands (0x08, 0x1E, 0x0A, 0x06) could only be probed, not
exercised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
2026-09-23 15:59:17 -04:00
serversdownandClaude Opus 4.8 154186a6cd Merge feat/event-timestamp-fix: exact waveform trigger time from the binary
read_blastware_file stamped waveforms with footer ts1 (the monitoring-session
start, hours off — vomit-list #3).  The event time is ts2 (recording stop) and
the trigger = ts2 - record time, a float32 in the recording-setup config block,
so the exact Blastware trigger is recovered from the binary alone (no .TXT).
Histograms keep ts1; a paired report's event_datetime stays authoritative.

Needs a re-decode backfill to correct existing stored events' timestamps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 23:13:19 +00:00
serversdownandClaude Opus 4.8 1765b3300d docs(changelog): waveform event-time fix (exact trigger from binary)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 22:32:13 +00:00
serversdownandClaude Opus 4.8 1e76d08b37 fix(decode): recover the exact waveform trigger from the binary (no .TXT)
Follow-up to the ts1→ts2 fix: get the trigger to the second from the binary
alone, instead of falling back to the stop time (~record-duration late) for
no-report events.

The configured post-trigger record time is a big-endian float32 in the
recording-setup config block, exactly 30 bytes before the "Standard Recording
Setup" marker.  _parse_record_time_seconds reads it; the waveform branch now
stamps trigger = ts2 - record_time.  Verified: the field reads 1.0 / 2.0 / 3.0 s
across different setups in the corpus, and all 7 BE12844 oracle events now
decode to their exact Blastware trigger (N844LQHB 10:33:29) from the binary,
no paired .TXT needed.  Falls back to ts2 (the stop) if the config block is
absent.  A paired report's event_datetime stays authoritative (clock drift).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 22:31:22 +00:00
serversdownandClaude Opus 4.8 a84a46e9d4 fix(decode): stamp waveform events with the event time, not the session start
read_blastware_file built ev.timestamp from footer ts1, which for a WAVEFORM is
the monitoring-session start (a unit arming at 06:00 stamps 06:00 on every event
that day) — so every waveform's time was hours off (vomit-list #3, "~4.5 h off").
The event time is footer ts2 (the recording stop); BW's displayed Date/Time is
the trigger = ts2 - record duration.

Root cause proven against the BE12844 oracle set: 5 of 7 events decoded to the
identical 06:00:13 (the shared session start); ts2 gives distinct plausible
event times (N844LQHB ts2 = 10:33:32, BW trigger 10:33:29 = ts2 - 3.0 s rectime).

  * read_blastware_file now uses ts2 for waveforms (discriminated by which codec
    decoded the body, not the filename — save_imported_bw passes a tmp name).
    Histograms keep ts1 (the ~24 h window start, which IS the event time).
  * Binary-only decode can't get the exact trigger: the STRT record-time byte is
    a misparsed record-type marker (0x46=70), so ts2 (the stop, ~record duration
    after the trigger) is the best estimate. A paired BW report carries the exact
    trigger — apply_report_to_event now overlays event.timestamp from
    report.event_datetime, matching the existing build-path override (line ~441).

Tests: waveform → ts2, histogram → ts1 unchanged, report → exact trigger.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-20 18:21:07 +00:00
serversdownandClaude Opus 5 ada5bc2a82 docs(changelog): Unreleased — cheap connect, Diagnostics tab, tool status
Written on dev as part of finishing the merge, per the convention adopted
2026-09-18: feature branches do not touch CHANGELOG.md, and the entry describes
what actually landed rather than what a branch intended.

First time through the new way rather than discovering the conflict afterward —
the merge was clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-20 17:12:49 +00:00
serversdownandClaude Opus 5 f1ab5b1e9d docs: record the 5A page-boundary bug, and assess SFM as a tool
Two things Brian asked for after the BE12599 work.

The known bug: the 5A walk discards the key's page byte, so once a unit has
recorded more than 64 KB since its last erase, an event spanning the boundary
reads an end_offset behind its own start. The chunk loop then fetches nothing
and TERM packs a negative offset_word, which is the 500. Reproduced on BE12599.
It hid this long because every capture the walk was verified against came from
a freshly-erased BE11529 — all three confirmed TERM examples sit inside page
0x11. Prod is unaffected; it ingests complete files and never runs this walk.

The status doc exists because "is SFM reliable?" has three different answers
depending on which tier is meant. The codec library and the data side are
production — verified per-sample at scale, carrying Terra-View daily. The
device side is emergency-grade: it works, but it is synchronous,
unauthenticated, and thinly tested. The lab is research artifacts. Most
confusion comes from answering for the wrong tier.

It covers all three of what Brian asked for: maturity per capability, an
operator-facing "what to use when" (the cheap probes are cheap and the event
walk is not), the known-issues table, and the gap analysis. That gap is mostly
auth, async and guardrails — not protocol work. The protocol is the finished
part.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-20 01:30:14 +00:00
serversdownandClaude Opus 5 6589da445b feat(webapp): cheap connect, opt-in event walk, and a Diagnostics tab
Connecting to a unit fired /device/events automatically, which walks the whole
event chain — every event header over a cellular link. On BE12599 that took
minutes and then 500'd outright, because its buffer has wrapped past 0xFFFF and
the uint16 offset arithmetic goes negative. Wanting to know whether ACH was on
should not require reading every event the unit has stored.

Connect now uses only cheap probes: /device/info (which already carries the
compliance config the event walk was re-reading) plus /device/events/storage_
range. The chain walk moves behind a "Load events" button in the Events
toolbar, and the Device tab gains an Event Chain card showing the first/last
keys.

Adds a Diagnostics tab for the endpoints that previously existed only as curl:
storage_range and events/index alongside monitor/status, then stop monitoring,
disable ACH (rescue?erase=false, so events survive), and erase. The wedged-unit
ladder — slow drip and blind stop — sits under its own heading pointing at the
runbook, with the reminder that slow_drip's success signal is bytes_received>0
and not a clean duration.

Erase is guarded by typing the unit's serial. Auth answers who, not whether you
meant it, and Swagger's try-it-out button on /device/events/erase is live on
:8200/docs — the realistic risk here is an accident.

Lifetime events is displayed but labelled unreliable: SUB 0x08 reports 0 on
units with years of history, which is a decode bug we have not chased yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-19 22:05:55 +00:00
serversdownandClaude Opus 5 0b58415fe2 chore(release): v0.31.0 — report parity + the inverted rescue
Cuts Unreleased to v0.31.0 and writes the theme now that the whole release is
visible, per the convention adopted today.

Two threads landed. Blastware Event/FFT-Report parity — the FFT, the USBM
RI8507 compliance chart, and the sensor self-check decoded for both series and
standardized into the .h5 (schema v2, /sensor_check). And the ach_server rescue
flags out of the BE12599 field emergency, which invert the wedged-unit recovery:
answer the unit's call instead of racing a Stop into the gaps between its
dial-outs.

Version stamped in pyproject.toml, CLAUDE.md and README.md. TOOL_VERSION was
already at 0.31.0 — it came in with the sensor-check work, and it is what makes
the backfill pick up the new /sensor_check group without --force.

⚠ This release owes prod a backfill: .h5 schema v1 -> v2, ~2 h on the NAS.
Stated in the Migration block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 20:40:48 +00:00
serversdownandClaude Opus 5 fa22bb9f59 Merge feat/sensor-check-h5 into dev
Sensor self-check standardized into the .h5 (schema v2, /sensor_check group),
decoded for both series-3 and series-4, plus the Thor backfill script.

CHANGELOG resolved per the convention adopted today: the incoming Unreleased
preamble was dropped rather than reconciled — no preamble under Unreleased, the
theme gets written at release time — and its load-bearing half was folded into
### Migration, which said "None" and is now false.

That block now states the real cost: .h5 schema v1 -> v2, TOOL_VERSION 0.31.0
so the standard backfill picks the traces up with no --force, and ~2 h on the
NAS. The FFT, the compliance chart and the ach_server rescue flags still owe
nothing.

The branch's rewritten "Sensor self-check — both series" entry merged cleanly
and is kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 20:16:13 +00:00
serversdown 27e9c56393 Merge pull request 'Feat/ach rescue on connect' (#38) from feat/ach-rescue-on-connect into dev
Reviewed-on: #38
2026-09-18 15:26:18 -04:00
serversdownandClaude Opus 5 0408c37866 docs: write the changelog on dev, not on feature branches
Reverses the "entry goes in with the work" rule from two commits ago. That was
wrong on the evidence: of the docs(changelog) commits in history, 3 of 4 in
seismo-relay and 2 of 4 in Terra-View were made directly on dev. The rule was
generalized from one unrepresentative commit rather than from the pattern.

It also caused the exact problem it was supposed to avoid. With four worktrees
in flight, every branch edits the same few lines at the top of CHANGELOG.md;
feat/ach-rescue-on-connect and feat/sensor-check-h5 collide on that file and
nothing else. Writing the entry once, on dev, after the merge removes the
whole conflict class.

The second benefit is accuracy: an entry written after the merge describes
what actually landed, including anything that changed during conflict
resolution. The sensor-check branch is a live example — its Unreleased
preamble describes a release that no longer looks like that.

The failure mode of writing it later is forgetting, so the merge is explicitly
not finished until Unreleased is updated — same sitting, reconstructed from the
branch commit messages.

Unchanged: no preamble under Unreleased, the mandatory operational consequence,
and cutting the version on dev when ready to ship to main.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 18:52:17 +00:00
serversdownandClaude Opus 5 a42e8d3651 docs: make the release cadence explicit
Brian described the practice: Unreleased is the staging area for what is going
into the next release, and the version bump happens when enough has
accumulated to be worth shipping — not per commit, not per merge. The
convention already implied it ("never touch the changelog at a merge
boundary") but never said it outright.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 18:04:54 +00:00
serversdownandClaude Opus 5 2fabf84d4d docs: adopt a changelog convention, and make Unreleased follow it
Brian asked what the standard is; there wasn't a written one, only a de facto
pattern in the history. This writes it down in CLAUDE.md and fixes the one
place the repo already diverged from it.

The rule: write the entry in the same commit as the work, under ## Unreleased;
cut the version on dev in a dedicated chore(release) commit; never touch the
changelog at a merge boundary. The entry goes in with the change because that
is the only moment you still know why.

Two additions beyond what the history already did:

No preamble under ## Unreleased. The themed opening paragraph gets written at
release time, when the whole release is visible and can be named honestly. The
current one proved the point — "Blastware Event/FFT-Report parity: the FFT,
the USBM compliance chart, and the sensor self-check" was accurate when the
first item landed and stopped being accurate once rescue-on-connect landed
under the same heading. Removed here; the release commit writes a new one
covering everything actually in the release.

And the operational consequence is now mandatory on any entry touching the
codec, the waveform store, or the DB — including when it is "none". This
repo's changelog is how future-you learns whether a deploy costs two hours on
the NAS, so silence is ambiguous and "none" is information. The old preamble's
load-bearing half is preserved as an explicit ### Migration block rather than
dropped with the prose around it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-18 17:39:56 +00:00
serversdownandClaude Opus 5 402bf30e37 docs(runbook): reframe as one disease with two cures, intercept first
The previous commit called BE12599 a second failure mode and claimed the
device "never enters S3 mode at all" and that no inbound work could reach it.
That was an overclaim built on a single slow_drip attempt, and Brian was right
to push back.

It is the same disease.  Method B's step 1 worked fine on BE12599 — clearing
the Destination did stop the dial-outs.  It was step 2 that did not land, on
one attempt, run ~90 s after a modem reboot with a dead session visible in the
log in that same window; BE9558H needed hours of attempts before one landed.
And the AT-init loop the ALEOS log revealed is almost certainly what BE9558H
was doing too — we just never turned on serial debug in May to look.  The
device speaks S3 fine; it handshook cleanly the moment it had a session.

What is genuinely new is the cure, and it deserves to be the default rather
than a footnote.  Racing a Stop into the gaps between dial-outs is a coin
flip.  Intercepting is deterministic: the unit dials every ~75 s, so give it
somewhere to dial and answer it.  It will not answer us because it is on the
phone — so be the one it calls.

Restructures accordingly: a "two cures" table up top, the intercept promoted
to Method A with its own procedure (listener before modem, stop at step 1.5,
drain before disabling ACH, restore the Destination and confirm it), and the
original inbound procedure kept intact as Method B for when there is no
listener the modem can reach.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-17 06:10:03 +00:00
serversdownandClaude Opus 5 c6fc3d0241 docs: BE12599 incident — the inverted rescue, plus a rescue-listener plan
The wedged_unit_recovery runbook covered exactly one failure mode.  BE12599
turned out to be a second one wearing the same symptoms, and the existing
procedure did not work on it.

Adds a "TWO failure modes" table up front so the next incident branches
correctly, and a full second-incident section covering what the ALEOS serial
debug log revealed: the device repeating a 29-byte AT modem-init string
(ATQ1/ATE0/ATS0=2, no ATD) every 75 s, never getting an OK because the modem
is in TCP data mode, and therefore never entering S3 mode at all.  Inbound
cannot win against that, no matter how well framed.

Also records the two red herrings, since together they cost ~90 minutes:
the RV50 trusted-IP whitelist drops non-listed sources silently (presents as
a connect timeout, and Brian's dynamic dev IP had rotated off the list), and
sfm/server.py returns 502 for BOTH "Protocol error:" and "Connection error:",
so a 502 was misread as "TCP connected, device mute" and a theory built on it.

And the gotchas worth never re-deriving: slow_drip's send_error=null plus a
full duration is not success (only bytes_received > 0 is); stopping monitoring
removes the call-in trigger, so it costs you the channel; --events-only skips
the device-info step, so the serial is never read and ach_state keys on
peer:ephemeral_port, silently breaking dedup and re-downloading the same event
every session.

The plan doc captures the tool Brian wants built out of this — a rescue
listener with a real lifecycle and, critically, a confirmation gate before
shutdown, because leaving the modem's Destination pointed at a dead listener
is worse than never having started.  Open questions are listed rather than
guessed at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-17 05:45:26 +00:00
serversdownandClaude Opus 5 9f1050b5e7 feat(ach): rescue-on-connect — stop monitoring / disable ACH from the server side
A unit whose geophone offset has grown past its trigger level records
back-to-back and, with ACH set to "after event recorded", re-dials every
time.  The wedged_unit_recovery runbook handles that by reaching the unit
inbound and clearing the modem's Destination Address so it stops dialing.

That fails when the device is wedged mid-modem-init.  BE12599 (2026-09-16)
sat repeating a 29-byte AT setup string — ATQ1/ATE0/ATS0=2, no ATD — every
75 s.  The modem is in TCP data mode, never interprets it, never answers OK,
so the device never progresses into S3 mode and ignores every frame we send.
Worse, each attempt makes ALEOS log "tcpmode trying to send to invalid
socket" and re-run "Initialize Auto answer on port 9034", which orphans any
held inbound session — slow_drip reports a clean 120 s hold with
bytes_received=0 because the modem stopped bridging after the first re-init.

Inbound cannot win that race.  But the modem auto-dials its Destination
whenever serial data arrives while closed, so pointing Destination at an
ach_server turns those 75 s attempts into a device-initiated session that
the modem bridges correctly.

Adds --stop-monitoring, --disable-ach and --rescue.  They run as step 1.5,
after the handshake and before the event walk, each independently guarded so
a failure does not abort the download.  Outcome is written to rescue.json.
Startup banner reports both, and warns when --restart-monitoring would undo
--stop-monitoring.

Prefer --stop-monitoring alone on first contact: --disable-ach stops the unit
calling, which is the only channel to a unit in this state, and halting the
recording ends the loop on its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qcu9ByJfuKBQxmrWb8rSrN
2026-09-16 20:54:46 +00:00
serversdownandClaude Opus 4.8 8265e32ad5 feat(backfill): regenerate .h5 with /sensor_check; TOOL_VERSION 0.31.0
Complete the sensor-check standardization: existing events need their .h5
regenerated to gain the v2 /sensor_check group.

  * backfill_thor_events.py attaches the decoded series-4 traces
    (micromate.sensor_check) on its own IDF decode path, mirroring
    save_imported_idf, so regenerated Thor .h5 files get the group. Series-3
    backfill needs no change — it re-decodes via read_blastware_file, which now
    attaches the traces itself.
  * TOOL_VERSION 0.30.0 → 0.31.0 so the standard backfill regenerates every
    event (no --force): the tool now produces the /sensor_check group. Purely
    additive — no decoded value changes.
  * CHANGELOG (Unreleased): sensor-check now series-3 + series-4, standardized
    into the .h5 (schema v2), with the ⚠ backfill note; FFT + compliance stay
    no-backfill (they read existing .h5 samples).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-16 06:53:33 +00:00
serversdownandClaude Opus 4.8 8d3cdba1b5 feat(h5): standardize sensor-check into the .h5 (schema v2); SFM reads it
Make the sensor self-check a first-class part of the standardized decoded event
so SFM stops decoding it at report time — device-agnostic, per the store's
decoder→standardized-.h5→SFM model.

  * Event gains a `sensor_check` field; both decoders attach the traces where
    they set raw_samples — series-3 in event_file_io.read_blastware_file
    (minimateplus.sensor_check), series-4 in waveform_store's IDF path
    (micromate.sensor_check).  Covers ingest and backfill (both re-decode).
  * event_hdf5 bumps schema_version 1→2 and writes an optional /sensor_check
    group (raw counts, int32, per channel present).  read_event_hdf5 returns
    it; plot_json_from_hdf5 carries it as a top-level key.  Old v1 files still
    read cleanly (no group → None), so nothing breaks before the backfill.
  * gather_report_data reads sensor_check_waveforms from the .h5 and drops the
    report-time series-3 decode — the report no longer reaches into a decoder,
    and a series-4 event now lights up the same strip automatically.

Stored as raw counts (a shape diagnostic, rendered fit-to-box): the per-series
count scale differs and a physical mic unit is ill-defined, so conversion would
add complexity for no display benefit — easy to add later if a numeric use
appears.

Tests: .h5 roundtrip + backward-compat + plot_json + real series-3 decode
attaches to the Event.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-16 06:49:55 +00:00
serversdownandClaude Opus 4.8 685a17d180 feat(series4): decode sensor self-check waveforms from the IDFW binary
The Thor/Micromate (series-4) IDFW binary carries the sensor self-check in its
fixed-header region (before the waveform body), as up to four records tagged
01 0e 3c/3d/3e/3f — the SAME channel ids as series-3 (Tran/Vert/Long/MicL).
Unlike series-3's delta-coded trailing block, series-4 stores each trace as a
raw int16-BE array after an 18-byte record header (2-byte sample count at
offset +8). Three-channel (mic-disabled) units carry only 3c/3d/3e.

New micromate/sensor_check.py: decode_idf_sensor_check(raw) locates the record
chain (id-ordered marker run, so a stray body match can't chain) and reads each
trace's int16 samples → {Tran,Vert,Long[,MicL]: [counts]}, or {} when absent.

Reverse-engineered + validated against 4 UM oracle events (added as fixtures):
clean geophone ring-downs on all, mic pulse trains on the 4-channel units,
correctly no MicL on the two 3-channel units. Validated by shape + cross-event
consistency (no Thor report strip to exact-match, unlike series-3's BW reports).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 20:01:39 +00:00
serversdownandClaude Opus 4.8 dcd9ad6f48 Merge feat/fft-series3: Blastware FFT, USBM compliance chart, sensor self-check
Reverse-engineered Blastware Event/FFT-Report parity, all additive (reads the
existing .h5 samples + retained raw binary, no DB/.h5 change or backfill):
 - Blastware-compatible channel FFT (waveform_fft)
 - USBM RI8507/OSMRE compliance chart on the event-report PDF (sfm/compliance)
 - sensor self-check strip decoded from the series-3 binary trailing block
   (minimateplus/sensor_check) + Frequency/Overswing sub-rows
 - seismo_lab Inspector hex reader (minimateplus/binary_annotate)
 - report fixes: stacked-lane y-tick collision, header serial fit

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 14:37:44 +00:00
serversdownandClaude Opus 4.8 4f73e919a0 docs(changelog): Unreleased — FFT, USBM compliance chart, sensor self-check
Document the feat/fft-series3 work under Unreleased: Blastware-compatible
channel FFT, the USBM RI8507/OSMRE compliance chart on the event-report PDF,
the decoded sensor self-check strip + Frequency/Overswing sub-rows, and the
seismo_lab Inspector hex reader — plus the two report-panel fixes (tick
collision, header serial fit). Additive, no .h5/DB change or backfill.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 14:34:04 +00:00
serversdownandClaude Opus 4.8 2a747f6893 fix(report): attach sensor-check strip to the waveform panel; move "0.0"
Match Blastware's layout, measured off the reference PDF: the sensor-check
strip shares a border with the main waveform panel (no gap between them), and
the per-lane "0.0" baseline labels sit to the RIGHT of the strip. Previously
the strip floated with a gap and the "0.0" label overprinted the strip's left
edge. Purely layout — the traces and decode are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 05:48:31 +00:00
serversdownandClaude Opus 4.8 2c5c20cfd7 fix(report): fit sensor-check mini-plots to their boxes
The sensor-check strip used a symmetric ±max scale, so the one-sided geophone
ring-downs (a dip to ~-990 with the baseline at 0) sat in the bottom half of
each mini-box with the top half blank — visibly off next to Blastware. Scale
each mini-plot to its actual data range with a small pad instead, and draw a
faint zero baseline, so the ring-downs and the mic pulse train fill their boxes
the way BW draws them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDXjZCr4RqT2U3QvMDhgzf
2026-09-15 05:39:53 +00:00