The field failure on UM12947 ("wouldn't stay connected, refresh did nothing, no
way to view a connection attempt") reproduced on the bench, with timestamps.
THOR was mid-bulk-download when the link was faulted. Sequence:
13:02:50 connection dies mid-transfer (165 BULK_DOWNLOAD frames in)
13:03:10 THOR reconnects once after 20 s, sends SUB_1F to resume, fails
13:03:33 link fully restored and healthy
13:04:21 operator clicks Refresh -> full 11-command check, all correct
13:06:27 still nothing. That refresh is the ONLY connection since 13:03:10.
Before the fault THOR had connected every 30 s without a miss for over an hour.
Establishes four things:
* one retry then give up -- no backoff, no further attempts
* the automatic poll loop dies too, not just the download
* it does not recover when the link returns (3 min of healthy link, nothing)
* Refresh works but only once -- it does NOT restart the automatic loop
The fourth is the dangerous one: Refresh makes the UI report a healthy unit while
nothing is watching it. Silent failure that looks like success. It also explains
why the field symptom resists characterisation -- the unit is reachable the whole
time; THOR has simply stopped asking and says nothing about it.
CAVEAT, recorded prominently: what THOR experienced was a TCP close mid-download,
not the silent link intended. mm_link.py mistook socket.timeout (which subclasses
OSError) for a closed socket, so 200 ms of quiet closed the connection -- the
relay killed the link it was meant to be faking a fault on. Fixed in this commit.
The run stands as a drop-mid-download test, arguably the more realistic case.
Single trial; true blackhole and clean drop not yet tested.
Adds four design consequences for SFM: unbounded retry with backoff; a manual
check must restart the automatic loop or the UI must say it is stopped; surface
the poll loop's own state (last success, last attempt, next attempt, consecutive
failures -- all four invisible here); and distinguish "unit unreachable" from "we
stopped checking", which present identically in THOR.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru8Lg9HkkYvX9VWWo65SmL
serversdownandClaude Opus 5
04b1ef3e04