Brian noticed BE12599's 2026-08-09 event reports no ZC frequency because the trace never crosses zero. That is the best detector in this investigation. A geophone has no DC response, so its output must integrate to ~zero over a record. |mean|/peak is therefore ~0 for real motion and ~1 for anything electrical. Across 12,068 channel-events with peak >= 0.05 in/s the statistic is bimodal with a 1.09% dead zone, and at mp >= 0.8 it returns exactly the five confirmed units -- from physics rather than a tuned threshold. Two detectors on different principles agreeing is the strongest corroboration the list has had. It also settles BE11007 as NOT an offset: mp 0.75-0.89 but frac_neg 0.99 at peaks of 7.4-9.4 in/s, i.e. a one-sided near-full-scale blast. Journal 8e diagnoses BE12599 specifically. Its August waveforms are unipolar impulses with an RC tail (26 ms -> 118 ms -> never recovers over 14 days), and the fault MOVES between Long and Tran while the sensor self-check passes on every event. A failing element cannot hop channels; a connector can -- which also explains why the swing test never fails and why an autozero rarely helps. Corrects 8c's claim that the spread gate is blind to onsets: of 87 BE18438|Vert events it rejected one, the transitional record. Narrower than stated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog
988 lines
52 KiB
Markdown
988 lines
52 KiB
Markdown
# The "offset" fault — investigation journal
|
||
|
||
> ## ⚠ CORRECTED 2026-08-28 (same day) — the v1 detector was wrong
|
||
>
|
||
> Brian pushed back on the finding that offsets "come and go": in the field,
|
||
> once a unit develops one it stays broken until the geophone is replaced.
|
||
> He was right, and the challenge exposed **two real flaws** in the v1 detector:
|
||
>
|
||
> 1. **It scored only the axis with the largest peak.** A real event on one axis
|
||
> hid a persistent pedestal on another. BE12599 on 2026-08-21 read "clean"
|
||
> solely because Long had a 1.065 in/s event — Tran was sitting at
|
||
> **+0.4732 in/s** at that moment and was never examined.
|
||
> 2. **It used the MEAN**, which a real transient perturbs. The **median** is the
|
||
> resting baseline — most samples sit at it, so a blast does not move it.
|
||
> Same event, Long channel: mean **+0.0783** vs median **-0.0050**.
|
||
>
|
||
> Both flaws manufactured false recoveries. The corrected detector
|
||
> (`scratch/offset_scan2.py`, per-channel median) shows the pedestal is
|
||
> **persistent**, exactly as the field experience says. See §2b and §3b.
|
||
>
|
||
> **Then Brian proposed a better detector still** — measure the floor during
|
||
> the *pre-trigger* window, and require it to hold across pre/middle/end.
|
||
> That is now the detector of record (§2c). Final answer: **5 of 45 units
|
||
> (11%)**, stable across a 2x threshold range.
|
||
>
|
||
> Sections below that were written against v1 are marked; v1 numbers are kept
|
||
> for the reasoning trail, not as current fact.
|
||
|
||
A running record of the **offset** hardware fault on Instantel Series III
|
||
seismographs: a geophone channel whose trace sits displaced from zero rather
|
||
than centred on it.
|
||
|
||
This is a *journal*, not a spec. Findings are dated, dead ends are kept with
|
||
the reason they died, and every number says where it came from. When something
|
||
here is superseded, strike it and say why rather than deleting it — the point
|
||
is that a future session can tell what was actually established from what was
|
||
merely believed at the time.
|
||
|
||
Companion material:
|
||
- `scratch/offset_scan.py` — the detector
|
||
- `scratch/verify_against_ascii.py` — decoder verification harness
|
||
- `docs/instantel_protocol_reference.md` — wire protocol, incl. the
|
||
unimplemented `SUB 0x0E` this investigation now wants
|
||
|
||
---
|
||
|
||
## TL;DR (current state, 2026-08-28)
|
||
|
||
- **It is real device data, not a decode bug.** Settled early and confirmed
|
||
against Blastware's own ASCII exports.
|
||
- **Base rate: 5–6 of 45 units (11–13%)** across the full DL2 archive,
|
||
2018–2026. This *confirms* the earlier 2-of-21 (9.5%) estimate from the much
|
||
smaller Terra-View DB — survivorship bias from deleted events had **not**
|
||
concealed a wave of cases.
|
||
- **The fault is bimodal, not a drift continuum.** A unit is either clean or
|
||
grossly off. Loosening the amplitude threshold 11× adds no new units.
|
||
- **The unit's own sensor check cannot see it.** 102 offset events, zero
|
||
sensor-check failures. Do not try to use it as a screen.
|
||
- **Cause is still unsettled.** Instantel's autozero fixes the minority of
|
||
cases; the rest are hardware. We cannot yet tell which is which remotely.
|
||
- **The histogram corpus (63,535 files, 9.7x the waveforms) is now scanned too** —
|
||
see §8b. It independently confirms BE18438 and BE9558 with a clean 2.5x
|
||
separation, but detects only **2 of the 5** confirmed units, cannot attribute a
|
||
channel, and resolves time to ~a month. **A negative histogram result is not
|
||
evidence of health** — DC leakage into the interval peak varies 45x between units.
|
||
- **`offset_scan3.py` has a label defect** (§8b): its spread gate discards 18.8% of
|
||
high-|pre| rows onto units currently counted as clean. Re-cut before quoting any
|
||
precision number again.
|
||
- **Best open lead:** `SUB 0x0E` (channel sensor data, 8 channels × 10 bytes,
|
||
unimplemented) may carry the very numbers Instantel says to check against
|
||
**2027–2069**. Untested.
|
||
|
||
---
|
||
|
||
## 1. What the fault looks like
|
||
|
||
A healthy geophone trace is centred on zero. An offset channel is parked away
|
||
from zero, so the channel **mean approaches its own peak**. In Blastware the
|
||
signature is "parallel lines above or below the zero line" (Instantel's own
|
||
wording).
|
||
|
||
Consequences observed in the field:
|
||
- The unit can **self-trigger on its own offset** when the displacement exceeds
|
||
the geo trigger level, producing streams of junk events with no ground
|
||
motion. Instantel has a separate FAQ for this symptom (13-0-22, *"Unit
|
||
triggers continuously without activity"*).
|
||
- Recorded PPV for that channel is meaningless while the fault persists.
|
||
|
||
---
|
||
|
||
## 2. The detector
|
||
|
||
Implemented in `scratch/offset_scan.py`. Operates on raw BW binaries only — no
|
||
DB, no sidecars.
|
||
|
||
```
|
||
for each series-3 waveform binary:
|
||
decode -> per-channel ADC counts
|
||
dominant axis = channel with the largest |peak|
|
||
flag when |mean| / peak > 0.70
|
||
and |mean| >= 0.90 x the unit's geo trigger level
|
||
episodes = per-serial runs of flagged events, split on a >12 h gap
|
||
```
|
||
|
||
Why each term:
|
||
|
||
| term | purpose |
|
||
|---|---|
|
||
| `\|mean\|/peak > 0.7` | the discriminator. A DC-parked trace has mean ≈ peak. |
|
||
| `\|mean\| >= 0.9 × trigger` | amplitude floor — suppresses quiet traces where mean and peak are both tiny and the ratio is meaningless. |
|
||
| dominant axis only | the fault is per-channel; scoring all three dilutes it. |
|
||
| 12 h episode gap | separates deployments/visits rather than counting events. |
|
||
|
||
Trigger level comes from a paired `_ASCII.TXT` when one exists, else the
|
||
per-serial median learned from that unit's ASCII files, else 0.2 in/s.
|
||
|
||
**Known limitation.** Event traces contain real ground motion, so this can only
|
||
see offsets large enough to *dominate* the trace. A mild offset on a real blast
|
||
is invisible. Instantel's A/D-mode check (§5) is the only thing that sees the
|
||
mild end. Our base rate is therefore a **gross-offset** rate.
|
||
|
||
### 2b. Detector v2 — per-channel median (CURRENT)
|
||
|
||
`scratch/offset_scan2.py`. Supersedes the above.
|
||
|
||
```
|
||
for each series-3 waveform binary:
|
||
for each geo channel independently:
|
||
pedestal = median(samples) # resting baseline, robust to blasts
|
||
flag the CHANNEL when |pedestal| >= 0.025 in/s (5 A/D counts)
|
||
a unit has a real fault when a channel is flagged on >=3 CONSECUTIVE events
|
||
```
|
||
|
||
Why median: a DC pedestal shifts every sample, so it moves the median. A real
|
||
event moves only a minority of samples, so it does not. This removes the need
|
||
for the `m/p` ratio guard entirely — that guard existed only to compensate for
|
||
using the mean.
|
||
|
||
Why per-channel: the fault is on one geophone axis. Scoring only the dominant
|
||
axis means any event with motion elsewhere hides it.
|
||
|
||
Why "3 consecutive": the 0.025 in/s floor is only ~2x a healthy channel's
|
||
resting median (observed 0.010-0.015), so isolated flags are noise. Persistence
|
||
is the discriminator — and it is what the field experience predicts.
|
||
|
||
---
|
||
|
||
## 3b. Archive results, corrected (v2)
|
||
|
||
| | v1 (dominant axis, mean) | **v2 (per-channel median)** |
|
||
|---|---|---|
|
||
| units with any flagged event | 6 of 45 | 19 of 45 |
|
||
| **units with a sustained pedestal (>=3 consecutive)** | — | **8 of 45 (18%)** |
|
||
| runs of >=3 consecutive | — | 29 |
|
||
| runs of 1-2 events (noise) | — | 69 |
|
||
|
||
Units with a sustained pedestal: **BE9558, BA10895, BE11007, BE11529, BE12599,
|
||
BE13117, BE18003, BE18438**. BA10895 and BE18003 were invisible to v1.
|
||
|
||
**The affected channel is most often Vert**, which v1 got wrong — it named
|
||
whichever axis had the largest peak. BE13117 and BE18438 are both Vert faults.
|
||
|
||
Longest / clearest runs:
|
||
|
||
| unit | ch | span | events | median in/s |
|
||
|---|---|---|---|---|
|
||
| BE13117 | Vert | 2023-05-03 → 05-04 | 194 | 0.035 → **1.915** |
|
||
| BE18438 | Vert | 2026-02-25 → 02-26 | 75 | 0.180 → 0.370 |
|
||
| BE9558 | Vert | 2020-02-11 (6 h) | 33 | 0.065 → 0.090 |
|
||
| BE12599 | Tran | 2026-08-14 → 08-23 | 8 | 0.030 → **0.565** |
|
||
| BE18003 | Vert | 2021-03-17 → 06-11 | 3 | 0.040 → 0.060 |
|
||
|
||
BE12599 began **2026-08-14**, not 08-17 as v1 reported, and was still faulting
|
||
at the last event in the archive.
|
||
|
||
### 2c. Detector v3 — PRE-TRIGGER floor + constant-floor test (CURRENT)
|
||
|
||
`scratch/offset_scan3.py`. Brian's method, and better than v2 for a reason
|
||
worth naming: **the pre-trigger window is definitionally quiet** — it is the
|
||
buffer captured before the trigger fired — whereas a whole-record median is
|
||
merely *robust* to the event. `pretrig_samples` comes from the STRT record.
|
||
|
||
```
|
||
per channel:
|
||
pre = median of the first pretrig_samples samples
|
||
mid = median of the middle third
|
||
end = median of the final third
|
||
spread = max(pre,mid,end) - min(pre,mid,end)
|
||
|
||
offset when |pre| >= floor AND spread <= 0.02 in/s
|
||
real fault when a channel is flagged on >=3 CONSECUTIVE events
|
||
```
|
||
|
||
A DC offset is a **constant floor** — present before the trigger, during, and
|
||
after. The spread test rejects transients (settling, handling, a long event
|
||
tail) that move one segment relative to the others, which is what v2's
|
||
whole-record median could not do.
|
||
|
||
**The empirical noise floor justifies the threshold.** Across 19,244
|
||
non-flagged channel-events the pre-trigger floor distributes as:
|
||
|
||
| floor | share |
|
||
|---|---|
|
||
| −1 unit (−0.005) | 18.4% |
|
||
| **0.000** | **62.7%** |
|
||
| +1 unit (+0.005) | 13.4% |
|
||
|
||
**94.5% within ±1 quantisation unit; median exactly +0.0000, mean −0.0008.**
|
||
So there is **no systematic zero-point bias in the decoder** — an independent
|
||
confirmation of the 32000-count scale. A healthy channel really does read
|
||
0.000, and "any constant floor that is not 0.000" is the right signal, with
|
||
±1 unit of slack for quantisation.
|
||
|
||
**The result is threshold-insensitive**, which is what distinguishes a real
|
||
signal from a tuned one:
|
||
|
||
| floor | units flagged | sustained units |
|
||
|---|---|---|
|
||
| 2 units (0.010) | 34 | 15 ← into the noise |
|
||
| 3 units (0.015) | 26 | 8 |
|
||
| **4 units (0.020)** | 17 | **5** |
|
||
| **5 units (0.025)** — Instantel's | 12 | **5** |
|
||
| **8 units (0.040)** | 8 | **5** |
|
||
|
||
### FINAL RESULT: 5 of 45 units (11%)
|
||
|
||
**BE9558, BE11529, BE12599, BE13117, BE18438.**
|
||
|
||
Unchanged across a 2x threshold range. BE11007 and BA10895 drop out — the
|
||
spread test identifies them as transients, not pedestals.
|
||
|
||
The 11% headline happens to match v1's, but the reasoning and the unit list
|
||
differ: v1 included BE11007 and named the wrong *channel* on most units.
|
||
|
||
---
|
||
|
||
## 3. Archive results (2026-08-28)
|
||
|
||
Source: DL2 event export, 6,577 **unique** series-3 waveforms, 45 units.
|
||
See [`dl2-archive`](#8-data-and-tooling) for the `Sent/` mirror trap.
|
||
|
||
**283 suspect events, 15 episodes, 6 of 45 units (13.3%).**
|
||
Excluding BE11007 (§4, likely not an offset at all): **5 of 45 = 11.1%**.
|
||
|
||
### Threshold sensitivity — the bimodality result
|
||
|
||
Re-scoring the same corpus at a range of amplitude floors, with two
|
||
ratio cut-offs (1 A/D count = 0.005 in/s, see §5):
|
||
|
||
| \|offset\| floor | m/p > 0.7 | m/p > 0.9 |
|
||
|---|---|---|
|
||
| 5 cts (0.025 in/s) — *Instantel's own* | 333 ev / 6 units | 279 ev / **5 units** |
|
||
| 10 cts (0.050) | 294 / 6 | 274 / 5 |
|
||
| 20 cts (0.100) | 250 / 5 | 244 / 4 |
|
||
| 40 cts (0.200) | 209 / 5 | 203 / 4 |
|
||
| 80 cts (0.400) | 152 / 4 | 148 / 2 |
|
||
| 160 cts (0.800) | 144 / 2 | 141 / 1 |
|
||
|
||
Relaxing the floor by 11× (0.27 → 0.025 in/s) adds ~14% more events and **no
|
||
new units**. There is no population of mild offsets hiding below our threshold
|
||
*in event data*. Either a unit is clean or it is grossly off.
|
||
|
||
---
|
||
|
||
## 4. Per-unit case files
|
||
|
||
Ordered by severity. `m/p` medians are on the offending channel.
|
||
|
||
### BE13117 — one violent day, never again
|
||
`145 / 454 events (32%)`, **1 episode**, 2023-05-04, 6.8 h.
|
||
Offset climbed **0.393 → 1.875 in/s within the episode**. `m/p` median
|
||
**0.996** — the trace is almost pure DC. No recurrence in the rest of its 454
|
||
events. No ASCII files in the archive, so no calibration history.
|
||
|
||
### BE18438 — recurring, months apart
|
||
`87 / 293 (30%)`, **2 episodes**: 2025-11-15 (1.2 h, n=12, 0.279 → 0.369) and
|
||
2026-02-25 (**28.8 h**, n=75, 0.183 → 0.366). `m/p` median 0.967.
|
||
Clean across all 196 events preceding its 2025-08-12 calibration.
|
||
|
||
### BE9558 — six years apart
|
||
`38 / 196 (19%)`, **4 episodes**: 2020-02-11 (6.3 h, n=33, but only
|
||
0.068 → 0.086 — very mild), then 2026-04-14, 2026-04-29, 2026-05-04
|
||
(0.28–0.45). `m/p` median 0.919. Calibrated 2026-06-26; 0/7 events flagged
|
||
after, but n=7 is far too small to call it fixed.
|
||
|
||
### BE12599 — the live case ⚠
|
||
`6 / 77 (8%)`, **6 single-event episodes, one per day at exactly 05:00**,
|
||
2026-08-17 → 2026-08-23. Offset rose 0.383 → 0.565 then fell back to 0.345.
|
||
`m/p` ≈ 0.965, geo trigger 0.3 in/s — **the offset exceeds the trigger level,
|
||
so the unit is triggering on its own fault**. Last calibrated 2025-08-12.
|
||
|
||
This is the most recent and the most useful: a currently-faulting unit is the
|
||
natural experiment for the re-zero-vs-repair question (§7).
|
||
|
||
### BE11529 — marginal
|
||
`4 / 99 (4%)`, 1 episode 2025-07-08, 0.4 h, offsets only 0.051 → 0.058 in/s.
|
||
`m/p` median 0.959, so DC-dominated, but the magnitude is near the noise of
|
||
this method. Treat as unconfirmed.
|
||
|
||
### BE11007 — probably NOT an offset
|
||
`3 / 70 (4%)`, 1 episode 2022-01-17, offsets 5.500 → 6.904 in/s — by far the
|
||
largest. But `m/p` is only **0.719–0.738** against ≥0.9 for every other unit,
|
||
and the peaks are 7.6–9.4 in/s on a 10 in/s range. That reads as a **large
|
||
real blast with asymmetric ground motion**, not a parked trace. Excluded from
|
||
the headline base rate.
|
||
|
||
---
|
||
|
||
## 5. Instantel's own procedure and thresholds
|
||
|
||
From two Instantel technical-support FAQs supplied 2026-08-28
|
||
(answers **13-0-21** *"How to determine offsets"* and **12-0-10** *"Removing
|
||
offsets on an Instantel Series III monitor"*; created 2008/2007, last updated
|
||
2009-03-06).
|
||
|
||
### Identifying (13-0-21)
|
||
|
||
1. Create or use an event with the **manual minimum trigger** set for the
|
||
connected geophone and microphone — i.e. an event that recorded no real data.
|
||
2. Save it and open in Blastware.
|
||
3. An offset shows as **parallel lines above or below the zero line**.
|
||
4. Put the unit in **A/D mode** — on Series III, press and hold `OPTION`, then
|
||
press `START MONITOR`.
|
||
5. **Display counts higher than 5**, with no vibration or overpressure present,
|
||
indicate an offset.
|
||
|
||
### Removing — the autozero (12-0-10)
|
||
|
||
1. Be in a **quiet area with low vibration**.
|
||
2. Power on the Blastmate III / Minimate Plus.
|
||
3. Connect the geophone and microphone — **LINEAR mic only**.
|
||
⚠ *Do not connect an "A" weight microphone, regardless of what the monitor
|
||
displays.*
|
||
4. Press `Test`.
|
||
5. Wait for the **Sensor Check** results to appear.
|
||
6. Press `OPTION` and `START MONITOR` **simultaneously**.
|
||
7. `Performing Autozero` appears; press `Enter`.
|
||
8. Confirm the sensors are properly connected; press `Enter`.
|
||
9. Wait for the autozero to complete.
|
||
10. Press `Enter` twice → Main Menu, *Ready To Monitor*, offset corrected.
|
||
|
||
### The go/no-go number — 2027 to 2069
|
||
|
||
> When you perform an Autozero on any Series III unit, the lists of numbers in
|
||
> the **X1 and X8 gains should all be between 2027 and 2069**. If not, repeat
|
||
> the Autozero. **If the numbers are extremely out of the specified range, then
|
||
> the unit should be sent in for repair.**
|
||
>
|
||
> If this process does not remove the offset problem, return the unit **and
|
||
> sensors** to Instantel for repair.
|
||
|
||
This is the documented explanation for the field experience (Brian's dad,
|
||
2026-08-28) that **a re-zero works maybe 10% of the time** — the autozero only
|
||
recovers units whose zero reference is still near-correct.
|
||
|
||
### Scale derivation (inference, well-supported — not proven)
|
||
|
||
2048 is 12-bit midscale. Our codec's geo full scale is 32000 internal counts =
|
||
10 in/s, with 1 decoder unit = 16 counts = exactly 0.005 in/s
|
||
(`geo-full-scale-is-32000-counts`). ±2000 A/D counts about 2048 therefore maps
|
||
to ±10 in/s at **0.005 in/s per A/D count**. That makes:
|
||
|
||
- Instantel's ">5 counts" threshold ≈ **0.025 in/s**
|
||
- the 2027–2069 window = **±21 counts = ±0.105 in/s** of tolerated zero error
|
||
|
||
Consistent and mutually corroborating, but we have not confirmed the A/D-count
|
||
scale directly from a device reading.
|
||
|
||
---
|
||
|
||
## 6. Ruled out — keep these dead
|
||
|
||
### Condensation / humidity — DEAD (2026-08-25)
|
||
Proposed, then killed by its own controls: BE18438 stayed flat across a 10-hour
|
||
overnight gap, and only 2 of 21 units showed the fault while 19 sat in the same
|
||
weather. The apparent "diurnal cycle" was an artifact of binning by hour-of-day
|
||
across two days. See `waveform-dc-offset-is-real-device-data`.
|
||
|
||
### Clipping as a false-positive source — RULED OUT (2026-08-28)
|
||
A rail-hitting trace would fake an offset (mean → peak). It isn't happening:
|
||
median suspect peak is only **10% of full scale**, p90 is 18.6%. Only BE11007's
|
||
3 events exceed 50% FS, and none reach 98%.
|
||
|
||
### The sensor check as a predictor — DOES NOT WORK (2026-08-28)
|
||
Tested on 102 offset events across 4 units:
|
||
|
||
| unit | state | n | failed | median ratio | median freq |
|
||
|---|---|---|---|---|---|
|
||
| BE11529 | offset | 4 | **0** | 3.90 | 7.6 |
|
||
| BE11529 | clean | 14 | 0 | 3.80 | 7.5 |
|
||
| BE12599 | offset | 6 | **0** | 4.00 | 7.4 |
|
||
| BE12599 | clean | 13 | 0 | 4.00 | 7.6 |
|
||
| BE18438 | offset | 87 | **0** | 3.70 | 7.6 |
|
||
| BE18438 | clean | 25 | 0 | 3.80 | 7.5 |
|
||
| BE9558 | offset | 5 | **0** | 3.90 | 7.8 |
|
||
| BE9558 | clean | 44 | 0 | 3.80 | 7.5 |
|
||
|
||
Zero failures on either side and indistinguishable ratios/frequencies. The
|
||
swing test measures geophone frequency response and damping — it never examines
|
||
DC zero. **A grossly offset unit passes its own self-check.** This is why the
|
||
fault goes unnoticed until somebody looks at waveforms.
|
||
|
||
### "Offsets are transient / come and go on their own" — RETRACTED 2026-08-28
|
||
v1 reported episodes lasting hours that ended spontaneously. **This was an
|
||
artifact of the v1 detector** (see the banner at the top). With the per-channel
|
||
median, the pedestal persists. Every clear case reads clean again only after a
|
||
multi-day-to-multi-month gap consistent with service: BE13117 6 days, BE18438
|
||
24 days, BE9558 63 days **with a confirmed Instantel calibration inside the
|
||
gap**. BE12599 never reads clean — it is still faulting at the end of the
|
||
archive. This matches the operational experience: once a unit develops an
|
||
offset it stays broken until the geophone is replaced.
|
||
|
||
### "Offsets develop N months after calibration" — CONFOUNDED, NOT A FINDING
|
||
Tempting, and it looked strong:
|
||
|
||
| unit | suspect before latest cal | after |
|
||
|---|---|---|
|
||
| BE18438 | 0 / 196 | 87 / 97 |
|
||
| BE12599 | 0 / 62 | 6 / 15 |
|
||
| BE11529 | 0 / 82 | 4 / 17 |
|
||
| BE9558 | 38 / 189 | 0 / 7 |
|
||
|
||
But bucketing suspects by months-since-calibration gives **one unit per bucket**:
|
||
`0–3mo={BE11529}`, `3–6 & 6–9mo={BE18438}`, `9–12mo={BE9558}`,
|
||
`12–15mo={BE12599}`. The apparent "51% failure rate at 6–9 months" is entirely
|
||
BE18438's single February 2026 episode. Five units with roughly one episode
|
||
each cannot support a population trend. **Do not re-derive this.**
|
||
|
||
Also note: all affected units are calibrated on a **~12–13 month cadence**, so
|
||
"sent to Instantel" is the routine annual schedule, not evidence of a
|
||
fault-driven return.
|
||
|
||
---
|
||
|
||
## 7. Open questions
|
||
|
||
### Q1 — Is it a latched bad zero or analog degradation?
|
||
The question that decides everything. A latched zero is correctable (possibly
|
||
over the wire); degradation means a repair. Instantel's 2027–2069 rule implies
|
||
*both* populations exist, with the split roughly 10/90 in the field.
|
||
|
||
**BE12599 is the natural experiment** — faulting as of 2026-08-23. Read its
|
||
values, run the autozero, read them again.
|
||
|
||
### Q2 — Can we read the autozero numbers over the wire? (best lead)
|
||
Instantel says to check *"the lists of numbers in the **X1 and X8 gains**"* —
|
||
4 sensors × 2 gains = **8 channels**. The protocol reference already documents
|
||
an unimplemented command with exactly that shape:
|
||
|
||
```
|
||
SUB 0x0E -> RSP 0xF1 "channel sensor data"
|
||
2-step read; channel selector in params[6:8] = 0x0000..0x0007
|
||
data length 0x0A (10 bytes) per channel
|
||
```
|
||
|
||
Blastware's *Unit Channel Test* sequence:
|
||
`POLL×N → 0x15 → 0x01 → 0x08 → 0x01 → 0x0E×8 → 0x98×2 → 0x0E×8`
|
||
— note the **second `0x0E` pass carries live ADC readings**.
|
||
|
||
**Hypothesis (untested):** `0x0E` returns the numbers Instantel wants compared
|
||
against 2027–2069. If true, SFM could diagnose an offset remotely *and* predict
|
||
whether a re-zero will succeed — converting a 10%/90% shipping gamble into a
|
||
decision made before packing a box.
|
||
|
||
**How to test.** `bridges/ach_mitm.py` is a generic TCP proxy:
|
||
|
||
```bash
|
||
python bridges/ach_mitm.py --bw-host <MODEM_IP> --bw-port 9034 --listen-port 9999
|
||
```
|
||
|
||
Point Blastware at the proxy and run **Unit Channel Test**.
|
||
⚠ In this topology the output filenames are reversed — the tool labels the
|
||
*connecting* side "unit", so `raw_s3_*.bin` holds Blastware's bytes and
|
||
`raw_bw_*.bin` the unit's.
|
||
|
||
Capture priority: (1) BE12599 while faulting, (2) a known-good unit as control,
|
||
(3) before/after an autozero on the same unit. Eight 10-byte payloads with an
|
||
expected value near 2048 is a very constrained puzzle.
|
||
|
||
### Q3 — What is the mild-offset rate?
|
||
Unmeasurable from event files (§2). Only the A/D-mode check sees it. Would
|
||
need a fleet sweep in A/D mode, or Q2 to succeed.
|
||
|
||
### Q4 — Does an offset recur on the same unit after service?
|
||
BE9558 shows episodes in 2020 and 2026; BE18438 twice in four months. Suggestive
|
||
of recurrence, but service records aren't in the data — only calibration dates.
|
||
|
||
---
|
||
|
||
## 8. Data and tooling
|
||
|
||
| what | where |
|
||
|---|---|
|
||
| detector | `scratch/offset_scan.py` |
|
||
| current results | `/home/serversdown/dl2-archive/offset_archive.csv` |
|
||
| earlier candidate list (Terra-View DB, 274 events) | `scratch/offset_candidates.csv` |
|
||
| archive working copy | `/home/serversdown/dl2-archive/files/` |
|
||
| archive source | NAS `DeathStar` 10.0.0.2, `/volume1/Uploads/TMI/DL2-Event-backup-8-25-26/Event/autocall home/` |
|
||
|
||
⚠ **The DL2 export keeps a byte-identical `Sent/` mirror of its root.** 13,077
|
||
waveform paths are 6,577 distinct files. Always dedupe by basename — this
|
||
doubled two reported figures before it was caught.
|
||
|
||
---
|
||
|
||
## 8b. The histogram corpus — the other 90% of the archive (2026-09-04)
|
||
|
||
Every result above §8 comes from **waveform** files. `offset_scan3.py` filters on
|
||
`\.[A-Za-z0-9]{2}0[Ww]$`, so the corpus it scanned is 6,577 unique binaries. The
|
||
archive also holds **63,535 unique histograms** — 9.7x more files — which the
|
||
pre-trigger method cannot touch, because a histogram carries no samples: only a
|
||
per-interval, per-channel peak and half-period.
|
||
|
||
`scratch/offset_hist_scan.py` scans them. **63,505 of 63,535 decoded (99.95%),
|
||
43 units, 77.9M intervals.** Two of the 45 units have no histograms at all.
|
||
Output: `/home/serversdown/dl2-archive/offset_hist.csv` (190,515 channel-rows).
|
||
|
||
### The premise, and how far it actually holds
|
||
|
||
A histogram file is hours of continuous monitoring, so most of its intervals are
|
||
definitionally quiet, and a channel parked off zero cannot report a peak below
|
||
its own displacement. The signal is real — two within-unit contrasts, siblings
|
||
unmoved in both:
|
||
|
||
| unit | channel | in-episode floor | outside | waveform \|pre\| same window |
|
||
|---|---|---|---|---|
|
||
| BE18438 | Vert | 0.0350 | 0.0050 | +0.18 .. +0.37 |
|
||
| BE12599 | Tran | 0.0250 | 0.0050 | +0.03 .. +0.49 |
|
||
|
||
But the **leakage from a waveform pedestal into the histogram floor is bimodal,
|
||
not merely partial**: measured ratio ~0.9 on BE18438 Vert, ~0.7 on BE9558,
|
||
**~0.02 on BE12599** — two orders of magnitude on one instrument. The device
|
||
evidently measures each interval peak against a running baseline, and how much
|
||
DC survives that varies per unit. **Consequence: a negative histogram result
|
||
carries almost no information.** Do not read "clean in the histograms" as clean.
|
||
|
||
### The detector that survived
|
||
|
||
dmin(file, ch) = min[ch] - min over the other two geo channels, SAME file
|
||
gates (both hard): n_intervals >= 60 AND mic_p5 <= 5 raw counts
|
||
day statistic: median of dmin over that day's qualifying files
|
||
flag day at dmin >= 0.020 in/s (4 A/D counts)
|
||
episode at >= 3 CONSECUTIVE observed days
|
||
|
||
**Result: BE18438|Vert, BE9558|Tran, BE9558|Long.** Threshold-insensitive —
|
||
the journal's own test for a real signal against a tuned one — and this is the
|
||
first operating point in the investigation that passes it cleanly. The identical
|
||
answer holds across: statistic `min` or `p5`; length gate 10/30/60/120/300; mic
|
||
gate 3/5/8/10; threshold 0.015–0.035 (a 2.3x span); persistence K = 2,3,4,5,7.
|
||
|
||
Separation, ranked by highest floor sustained over 3 consecutive gated days
|
||
across all 135 unit-channels:
|
||
|
||
| unit-channel | best3 |
|
||
|---|---|
|
||
| BE18438 Vert | 0.1650 |
|
||
| BE9558 Long | 0.0350 |
|
||
| BE9558 Tran | 0.0250 |
|
||
| *(2.5x gap)* | |
|
||
| BE7145 Tran | 0.0100 |
|
||
| entire rest of fleet | <= 0.0050 (one quantisation count) |
|
||
|
||
Day-level false alarm: **37 of 99,432 gated unit-channel-days = 0.037%.**
|
||
|
||
### What it does NOT do — read this before trusting it
|
||
|
||
- **It finds 2 of the 5 confirmed units, not 5.** The site-quiet gate is what
|
||
makes it work and it is also what costs BE11529 and BE12599. BE11529's
|
||
four-day single-axis ramp (Tran 0.025 -> 0.055, both siblings pinned at 0.005)
|
||
is the most offset-shaped thing in the corpus outside the two detections, and
|
||
the gate discards it.
|
||
- **The positive class is two units.** Every threshold here is fitted to
|
||
BE18438 and BE9558, which contribute 22 of the 37 flagged days in the entire
|
||
corpus. No cross-validation is possible at n=2.
|
||
- **Per-channel attribution is NOT established.** Rotating the three geo channel
|
||
labels within each file — preserving every value, file and day, destroying
|
||
only channel identity — reproduces the episode *count* with p = 0.769 and the
|
||
label agreement at p = 0.038–0.077. Report a **unit and a window**; do not
|
||
name a geophone axis on the strength of this detector alone.
|
||
- **Timing resolution is ~1 month, not ~1 day.** A 30-day label shift still
|
||
scores 2 of 9 episode hits; the signal dies only past ~60 days. The day-level
|
||
series look far crisper than they are.
|
||
- **Ground truth here is a sibling detector, not a service record.** Agreement
|
||
between the two corpora is corroboration of a shared method. Nothing in this
|
||
section has been checked against an actual repair, calibration or RMA.
|
||
|
||
### Dead ends — keep these dead
|
||
|
||
- **Absolute floor (min / p1 / p5 / p10 / p25, thresholded alone) — RETIRED.**
|
||
Not fleet-comparable and mostly not about the channel. Scoring each cell using
|
||
*only the other two channels* — a statistic containing zero information about
|
||
the suspect channel — reaches AUC 0.746 against the same labels, versus 0.872
|
||
for the absolute floor itself. **66% of its apparent discrimination is "that
|
||
day was noisy at that site."** Interval size alone moves its p99 7x (0.0350 at
|
||
1 min vs 0.0050 at 2 s). And of all files with any channel above 0.025, 56.5%
|
||
have **all three** channels above it — common-mode, i.e. the wrong physics.
|
||
- **Zero-fraction — STRUCTURALLY IMPOSSIBLE, not merely weak.** The device never
|
||
reports a zero histogram interval peak. The value is a max over hundreds of
|
||
samples of a channel that always carries at least 1 count of noise, so it is
|
||
clamped at 1 A/D count (0.005 in/s). There is no zero to count.
|
||
- **Interval size, sample rate, geo range, firmware — refuted as confounds for
|
||
the differential.** All four are *file-level scalars*: they move all three geo
|
||
channels together, so they cannot produce a single-channel lift and the
|
||
within-file differential is immune to them by construction. Geo range is
|
||
identical across the three geo channels in **63,535 of 63,535** binaries.
|
||
(Interval size remains fatal to the *absolute*-floor version, above.)
|
||
|
||
### Two findings that are independent of the histogram detector
|
||
|
||
**1. `offset_scan3.py`'s `spread <= 0.02` gate is discarding real signal.**
|
||
It rejects **113 of the 600 channel-rows with |pre| >= 0.025 (18.8%)**, and the
|
||
rejections are not random — 92 of them fall across 41 unit-channels currently
|
||
labelled NEGATIVE. Four would become sustained positives under an
|
||
amplitude-only >=3-consecutive rule: **BE12599|Long (run of 8), BE18003|Vert
|
||
(4), BA10895|Vert (3), BE12844|Tran (3).** Until this is re-cut, the fleet label
|
||
is **three-state — POSITIVE / NEGATIVE / SPREAD-REJECTED(unknown)** — and the
|
||
third state should be excluded from both TP and FP counts rather than silently
|
||
scored as healthy. Every precision figure computed against the two-state label,
|
||
in this section and in §3, is affected.
|
||
|
||
**2. The waveform corpus sees ~7% of the days a unit was deployed.** 2,627
|
||
(unit, day) observations against the histogram corpus's 35,105 — 13.4x — with a
|
||
per-unit median ratio of 0.070. BE12599, a confirmed unit, is waveform-observed
|
||
on 39 of its 1,666 histogram-observed days (**2.3%**). Any statement of the form
|
||
"the fault was absent before date X" that rests on waveform coverage alone is
|
||
much weaker than its event count suggests.
|
||
|
||
### BA10895 — reclassified (see also §4)
|
||
|
||
Previously dismissed as a transient. The histogram record shows its **Vert**
|
||
quiet-minute floor at 0.005 on 62/62 qualifying files from 2023-07-07, then
|
||
0.010–0.015 on 48/58 files from 2023-08-03 to 08-27, while Tran moves on 2/58
|
||
and Long on 9/58 and the site mic floor never leaves 1–3 counts. Independently,
|
||
**42 of its 85 waveform events (49.4%) are single-axis-dominant** — one geo peak
|
||
>= 10x both siblings and >= 0.05 in/s — the **highest rate in the 45-unit
|
||
fleet** (BE13117 36.1%, BE18438 29.4%), and **100% of it on Vert**. Vert
|
||
excursions of 0.1–1.5 in/s with Tran/Long at 0.005–0.035 are not ground motion.
|
||
|
||
This is a genuine Vert-channel hardware fault, but **not the classic pedestal** —
|
||
the differential is only one A/D count. Caveat: its entire histogram record is a
|
||
single 52-day deployment ending 2023-08-27, so nothing says whether it
|
||
persisted, was serviced, or resolved.
|
||
|
||
The other six marginal units — BE11007, BE17354, BE18004, BE18104, BE9557,
|
||
BE18003 — are **clean**. All seven cap at +0.005 to +0.007 (one A/D count)
|
||
lifetime under the quiet-site gate, against +0.175 for BE18438 Vert and +0.062
|
||
for BE9558 Long. Three individual waveform flags fall in windows with **zero**
|
||
histogram coverage and are NO-DATA, not clean: BE18004|Tran 2024-10-16,
|
||
BE9557|Tran 2021-06-28, BE9557|Vert 2025-06-12.
|
||
|
||
### Still open in this section
|
||
|
||
- **The 11 thin-coverage units were not screened** (BE10202, BE11462, BE13779,
|
||
BE15760, BA15957, BE16754, BE16758, BE8081, BE8626, BA9229, BE9887 — each
|
||
under 20 waveform events, several with hundreds of histograms). This is the
|
||
population most likely to hold a previously unknown offset, and it is the one
|
||
slice of the plan that did not run. BE11462 was incidentally scored clean by
|
||
the full-archive pass; BE10202 has no histogram files at all.
|
||
- **No completeness audit was run** over the above.
|
||
- Re-cutting the ground truth three-state (finding 1) and re-scoring everything
|
||
against it.
|
||
|
||
---
|
||
|
||
## 8c. Mechanism — five hypotheses tested, all dead (2026-09-06)
|
||
|
||
**The mechanism is still unknown.** Five campaigns, ~105 effectively independent
|
||
tests, seven nominally significant results against **5.2 expected by chance**
|
||
under a global null. Every one died to its own confound analysis. What the
|
||
campaign bought is a set of *shape constraints* and a long list of dead ends.
|
||
|
||
### ⚠ Two things retracted from this journal
|
||
|
||
**1. "Polarity is perfectly consistent — 11 of 11, zero mixed cases."** That is
|
||
a **tautology of the spread gate**, not a property of the fault. `spread <= 0.02`
|
||
requires pre/mid/end to agree, which forces one sign. Amplitude-only at the same
|
||
0.025 threshold: **12 of 53 unit-channels are mixed**, including BE18438|Vert
|
||
(88+/1−) and BE9558|Vert (1+/35−). Withdrawn.
|
||
|
||
**2. "5 of 45 units, unchanged across a 2x threshold range."** The
|
||
threshold-insensitivity is also a property of the gate. Amplitude-only gives
|
||
**9 units at 0.020, 8 at 0.025** (adding BA10895, BE12844, BE18003), 5 at 0.040.
|
||
The fleet is **8–9 units, not 5**.
|
||
|
||
**3. "Persistent — it stays until the geophone is serviced."** Weakened, not
|
||
withdrawn. There are **23 recoveries after runs of >=3 flagged events, median
|
||
gap 6.03 days**, three inside ten minutes. BE18438|Vert reads `pre=mid=end=
|
||
+0.0000` on 2026-02-10, +0.185→+0.370 across 02-25/26, and `+0.0000` again on
|
||
2026-03-22 — identical Project, Seis Loc, calibration date, geo range and
|
||
trigger throughout. The one thing that cannot be excluded is a **field
|
||
autozero**: it is a button sequence at the unit and writes nothing into the
|
||
event header. So "persistent" may be "persistent unless somebody pressed the
|
||
buttons," and the archive cannot tell those apart.
|
||
|
||
### The one positive finding: onset is a RAMP, minutes to hours
|
||
|
||
Both onsets resolvable at minute cadence are ramps. **BE18438|Vert,
|
||
2026-02-20** — the histogram corpus collapses a 14 d 21 h waveform bracket to
|
||
**one minute**:
|
||
|
||
```
|
||
~14,200 consecutive quiet minutes at 0.000–0.005 (ten full daily files)
|
||
09:32 +0.005 09:39 +0.045 10:20 +0.125 16:00 +0.165
|
||
09:33 +0.010 09:42 +0.070 13:13 +0.150 20:17 +0.185 plateau
|
||
```
|
||
|
||
**50% of the excursion in 7 minutes**, the rest asymptotic over ~10 h, **>=25
|
||
distinct one-minute intermediates**. Validated **75/75** against Blastware's own
|
||
ASCII export. Its 2025-11-15 onset is the same shape over 2.7 h. BE13117 stage B
|
||
is a 91-minute monotone rise, +0.035 → +1.745 in/s over ~40 samples.
|
||
|
||
**This kills both poles of the original dichotomy** (journal Q1): not an
|
||
instantaneous latched step (a bad autozero, a stuck trim-DAC), and not slow
|
||
component degradation over days or weeks.
|
||
|
||
⚠ It rests on **2 of 45 instruments**. Clopper-Pearson on 4/4 resolved onsets
|
||
gives 95% CI [0.40, 1.00] — a mixed population with up to 60% true steps is not
|
||
excluded. BE13117 has zero paired ASCII, so its ramp rests on our decoder alone.
|
||
|
||
### The methodological corollary — more important than the finding
|
||
|
||
**A waveform-only bracket manufactures the appearance of a step, and the spread
|
||
gate is blind to onsets by construction.**
|
||
|
||
The offset is what fires the trigger, so no waveform event can exist until the
|
||
ramp has nearly reached the trigger level. BE18438's first event of each episode
|
||
sits at 0.280 against a 0.300 trigger, and 0.185 against 0.200. At daily cadence
|
||
against a 3 h ramp, P(catching an intermediate) = **0.125**.
|
||
|
||
And `spread <= 0.02` rejects any record in which the floor is *moving* — which
|
||
is exactly what an onset is. **The gate rejected the very BE18438 record where
|
||
the ramp is visible.** If the operational goal is catching a fault early, before
|
||
the unit floods the store with junk events, the current detector is the wrong
|
||
shape for the job.
|
||
|
||
### The surviving shape
|
||
|
||
An **electrical, reversible, two-time-constant settling process** (~10 min and
|
||
~hours), saturating at a ceiling, with occasional sub-3-minute discrete jumps
|
||
superposed (BE18438 2026-02-26: 13:24 pre +0.180 / mid +0.240 / end +0.255 →
|
||
13:27 +0.325, identical metadata). That is the signature of a **bias or leakage
|
||
path charging a high-impedance node** — the class of fault Instantel's autozero
|
||
recovers ~10% of the time, and what the X1/X8 gains measure.
|
||
|
||
**It is a shape constraint, not a mechanism. Do not write it up as one.**
|
||
|
||
### Dead — with the evidence, so none of this is re-derived
|
||
|
||
| Killed | Evidence |
|
||
|---|---|
|
||
| **Latched step at onset** | >=25 one-minute intermediates over ~10 h, ASCII-validated. Direct observation, not a test. |
|
||
| **Slow degradation over days/weeks** | Same observation — bulk of the excursion in 7 min to 2.7 h. |
|
||
| **Thermal driving of pedestal magnitude** | BE13117, 365-count pedestal, n=128: full-day modulation **−0.42% ± 0.42%**, 95% CI [−1.25%, +0.40%]. Healthy-fleet seasonal zero drift totals **~0.3 A/D counts** — 15x to 1200x too small. Best-powered result in the campaign. |
|
||
| **Ground-motion shock** | 30-day window-max percentile ranks 0.03/0.98/0.15/0.01/0.68/0.24, median **0.194** against a null of 0.5. **0 of 7 events >=9 in/s** was followed by an onset within 30 d. BE12599 hit 10.220 in/s (2023-11) and 10.005 (2025-04) and did not onset until 2026-08-14. |
|
||
| **Handling / redeployment** | **0 of 9** onsets had a Project/Client/Seis Loc change. Widened to 30 d: 2 observed vs 4.90 expected, P(X>=2)=0.995 — *depleted*, the wrong direction. The apparent gap effect (p=0.035) died on histogram coverage: BE18438's "59.7-day gap" contains 122 histogram files; true silence 0.52 d. |
|
||
| **Mechanical resonance / damping change** | BE18438|Vert at a 64-count pedestal (3x outside Instantel's ±21): ΔTest-Freq **CI [−0.090, +0.021]** against 0.127 Hz for a real calibration. Block permutation p=0.658. |
|
||
| **Accumulated-duty threshold** | ~4 clean units logged more monitoring than the largest positive onset dose; BE18193 logged **13.45M intervals, 6.2x**. A counterexample — no power argument weakens it. |
|
||
| **Firmware** | **14,338 of 14,340** exports read `V 10.72-8.17`. A constant cannot explain a variable. |
|
||
| **Unit age** | Serial rank-sum 118.0 vs null 115.0, p=0.549; unchanged on the 8-unit re-cut (p=0.586). Serial is a poor age proxy anyway (Spearman +0.113 against archive entry). |
|
||
| **Strong seasonal clustering** | 25 onsets, exposure-weighted permutation **p=0.59**. Excludes >=75%-in-one-season only; a 2x seasonal hazard is *not* excluded. |
|
||
|
||
Also retire two overstated bounds. H6's dose-response exclusion "|r| > 0.03" is
|
||
a **10x overstatement** once clustering is corrected — the honest bound is
|
||
|r| > 0.1–0.3, so a real r=0.2 is not excluded. And **any statistic quoted
|
||
per-event**: 512 flagged channel-events collapse to **4.9 effective independent
|
||
observations** (unequal-cluster design effect 104.6 at ICC=1), and **55% of the
|
||
flagged corpus is one instrument on two calendar days** (BE13117, 2023-05-03/04).
|
||
|
||
### Power — read every negative in this section as bounded
|
||
|
||
Fisher exact, 5 positives of 45, one-sided α=0.05, exposure a third of the fleet:
|
||
|
||
| relative risk | power |
|
||
|---|---|
|
||
| 1.5 | 0.059 |
|
||
| 2 | 0.112 |
|
||
| 3 | 0.231 |
|
||
| 6 | 0.497 |
|
||
| 15 | 0.753 |
|
||
|
||
80% power needs **RR ≈ 13–20**. Even a *perfect* split reaches p<0.05 only if
|
||
the exposed group is <=25 of 45 units. **This archive can detect only
|
||
near-deterministic unit-level causes.** Every negative above excludes a strong
|
||
effect, not a real one.
|
||
|
||
### What this archive can NEVER answer
|
||
|
||
- **The A/D zero and the X1/X8 gains.** The 2027–2069 numbers appear in no file,
|
||
header or decoded record. They exist only on a live device behind `SUB 0x0E`.
|
||
Q1 is structurally unanswerable from data.
|
||
- **Unit-level vs component-level cause.** **Zero of 14,340** exports carry a
|
||
geophone or sensor serial. Q4 is dead — there is no way to know whether the
|
||
same physical geophone came back after service.
|
||
- **Service history.** The only service-adjacent field is `Calibration: <date>`
|
||
— 30 distinct dates fleet-wide, none before 2023, ASCII corpus entirely
|
||
2025–26. BE9558's 2020 and BE13117's 2023 episodes have no calibration record.
|
||
- **Temperature.** Zero exports carry it. Battery Level is a verified coarse
|
||
thermometer (+0.204 V winter over summer, 20/20 unit-years, p=9.5e−7, matching
|
||
lead-acid tempco) but quantised at 0.1 V ≈ 10 °C — useless within a day. The
|
||
archive can *bound* thermal; it can never *test* it.
|
||
- **BE13117 specifically** — 55% of the flagged corpus, the largest pedestal at
|
||
1.92 in/s, **zero** ASCII exports, histogram record ending eight months before
|
||
its episode. The most informative case in the archive is permanently outside
|
||
every metadata test.
|
||
- **The mild-offset rate**, and therefore the base rate's denominator. Event
|
||
files only see offsets large enough to dominate the trace.
|
||
|
||
### The experiment to run — `SUB 0x0E`, one afternoon
|
||
|
||
Point Blastware at `bridges/ach_mitm.py` and run **Unit Channel Test** against
|
||
(1) a faulting unit, (2) a known-good control, (3) the same unit before and
|
||
after an autozero. BW's sequence is `0x0E x8 → 0x98 x2 → 0x0E x8`, the second
|
||
pass carrying live ADC. Eight 10-byte payloads with expected values near 2048 is
|
||
a very constrained puzzle.
|
||
|
||
- **Proves:** whether the X1/X8 gains are readable over the wire, and whether
|
||
the fault sits at or upstream of the ADC zero reference. Gains walk out of
|
||
2027–2069 with the pedestal → the fault *is* the zero reference, Q1 answered.
|
||
Gains hold while the trace moves → the fault is downstream, look at the front
|
||
end.
|
||
- **§8c hands it a falsifiable time course:** poll at ~1-minute cadence and the
|
||
numbers should **ramp over minutes-to-hours, not step**. If they step while
|
||
the trace ramps, the two are decoupled.
|
||
- **Payoff:** converts the 10%/90% ship-it-or-not gamble into a decision made
|
||
before packing a box, remotely, for the whole fleet.
|
||
- ⚠ In the MITM topology filenames are reversed — `raw_s3_*.bin` holds
|
||
Blastware's bytes.
|
||
|
||
**Second: swap the geophone** between a faulted base and a healthy one. Fault
|
||
follows the sensor → element or cable. Fault stays with the base → front-end
|
||
board. One afternoon, zero code, and it settles the one question the archive is
|
||
permanently blind to.
|
||
|
||
**Third: log a faulting unit for 72 h untouched.** Every recovery we have is
|
||
confounded by a possible field autozero. A shelf and a logger settles whether
|
||
the fault genuinely self-reverses.
|
||
|
||
**Fourth, free: re-cut the fleet label** — drop the spread gate, re-score
|
||
amplitude-only, screen the 11 unscreened thin-coverage units. Might reach 9–10
|
||
positives. Be honest about the gain: power against "older half carries 3x the
|
||
hazard" rises only 0.23 → 0.30.
|
||
|
||
**Highest-value item overall, and not an experiment: the RMA/repair records.**
|
||
Which unit went back, when, what was done (autozero vs geophone replaced vs
|
||
board), and the geophone serial fitted. "Same channel after a documented
|
||
geophone *replacement*" is component-level-negative in one observation.
|
||
|
||
---
|
||
|
||
### 8d. The non-motion test — Brian's "it doesn't cross zero" (2026-09-07)
|
||
|
||
Looking at BE12599's 2026-08-09 event, Brian noted it reports no ZC frequency
|
||
**because the trace never crosses zero**. That observation is the best detector
|
||
in this investigation, and it comes from physics rather than a threshold.
|
||
|
||
A geophone is a velocity sensor with no DC response, so its output over a record
|
||
must integrate to ~zero — the ground does not relocate. Real motion therefore
|
||
sits roughly half below zero. Anything electrical is one-sided.
|
||
|
||
mp = |mean| / peak ~0 for motion, ~1 for a fault
|
||
frac_neg = share of samples < 0
|
||
|
||
`scratch/nonmotion_scan.py`, all 6,577 waveforms, 19,731 channel-rows.
|
||
Restricted to peak >= 0.05 in/s (n = 12,068), the distribution is **bimodal
|
||
with an empty middle**:
|
||
|
||
| mp band | channel-events |
|
||
|---|---|
|
||
| 0.0–0.1 | 11,384 |
|
||
| 0.1–0.2 | 293 |
|
||
| **0.15–0.85 (dead zone)** | **131 = 1.09%** |
|
||
| 0.9–1.0 | 278 |
|
||
|
||
At `mp >= 0.8` with >=3 events it returns **exactly the five confirmed units** —
|
||
BE9558, BE11529, BE12599, BE13117, BE18438 — stable from 0.5 to 0.9. Two
|
||
detectors on entirely different principles agreeing on the unit list is the
|
||
strongest corroboration that list has.
|
||
|
||
**BE11007 is settled: NOT an offset.** It reaches mp 0.75–0.89, but with
|
||
`frac_neg = 0.99` at peaks of **7.4–9.4 in/s** — parked *negative* during a
|
||
near-full-scale blast. §4's guess was right. `mp` alone cannot separate a
|
||
pedestal from a large one-sided blast; pair it with a peak ceiling or with
|
||
sign-consistency across events.
|
||
|
||
⚠ **Not a rediscovery of the retracted v1 detector.** v1 scored only the
|
||
largest-peak axis and used the mean as a *baseline estimator* where the median
|
||
was required. Here the mean is the signal itself, per channel — that is what the
|
||
physics licenses.
|
||
|
||
**Correction to §8c.** That section says the spread gate is "blind to onsets by
|
||
construction." Too strong: of 87 BE18438|Vert events at mp >= 0.5 the gate
|
||
rejected **one** — the transitional record. It does not lose onsets
|
||
systematically; it loses the transition specifically.
|
||
|
||
### 8e. BE12599 — a connector, not a geophone (2026-09-07)
|
||
|
||
Waveform shapes across its August episode, measured rather than eyeballed:
|
||
|
||
| date | channel | shape |
|
||
|---|---|---|
|
||
| Aug 09 05:29 | Long | **unipolar +**, 0/2304 samples below zero, decay tau **26 ms** |
|
||
| Aug 09 05:35 | Long | unipolar +, 3 spikes at irregular gaps (744, 1032 ms), tau **38 ms** |
|
||
| Aug 14 05:00 | Long | single lobe, bipolar, tau **118 ms** |
|
||
| Aug 17–23 | Tran | **flat DC pedestal**, sd/level 0.015–0.020, 0 zero crossings |
|
||
|
||
**Unipolar impulses with an RC tail are not mechanical.** Fast rise, exponential
|
||
decay, one polarity, irregular timing — that is charge dumped into a
|
||
capacitively-coupled input and draining through the input resistance. The
|
||
progression 26 ms -> 118 ms -> never recovers, over 14 days, is a leakage path
|
||
worsening.
|
||
|
||
**And the fault moved channels** — Long on Aug 9/14, Tran on Aug 17–23, Long
|
||
again on Aug 21 (1.065 in/s) while Tran held its pedestal. Vert stayed clean
|
||
throughout. **A failing geophone element cannot hop channels. A connector can.**
|
||
|
||
That single fact explains what had been puzzling:
|
||
- **The sensor self-check keeps passing** (7.4/7.5/7.6 Hz, ratios 3.6–4.2, all
|
||
four channels Passed, on the very events where Long throws 0.5 in/s spikes).
|
||
The swing test drives the element; the element is fine. The fault is in the
|
||
wiring to it.
|
||
- **Why Instantel's autozero fixes only ~10%** — it cannot fix a connector.
|
||
- **Why onset "ramps" over minutes to hours** — contact resistance drifting.
|
||
|
||
All seven Aug 17–23 events are stamped **05:00:14**, the same second, and their
|
||
filename extensions run `8E → WE → KE → 8E → WE → KE → 8E` — the documented
|
||
3-day cycle for a fixed daily time. Clock-scheduled, not physically triggered:
|
||
the modem powers up, draws a surge, and a marginal connection responds.
|
||
|
||
**Field action: inspect and photograph the geophone connector BEFORE reseating
|
||
anything** — an intermittent contact clears the moment it is disturbed.
|
||
|
||
⚠ Scoped to BE12599. BE18438's onset was a smooth 7-minute ramp with no spikes,
|
||
which looks like a different failure mode wearing the same signature.
|
||
|
||
---
|
||
|
||
### ⚠ Serial prefixes — four of these units are BlastMates, not MiniMates
|
||
|
||
Corrected 2026-09-06, after Brian queried "BA10895?" against a report that
|
||
said BE10895. He was right. The BW filename encodes the serial **number
|
||
only** — `L895` -> 10895 — and every offset scanner synthesised the family
|
||
prefix as `"BE"`. Four of the 43 archive units are **BA** (BlastMate, the
|
||
MiniMate Plus's bigger sibling; same Series III, byte-identical data):
|
||
|
||
**BA9229, BA10060, BA10895, BA15957.**
|
||
|
||
Read off the file bodies, which carry the serial verbatim. No analysis
|
||
changed — grouping was always on the numeric part, and no unit number maps
|
||
to two serials — but every earlier reference to "BE10895" and the other
|
||
three is a label error and has been corrected throughout this document.
|
||
|
||
The same assumption was live in two production sites and is fixed
|
||
(`sfm/waveform_store.py`, `minimateplus/client.py`): the store would have
|
||
filed a BlastMate under a unit that does not exist, and the monitor-log
|
||
decoder lost the geo threshold along with the serial. See commit `9ceff65`.
|
||
|
||
---
|
||
|
||
## 9. Chronology
|
||
|
||
| date | event |
|
||
|---|---|
|
||
| 2026-08-25 | Reported as a *waveform decode bug* — traces with a DC offset. Investigation shows the offset is **real device data**; the decoder is correct. |
|
||
| 2026-08-25 | Brian relays his dad's description: a known hardware fault called an "offset"; usually sent to Instantel. |
|
||
| 2026-08-25 | First detection pass over the Terra-View DB: **2 of 21 units**, 274 events, 11 months. Flagged as vulnerable to survivorship bias — flooded events were routinely deleted. |
|
||
| 2026-08-25 | Condensation hypothesis proposed, then **killed by its own controls**. |
|
||
| 2026-08-25 | Parked pending the multi-year archive. |
|
||
| 2026-08-28 | DL2 archive pulled (33 GB, 546k files; 6.6 GB working set). |
|
||
| 2026-08-28 | Archive scan: **6 of 45 units**, 283 events, 15 episodes. Prior base rate **confirmed**, not overturned. |
|
||
| 2026-08-28 | Clipping ruled out; `m/p` established as the discriminator; BE11007 reclassified as probably a real blast. |
|
||
| 2026-08-28 | Calibration-timing correlation attempted and **rejected as confounded**. |
|
||
| 2026-08-28 | Instantel FAQs supplied: autozero procedure, the **2027–2069** window, the **>5 counts** threshold. Explains the ~10% re-zero success rate. |
|
||
| 2026-08-28 | Bimodality established; sensor check proven **blind** to offsets; `SUB 0x0E` identified as the best open lead. |
|
||
| 2026-08-28 | **v1 detector retracted.** Brian challenged the "come and go" finding against field experience. Two flaws found: dominant-axis-only scoring and mean-instead-of-median. Corrected detector shows persistent pedestals on **8 of 45 units**, and the gaps are service windows. |
|
||
| 2026-08-28 | **Detector v3 (Brian's method):** pre-trigger floor + pre/mid/end consistency. Healthy channels proven to sit at 0.000 +/-1 unit (94.5%), confirming no decoder zero-point bias. Final: **5 of 45 units (11%)**, threshold-insensitive. |
|
||
| 2026-09-04 | **Histogram corpus scanned** — 63,505 of 63,535 files, 43 units, 77.9M intervals (9.7x the waveform corpus). `scratch/offset_hist_scan.py`. |
|
||
| 2026-09-04 | Absolute-floor statistic **retired**: 66% of its discrimination is a day/site confound (other-channels-only AUC 0.746 vs 0.872). Zero-fraction shown **structurally impossible** — the device clamps every interval peak at >= 1 count. |
|
||
| 2026-09-04 | Site-quiet-gated cross-channel differential established: **BE18438 Vert, BE9558 Tran+Long**, threshold-insensitive over a 2.3x span. Finds only **2 of the 5** confirmed units — leakage into the histogram floor is bimodal (0.9 to 0.02), so a negative result carries almost no information. Per-channel attribution **not** established (channel-scramble p = 0.769). |
|
||
| 2026-09-04 | **BA10895 reclassified** from transient to a genuine Vert fault of a different subtype — 49.4% single-axis-dominant events, the highest in the fleet, 100% on Vert. The other six marginal units are clean. |
|
||
| 2026-09-04 | **Defect found in `offset_scan3.py`**: its `spread <= 0.02` gate discards 18.8% of rows with \|pre\| >= 0.025, concentrated on 41 negative unit-channels; 4 would be sustained positives without it. The fleet label is three-state, not two. |
|
||
| 2026-09-06 | **Four units relabelled BA, not BE** — BA9229, BA10060, BA10895, BA15957 are BlastMates. The BW filename carries only the serial number; the family prefix must be read from the file body. Fixed in the scanners and in two production sites. |
|
||
| 2026-09-06 | **Mechanism campaign — five hypotheses, all dead.** Thermal, ground-motion shock, handling/redeployment, accumulated duty, unit age, firmware and a mechanical element fault are each refuted or bounded. 7 nominally significant results against 5.2 expected by chance. |
|
||
| 2026-09-06 | **Onset is a RAMP of minutes-to-hours, not a step** — BE18438 Vert resolved to one-minute cadence, 50% of the excursion in 7 min, >=25 intermediates, ASCII-validated 75/75. Kills both a latched digital step AND slow component degradation. Surviving shape: a reversible two-time-constant settling process — a bias/leakage path charging a high-impedance node. |
|
||
| 2026-09-06 | **Polarity consistency RETRACTED** (a tautology of the spread gate; amplitude-only gives 12 of 53 unit-channels mixed) and the fleet **re-cut to 8–9 units, not 5**. "Persistent until serviced" weakened: 23 recoveries, median gap 6 days — though a field autozero cannot be excluded. |
|
||
| 2026-09-06 | The spread gate is **blind to onsets by construction** — it rejects a moving floor, which is what an onset is. It rejected the very record in which the ramp is visible. |
|
||
| 2026-09-07 | **The non-motion test** (Brian: "it doesn't cross zero"). `\|mean\|/peak` is bimodal with a 1.09% dead zone and returns exactly the 5 confirmed units from physics, not a threshold. Independent corroboration of the unit list. **BE11007 settled as NOT an offset** — a one-sided 9 in/s blast. |
|
||
| 2026-09-07 | **BE12599 is a connector fault, not a geophone fault.** Unipolar spikes with a 26→118 ms RC tail progressing to a flat pedestal, and the fault MOVES between Long and Tran while the sensor self-check passes on every event. An element cannot hop channels; a connector can. Inspect before reseating. |
|