From e07f76dd31cc10c5115a5e401a9e3a1daf389d5a Mon Sep 17 00:00:00 2001 From: serversdown Date: Sat, 29 Aug 2026 06:12:19 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20correct=20the=20v0.27.0=20backfill=20cl?= =?UTF-8?q?aim=20=E2=80=94=20prod=20needs=20no=20backfill?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v0.27.0 notes said prod held 4 histograms that would stay empty until a backfill. That was wrong, and asserted without checking. Verified: all four recovered files (K440HJCN.3C0H, K557IF1U.8K0H, T191HVNP.0S0H, T193L0XM.CI0H) are archive-only — none appears in the production store or the events DB. Re-running stride detection over the prod store's 10,215 histogram binaries under both the old and new code shows 0 files whose decode changes. So the partial-final-block fix is forward-looking: it matters for future ingests of sub-minute histograms with a partial final block, not for anything already stored. TOOL_VERSION still moves with the release, so a future backfill run will regenerate the whole store instead of skipping. Harmless — byte-identical output for every stored file — but it costs the full ~2 hours on the NAS, so it should not be started casually. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01HgTe8CamXAHcAmaQ6QNcog --- CHANGELOG.md | 14 +++++++++++--- CLAUDE.md | 5 +++-- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 591a5de..6b4fad0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,9 +39,17 @@ carried, and it found one real codec bug (below). walk double-counts every binary — 127,035 paths are 63,535 distinct files. The ASCII exports are *not* mirrored, so the 14,340 pair count is already distinct.) - ⚠ Prod stores hold `.h5` files generated before this fix. Those 4 events stay - empty until `backfill_sidecars.py` is re-run — not worth a two-hour prod backfill - on its own; fold it into the next one. + **No prod backfill is required for this.** Verified after the fact: all four + recovered files are archive-only — none exists in the production store or the + events DB — and re-running stride detection over the production store's + **10,215** histogram binaries shows **0 files whose decode changes**. The fix + matters for future ingests of sub-minute histograms with a partial final block, + not for anything already stored. + + (`TOOL_VERSION` moves with the release, so whenever a backfill *is* next run for + some other reason it will regenerate the whole store rather than skipping. That + is harmless — the output is byte-identical for every currently-stored file — but + it means the run takes its full ~2 hours on the NAS.) - **Histogram/waveform twin matching is now interval-based** (`find_twins`). A real trigger is recorded twice — as a triggered waveform (stamped at the trigger instant) diff --git a/CLAUDE.md b/CLAUDE.md index 2e3ae32..0371b47 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -33,8 +33,9 @@ Read this first when picking the project back up. (it gates regeneration). ⚠ On the office NAS this takes **~2 hours** (~1.5 files/sec vs 85/sec on the dev box — gzip-4 in `sfm/event_hdf5.py` against a Synology CPU). Budget it up front. - **v0.27.0 owes prod a backfill:** the partial-final-block fix recovers 4 - histograms that are still empty in the store. + **v0.27.0 does NOT owe prod a backfill** — verified: the partial-final-block + fix changes 0 of the 10,215 histograms in the prod store (the 4 recovered + files are archive-only and were never ingested). - **The "offset" hardware fault has its own journal** -- `docs/offset_investigation.md`. **5 of 45 units (11%)**, and the fault is **persistent** — it stays until the geophone is serviced. Detect it with