From 8710b8f327afb06c86c5fbbe2b0b35e9b89d976e Mon Sep 17 00:00:00 2001
From: serversdown <brian@serversdown.net>
Date: Fri, 22 May 2026 21:02:13 +0000
Subject: [PATCH] docs: record three known issues discovered during prod
 deployment
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

1. bw_ascii_report parser misses PPV/vector_sum fields on certain TXT
   formats (5 events in prod).  Parser extracts every OTHER field for
   the same channels — likely a regex / format mismatch specific to
   some firmware-or-event-type combination.

2. NULL-timestamp duplicate rows.  events.timestamp can come back as
   NULL when the codec can't extract a footer timestamp; UNIQUE(serial,
   timestamp) doesn't fire on NULL, so backfills create new rows
   instead of upserting.  2 affected events on prod, easy SQL cleanup.

3. Histogram body sub-format with byte[5] != 0.  ~3 events on prod
   (T190LD5Q, O121L4L1) use a histogram body the walker doesn't
   recognize.  Codec returns 0 valid blocks; DB peaks come from the
   bw_report ASCII overlay so DB columns are correct, only the .h5
   plot is empty.  Cracking the sub-format unlocks the plot.

All three are pre-existing issues that today's deployment surfaced
during validation; none are regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---
 README.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/README.md b/README.md
index 6433158..d62c1e5 100644
--- a/README.md
+++ b/README.md
@@ -536,6 +536,7 @@ Implementation steps (concrete):
 
 ### BW ASCII report parser enhancements (built in v0.16.0)
 
+- [ ] **PPV field misses on certain TXT formats.**  Discovered 2026-05-22 during the histogram-codec backfill validation: a handful of events (5 in prod) have a `bw_report` block where `peaks.{tran,vert,long}.ppv_ips` and `peaks.vector_sum.ips` are all `None`, despite the parser correctly extracting every OTHER field for the same channels (zc_freq_hz, time_of_peak_s, peak_accel_g, peak_disp_in).  Symptom on the DB side: `peak_vector_sum=0` after a `--force` backfill that overlays from the parsed bw_report dict.  Affected events on prod include `T190LD5Q.LK0W`, `T438L713.RY0W`, `K557L3YM.OE0W`.  Root cause likely a regex or format mismatch for the "PPV" header line in those specific firmware/event-type outputs.  Once fixed, re-forwarding the events from series3-watcher will re-populate the `bw_report` blocks correctly.
 - [ ] **Histogram-specific structural fields.**  Current parser handles the shared fields (PPV, ZC Freq, sensor self-check, project) but silently drops histogram-only fields: `Histogram Start/Stop Time`, `Histogram Start/Stop Date`, `Number of Intervals`, `Interval Size`, per-channel `Peak Time` + `Peak Date` (absolute timestamps rather than the waveform's `Time of Peak` relative seconds).
 - [ ] **Histogram interval bin-table parsing.**  Trailing 792-row table (per-interval Peak/Freq per channel + MicL) in histogram TXTs is unparsed.  Probably too big for the sidecar JSON; may want a separate `.histogram.h5` companion file.
 - [ ] **`>100 Hz` value parsing.**  Histogram TXTs use `>100 Hz` for out-of-range ZC freq; current `_parse_number()` returns `None` for these (loses information).
@@ -564,3 +565,5 @@ Implementation steps (concrete):
 - [ ] Locate "Sensor Check" byte in compliance config (need capture with Disabled vs Before-monitoring).
 - [ ] Call Home — map time slots 3/4 offsets; confirm `modem_power_relay_enabled`.
 - [ ] RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred).
+- [ ] **NULL-timestamp duplicate-row dedup.**  A small handful of events (2 known on prod as of 2026-05-22) have `events.timestamp IS NULL` because the codec couldn't extract a timestamp from the binary footer.  The `UNIQUE(serial, timestamp)` constraint doesn't fire on `NULL` (SQL semantics: `NULL ≠ NULL`), so every `--force` backfill INSERTs a new row instead of UPSERTing the existing one.  Cleanup: a one-shot SQL query that keeps only the newest row per `(serial, blastware_filename)` and deletes the rest.  Longer-term: extend the unique key to `(serial, COALESCE(timestamp, blastware_filename))` or reject inserts with NULL timestamp.
+- [ ] **Histogram body sub-format with `byte[5] != 0`.**  ~3 events on prod (`T190LD5Q.LD0H`, `O121L4L1.GU0H`) use a histogram body my walker doesn't recognize — the first block has `byte[5] = 0x01` or `0x07` instead of `0x00`, and the entire body lacks the `1e 0a 00 00` tail signature.  Codec returns 0 valid blocks; their DB PVS comes from the bw_report ASCII overlay (which BW computed from the same binary, so the DB columns are correct).  Only the `.h5` waveform plot is empty.  Cracking the sub-format would unlock the plot.  Needs binary+ASCII pairs from a few `byte[5]!=0` events; same RE approach as the K558 case.