Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 86325b9bab | |||
| 6381dcb312 |
+12
-1
@@ -2,10 +2,21 @@ FROM python:3.11-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# tzdata is required for the TZ env var to take effect (python:slim
|
||||
# omits the timezone database). Without it, datetime.now() / logging
|
||||
# / matplotlib all stay in UTC regardless of TZ. Default zone gets
|
||||
# set further down via ENV; users override per-deployment via the
|
||||
# `TZ` env var in docker-compose.
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends curl && \
|
||||
apt-get install -y --no-install-recommends curl tzdata && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Default display timezone — applied to server logs, datetime.now(),
|
||||
# matplotlib rendered timestamps, and any naïve-vs-aware datetime
|
||||
# conversions in the PDF renderer. Override via TZ env var in
|
||||
# docker-compose; storage in the DB is always UTC regardless.
|
||||
ENV TZ=America/New_York
|
||||
|
||||
COPY pyproject.toml requirements.txt ./
|
||||
COPY minimateplus ./minimateplus
|
||||
COPY micromate ./micromate
|
||||
|
||||
@@ -567,4 +567,5 @@ Implementation steps (concrete):
|
||||
- [ ] RV55 DCD/DTR — newer RV55 firmware doesn't assert DCD by default; units don't resume monitoring after call-home disconnect (`--restart-monitoring` flag deferred).
|
||||
- [ ] **NULL-timestamp duplicate-row dedup.** A small handful of events (2 known on prod as of 2026-05-22) have `events.timestamp IS NULL` because the codec couldn't extract a timestamp from the binary footer. The `UNIQUE(serial, timestamp)` constraint doesn't fire on `NULL` (SQL semantics: `NULL ≠ NULL`), so every `--force` backfill INSERTs a new row instead of UPSERTing the existing one. Cleanup: a one-shot SQL query that keeps only the newest row per `(serial, blastware_filename)` and deletes the rest. Longer-term: extend the unique key to `(serial, COALESCE(timestamp, blastware_filename))` or reject inserts with NULL timestamp.
|
||||
- [ ] **Histogram body sub-format with `byte[5] != 0`.** ~3 events on prod (`T190LD5Q.LD0H`, `O121L4L1.GU0H`) use a histogram body my walker doesn't recognize — the first block has `byte[5] = 0x01` or `0x07` instead of `0x00`, and the entire body lacks the `1e 0a 00 00` tail signature. Codec returns 0 valid blocks; their DB PVS comes from the bw_report ASCII overlay (which BW computed from the same binary, so the DB columns are correct). Only the `.h5` waveform plot is empty. Cracking the sub-format would unlock the plot. Needs binary+ASCII pairs from a few `byte[5]!=0` events; same RE approach as the K558 case.
|
||||
- [ ] **Histogram body sub-format with `byte[5] == 0x00` but undecodable.** Observed 2026-05-28 on BE17353 (S353) events: `S353L4H2.FZ0H`, `S353L4H2.P00H`, `S353L4H3.7O0H`, `S353L4H3.E10H`. Body starts `00 00 00 01 0a 00 XX 00 ...` which LOOKS like a valid histogram block header (marker 0x000a at byte[4:6] ✓, byte[5]=0x00 normal-format ✓), but the walker finds zero data blocks across the whole body. Likely an extra header before the block stream OR a different tail signature than `1e 0a 00 00`. Smaller body lengths (1900-2100 bytes) suggest these may be short-recording histogram variants. Same operational impact as the byte[5]!=0 case: event ingests cleanly, DB peaks correct via bw_report overlay, only the chart is empty. Worth dumping a hex view of one body to diagnose.
|
||||
- [ ] **Sensor-check waveform extraction from the BW binary.** BW's Event Report PDFs include a narrow panel on the right side of the waveform plot showing each channel's response to the sensor self-check signal (a damped sinusoid for geo, sawtooth-at-test-freq for mic). Our parser captures the test RESULTS (`test_freq_hz`, `test_ratio`, `test_amplitude_mv`, `test_results` pass/fail) and the PDF + modal display them as text — but BW's per-sample sensor-check waveform isn't accessible to us today. Two paths to add it: (a) RE the binary to find where the sensor-check samples are stored — could be a section before STRT, after the footer, or in a separate sub-record; protocol reference doesn't currently mention it. (b) If samples aren't in the binary, synthesize a representative waveform from the test parameters (damped sinusoid at `test_freq_hz` with damping from `test_ratio`). Path (a) is the honest answer; path (b) is decorative. Until either lands, the text-only sensor-check display in the report is fine.
|
||||
|
||||
+34
-13
@@ -348,9 +348,11 @@ def render_event_report_pdf(rd: ReportData) -> bytes:
|
||||
# Page footer (common to both layouts) — Created date + event id.
|
||||
# Pushed to the very page bottom so it doesn't collide with the
|
||||
# waveform footer scale / trigger legend lines just above.
|
||||
# Convert UTC server_received_at to local for display.
|
||||
created_local = _fmt_iso_to_bw(rd.server_received_at) if rd.server_received_at else "—"
|
||||
fig.text(
|
||||
0.07, 0.005,
|
||||
f"Created: {rd.server_received_at or '—'} • seismo-relay",
|
||||
f"Created: {created_local} • seismo-relay",
|
||||
fontsize=6, color="#888", ha="left",
|
||||
)
|
||||
fig.text(
|
||||
@@ -419,31 +421,50 @@ def _render_histogram_layout(fig, rd: ReportData) -> None:
|
||||
_draw_histogram_subplot(fig, gs[3], rd)
|
||||
|
||||
|
||||
def _to_display_local(iso: str):
|
||||
"""Parse an ISO timestamp and return a datetime in the system's local
|
||||
timezone (set by the TZ env var, default America/New_York via the
|
||||
Dockerfile).
|
||||
|
||||
Behaviour:
|
||||
- "...Z" or "...+HH:MM" suffix → tz-aware UTC → converted to local
|
||||
- Naïve "YYYY-MM-DDTHH:MM:SS" (no tz) → returned as-is. This
|
||||
matches the convention used elsewhere in seismo-relay: BW's
|
||||
recorded-at timestamps are naïve and ALREADY in the unit's
|
||||
local clock; we don't second-guess them.
|
||||
"""
|
||||
import datetime as _dt
|
||||
dt = _dt.datetime.fromisoformat(iso.replace("Z", "+00:00"))
|
||||
if dt.tzinfo is not None:
|
||||
# Convert from UTC (or other tz) → local per the TZ env var.
|
||||
# astimezone() without arg uses the system timezone.
|
||||
dt = dt.astimezone()
|
||||
return dt
|
||||
|
||||
|
||||
def _fmt_iso_to_bw(iso: Optional[str]) -> Optional[str]:
|
||||
"""Convert a ISO-8601 timestamp like '2026-05-16T22:30:37' to BW's
|
||||
display format '22:30:37 May 16, 2026'. Returns input unchanged if
|
||||
it doesn't look like ISO."""
|
||||
"""Convert an ISO-8601 timestamp to BW's display format
|
||||
'22:30:37 May 16, 2026'. UTC inputs (with Z suffix) are
|
||||
converted to the system's local timezone first; naïve inputs
|
||||
are formatted as-is. Returns input unchanged on parse failure."""
|
||||
if not iso or "T" not in iso:
|
||||
return iso
|
||||
try:
|
||||
import datetime as _dt
|
||||
dt = _dt.datetime.fromisoformat(iso.replace("Z", "+00:00"))
|
||||
return dt.strftime("%H:%M:%S %B %d, %Y").replace(" 0", " ")
|
||||
return _to_display_local(iso).strftime("%H:%M:%S %B %d, %Y").replace(" 0", " ")
|
||||
except Exception:
|
||||
return iso
|
||||
|
||||
|
||||
def _split_iso_to_date_time(iso: Optional[str]) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Split an ISO timestamp into BW-formatted ("May 27 /26", "06:06:14")
|
||||
"""Split an ISO timestamp into BW-formatted ('May 27 /26', '06:06:14')
|
||||
date+time strings. Used for the histogram stats table where the
|
||||
Date and Time rows are presented separately. Returns (None, None)
|
||||
if the input isn't a valid ISO datetime."""
|
||||
Date and Time rows are presented separately. UTC inputs are
|
||||
converted to local time first. Returns (None, None) on parse failure."""
|
||||
if not iso:
|
||||
return (None, None)
|
||||
try:
|
||||
import datetime as _dt
|
||||
dt = _dt.datetime.fromisoformat(iso.replace("Z", "+00:00"))
|
||||
# BW format: "May 27 /26" (3-letter month + 2-digit year)
|
||||
dt = _to_display_local(iso)
|
||||
# BW format: 'May 27 /26' (3-letter month + 2-digit year)
|
||||
date_str = dt.strftime("%b %d /%y").replace(" 0", " ")
|
||||
time_str = dt.strftime("%H:%M:%S")
|
||||
return (date_str, time_str)
|
||||
|
||||
Reference in New Issue
Block a user