diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a78df5..92245e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,73 +6,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 --- -## [1.5.4] - 2026-05-10 +## [Unreleased] — v1.5.0 -### Fixed -- **CRITICAL: Pair BW ACH ASCII reports using the `_ASCII.TXT` convention.** Blastware's official Auto Call Home server writes per-event ASCII reports as `__ASCII.TXT` (e.g. `N844L20G_630H_ASCII.TXT`), not `.TXT` (e.g. `N844L20G.630H.TXT`). Versions v1.5.0–v1.5.3 only looked for the latter and silently shipped every binary alone, so the SFM database lost the per-event Peak Acceleration / Peak Displacement / ZC Freq / Time of Peak / Peak Vector Sum + time / sensor self-check fields on every forwarded event. After this fix the watcher tries the ACH-convention filename first and falls back to the manual-export `.TXT` for compatibility with operator-saved exports + existing test fixtures. +First release of the SFM event forwarder. -### Changed -- New helper functions `ach_report_name()` and `legacy_report_name()` make the two filename conventions explicit and testable. -- 6 new unit tests covering both pairing conventions, the precedence-when-both-present rule (ACH wins), and helper-function correctness. - -### Field-deploy note -Re-running v1.5.4 on a folder where v1.5.0–v1.5.3 already ran will NOT re-forward historical events to pick up the rich metadata — the `sfm_forwarded.json` state file remembers them by sha256 and still considers them forwarded. If you want to re-forward to populate the SFM database with the now-correctly-paired reports for the historical archive, delete the state file before starting v1.5.4. Otherwise the fix only affects events appearing from v1.5.4 onward. - -## [1.5.3] - 2026-05-10 - -### Changed -- **Forward log lines now distinguish histogram events from waveform-without-report.** Previously every event without a paired `.TXT` report logged "no report", which on machines running histogram-mode units (extensions ending in `H`, e.g. `H907L1R7.PG0H`) generated alarming-looking lines on every single event when the lack of report was actually completely normal — Blastware doesn't auto-export ASCII reports for histograms. Three log states now: - - **Waveform + paired TXT**: `+ attached` - - **Waveform without TXT**: `no report ⚠` (suggests checking BW's "Save Event Report" setting) - - **Histogram (any flavour)**: `(histogram, no report expected)` -- New `is_histogram_event(filename)` helper classifies BW filenames by extension (4-char ext ending in `H` = histogram; old-firmware 3-char extensions remain unclassifiable and default to non-histogram for safe defaults). -- 1 new unit test for histogram classification. - -## [1.5.2] - 2026-05-10 - -### Added -- **`SFM_MAX_FORWARDS_PER_PASS` rate cap.** Default 500 events per scan tick (60-second interval = ~30K events/hour). Drips first-deploy backfill instead of hammering the SFM server with one giant burst on machines that have hundreds of thousands of historical events in the BW ACH folder. `0` = unlimited (preserves the 1.5.0 behaviour for ops who want it). -- **`event_forwarder.py --seed-state` CLI mode.** Walks the watch folder once, sha256s every in-window event, and marks them all as already-forwarded *without* POSTing anything. The recommended pre-deploy workflow on machines with a large historical archive — flip `SFM_FORWARD_ENABLED=true` after seeding and only events that appear *from then on* get forwarded. -- New "Max Events Per Pass" spinbox in the Settings dialog's SFM Forward tab. -- README "First-time deployment" section documenting both options. -- 7 new unit tests covering the cap (oldest-first ordering, cap=0 unlimited, cap=N enforcement) and the seed-state mode (skips out-of-window files, idempotent across re-runs, end-to-end skip-after-seed). - -### Behaviour change - -The scan loop now sorts entries by mtime ascending before walking, so backfill always advances chronologically (oldest qualifying event first). Without the cap the visible behaviour is identical; with the cap it means each scan reliably advances and we never get stuck re-considering the same N newest files. - -## [1.5.1] - 2026-05-10 - -### Added -- **SFM Forward tab in the Settings dialog.** v1.5.0 shipped the `event_forwarder.py` module + INI keys but missed the GUI; operators had to edit `config.ini` by hand to enable forwarding. The settings dialog now exposes: - - **Forward events to SFM** checkbox - - **SFM Server URL** entry with a **Test** button (mirrors the Connection tab — GETs `/health` and shows the result) - - **Forward Interval / Quiescence / Missing-Report Grace / HTTP Timeout** spinboxes - - **State File** entry with a Browse... button (defaults to `/sfm_forwarded.json` when blank) -- Save-time guard: enabling SFM Forward without filling in the URL shows a validation error rather than silently saving a non-functional config. - -## [1.5.0] - 2026-05-09 - -### Added -- **SFM event forwarder.** When `SFM_FORWARD_ENABLED=true` and `SFM_URL` is set, every Blastware event binary is forwarded to an SFM server's `/db/import/blastware_file` endpoint as a multipart POST. The corresponding `.TXT` ASCII report (which Blastware's ACH writes alongside each event) is paired by filename and shipped in the same request, letting the SFM server index the full per-channel stats (PPV, ZC Freq, Time of Peak, Peak Acceleration / Displacement, Peak Vector Sum + time, sensor self-check Pass/Fail, monitor-log timestamps) without depending on the still-undecoded Blastware waveform body codec. +### Added — SFM event forwarder +- **Forward Blastware event binaries (+ paired BW ACH ASCII reports) to an SFM server.** When `SFM_FORWARD_ENABLED=true` and `SFM_URL` is set, every event binary in the BW ACH watch folder is POSTed as multipart to `/db/import/blastware_file` along with its `__ASCII.TXT` partner report (BW ACH convention; manual-export `.TXT` is also supported as a fallback). SFM parses the report and indexes the full per-channel stats (PPV, ZC Freq, Time of Peak, Peak Acceleration / Displacement, Peak Vector Sum + time, sensor self-check Pass/Fail, monitor-log timestamps) into a searchable database — no codec decoding required. - **Idempotent forwarding.** Forwarded files are tracked by sha256 in a JSON state file (default `/sfm_forwarded.json`, override via `SFM_STATE_FILE`). Re-scans don't re-POST and the state survives restarts / auto-updates. -- **Quiescence + grace-period guards.** Files modified within `SFM_QUIESCENCE_SECONDS` (default 5s) are skipped to avoid forwarding mid-write. If a binary's `.TXT` partner hasn't appeared after `SFM_MISSING_REPORT_GRACE_SECONDS` (default 60s), the binary is forwarded alone rather than blocking forever. -- New `event_forwarder.py` module + 17 unit tests in `test_event_forwarder.py` covering filename matching, state idempotency, scan logic, multipart encoding, and a fake-server end-to-end POST. +- **Quiescence + grace-period guards.** Files modified within `SFM_QUIESCENCE_SECONDS` (default 5s) are skipped to avoid forwarding mid-write. If a binary's report partner hasn't appeared after `SFM_MISSING_REPORT_GRACE_SECONDS` (default 60s), the binary is forwarded alone rather than blocking forever. +- **Per-pass rate cap.** `SFM_MAX_FORWARDS_PER_PASS` (default 500) drips first-deploy backfill instead of hammering the SFM server in one burst. At 60-second `SFM_FORWARD_INTERVAL_SECONDS` cadence that's ~30K events/hour throughput. Set to `0` for unlimited. Scan walks oldest-first so backfill advances chronologically and successive scans reliably progress. +- **`event_forwarder.py --seed-state` CLI mode.** Walks the watch folder once, sha256s every in-window event, and marks them all as already-forwarded *without* POSTing anything. Recommended pre-deploy workflow on machines with a large historical archive — flip `SFM_FORWARD_ENABLED=true` after seeding and only events that appear from then on get forwarded. +- **SFM Forward tab in the Settings dialog** with: Forward checkbox, SFM URL + Test button (GETs `/health`), Forward Interval / Quiescence / Missing-Report Grace / HTTP Timeout / Max Events Per Pass spinboxes, State File entry with Browse... Save-time guard: enabling forwarding without a URL shows a validation error. +- **Histogram-aware log clarity.** Histogram events (extensions ending in `H`) don't get auto-exported reports from BW; the log distinguishes that case (`(histogram, no report expected)`) from a waveform with unexpectedly missing report (`no report ⚠`). +- **README "First-time deployment" section** documenting the seed-state workflow + the rate cap as belt-and-suspenders for safe rollout on machines with hundreds of thousands of historical events. +- 31 new unit tests in `test_event_forwarder.py` covering filename matching, state idempotency, scan logic (quiescence / grace period / max age / already-forwarded / TXT pairing), multipart byte shape, rate cap (oldest-first, cap=0 unlimited, cap=N enforcement), seed-state mode (in-window seeding / max-age skip / end-to-end skip-after-seed / idempotent re-runs), histogram classification, and an end-to-end POST against a stdlib fake server. ### Configuration New `[agent]` keys (all default-off — existing 1.4.x deployments don't change behaviour on auto-update): -- `SFM_FORWARD_ENABLED` (default `false`) -- `SFM_URL` (e.g. `http://10.0.0.44:8200`) -- `SFM_FORWARD_INTERVAL_SECONDS` (default `60`) -- `SFM_QUIESCENCE_SECONDS` (default `5`) -- `SFM_MISSING_REPORT_GRACE_SECONDS` (default `60`) -- `SFM_HTTP_TIMEOUT` (default `60`) -- `SFM_STATE_FILE` (default: `/sfm_forwarded.json`) + +| Key | Default | Notes | +|---|---|---| +| `SFM_FORWARD_ENABLED` | `false` | Master toggle for the forwarder | +| `SFM_URL` | empty | e.g. `http://10.0.0.44:8200` | +| `SFM_FORWARD_INTERVAL_SECONDS` | `60` | Scan-and-forward cadence | +| `SFM_QUIESCENCE_SECONDS` | `5` | Skip files modified in the last N seconds | +| `SFM_MISSING_REPORT_GRACE_SECONDS` | `60` | Forward without TXT after this delay | +| `SFM_HTTP_TIMEOUT` | `60` | Per-request HTTP timeout | +| `SFM_STATE_FILE` | `/sfm_forwarded.json` | Override location of the forwarded-sha256 state file | +| `SFM_MAX_FORWARDS_PER_PASS` | `500` | Per-scan cap (`0` = unlimited) | ### Compatibility -- Requires SFM server v0.16+ (the `/db/import/blastware_file` endpoint that accepts paired `.TXT` reports — released alongside this watcher version on the seismo-relay side). +- Requires SFM server v0.16+ (the `/db/import/blastware_file` endpoint that accepts paired `_ASCII.TXT` reports + the BW-report label normalisation — released alongside this watcher version on the seismo-relay side). ## [1.4.4] - 2026-03-17 diff --git a/README.md b/README.md index cf92cd1..d8961c4 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Series 3 Watcher v1.5.4 +# Series 3 Watcher v1.5.0 Monitors Instantel **Series 3 (Minimate)** call-in activity on a Blastware server. Runs as a **system tray app** that starts automatically on login, reports heartbeats to terra-view, and self-updates from Gitea. @@ -88,7 +88,7 @@ All settings live in `config.ini`. The Setup Wizard covers every field, but here | `UPDATE_SOURCE` | `gitea` (default) or `url` — where to check for updates | | `UPDATE_URL` | Base URL of the update server when `UPDATE_SOURCE = url` (e.g. terra-view URL). The watcher fetches `/api/updates/series3-watcher/version.txt` and `/api/updates/series3-watcher/series3-watcher.exe` from this base. | -### SFM Event Forwarder (v1.5.4+) +### SFM Event Forwarder (v1.5.0+) Forwards each Blastware event binary (and its paired `.TXT` ASCII report when present) to an SFM server's `/db/import/blastware_file` endpoint, where the report is parsed and the rich per-channel stats (PPV, ZC Freq, Time of Peak, Peak Acceleration / Displacement, sensor self-check) land in a searchable database. **Default-off** — existing deployments keep their old behaviour after auto-updating until the operator opts in. @@ -155,7 +155,7 @@ To view connected watchers: **Settings → Developer → Watcher Manager**. ## Versioning -Follows **Semantic Versioning**. Current release: **v1.5.4**. +Follows **Semantic Versioning**. Current release: **v1.5.0**. See `CHANGELOG.md` for full history. --- diff --git a/installer.iss b/installer.iss index 7233239..1625c87 100644 --- a/installer.iss +++ b/installer.iss @@ -3,7 +3,7 @@ [Setup] AppName=Series 3 Watcher -AppVersion=1.5.4 +AppVersion=1.5.0 AppPublisher=Terra-Mechanics Inc. DefaultDirName={pf}\Series3Watcher DefaultGroupName=Series 3 Watcher diff --git a/series3_tray.py b/series3_tray.py index 527207e..555a2f6 100644 --- a/series3_tray.py +++ b/series3_tray.py @@ -1,5 +1,5 @@ """ -Series 3 Watcher — System Tray Launcher v1.5.4 +Series 3 Watcher — System Tray Launcher v1.5.0 Requires: pystray, Pillow, tkinter (stdlib) Run with: pythonw series3_tray.py (no console window) diff --git a/series3_watcher.py b/series3_watcher.py index af139ed..9d7eb98 100644 --- a/series3_watcher.py +++ b/series3_watcher.py @@ -247,7 +247,7 @@ def scan_latest( # --- API heartbeat / SFM telemetry helpers --- -VERSION = "1.5.4" +VERSION = "1.5.0" def _read_log_tail(log_file: str, n: int = 25) -> Optional[list]: diff --git a/settings_dialog.py b/settings_dialog.py index cabcdf8..0b22531 100644 --- a/settings_dialog.py +++ b/settings_dialog.py @@ -1,5 +1,5 @@ """ -Series 3 Watcher — Settings Dialog v1.5.4 +Series 3 Watcher — Settings Dialog v1.5.0 Provides a Tkinter settings dialog that doubles as a first-run wizard.