When a binary is forwarded WITHOUT its paired _ASCII.TXT (because
the TXT wasn't quiescent within the grace period — BW slow to
write, AV scanning, etc.), the old behaviour was to permanently
mark the binary as "done" in the state file, even though the TXT
might land seconds later. Result: that event lived in SFM forever
with broken-codec peak values and no project info.
Fix: state entries now carry a had_report flag. Forwards without
a TXT set had_report=False. On subsequent scans, the watcher
treats had_report=False entries as re-pair candidates — they get
re-forwarded once the TXT appears, and the SFM server's upsert
path (in seismo-relay's insert_events IntegrityError handler)
refreshes the DB row with the report's authoritative values.
Three status states in ForwardState.status(sha256):
None — never forwarded. First-forward path.
True — forwarded successfully WITH report (or legacy entry
without the had_report field). Permanently done.
False — forwarded WITHOUT report. Re-pair if TXT now exists.
Backward compat: legacy state-file entries (no had_report key)
default to True so existing deployments don't unexpectedly
re-forward every entry on upgrade.
Tests cover:
- re-pair when TXT appears after a had_report=False forward
- had_report=True entries stay skipped permanently
- legacy entries (missing field) treated as fully forwarded
- state.status() returns None for unknown sha
- re-marking had_report=False then True promotes to fully-done
36 watcher tests pass (was 31, +5 new).
Blastware's official Auto Call Home server writes per-event ASCII
reports as <stem>_<ext>_ASCII.TXT (e.g. N844L20G_630H_ASCII.TXT),
not <binary>.TXT (e.g. N844L20G.630H.TXT). Versions v1.5.0–v1.5.3
only looked for the latter and silently shipped every binary alone,
so the SFM database lost the per-event Peak Acceleration / Peak
Displacement / ZC Freq / Time of Peak / Peak Vector Sum + time /
sensor self-check fields on every forwarded event.
Fix: pair-finding logic now tries the ACH-convention filename first
and falls back to <binary>.TXT for compatibility with operator-saved
manual exports and existing test fixtures.
ach_report_name("M529LK44.AB0") → "M529LK44_AB0_ASCII.TXT"
legacy_report_name("M529LK44.AB0") → "M529LK44.AB0.TXT"
When both files exist (operator manually saved + ACH auto-exported),
ACH wins because that's the canonical name on modern BW deployments.
Both candidates checked case-insensitively against the cached
directory listing — no extra stat() calls.
6 new unit tests cover the new pairing logic, helper-function
correctness, and the precedence rule. Total now 31 tests, all green.
Field-deploy note: re-running v1.5.4 on a folder where v1.5.0–v1.5.3
already ran will NOT re-forward historical events — the
sfm_forwarded.json state file remembers them by sha256. To re-forward
historical events to populate SFM with the now-correctly-paired
reports, delete the state file before starting v1.5.4.
On machines running histogram-mode units (extensions ending in H,
e.g. H907L1R7.PG0H), every forwarded event was logging "no report"
even though histograms never get auto-exported ASCII reports from
Blastware — making the log look like every forward was misconfigured
when in fact things were working correctly.
Three log states now:
- Waveform + paired TXT
→ "+ <txt> attached"
- Waveform without TXT (likely BW config issue)
→ "no report ⚠"
- Histogram (any flavour)
→ "(histogram, no report expected)"
New is_histogram_event() helper classifies by BW filename extension:
4-char ext ending in H = histogram; old-firmware 3-char extensions
default to non-histogram (safe default — we'd rather flag a missing
report than silently suppress the warning on a real waveform event).
Forwarding logic itself is unchanged — this is purely log clarity.
Two safety nets for first-deploy on Blastware ACH machines that
have accumulated tens or hundreds of thousands of historical events
in the watch folder.
1. SFM_MAX_FORWARDS_PER_PASS (default 500, 0=unlimited)
---------------------------------------------------
Cap on the number of events forwarded per scan tick. At the
60-second default interval that's ~30K events/hour throughput —
the SFM server gets a steady drip instead of one giant burst.
Scan now sorts by mtime ascending so backfill advances
chronologically (oldest first) and successive scans always
make progress instead of re-considering the same N newest files.
Wired into:
- event_forwarder.find_pending_events / forward_pending
- series3_watcher.run_watcher loop
- config-template.ini
- settings_dialog SFM Forward tab (new "Max Events Per Pass"
spinbox, validated in _on_save)
2. event_forwarder.py --seed-state CLI
-----------------------------------
One-shot mode that walks the watch folder, sha256s every in-window
event binary, and marks them all as already-forwarded WITHOUT
POSTing anything. Run before flipping SFM_FORWARD_ENABLED=true
to skip the historical backfill entirely — the watcher then only
forwards events that appear AFTER the seed.
Usage:
python event_forwarder.py --seed-state \
--watch "C:\Blastware 10\Event\autocall home" \
--state "C:\...\sfm_forwarded.json" \
[--max-age-days 365]
7 new unit tests:
- max_per_pass cap enforcement (=N, =0 unlimited, oldest-first
ordering)
- seed-state mode (in-window seeding, max-age skip,
end-to-end skip-after-seed, idempotent re-runs)
README adds a "First-time deployment" section walking through both
options. Bumps to v1.5.2.
When SFM_FORWARD_ENABLED=true and SFM_URL is set, every Blastware
event binary in the ACH watch folder is forwarded to an SFM server's
/db/import/blastware_file endpoint as a multipart POST. The paired
<binary>.TXT ASCII report (which Blastware's ACH writes alongside
each event) is shipped in the same request, letting the SFM server
index the full per-channel stats — PPV, ZC Freq, Time of Peak, Peak
Acceleration / Displacement, Peak Vector Sum + time, sensor
self-check Pass/Fail per channel, and monitor-log timestamps —
without depending on the still-undecoded BW waveform body codec.
New module event_forwarder.py:
- is_event_binary() filename matcher (BW's <P><serial3><stem>.<ext>
scheme; rejects .MLG, .TXT, .log, .ini, .h5, etc.)
- ForwardState (.json file keyed by sha256 — idempotent across
restarts and auto-updates)
- find_pending_events() with quiescence + grace-period guards
- Hand-rolled multipart encoder (stdlib-only)
- forward_event_pair() / forward_pending() — POST loop with
structured per-event outcomes
Wired into series3_watcher.run_watcher() on its own cadence
(SFM_FORWARD_INTERVAL_SECONDS, default 60s) so it doesn't slow the
existing 5-min heartbeat scan.
Default-off: existing 1.4.x deployments keep their old behaviour
after auto-updating until an operator sets SFM_URL +
SFM_FORWARD_ENABLED=true and restarts.
17 unit tests in test_event_forwarder.py cover filename matching,
state idempotency, scan logic (quiescence, grace, max age,
already-forwarded, .TXT pairing), multipart byte shape, and an
end-to-end POST against a tiny stdlib http.server fake.
Bumps version 1.4.4 → 1.5.0 (minor — additive feature, no API break).
Requires SFM server v0.16+ for the paired-.TXT import endpoint.
serversdown