serversdown
815c643fb2
Two safety nets for first-deploy on Blastware ACH machines that
have accumulated tens or hundreds of thousands of historical events
in the watch folder.
1. SFM_MAX_FORWARDS_PER_PASS (default 500, 0=unlimited)
---------------------------------------------------
Cap on the number of events forwarded per scan tick. At the
60-second default interval that's ~30K events/hour throughput —
the SFM server gets a steady drip instead of one giant burst.
Scan now sorts by mtime ascending so backfill advances
chronologically (oldest first) and successive scans always
make progress instead of re-considering the same N newest files.
Wired into:
- event_forwarder.find_pending_events / forward_pending
- series3_watcher.run_watcher loop
- config-template.ini
- settings_dialog SFM Forward tab (new "Max Events Per Pass"
spinbox, validated in _on_save)
2. event_forwarder.py --seed-state CLI
-----------------------------------
One-shot mode that walks the watch folder, sha256s every in-window
event binary, and marks them all as already-forwarded WITHOUT
POSTing anything. Run before flipping SFM_FORWARD_ENABLED=true
to skip the historical backfill entirely — the watcher then only
forwards events that appear AFTER the seed.
Usage:
python event_forwarder.py --seed-state \
--watch "C:\Blastware 10\Event\autocall home" \
--state "C:\...\sfm_forwarded.json" \
[--max-age-days 365]
7 new unit tests:
- max_per_pass cap enforcement (=N, =0 unlimited, oldest-first
ordering)
- seed-state mode (in-window seeding, max-age skip,
end-to-end skip-after-seed, idempotent re-runs)
README adds a "First-time deployment" section walking through both
options. Bumps to v1.5.2.
75 lines
2.7 KiB
INI
75 lines
2.7 KiB
INI
[agent]
|
||
|
||
# --- API Heartbeat Settings ---
|
||
API_ENABLED = true
|
||
API_URL =
|
||
API_INTERVAL_SECONDS = 300
|
||
SOURCE_ID = #computer that is running agent.
|
||
SOURCE_TYPE = series3_watcher
|
||
|
||
# Paths
|
||
SERIES3_PATH = C:\Blastware 10\Event\autocall home
|
||
MAX_EVENT_AGE_DAYS = 365
|
||
|
||
|
||
# Scanning
|
||
SCAN_INTERVAL_SECONDS = 30
|
||
|
||
# Logging
|
||
ENABLE_LOGGING = True
|
||
LOG_FILE = C:\Users\%USERNAME%\AppData\Local\Series3Watcher\agent_logs\series3_watcher.log
|
||
LOG_RETENTION_DAYS = 30
|
||
|
||
# .MLG parsing
|
||
MLG_HEADER_BYTES = 2048 ; used for unit-id extraction
|
||
|
||
# Deep sniff
|
||
DEEP_SNIFF = True ; toggle deep sniff on/off
|
||
SNIFF_BYTES = 65536 ; max bytes to scan for Notes/Cal
|
||
|
||
# Auto-updater source: gitea (default) or url
|
||
UPDATE_SOURCE = gitea
|
||
# If UPDATE_SOURCE = url, set UPDATE_URL to the base URL of the update server (e.g. terra-view)
|
||
UPDATE_URL =
|
||
|
||
# --- SFM Event Forwarder ---
|
||
# When enabled, every Blastware event binary (and its paired .TXT
|
||
# report when present) is forwarded to an SFM server's
|
||
# /db/import/blastware_file endpoint as a multipart POST. The SFM
|
||
# server parses the .TXT and indexes the event's full per-channel
|
||
# stats (PPV, ZC Freq, Time of Peak, Peak Acceleration, Peak
|
||
# Displacement, sensor self-check) for sortable / filterable review.
|
||
#
|
||
# Default-off so existing deployments don't change behaviour after an
|
||
# auto-update. To enable on a field machine: set SFM_URL, then flip
|
||
# SFM_FORWARD_ENABLED to true and restart the watcher.
|
||
SFM_FORWARD_ENABLED = false
|
||
SFM_URL = ; e.g. http://10.0.0.44:8200
|
||
SFM_FORWARD_INTERVAL_SECONDS = 60 ; scan-and-forward cadence (independent of heartbeat)
|
||
|
||
# Files modified within the last N seconds are skipped (BW may still
|
||
# be writing them). Defence against truncated uploads.
|
||
SFM_QUIESCENCE_SECONDS = 5
|
||
|
||
# If a binary's .TXT report hasn't appeared after this many seconds,
|
||
# forward the binary alone rather than blocking forever waiting.
|
||
SFM_MISSING_REPORT_GRACE_SECONDS = 60
|
||
|
||
# Per-request HTTP timeout (seconds).
|
||
SFM_HTTP_TIMEOUT = 60
|
||
|
||
# Path to the JSON state file tracking which events have been
|
||
# forwarded (sha256-keyed, idempotent across restarts). Leave blank
|
||
# to default to <log dir>/sfm_forwarded.json.
|
||
SFM_STATE_FILE =
|
||
|
||
# Per-pass cap — forward at most N events per scan tick. 0 = unlimited.
|
||
# Default 500 throttles first-deploy backfill on machines with large
|
||
# historical archives (tens or hundreds of thousands of events) so
|
||
# the SFM server isn't hammered with one giant burst. At 60s scan
|
||
# interval × 500 events/pass that's 30K events/hour throughput.
|
||
# See README "First-time deployment" for the recommended
|
||
# `--seed-state` workflow that skips the historical backfill entirely.
|
||
SFM_MAX_FORWARDS_PER_PASS = 500
|
||
|