# Series 3 Watcher v1.5.0

Monitors Instantel **Series 3 (Minimate)** call-in activity on a Blastware server. Runs as a **system tray app** that starts automatically on login, reports heartbeats to terra-view, and self-updates from Gitea.

---

## Deployment (Recommended — Installer)

The easiest way to deploy to a field machine is the pre-built Windows installer.

1. Download `series3-watcher-setup.exe` from the [latest release](https://gitea.serversdown.net/serversdown/series3-watcher/releases) on Gitea.
2. Run the installer on the target machine. It installs to `C:\Program Files\Series3Watcher\` and adds a shortcut to the user's Startup folder.
3. On first launch the **Setup Wizard** opens automatically — fill in the terra-view URL and Blastware path, then click **Save & Start**.
4. A coloured dot appears in the system tray. Done.

The watcher will auto-start on every login from that point on.

### Auto-Updates

The watcher checks [Gitea](https://gitea.serversdown.net/serversdown/series3-watcher) for a newer release approximately every 5 minutes. When a newer `.exe` is found it downloads it silently, swaps the file, and relaunches — no user action required.

Updates can also be pushed remotely from terra-view → **Settings → Developer → Watcher Manager**.

---

## Building & Releasing

See [BUILDING.md](BUILDING.md) for the full step-by-step process covering:
- First-time build and installer creation
- Publishing a release to Gitea
- Releasing hotfix updates (auto-updater picks them up automatically)

---

## Running Without the Installer (Dev / Debug)

```
pip install -r requirements.txt
python series3_tray.py        # tray app (recommended)
python series3_watcher.py     # console-only, no tray
```

`config.ini` must exist in the same directory. Copy `config-template.ini` to `config.ini` and edit it, or just run `series3_tray.py` — the wizard will create it on first run.

---

## Configuration

All settings live in `config.ini`. The Setup Wizard covers every field, but here's the reference:

### API / terra-view

| Key | Description |
|-----|-------------|
| `API_ENABLED` | `true` to send heartbeats to terra-view |
| `API_URL` | Terra-View base URL, e.g. `http://192.168.1.10:8000` — the `/api/series3/heartbeat` endpoint is appended automatically |
| `API_INTERVAL_SECONDS` | How often to POST (default `300`) |
| `SOURCE_ID` | Identifier for this machine (defaults to hostname) |
| `SOURCE_TYPE` | Always `series3_watcher` |

### Paths

| Key | Description |
|-----|-------------|
| `SERIES3_PATH` | Blastware autocall folder, e.g. `C:\Blastware 10\Event\autocall home` |
| `MAX_EVENT_AGE_DAYS` | Ignore `.MLG` files older than this (default `365`) |
| `LOG_FILE` | Path to the log file |

### Scanning

| Key | Description |
|-----|-------------|
| `SCAN_INTERVAL_SECONDS` | How often to scan the folder (default `300`) |
| `MLG_HEADER_BYTES` | Bytes to read from each `.MLG` header for unit ID (default `2048`) |
| `RECENT_WARN_DAYS` | Log unsniffable files newer than this window |

### Logging

| Key | Description |
|-----|-------------|
| `ENABLE_LOGGING` | `true` / `false` |
| `LOG_RETENTION_DAYS` | Auto-clear log after this many days (default `30`) |

### Auto-Updater

| Key | Description |
|-----|-------------|
| `UPDATE_SOURCE` | `gitea` (default) or `url` — where to check for updates |
| `UPDATE_URL` | Base URL of the update server when `UPDATE_SOURCE = url` (e.g. terra-view URL). The watcher fetches `/api/updates/series3-watcher/version.txt` and `/api/updates/series3-watcher/series3-watcher.exe` from this base. |

### SFM Event Forwarder (v1.5.0+)

Forwards each Blastware event binary (and its paired `<binary>.TXT` ASCII report when present) to an SFM server's `/db/import/blastware_file` endpoint, where the report is parsed and the rich per-channel stats (PPV, ZC Freq, Time of Peak, Peak Acceleration / Displacement, sensor self-check) land in a searchable database. **Default-off** — existing deployments keep their old behaviour after auto-updating until the operator opts in.

| Key | Description |
|-----|-------------|
| `SFM_FORWARD_ENABLED` | `true` to enable forwarding (default `false`) |
| `SFM_URL` | Base URL of the SFM server, e.g. `http://10.0.0.44:8200` |
| `SFM_FORWARD_INTERVAL_SECONDS` | Scan-and-forward cadence (default `60`); independent of the heartbeat interval |
| `SFM_QUIESCENCE_SECONDS` | Skip files modified within this many seconds (default `5`) — avoids forwarding mid-write |
| `SFM_MISSING_REPORT_GRACE_SECONDS` | If a `.TXT` partner hasn't appeared after this many seconds, forward the binary alone (default `60`) |
| `SFM_HTTP_TIMEOUT` | Per-request HTTP timeout in seconds (default `60`) |
| `SFM_STATE_FILE` | Path to the JSON state file tracking sha256 of forwarded events. Leave blank to default to `<log dir>/sfm_forwarded.json` |
| `SFM_MAX_FORWARDS_PER_PASS` | Max events forwarded per scan tick (default `500`, `0` = unlimited). Drip-feeds backfill so a folder with thousands of qualifying events doesn't hammer the SFM server in one giant burst. |

Forwarded files are tracked by sha256 in the state file, so re-scans / restarts / auto-updates never re-POST the same content. A failed POST stays in the pending pool and is retried on the next interval.

#### First-time deployment on a folder with a large historical archive

If you're enabling SFM forwarding on a Blastware ACH machine that's been accumulating events for years (tens or hundreds of thousands of files in the watch folder), you almost certainly **don't** want the watcher to forward all of them on first run. Two options:

1. **Skip the historical backfill (recommended).** Run the seed-state CLI once before flipping `SFM_FORWARD_ENABLED=true`. It walks the folder, sha256s every existing in-window event, and marks them all as already-forwarded — without POSTing anything. The watcher then only forwards events that appear *after* the seed run.

   ```
   python event_forwarder.py --seed-state ^
       --watch "C:\Blastware 10\Event\autocall home" ^
       --state "C:\Users\<you>\AppData\Local\Series3Watcher\agent_logs\sfm_forwarded.json" ^
       --max-age-days 365
   ```

2. **Throttle the backfill.** Leave `SFM_MAX_FORWARDS_PER_PASS` at its 500 default and let the watcher drip-feed. With a 60-second `SFM_FORWARD_INTERVAL_SECONDS` that's ~30K events/hour throughput. Backfill of 30K events takes about an hour, 100K takes ~3.5 hours. The cap fires per scan, so heartbeat and forwarding share the watcher's main loop without saturating it.

Combine both for a fully controlled rollout: seed-state to skip the deep archive, then leave the cap on as a steady-state safety net.

---

## Tray Icon

| Colour | Meaning |
|--------|---------|
| Grey | Starting / no scan yet |
| Green | All detected units OK |
| Yellow | At least one unit Pending |
| Red | At least one unit Missing, or error |

Right-click the icon for: status, per-unit list, Settings, Open Log Folder, Exit.

---

## terra-view Integration

When `API_ENABLED = true`, the watcher POSTs a telemetry payload to terra-view on each heartbeat interval. terra-view updates the emitter table and tracks the watcher process itself (version, last seen, log tail) in the Watcher Manager.

To view connected watchers: **Settings → Developer → Watcher Manager**.

---

## Requirements

- Windows 7 or later
- Python 3.8 (only needed if running from source — not needed with the installer)
- Blastware 10 event folder accessible on the local machine

---

## Roadmap (Future)

Deferred work — parked but worth tracking.  Pairs with seismo-relay's
[Roadmap (Future)](https://gitea.serversdown.net/serversdown/seismo-relay#roadmap-future)
where the corresponding server-side work lives.

- [ ] **File archive manager.**  Move BW autocall-home events older than ~90 days into `<watch_folder>_archive/<year>/<month>/` subfolders so the active watch directory doesn't accumulate hundreds of thousands of entries (filesystem dir lookups slow at 100K+, BW UI hangs opening the folder, watcher's own scandir gets expensive).  Plan drafted in the codec-RE branch's plan-mode session, including a critical pre-coding test (does Blastware UI walk subfolders or only see the flat watch folder?) that determines the archive layout (in-place subfolders vs sibling archive).  Default-off, dry-run mode, opt-in per machine.
- [ ] **MLG forwarding.**  Currently the watcher's `is_event_binary()` filter explicitly excludes `.MLG` per-unit monitor log files — only event binaries (`.AB0` / `.PG0H` / etc.) and their paired `_ASCII.TXT` reports get forwarded.  Adding an `POST /db/import/mlg_file` SFM endpoint + a parallel `.MLG` scan path on the watcher would populate `monitor_log` rows for non-ACH-routed units (coverage queries, "was this unit monitoring on date X" lookups).  MLG files are append-only / mutable so the watcher needs a different dedup strategy than the per-event sha256 state file — better to forward whole file every scan and let the server dedup by `(serial, start_time)` on insert.
- [ ] **Pre-deploy seed-state UX in the Settings dialog.**  Currently `event_forwarder.py --seed-state` is a CLI-only operation.  A "Skip backfill" button next to the SFM Forward checkbox would let operators opt-out of re-forwarding the historical archive without dropping to a command line.

---

## Versioning

Follows **Semantic Versioning**. Current release: **v1.5.0**.
See `CHANGELOG.md` for full history.

---

## License

Private / internal — Terra-Mechanics Inc.