Compare commits

..

2 Commits

Author SHA1 Message Date
serversdwn
8c133f2d6f chore: version bump to 0.2.0, gitignore setup properly
doc: readme updates to 0.2.0.
2026-03-20 18:13:24 -04:00
serversdwn
81fca88475 Feat: full installation wizard/execuctable builder batch file created (to match series3-watcher.
fix: timezone issue. now sends utc.
2026-03-20 17:34:51 -04:00
9 changed files with 1531 additions and 486 deletions

BIN
.gitignore vendored

Binary file not shown.

View File

@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.0] - 2026-03-20
### Added
- `thor_tray.py` — system tray launcher with status icon (green/amber/red/grey), Settings and Open Log Folder menu items
- `thor_settings_dialog.py` — Tkinter settings dialog with first-run wizard; tabs for Connection, Paths, Scanning, Logging, Updates
- Hardened auto-updater: three-layer download validation (100 KB floor, 50% relative size floor, MZ magic bytes), safer swap bat with 5-retry cap and `:fail` exit, `.exe.old` backup
- Configurable update source: `update_source` (gitea / url / disabled), `update_url` for custom server
- Remote push support: `update_available` flag in API heartbeat response triggers update regardless of `update_source` setting
- `build.bat` — PyInstaller build script; outputs versioned exe for Gitea and plain copy for Inno Setup
- `installer.iss` — Inno Setup installer script with startup shortcut
- `_update_log()` helper writes timestamped `[updater]` lines to the watcher log
- `log_tail` included in heartbeat payload (last 25 lines) for terra-view display
- `run_watcher(state, stop_event)` pattern in `series4_ingest.py` for background thread use from tray
### Changed
- `series4_ingest.py` refactored into tray-friendly background thread module; `main()` retained for standalone use
- Config key `sfm_endpoint` renamed to `api_url` for consistency with series3-watcher
- Heartbeat payload now uses `source_id`, `source_type`, `version` fields matching terra-view WatcherAgent model
- AppData folder: `ThorWatcher` (was not previously defined)
## [0.1.1] - 2025-12-08
### Changed

262
README.md
View File

@@ -1,205 +1,157 @@
# Series 4 Ingest Agent
# Thor Watcher
**Version:** 0.1.2
**Version:** 0.2.0
Micromate (Series 4) ingest agent for Seismo Fleet Manager (SFM).
Micromate (Series 4) watcher agent for Terra-View fleet management. Runs as a Windows system tray application, scans THORDATA for Micromate unit activity, and sends heartbeat data to Terra-View.
---
## Overview
Series 4 Ingest Agent is a Python-based monitoring tool that scans for Micromate unit activity logs and reports their status. It monitors MLG (Micromate Log) files in the THORDATA directory structure, determines unit health based on last activity, and optionally transmits heartbeat data to a Seismo Fleet Manager backend.
Thor Watcher monitors `C:\THORDATA\<Project>\<UM####>\*.MLG` files, determines each unit's last activity from the MLG filename timestamp, and periodically posts a heartbeat payload to the Terra-View backend. It runs silently in the system tray and auto-starts on login.
## Features
- **Automatic MLG File Discovery**: Scans `C:\THORDATA\<Project>\<UM####>\*.MLG` for Micromate units
- **Intelligent Unit Tracking**: Automatically identifies the newest MLG file per unit based on timestamp
- **Status Classification**: Categorizes units as OK, LATE, or STALE based on configurable age thresholds
- **Console Heartbeat Display**: Real-time status dashboard with formatted output
- **SFM Backend Integration**: Optional HTTP POST of JSON telemetry to Fleet Manager
- **External Configuration**: JSON-based configuration file for easy customization
- **Robust Error Handling**: Graceful degradation with informative warnings
- **No External Dependencies**: Uses Python standard library only (urllib, json, datetime, etc.)
## Requirements
- Python 3.6 or higher
- Windows OS (designed for THOR PC/VM environments)
- Access to THORDATA directory structure
---
## Installation
1. Clone or copy this repository to your THOR PC/VM
2. Ensure Python 3.6+ is installed
3. Configure `config.json` (see Configuration section)
4. Run the agent:
1. Run `thor-watcher-setup.exe`
2. On first launch the setup wizard will appear — enter your THORDATA path and Terra-View URL
3. The app starts in the system tray and launches automatically on login
```bash
python series4_ingest.py
---
## Building from Source
Requires Python 3.10+ and pip on PATH.
```bat
build.bat
```
Produces:
- `dist\thor-watcher-0.2.0.exe` — upload to Gitea release
- `dist\thor-watcher.exe` — use with Inno Setup
Then run Inno Setup Compiler on `installer.iss` to produce `thor-watcher-setup.exe`.
---
## Configuration
All configuration is managed through `config.json` in the same directory as the script.
Config is stored at:
```
%LOCALAPPDATA%\ThorWatcher\config.json
```
### config.json Structure
Managed through the Settings dialog (right-click tray icon → Settings). A `config.example.json` is included as reference.
### Config Keys
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `thordata_path` | string | `C:\THORDATA` | Root THORDATA directory |
| `scan_interval` | integer | `60` | Seconds between scans |
| `api_url` | string | `""` | Terra-View heartbeat URL (e.g. `http://10.0.0.40:8000/api/series4/heartbeat`) |
| `api_timeout` | integer | `5` | HTTP request timeout in seconds |
| `api_interval` | integer | `300` | Seconds between API heartbeat POSTs |
| `source_id` | string | hostname | Identifier for this machine in Terra-View |
| `source_type` | string | `series4_watcher` | Agent type (do not change) |
| `local_timezone` | string | `America/New_York` | Timezone of the field machine — used to convert MLG timestamps to UTC |
| `enable_logging` | boolean | `true` | Write log file |
| `log_file` | string | `%LOCALAPPDATA%\ThorWatcher\agent_logs\thor_watcher.log` | Log file path |
| `log_retention_days` | integer | `30` | Days before log is auto-cleared |
| `update_source` | string | `gitea` | Auto-update source: `gitea`, `url`, or `disabled` |
| `update_url` | string | `""` | Base URL for `url` mode (e.g. Terra-View server) |
---
## Tray Icon Colors
| Color | Meaning |
|-------|---------|
| Green | Running, API reporting OK |
| Amber | Running, API disabled or not configured |
| Red | Running, API failing |
| Purple | Error — check logs |
| Grey | Starting up |
---
## Auto-Updater
Thor Watcher checks for updates every ~5 minutes. When a new release is found it downloads and validates the exe, then relaunches via a swap bat — no manual intervention needed.
**Update sources:**
- `gitea` — checks the Gitea release page (default)
- `url` — fetches `version.txt` and `thor-watcher.exe` from a custom server (e.g. Terra-View)
- `disabled` — no automatic checks; remote push from Terra-View still works
**Download validation:** 100 KB minimum size, 50% relative size floor vs current exe, MZ magic bytes check.
Remote update push from Terra-View Watcher Manager works regardless of `update_source` setting.
---
## Heartbeat Payload
Posted to `api_url` on each API interval:
```json
{
"thordata_path": "C:\\THORDATA",
"scan_interval": 60,
"late_days": 2,
"stale_days": 60,
"sfm_endpoint": "http://<sfm backend ip>:8001/api/series4/heartbeat"
"sfm_timeout": 5,
"debug": true
}
```
### Configuration Options
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `thordata_path` | string | `C:\THORDATA` | Root directory containing project/unit MLG files |
| `scan_interval` | integer | `60` | Time in seconds between scans |
| `late_days` | integer | `2` | Days before a unit is marked as LATE |
| `stale_days` | integer | `60` | Days before a unit is marked as STALE |
| `sfm_endpoint` | string | `""` | SFM backend URL (leave empty to disable HTTP) |
| `sfm_timeout` | integer | `5` | HTTP request timeout in seconds |
| `debug` | boolean | `true` | Enable/disable debug logging |
### Fallback Behavior
If `config.json` is missing or malformed, the agent will:
- Display a warning message
- Use built-in default values
- Continue running normally
## Usage
### Start the Agent
```bash
python series4_ingest.py
```
### Console Output Example
```
Series 4 Ingest Agent — Micromate Heartbeat (v0.1.2)
THORDATA root: C:\THORDATA
Now: 2025-12-08 14:30:00
--------------------------------------------------------------------------------
UM11719 OK Age: 1h 12m Last: 2025-12-08 13:18:00 Project: Clearwater - ECMS 57940
UM12345 LATE Age: 3d 5h Last: 2025-12-05 09:15:30 Project: Site Alpha
UM98765 STALE Age: 65d Last: 2025-10-04 08:22:15 Project: Legacy Site
--------------------------------------------------------------------------------
Total units: 3
Next scan in 60 seconds...
```
### Stop the Agent
Press `Ctrl+C` to gracefully stop the agent.
## Status Classification
Units are classified based on the age of their last MLG file:
- **OK**: Last activity within `late_days` threshold (default: < 2 days)
- **LATE**: Last activity between `late_days` and `stale_days` (default: 2-60 days)
- **STALE**: Last activity exceeds `stale_days` threshold (default: ≥ 60 days)
## SFM Backend Integration
When `sfm_endpoint` is configured, the agent sends JSON payloads to the Seismo Fleet Manager:
### Payload Structure
```json
{
"source": "series4_ingest",
"generated_at": "2025-12-08T14:30:00",
"source_id": "THOR-PC",
"source_type": "series4_watcher",
"version": "0.2.0",
"generated_at": "2026-03-20T14:30:00Z",
"log_tail": ["...last 25 log lines..."],
"units": [
{
"unit_id": "UM11719",
"type": "micromate",
"project_hint": "Clearwater - ECMS 57940",
"last_call": "2025-12-08T13:18:00",
"status": "OK",
"age_days": 0.05,
"age_hours": 1.2,
"mlg_path": "C:\\THORDATA\\Clearwater - ECMS 57940\\UM11719\\UM11719_20251208131800.MLG"
"last_call": "2026-03-20T13:18:00Z",
"age_minutes": 72,
"mlg_path": "C:\\THORDATA\\Project A\\UM11719\\UM11719_20260320131800.MLG",
"project_hint": "Project A"
}
]
}
```
### Disabling SFM Integration
---
Set `sfm_endpoint` to an empty string in `config.json`:
```json
{
"sfm_endpoint": ""
}
```
## MLG File Format
The agent expects MLG files to follow this naming convention:
```
UM<unit_number>_YYYYMMDDHHMMSS.MLG
```
Example: `UM11719_20251208131800.MLG`
- `UM11719`: Unit ID
- `20251208131800`: Timestamp (2025-12-08 13:18:00)
- `.MLG`: Micromate Log file extension
## Directory Structure
Expected THORDATA folder structure:
## THORDATA Directory Structure
```
C:\THORDATA\
├── Project A\
│ ├── UM11719\
│ │ ├── UM11719_20251208131800.MLG
│ │ ── UM11719_20251207095430.MLG
│ │ └── ...
│ │ ├── UM11719_20260320131800.MLG
│ │ ── UM11719_20260319095430.MLG
│ └── UM12345\
│ └── UM12345_20251205091530.MLG
│ └── UM12345_20260318091530.MLG
└── Project B\
└── UM98765\
└── UM98765_20251004082215.MLG
└── UM98765_20260301082215.MLG
```
---
## Troubleshooting
### Config file not found
If you see `[WARN] Config file not found`, create `config.json` in the same directory as `series4_ingest.py`.
**Tray icon is amber:** API URL is not configured or disabled — open Settings and enter Terra-View URL.
### THORDATA path doesn't exist
Verify the `thordata_path` in `config.json` points to the correct directory.
**Tray icon is red:** API is failing — check Terra-View is reachable, URL is correct, and the network is up.
### No units found
Ensure MLG files exist in the expected directory structure and follow the naming convention.
**Units showing wrong time in Terra-View:** Check `local_timezone` in Settings matches the field machine's timezone.
### SFM POST failures
Check that:
- `sfm_endpoint` URL is correct and accessible
- Network connectivity is available
- SFM backend is running and accepting requests
**No units found:** Verify `thordata_path` is correct and MLG files exist following the `UM####_YYYYMMDDHHMMSS.MLG` naming convention.
## License
**Auto-updater not working:** Check the log file for `[updater]` lines. On first deploy, verify the Gitea release has a `thor-watcher-X.X.X.exe` asset (not a setup exe).
Proprietary - Internal use only
---
## Version History
See [CHANGELOG.md](CHANGELOG.md) for detailed version history.
## Support
---
For issues or questions, contact the Seismo development team.
*Proprietary — Terra-Mechanics Inc. Internal use only.*

31
build.bat Normal file
View File

@@ -0,0 +1,31 @@
@echo off
echo Building thor-watcher.exe...
pip install pyinstaller pystray Pillow
REM Extract version from series4_ingest.py (looks for: VERSION = "0.2.0")
for /f "tokens=3 delims= " %%V in ('findstr /C:"VERSION = " series4_ingest.py') do set RAW_VER=%%V
set VERSION=%RAW_VER:"=%
set EXE_NAME=thor-watcher-%VERSION%
echo Version: %VERSION%
echo Output: dist\%EXE_NAME%.exe
REM Check whether icon.ico exists alongside this script.
if exist "%~dp0icon.ico" (
pyinstaller --onefile --windowed --name "%EXE_NAME%" ^
--icon="%~dp0icon.ico" ^
--add-data "%~dp0icon.ico;." ^
thor_tray.py
) else (
echo [INFO] icon.ico not found -- building without custom icon.
pyinstaller --onefile --windowed --name "%EXE_NAME%" thor_tray.py
)
REM Copy versioned exe to plain name for Inno Setup
copy /Y "dist\%EXE_NAME%.exe" "dist\thor-watcher.exe"
echo.
echo Done.
echo Gitea upload: dist\%EXE_NAME%.exe
echo Inno Setup: dist\thor-watcher.exe (copy of above)
pause

View File

@@ -1,9 +1,18 @@
{
"thordata_path": "C:\\THORDATA",
"scan_interval": 60,
"late_days": 2,
"stale_days": 60,
"sfm_endpoint": "http://<server_address>:8001/api/series4/heartbeat",
"sfm_timeout": 5,
"debug": true
"api_url": "",
"api_timeout": 5,
"api_interval": 300,
"source_id": "",
"source_type": "series4_watcher",
"local_timezone": "America/New_York",
"enable_logging": true,
"log_file": "C:\\Users\\%USERNAME%\\AppData\\Local\\ThorWatcher\\agent_logs\\thor_watcher.log",
"log_retention_days": 30,
"update_source": "gitea",
"update_url": ""
}

41
installer.iss Normal file
View File

@@ -0,0 +1,41 @@
; Inno Setup script for Thor Watcher
; Run through Inno Setup Compiler after building dist\thor-watcher.exe
[Setup]
AppName=Thor Watcher
AppVersion=0.2.0
AppPublisher=Terra-Mechanics Inc.
DefaultDirName={pf}\ThorWatcher
DefaultGroupName=Thor Watcher
OutputBaseFilename=thor-watcher-setup
Compression=lzma
SolidCompression=yes
; Require admin rights so we can write to Program Files
PrivilegesRequired=admin
[Tasks]
Name: "desktopicon"; Description: "Create a &desktop icon"; GroupDescription: "Additional icons:"; Flags: unchecked
[Dirs]
; Create the agent_logs folder so the watcher can write logs on first run
Name: "{app}\agent_logs"
[Files]
; Main executable — built by build.bat / PyInstaller
Source: "dist\thor-watcher.exe"; DestDir: "{app}"; Flags: ignoreversion
[Icons]
; Start Menu shortcut
Name: "{group}\Thor Watcher"; Filename: "{app}\thor-watcher.exe"
; Start Menu uninstall shortcut
Name: "{group}\Uninstall Thor Watcher"; Filename: "{uninstallexe}"
; Desktop shortcut (optional — controlled by [Tasks] above)
Name: "{commondesktop}\Thor Watcher"; Filename: "{app}\thor-watcher.exe"; Tasks: desktopicon
; Startup folder shortcut so the tray app launches on login
Name: "{userstartup}\Thor Watcher"; Filename: "{app}\thor-watcher.exe"
[Run]
; Offer to launch the app after install (unchecked by default)
Filename: "{app}\thor-watcher.exe"; \
Description: "Launch Thor Watcher"; \
Flags: nowait postinstall skipifsilent unchecked

View File

@@ -1,145 +1,130 @@
"""
Series 4 Ingest Agent v0.1.2
Thor Watcher — Series 4 Ingest Agent v0.2.0
Micromate (Series 4) ingest agent for Seismo Fleet Manager (SFM).
Micromate (Series 4) ingest agent for Terra-View.
Behavior:
- Scans C:\THORDATA\<Project>\<UM####>\*.MLG
- For each UM####, finds the newest .MLG by timestamp in the filename
- Computes "age" from last_call -> now
- Classifies status as OK / LATE / STALE
- Prints a console heartbeat
- (Optional) Posts JSON payload to SFM backend
No roster. SFM backend decides what to do with each unit.
- Posts JSON heartbeat payload to Terra-View backend
- Tray-friendly: run_watcher(state, stop_event) for background thread use
"""
import os
import re
import sys
import time
import json
import sys
from datetime import datetime, timedelta, timezone
from typing import Dict, Any, Optional, Tuple
from zoneinfo import ZoneInfo
try:
# urllib is in stdlib; used instead of requests for portability
import threading
import urllib.request
import urllib.error
except ImportError:
urllib = None # type: ignore
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional, Tuple
from socket import gethostname
# ---------------- Config ----------------
def load_config(config_path: str = "config.json") -> Dict[str, Any]:
# ── Version ───────────────────────────────────────────────────────────────────
VERSION = "0.2.0"
# ── Config ────────────────────────────────────────────────────────────────────
def load_config(config_path: str) -> Dict[str, Any]:
"""
Load configuration from a JSON file.
Falls back to defaults if file doesn't exist or has errors.
Load configuration from config.json.
Merges with defaults so any missing key is always present.
Raises on file-not-found or malformed JSON (caller handles).
"""
defaults = {
defaults: Dict[str, Any] = {
"thordata_path": r"C:\THORDATA",
"scan_interval": 60,
"late_days": 2,
"stale_days": 60,
"sfm_endpoint": "",
"sfm_timeout": 5,
"debug": True
"api_url": "",
"api_timeout": 5,
"api_interval": 300,
"source_id": "",
"source_type": "series4_watcher",
"local_timezone": "America/New_York",
"enable_logging": True,
"log_file": os.path.join(
os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or "C:\\",
"ThorWatcher", "agent_logs", "thor_watcher.log"
),
"log_retention_days": 30,
"update_source": "gitea",
"update_url": "",
"debug": False,
}
# Try to find config file relative to script location
script_dir = os.path.dirname(os.path.abspath(__file__))
full_config_path = os.path.join(script_dir, config_path)
with open(config_path, "r", encoding="utf-8") as f:
raw = json.load(f)
if not os.path.exists(full_config_path):
print(f"[WARN] Config file not found at {full_config_path}, using defaults", file=sys.stderr)
return defaults
return {**defaults, **raw}
# ── Logging ───────────────────────────────────────────────────────────────────
def log_message(path: str, enabled: bool, msg: str) -> None:
if not enabled:
return
try:
with open(full_config_path, 'r') as f:
config = json.load(f)
# Merge with defaults to ensure all keys exist
return {**defaults, **config}
except json.JSONDecodeError as e:
print(f"[WARN] Invalid JSON in config file: {e}, using defaults", file=sys.stderr)
return defaults
except Exception as e:
print(f"[WARN] Error loading config file: {e}, using defaults", file=sys.stderr)
return defaults
# Load configuration
config = load_config()
THORDATA_PATH = config["thordata_path"]
SCAN_INTERVAL = config["scan_interval"]
LATE_DAYS = config["late_days"]
STALE_DAYS = config["stale_days"]
SFM_ENDPOINT = config["sfm_endpoint"]
SFM_TIMEOUT = config["sfm_timeout"]
DEBUG = config["debug"]
# Regex: UM12345_YYYYMMDDHHMMSS.MLG
MLG_PATTERN = re.compile(r"^(UM\d+)_([0-9]{14})\.MLG$", re.IGNORECASE)
d = os.path.dirname(path) or "."
if not os.path.exists(d):
os.makedirs(d)
with open(path, "a", encoding="utf-8") as f:
f.write("{} {}\n".format(datetime.now(timezone.utc).isoformat(), msg))
except Exception:
pass
# ---------------- Helpers ----------------
def _read_log_tail(log_file: str, n: int = 25) -> Optional[List[str]]:
"""Return the last n lines of the log file as a list, or None."""
if not log_file:
return None
try:
with open(log_file, "r", errors="replace") as f:
lines = f.readlines()
return [line.rstrip("\n") for line in lines[-n:]]
except Exception:
return None
def debug(msg: str) -> None:
if DEBUG:
print(f"[DEBUG] {msg}", file=sys.stderr, flush=True)
# ── MLG filename parsing ──────────────────────────────────────────────────────
# Matches: UM12345_20251204193042.MLG
_MLG_PATTERN = re.compile(r"^(UM\d+)_([0-9]{14})\.MLG$", re.IGNORECASE)
def parse_mlg_filename(name: str) -> Optional[Tuple[str, datetime]]:
"""
Parse a Micromate MLG filename of the form:
UM12345_20251204193042.MLG
Returns:
(unit_id, timestamp) or None if pattern doesn't match.
"""
m = MLG_PATTERN.match(name)
"""Parse UM####_YYYYMMDDHHMMSS.MLG -> (unit_id, timestamp) or None."""
m = _MLG_PATTERN.match(name)
if not m:
return None
unit_id_raw = m.group(1) # e.g. "UM12345"
ts_str = m.group(2) # "YYYYMMDDHHMMSS"
unit_id = m.group(1).upper()
try:
ts = datetime.strptime(ts_str, "%Y%m%d%H%M%S")
ts = datetime.strptime(m.group(2), "%Y%m%d%H%M%S")
except ValueError:
return None
# Normalize unit_id to uppercase for consistency
return unit_id_raw.upper(), ts
return unit_id, ts
# ── THORDATA scanner ──────────────────────────────────────────────────────────
def scan_thordata(root: str) -> Dict[str, Dict[str, Any]]:
"""
Scan THORDATA folder for Micromate MLG files.
Expected structure:
C:\THORDATA\<Project>\<UM####>\*.MLG
Scan THORDATA folder structure: <root>/<Project>/<UM####>/*.MLG
Returns:
unit_map: {
"UM12345": {
"unit_id": "UM12345",
"project": "Clearwater - ECMS 57940",
"last_call": datetime(...),
"mlg_path": "C:\\THORDATA\\Clearwater...\\UM12345_....MLG"
},
...
}
{ "UM12345": { "unit_id", "project", "last_call" (datetime naive local), "mlg_path" }, ... }
"""
unit_map: Dict[str, Dict[str, Any]] = {}
if not os.path.isdir(root):
debug(f"THORDATA_PATH does not exist or is not a directory: {root}")
return unit_map
try:
project_names = os.listdir(root)
except OSError as e:
debug(f"Failed to list THORDATA root '{root}': {e}")
except OSError:
return unit_map
for project_name in project_names:
@@ -147,11 +132,9 @@ def scan_thordata(root: str) -> Dict[str, Dict[str, Any]]:
if not os.path.isdir(project_path):
continue
# Each project contains UM#### subfolders
try:
unit_dirs = os.listdir(project_path)
except OSError as e:
debug(f"Failed to list project '{project_path}': {e}")
except OSError:
continue
for unit_name in unit_dirs:
@@ -159,25 +142,19 @@ def scan_thordata(root: str) -> Dict[str, Dict[str, Any]]:
if not os.path.isdir(unit_path):
continue
# We expect folder names like "UM12345"
# but we'll parse filenames anyway, so we don't rely on folder naming.
try:
files = os.listdir(unit_path)
except OSError as e:
debug(f"Failed to list unit folder '{unit_path}': {e}")
except OSError:
continue
for fname in files:
if not fname.upper().endswith(".MLG"):
continue
parsed = parse_mlg_filename(fname)
if not parsed:
continue
unit_id, ts = parsed
full_path = os.path.join(unit_path, fname)
current = unit_map.get(unit_id)
if current is None or ts > current["last_call"]:
unit_map[unit_id] = {
@@ -190,249 +167,221 @@ def scan_thordata(root: str) -> Dict[str, Dict[str, Any]]:
return unit_map
def determine_status(last_call: datetime, now: Optional[datetime] = None) -> Tuple[str, float]:
"""
Determine status (OK / LATE / STALE) based on age in days.
# ── API payload ───────────────────────────────────────────────────────────────
Returns:
(status, age_days)
"""
if now is None:
now = datetime.now()
age = now - last_call
# Protect against clocks being off; don't go negative.
if age.total_seconds() < 0:
age = timedelta(seconds=0)
age_days = age.total_seconds() / 86400.0
if age_days < LATE_DAYS:
status = "OK"
elif age_days < STALE_DAYS:
status = "LATE"
else:
status = "STALE"
return status, age_days
def format_age(td: timedelta) -> str:
"""
Format a timedelta into a human-readable age string.
Examples:
1d 2h
3h 15m
42m
"""
total_seconds = int(td.total_seconds())
if total_seconds < 0:
total_seconds = 0
days, rem = divmod(total_seconds, 86400)
hours, rem = divmod(rem, 3600)
minutes, _ = divmod(rem, 60)
parts = []
if days > 0:
parts.append(f"{days}d")
if hours > 0:
parts.append(f"{hours}h")
if days == 0 and minutes > 0:
parts.append(f"{minutes}m") # only show minutes if < 1d
if not parts:
return "0m"
return " ".join(parts)
def clear_console() -> None:
"""Clear the console screen (Windows / *nix)."""
if os.name == "nt":
os.system("cls")
else:
os.system("clear")
def print_heartbeat(unit_map: Dict[str, Dict[str, Any]]) -> None:
"""
Print a console heartbeat table of all units.
Example:
UM11719 OK Age: 1h 12m Last: 2025-12-04 19:30:42 Project: Clearwater - ECMS 57940
"""
now = datetime.now()
clear_console()
print("Series 4 Ingest Agent — Micromate Heartbeat (v0.1.2)")
print(f"THORDATA root: {THORDATA_PATH}")
print(f"Now: {now.strftime('%Y-%m-%d %H:%M:%S')}")
print("-" * 80)
if not unit_map:
print("No units found (no .MLG files detected).")
return
# Sort by unit_id for stable output
for unit_id in sorted(unit_map.keys()):
entry = unit_map[unit_id]
last_call = entry["last_call"]
project = entry["project"]
age_td = now - last_call
status, _age_days = determine_status(last_call, now)
age_str = format_age(age_td)
last_str = last_call.strftime("%Y-%m-%d %H:%M:%S")
print(
f"{unit_id:<8} {status:<6} Age: {age_str:<8} "
f"Last: {last_str} Project: {project}"
)
print("-" * 80)
print(f"Total units: {len(unit_map)}")
print(f"Next scan in {SCAN_INTERVAL} seconds...")
sys.stdout.flush()
def build_sfm_payload(unit_map: Dict[str, Dict[str, Any]]) -> Dict[str, Any]:
"""
Build a JSON-serializable payload for SFM backend.
All timestamps are converted to UTC for transmission (standard practice).
Terra-View stores UTC and converts to local time for display.
Structure (example):
{
"source": "series4_ingest",
"generated_at": "2025-12-04T20:01:00Z",
"units": [
{
"unit_id": "UM11719",
"type": "micromate",
"project_hint": "Clearwater - ECMS 57940",
"last_call": "2025-12-05T00:30:42Z",
"status": "OK",
"age_days": 0.04,
"age_hours": 0.9,
"mlg_path": "C:\\THORDATA\\Clearwater...\\UM11719_....MLG"
},
...
]
}
"""
now_local = datetime.now()
def build_api_payload(unit_map: Dict[str, Dict[str, Any]], cfg: Dict[str, Any]) -> dict:
"""Build the Terra-View JSON heartbeat payload."""
now_utc = datetime.now(timezone.utc)
local_tz = ZoneInfo("America/New_York")
payload_units = []
now_local = datetime.now()
source_id = (cfg.get("source_id") or "").strip() or gethostname()
# Resolve local timezone for MLG timestamp conversion
try:
from zoneinfo import ZoneInfo
local_tz = ZoneInfo(cfg.get("local_timezone") or "America/New_York")
except Exception:
local_tz = None
units = []
for unit_id, entry in unit_map.items():
last_call: datetime = entry["last_call"]
project = entry["project"]
mlg_path = entry["mlg_path"]
age_seconds = max(0.0, (now_local - last_call).total_seconds())
age_minutes = int(age_seconds // 60)
# Use local time for status calculation (age comparison)
status, age_days = determine_status(last_call, now_local)
age_hours = age_days * 24.0
# Convert last_call from local time to UTC for transmission
# MLG timestamps are local naive — convert to UTC for transmission
try:
if local_tz is not None:
last_call_utc = last_call.replace(tzinfo=local_tz).astimezone(timezone.utc)
last_call_str = last_call_utc.strftime("%Y-%m-%dT%H:%M:%SZ")
else:
# Fallback: send as-is with Z and accept the inaccuracy
last_call_str = last_call.strftime("%Y-%m-%dT%H:%M:%SZ")
except Exception:
last_call_str = last_call.strftime("%Y-%m-%dT%H:%M:%SZ")
payload_units.append(
{
units.append({
"unit_id": unit_id,
"type": "micromate",
"project_hint": project,
"last_call": last_call_utc.strftime("%Y-%m-%dT%H:%M:%SZ"),
"status": status,
"age_days": age_days,
"age_hours": age_hours,
"mlg_path": mlg_path,
}
)
"last_call": last_call_str,
"age_minutes": age_minutes,
"mlg_path": entry["mlg_path"],
"project_hint": entry["project"],
})
payload = {
"source": "series4_ingest",
return {
"source_id": source_id,
"source_type": cfg.get("source_type", "series4_watcher"),
"version": VERSION,
"generated_at": now_utc.strftime("%Y-%m-%dT%H:%M:%SZ"),
"units": payload_units,
"units": units,
}
return payload
def emit_sfm_payload(unit_map: Dict[str, Dict[str, Any]]) -> None:
"""
Send heartbeat payload to SFM backend, if SFM_ENDPOINT is configured.
This is intentionally conservative:
- If SFM_ENDPOINT is empty -> do nothing
- If any error occurs -> print to stderr, but do not crash the agent
"""
if not SFM_ENDPOINT:
return
if urllib is None:
print(
"[WARN] urllib not available; cannot POST to SFM. "
"Install standard Python or disable SFM_ENDPOINT.",
file=sys.stderr,
)
return
payload = build_sfm_payload(unit_map)
def send_api_payload(payload: dict, api_url: str, timeout: int) -> Optional[dict]:
"""POST payload to Terra-View. Returns parsed JSON response or None on failure."""
if not api_url:
return None
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
SFM_ENDPOINT,
data=data,
api_url, data=data,
headers={"Content-Type": "application/json"},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
print("[API] POST success: {}".format(resp.status))
try:
return json.loads(resp.read().decode("utf-8"))
except Exception:
return {}
except urllib.error.URLError as e:
print("[API] POST failed: {}".format(e))
return None
except Exception as e:
print("[API] Unexpected error: {}".format(e))
return None
# ── Watcher loop (tray-friendly) ──────────────────────────────────────────────
def run_watcher(state: Dict[str, Any], stop_event: threading.Event) -> None:
"""
Main watcher loop. Runs in a background thread when launched from the tray.
state keys written each cycle:
state["status"] — "running" | "error" | "starting"
state["api_status"] — "ok" | "fail" | "disabled"
state["units"] — list of unit dicts for tray display
state["last_scan"] — datetime of last successful scan
state["last_error"] — last error string or None
state["log_dir"] — directory containing the log file
state["cfg"] — loaded config dict
state["update_available"] — set True when API response signals an update
"""
# Resolve config path
if getattr(sys, "frozen", False):
_appdata = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or ""
config_dir = os.path.join(_appdata, "ThorWatcher")
else:
config_dir = os.path.dirname(os.path.abspath(__file__)) or "."
config_path = os.path.join(config_dir, "config.json")
state["status"] = "starting"
state["units"] = []
state["last_scan"] = None
state["last_error"] = None
state["log_dir"] = None
state["cfg"] = {}
state["update_available"] = False
try:
with urllib.request.urlopen(req, timeout=SFM_TIMEOUT) as resp:
_ = resp.read() # we don't care about the body for now
debug(f"SFM POST OK: HTTP {resp.status}")
except urllib.error.URLError as e:
print(f"[WARN] Failed to POST to SFM: {e}", file=sys.stderr)
cfg = load_config(config_path)
except Exception as e:
print(f"[WARN] Unexpected error during SFM POST: {e}", file=sys.stderr)
state["status"] = "error"
state["last_error"] = "Config load failed: {}".format(e)
return
state["cfg"] = cfg
log_file = cfg["log_file"]
state["log_dir"] = os.path.dirname(log_file) or config_dir
THORDATA_PATH = cfg["thordata_path"]
SCAN_INTERVAL = int(cfg["scan_interval"])
API_URL = cfg["api_url"]
API_TIMEOUT = int(cfg["api_timeout"])
API_INTERVAL = int(cfg["api_interval"])
ENABLE_LOGGING = bool(cfg["enable_logging"])
log_message(log_file, ENABLE_LOGGING,
"[cfg] THORDATA_PATH={} SCAN_INTERVAL={}s API_INTERVAL={}s API={}".format(
THORDATA_PATH, SCAN_INTERVAL, API_INTERVAL, bool(API_URL)
)
)
print("[CFG] THORDATA_PATH={} SCAN_INTERVAL={}s API={}".format(
THORDATA_PATH, SCAN_INTERVAL, bool(API_URL)
))
last_api_ts = 0.0
while not stop_event.is_set():
try:
now_str = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
print("-" * 80)
print("Heartbeat @ {}".format(now_str))
print("-" * 80)
unit_map = scan_thordata(THORDATA_PATH)
now_local = datetime.now()
unit_list = []
for uid in sorted(unit_map.keys()):
entry = unit_map[uid]
last_call = entry["last_call"]
age_seconds = max(0.0, (now_local - last_call).total_seconds())
age_minutes = int(age_seconds // 60)
unit_list.append({
"uid": uid,
"age_minutes": age_minutes,
"last_call": last_call.strftime("%Y-%m-%d %H:%M:%S"),
"mlg_path": entry["mlg_path"],
"project": entry["project"],
})
line = "{uid:<8} Age: {h}h {m}m Last: {last} Project: {proj}".format(
uid=uid,
h=age_minutes // 60,
m=age_minutes % 60,
last=last_call.strftime("%Y-%m-%d %H:%M:%S"),
proj=entry["project"],
)
print(line)
log_message(log_file, ENABLE_LOGGING, line)
if not unit_list:
msg = "[info] No Micromate units found in THORDATA"
print(msg)
log_message(log_file, ENABLE_LOGGING, msg)
state["status"] = "running"
state["units"] = unit_list
state["last_scan"] = datetime.now()
state["last_error"] = None
# ── API heartbeat ──────────────────────────────────────────────────
if API_URL:
now_ts = time.time()
if now_ts - last_api_ts >= API_INTERVAL:
payload = build_api_payload(unit_map, cfg)
payload["log_tail"] = _read_log_tail(log_file, 25)
response = send_api_payload(payload, API_URL, API_TIMEOUT)
last_api_ts = now_ts
if response is not None:
state["api_status"] = "ok"
if response.get("update_available"):
state["update_available"] = True
else:
state["api_status"] = "fail"
else:
state["api_status"] = "disabled"
except Exception as e:
err = "[loop-error] {}".format(e)
print(err)
log_message(log_file, ENABLE_LOGGING, err)
state["status"] = "error"
state["last_error"] = str(e)
stop_event.wait(timeout=SCAN_INTERVAL)
# ── Standalone entry point ────────────────────────────────────────────────────
def main() -> None:
print("Starting Series 4 Ingest Agent (Micromate) v0.1.2")
print(f"THORDATA_PATH = {THORDATA_PATH}")
print(f"SCAN_INTERVAL = {SCAN_INTERVAL} seconds")
print(f"LATE_DAYS = {LATE_DAYS}, STALE_DAYS = {STALE_DAYS}")
if not os.path.isdir(THORDATA_PATH):
print(f"[WARN] THORDATA_PATH does not exist: {THORDATA_PATH}", file=sys.stderr)
loop_counter = 0
state: Dict[str, Any] = {}
stop_event = threading.Event()
try:
while True:
loop_counter += 1
print(f"\n[LOOP] Iteration {loop_counter} starting...", flush=True)
try:
unit_map = scan_thordata(THORDATA_PATH)
debug(f"scan_thordata found {len(unit_map)} units")
print_heartbeat(unit_map)
emit_sfm_payload(unit_map)
print("[LOOP] Iteration complete, entering sleep...", flush=True)
except Exception as e:
# Catch-all so a single error doesn't kill the loop
print(f"[ERROR] Exception in main loop: {e}", file=sys.stderr)
sys.stderr.flush()
# Sleep in 1-second chunks to avoid VM time drift weirdness
for i in range(SCAN_INTERVAL):
time.sleep(1)
print("[LOOP] Woke up for next scan", flush=True)
run_watcher(state, stop_event)
except KeyboardInterrupt:
print("\nSeries 4 Ingest Agent stopped by user.")
print("\nStopping...")
stop_event.set()
if __name__ == "__main__":

511
thor_settings_dialog.py Normal file
View File

@@ -0,0 +1,511 @@
"""
Thor Watcher — Settings Dialog v0.2.0
Provides a Tkinter settings dialog that doubles as a first-run wizard.
Public API:
show_dialog(config_path, wizard=False) -> bool
Returns True if the user saved, False if they cancelled.
"""
import os
import json
import tkinter as tk
from tkinter import ttk, filedialog, messagebox
from socket import gethostname
# ── Defaults (mirror config.example.json) ────────────────────────────────────
DEFAULTS = {
"thordata_path": r"C:\THORDATA",
"scan_interval": 60,
"api_url": "",
"api_timeout": 5,
"api_interval": 300,
"source_id": "",
"source_type": "series4_watcher",
"local_timezone": "America/New_York",
"enable_logging": True,
"log_file": os.path.join(
os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or "C:\\",
"ThorWatcher", "agent_logs", "thor_watcher.log"
),
"log_retention_days": 30,
"update_source": "gitea",
"update_url": "",
}
# ── Config I/O ────────────────────────────────────────────────────────────────
def _load_config(config_path):
"""Load existing config.json, merged with DEFAULTS for any missing key."""
values = dict(DEFAULTS)
if not os.path.exists(config_path):
return values
try:
with open(config_path, "r", encoding="utf-8") as f:
raw = json.load(f)
values.update(raw)
except Exception:
pass
return values
def _save_config(config_path, values):
"""Write values dict to config_path as JSON."""
config_dir = os.path.dirname(config_path)
if config_dir and not os.path.exists(config_dir):
os.makedirs(config_dir)
with open(config_path, "w", encoding="utf-8") as f:
json.dump(values, f, indent=2)
# ── Widget helpers ────────────────────────────────────────────────────────────
def _make_spinbox(parent, from_, to, width=8):
try:
sb = ttk.Spinbox(parent, from_=from_, to=to, width=width)
except AttributeError:
sb = tk.Spinbox(parent, from_=from_, to=to, width=width)
return sb
def _add_label_entry(frame, row, label_text, var, hint=None, readonly=False):
tk.Label(frame, text=label_text, anchor="w").grid(
row=row, column=0, sticky="w", padx=(8, 4), pady=4
)
state = "readonly" if readonly else "normal"
entry = ttk.Entry(frame, textvariable=var, width=42, state=state)
entry.grid(row=row, column=1, sticky="ew", padx=(0, 8), pady=4)
if hint and not var.get():
entry.config(foreground="grey")
entry.insert(0, hint)
def _on_focus_in(event, e=entry, h=hint, v=var):
if e.get() == h:
e.delete(0, tk.END)
e.config(foreground="black")
def _on_focus_out(event, e=entry, h=hint, v=var):
if not e.get():
e.config(foreground="grey")
e.insert(0, h)
v.set("")
entry.bind("<FocusIn>", _on_focus_in)
entry.bind("<FocusOut>", _on_focus_out)
return entry
def _add_label_spinbox(frame, row, label_text, var, from_, to):
tk.Label(frame, text=label_text, anchor="w").grid(
row=row, column=0, sticky="w", padx=(8, 4), pady=4
)
sb = _make_spinbox(frame, from_=from_, to=to, width=8)
sb.grid(row=row, column=1, sticky="w", padx=(0, 8), pady=4)
sb.delete(0, tk.END)
sb.insert(0, str(var.get()))
def _on_change(*args):
var.set(sb.get())
sb.config(command=_on_change)
sb.bind("<KeyRelease>", _on_change)
return sb
def _add_label_check(frame, row, label_text, var):
cb = ttk.Checkbutton(frame, text=label_text, variable=var)
cb.grid(row=row, column=0, columnspan=2, sticky="w", padx=(8, 8), pady=4)
return cb
def _add_label_browse_entry(frame, row, label_text, var, browse_fn):
tk.Label(frame, text=label_text, anchor="w").grid(
row=row, column=0, sticky="w", padx=(8, 4), pady=4
)
inner = tk.Frame(frame)
inner.grid(row=row, column=1, sticky="ew", padx=(0, 8), pady=4)
inner.columnconfigure(0, weight=1)
entry = ttk.Entry(inner, textvariable=var, width=36)
entry.grid(row=0, column=0, sticky="ew")
btn = ttk.Button(inner, text="Browse...", command=browse_fn, width=9)
btn.grid(row=0, column=1, padx=(4, 0))
return entry
# ── Main dialog class ─────────────────────────────────────────────────────────
class SettingsDialog:
def __init__(self, parent, config_path, wizard=False):
self.config_path = config_path
self.wizard = wizard
self.saved = False
self.root = parent
title = "Thor Watcher — Setup" if wizard else "Thor Watcher — Settings"
self.root.title(title)
self.root.resizable(False, False)
self.root.update_idletasks()
self._values = _load_config(config_path)
self._build_vars()
self._build_ui()
self.root.grab_set()
self.root.protocol("WM_DELETE_WINDOW", self._on_cancel)
# ── Variable setup ────────────────────────────────────────────────────────
def _build_vars(self):
v = self._values
# Connection
raw_url = str(v.get("api_url", ""))
_suffix = "/api/series4/heartbeat"
if raw_url.endswith(_suffix):
raw_url = raw_url[:-len(_suffix)]
self.var_api_url = tk.StringVar(value=raw_url)
self.var_api_interval = tk.StringVar(value=str(v.get("api_interval", 300)))
self.var_source_id = tk.StringVar(value=str(v.get("source_id", "")))
self.var_source_type = tk.StringVar(value=str(v.get("source_type", "series4_watcher")))
# Paths
self.var_thordata_path = tk.StringVar(value=str(v.get("thordata_path", r"C:\THORDATA")))
self.var_log_file = tk.StringVar(value=str(v.get("log_file", DEFAULTS["log_file"])))
# Scanning
self.var_scan_interval = tk.StringVar(value=str(v.get("scan_interval", 60)))
# Logging
en = v.get("enable_logging", True)
self.var_enable_logging = tk.BooleanVar(value=bool(en) if isinstance(en, bool) else str(en).lower() in ("true", "1", "yes"))
self.var_log_retention_days = tk.StringVar(value=str(v.get("log_retention_days", 30)))
# Updates
src = str(v.get("update_source", "gitea")).lower()
if src not in ("gitea", "url", "disabled"):
src = "gitea"
self.var_local_timezone = tk.StringVar(value=str(v.get("local_timezone", "America/New_York")))
self.var_update_source = tk.StringVar(value=src)
self.var_update_url = tk.StringVar(value=str(v.get("update_url", "")))
# ── UI construction ───────────────────────────────────────────────────────
def _build_ui(self):
outer = tk.Frame(self.root, padx=10, pady=8)
outer.pack(fill="both", expand=True)
if self.wizard:
welcome = (
"Welcome to Thor Watcher!\n\n"
"No configuration file was found. Please review the settings below\n"
"and click \"Save & Start\" when you are ready."
)
tk.Label(
outer, text=welcome, justify="left",
wraplength=460, fg="#1a5276", font=("TkDefaultFont", 9, "bold"),
).pack(fill="x", pady=(0, 8))
nb = ttk.Notebook(outer)
nb.pack(fill="both", expand=True)
self._build_tab_connection(nb)
self._build_tab_paths(nb)
self._build_tab_scanning(nb)
self._build_tab_logging(nb)
self._build_tab_updates(nb)
btn_frame = tk.Frame(outer)
btn_frame.pack(fill="x", pady=(10, 0))
save_label = "Save & Start" if self.wizard else "Save"
ttk.Button(btn_frame, text=save_label, command=self._on_save, width=14).pack(side="right", padx=(4, 0))
ttk.Button(btn_frame, text="Cancel", command=self._on_cancel, width=10).pack(side="right")
def _tab_frame(self, nb, title):
outer = tk.Frame(nb, padx=4, pady=4)
nb.add(outer, text=title)
outer.columnconfigure(1, weight=1)
return outer
def _build_tab_connection(self, nb):
f = self._tab_frame(nb, "Connection")
# URL row with Test button
tk.Label(f, text="Terra-View URL", anchor="w").grid(
row=0, column=0, sticky="w", padx=(8, 4), pady=4
)
url_frame = tk.Frame(f)
url_frame.grid(row=0, column=1, sticky="ew", padx=(0, 8), pady=4)
url_frame.columnconfigure(0, weight=1)
url_entry = ttk.Entry(url_frame, textvariable=self.var_api_url, width=32)
url_entry.grid(row=0, column=0, sticky="ew")
_hint = "http://192.168.x.x:8000"
if not self.var_api_url.get():
url_entry.config(foreground="grey")
url_entry.insert(0, _hint)
def _on_focus_in(e):
if url_entry.get() == _hint:
url_entry.delete(0, tk.END)
url_entry.config(foreground="black")
def _on_focus_out(e):
if not url_entry.get():
url_entry.config(foreground="grey")
url_entry.insert(0, _hint)
self.var_api_url.set("")
url_entry.bind("<FocusIn>", _on_focus_in)
url_entry.bind("<FocusOut>", _on_focus_out)
self._test_btn = ttk.Button(url_frame, text="Test", width=6,
command=self._test_connection)
self._test_btn.grid(row=0, column=1, padx=(4, 0))
self._test_status = tk.Label(url_frame, text="", anchor="w", width=20)
self._test_status.grid(row=0, column=2, padx=(6, 0))
_add_label_spinbox(f, 1, "API Interval (sec)", self.var_api_interval, 30, 3600)
source_id_hint = "Defaults to hostname ({})".format(gethostname())
_add_label_entry(f, 2, "Source ID", self.var_source_id, hint=source_id_hint)
_add_label_entry(f, 3, "Source Type", self.var_source_type, readonly=True)
_add_label_entry(f, 4, "Local Timezone", self.var_local_timezone,
hint="e.g. America/New_York, America/Chicago")
tk.Label(
f,
text="Used to convert MLG file timestamps (local time) to UTC for terra-view.",
justify="left", fg="#555555", wraplength=340,
).grid(row=5, column=0, columnspan=2, sticky="w", padx=(8, 8), pady=(0, 4))
def _test_connection(self):
import urllib.request
import urllib.error
self._test_status.config(text="Testing...", foreground="grey")
self._test_btn.config(state="disabled")
self.root.update_idletasks()
raw = self.var_api_url.get().strip()
if not raw or raw == "http://192.168.x.x:8000":
self._test_status.config(text="Enter a URL first", foreground="orange")
self._test_btn.config(state="normal")
return
url = raw.rstrip("/") + "/health"
try:
with urllib.request.urlopen(urllib.request.Request(url), timeout=5) as resp:
if resp.status == 200:
self._test_status.config(text="Connected!", foreground="green")
else:
self._test_status.config(text="HTTP {}".format(resp.status), foreground="orange")
except urllib.error.URLError as e:
reason = str(e.reason) if hasattr(e, "reason") else str(e)
self._test_status.config(text="Failed: {}".format(reason[:30]), foreground="red")
except Exception as e:
self._test_status.config(text="Error: {}".format(str(e)[:30]), foreground="red")
finally:
self._test_btn.config(state="normal")
def _build_tab_paths(self, nb):
f = self._tab_frame(nb, "Paths")
def browse_thordata():
d = filedialog.askdirectory(
title="Select THORDATA Folder",
initialdir=self.var_thordata_path.get() or "C:\\",
)
if d:
self.var_thordata_path.set(d.replace("/", "\\"))
_add_label_browse_entry(f, 0, "THORDATA Path", self.var_thordata_path, browse_thordata)
def browse_log():
p = filedialog.asksaveasfilename(
title="Select Log File",
defaultextension=".log",
filetypes=[("Log files", "*.log"), ("Text files", "*.txt"), ("All files", "*.*")],
initialfile=os.path.basename(self.var_log_file.get() or "thor_watcher.log"),
initialdir=os.path.dirname(self.var_log_file.get() or "C:\\"),
)
if p:
self.var_log_file.set(p.replace("/", "\\"))
_add_label_browse_entry(f, 1, "Log File", self.var_log_file, browse_log)
def _build_tab_scanning(self, nb):
f = self._tab_frame(nb, "Scanning")
_add_label_spinbox(f, 0, "Scan Interval (sec)", self.var_scan_interval, 10, 3600)
def _build_tab_logging(self, nb):
f = self._tab_frame(nb, "Logging")
_add_label_check(f, 0, "Enable Logging", self.var_enable_logging)
_add_label_spinbox(f, 1, "Log Retention (days)", self.var_log_retention_days, 1, 365)
def _build_tab_updates(self, nb):
f = self._tab_frame(nb, "Updates")
tk.Label(f, text="Auto-Update Source", anchor="w").grid(
row=0, column=0, sticky="w", padx=(8, 4), pady=(8, 2)
)
radio_frame = tk.Frame(f)
radio_frame.grid(row=0, column=1, sticky="w", padx=(0, 8), pady=(8, 2))
ttk.Radiobutton(
radio_frame, text="Gitea (default)",
variable=self.var_update_source, value="gitea",
command=self._on_update_source_change,
).grid(row=0, column=0, sticky="w", padx=(0, 12))
ttk.Radiobutton(
radio_frame, text="Custom URL",
variable=self.var_update_source, value="url",
command=self._on_update_source_change,
).grid(row=0, column=1, sticky="w", padx=(0, 12))
ttk.Radiobutton(
radio_frame, text="Disabled",
variable=self.var_update_source, value="disabled",
command=self._on_update_source_change,
).grid(row=0, column=2, sticky="w")
tk.Label(f, text="Update Server URL", anchor="w").grid(
row=1, column=0, sticky="w", padx=(8, 4), pady=4
)
self._update_url_entry = ttk.Entry(f, textvariable=self.var_update_url, width=42)
self._update_url_entry.grid(row=1, column=1, sticky="ew", padx=(0, 8), pady=4)
tk.Label(
f,
text=(
"Gitea: checks the Gitea release page automatically every 5 minutes.\n"
"Custom URL: fetches version.txt and thor-watcher.exe from a web\n"
"server — use when Gitea is not reachable (e.g. terra-view URL).\n"
"Disabled: no automatic update checks. Remote push from terra-view\n"
"still works when disabled."
),
justify="left", fg="#555555", wraplength=380,
).grid(row=2, column=0, columnspan=2, sticky="w", padx=(8, 8), pady=(4, 8))
self._on_update_source_change()
def _on_update_source_change(self):
if self.var_update_source.get() == "url":
self._update_url_entry.config(state="normal")
else:
self._update_url_entry.config(state="disabled")
# ── Validation ────────────────────────────────────────────────────────────
def _get_int_var(self, var, name, min_val, max_val):
raw = str(var.get()).strip()
try:
val = int(raw)
except ValueError:
messagebox.showerror("Validation Error",
"{} must be an integer (got: {!r}).".format(name, raw))
return None
if val < min_val or val > max_val:
messagebox.showerror("Validation Error",
"{} must be between {} and {} (got {}).".format(name, min_val, max_val, val))
return None
return val
# ── Save / Cancel ─────────────────────────────────────────────────────────
def _on_save(self):
checks = [
(self.var_api_interval, "API Interval", 30, 3600),
(self.var_scan_interval, "Scan Interval", 10, 3600),
(self.var_log_retention_days, "Log Retention Days", 1, 365),
]
int_values = {}
for var, name, mn, mx in checks:
result = self._get_int_var(var, name, mn, mx)
if result is None:
return
int_values[name] = result
source_id = self.var_source_id.get().strip()
if source_id.startswith("Defaults to hostname"):
source_id = ""
api_url = self.var_api_url.get().strip()
if api_url == "http://192.168.x.x:8000" or not api_url:
api_url = ""
else:
api_url = api_url.rstrip("/") + "/api/series4/heartbeat"
values = {
"thordata_path": self.var_thordata_path.get().strip(),
"scan_interval": int_values["Scan Interval"],
"api_url": api_url,
"api_timeout": 5,
"api_interval": int_values["API Interval"],
"source_id": source_id,
"source_type": self.var_source_type.get().strip() or "series4_watcher",
"local_timezone": self.var_local_timezone.get().strip() or "America/New_York",
"enable_logging": self.var_enable_logging.get(),
"log_file": self.var_log_file.get().strip(),
"log_retention_days": int_values["Log Retention Days"],
"update_source": self.var_update_source.get().strip() or "gitea",
"update_url": self.var_update_url.get().strip(),
}
try:
_save_config(self.config_path, values)
except Exception as e:
messagebox.showerror("Save Error", "Could not write config.json:\n{}".format(e))
return
self.saved = True
self.root.destroy()
def _on_cancel(self):
self.saved = False
self.root.destroy()
# ── Public API ────────────────────────────────────────────────────────────────
def show_dialog(config_path, wizard=False):
"""
Open the settings dialog.
Parameters
----------
config_path : str
Absolute path to config.json (read if exists, written on Save).
wizard : bool
If True, shows first-run welcome message and "Save & Start" button.
Returns
-------
bool
True if the user saved, False if they cancelled.
"""
root = tk.Tk()
root.withdraw()
top = tk.Toplevel(root)
top.deiconify()
dlg = SettingsDialog(top, config_path, wizard=wizard)
top.update_idletasks()
w = top.winfo_reqwidth()
h = top.winfo_reqheight()
sw = top.winfo_screenwidth()
sh = top.winfo_screenheight()
top.geometry("{}x{}+{}+{}".format(w, h, (sw - w) // 2, (sh - h) // 2))
root.wait_window(top)
root.destroy()
return dlg.saved

532
thor_tray.py Normal file
View File

@@ -0,0 +1,532 @@
"""
Thor Watcher — System Tray Launcher v0.2.0
Requires: pystray, Pillow, tkinter (stdlib)
Run with: pythonw thor_tray.py (no console window)
or: python thor_tray.py (with console, for debugging)
Put a shortcut to this in shell:startup for auto-start on login.
"""
import os
import sys
import json
import subprocess
import tempfile
import threading
import urllib.request
import urllib.error
from datetime import datetime
import pystray
from PIL import Image, ImageDraw
import series4_ingest as watcher
# ── Auto-updater ──────────────────────────────────────────────────────────────
GITEA_BASE = "https://gitea.serversdown.net"
GITEA_USER = "serversdown"
GITEA_REPO = "thor-watcher"
GITEA_API_URL = "{}/api/v1/repos/{}/{}/releases?limit=1&page=1".format(
GITEA_BASE, GITEA_USER, GITEA_REPO
)
_CURRENT_VERSION = getattr(watcher, "VERSION", "0.0.0")
def _version_tuple(v):
"""Convert '0.2.0' -> (0, 2, 0) for comparison."""
parts = []
for p in str(v).lstrip("v").split(".")[:3]:
try:
parts.append(int(p))
except ValueError:
parts.append(0)
while len(parts) < 3:
parts.append(0)
return tuple(parts)
def _update_log(msg):
"""Append a timestamped [updater] line to the watcher log."""
try:
log_path = os.path.join(
os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or "",
"ThorWatcher", "agent_logs", "thor_watcher.log"
)
os.makedirs(os.path.dirname(log_path), exist_ok=True)
with open(log_path, "a") as f:
f.write("[{}] [updater] {}\n".format(
datetime.now().strftime("%Y-%m-%d %H:%M:%S"), msg
))
except Exception:
pass
def _check_for_update_gitea():
"""Query Gitea API for latest release. Returns (tag, download_url) or (None, None)."""
try:
req = urllib.request.Request(
GITEA_API_URL,
headers={"User-Agent": "thor-watcher/{}".format(_CURRENT_VERSION)},
)
with urllib.request.urlopen(req, timeout=8) as resp:
releases = json.loads(resp.read().decode("utf-8"))
if not releases:
return None, None
latest = releases[0]
tag = latest.get("tag_name", "")
if _version_tuple(tag) <= _version_tuple(_CURRENT_VERSION):
return None, None
assets = latest.get("assets", [])
for asset in assets:
name = asset.get("name", "").lower()
if name.endswith(".exe") and "setup" not in name:
return tag, asset.get("browser_download_url")
_update_log("Newer release {} found but no valid .exe asset".format(tag))
return tag, None
except Exception as e:
_update_log("check_for_update (gitea) failed: {}".format(e))
return None, None
def _check_for_update_url(base_url):
"""Query a custom URL server for latest version. Returns (tag, download_url) or (None, None)."""
if not base_url:
_update_log("update_source=url but update_url is empty — skipping")
return None, None
try:
ver_url = base_url.rstrip("/") + "/api/updates/thor-watcher/version.txt"
req = urllib.request.Request(
ver_url,
headers={"User-Agent": "thor-watcher/{}".format(_CURRENT_VERSION)},
)
with urllib.request.urlopen(req, timeout=8) as resp:
tag = resp.read().decode("utf-8").strip()
if not tag:
return None, None
if _version_tuple(tag) <= _version_tuple(_CURRENT_VERSION):
return None, None
exe_url = base_url.rstrip("/") + "/api/updates/thor-watcher/thor-watcher.exe"
return tag, exe_url
except Exception as e:
_update_log("check_for_update (url mode) failed: {}".format(e))
return None, None
def check_for_update():
"""
Check for an update using the configured source (gitea, url, or disabled).
Reads update_source and update_url from config.json at check time.
Returns (tag, download_url) if an update is available, else (None, None).
"""
try:
cfg = _read_config()
update_source = str(cfg.get("update_source", "gitea")).strip().lower()
update_url = str(cfg.get("update_url", "")).strip()
except Exception:
update_source = "gitea"
update_url = ""
if update_source == "disabled":
return None, None
_update_log("Checking for update (source={}, version={})".format(
update_source, _CURRENT_VERSION
))
if update_source == "url":
return _check_for_update_url(update_url)
else:
return _check_for_update_gitea()
def apply_update(download_url):
"""
Download new .exe, validate it, write swap .bat, launch it, exit.
Backs up old exe to .exe.old before replacing.
"""
exe_path = os.path.abspath(sys.executable if getattr(sys, "frozen", False) else sys.argv[0])
try:
tmp_fd, tmp_path = tempfile.mkstemp(suffix=".exe", prefix="tw_update_")
os.close(tmp_fd)
_update_log("Downloading update from: {}".format(download_url))
req = urllib.request.Request(
download_url,
headers={"User-Agent": "thor-watcher/{}".format(_CURRENT_VERSION)},
)
with urllib.request.urlopen(req, timeout=60) as resp:
with open(tmp_path, "wb") as f:
f.write(resp.read())
# Three-layer validation
try:
dl_size = os.path.getsize(tmp_path)
current_size = os.path.getsize(exe_path)
_update_log("Download complete ({} bytes), validating...".format(dl_size))
if dl_size < 100 * 1024:
_update_log("Validation failed: too small ({} bytes) — aborting".format(dl_size))
os.remove(tmp_path)
return False
if current_size > 0 and dl_size < current_size * 0.5:
_update_log("Validation failed: suspiciously small ({} vs {} bytes) — aborting".format(
dl_size, current_size))
os.remove(tmp_path)
return False
with open(tmp_path, "rb") as _f:
magic = _f.read(2)
if magic != b"MZ":
_update_log("Validation failed: not a valid Windows exe — aborting")
os.remove(tmp_path)
return False
_update_log("Validation passed ({} bytes, MZ ok)".format(dl_size))
except Exception as e:
_update_log("Validation error: {} — aborting".format(e))
try:
os.remove(tmp_path)
except Exception:
pass
return False
bat_fd, bat_path = tempfile.mkstemp(suffix=".bat", prefix="tw_swap_")
os.close(bat_fd)
bat_content = (
"@echo off\r\n"
"ping 127.0.0.1 -n 4 > nul\r\n"
"copy /Y \"{exe}\" \"{exe}.old\"\r\n"
"set RETRIES=0\r\n"
":retry\r\n"
"copy /Y \"{new}\" \"{exe}\"\r\n"
"if errorlevel 1 (\r\n"
" set /a RETRIES+=1\r\n"
" if %RETRIES% GEQ 5 goto fail\r\n"
" ping 127.0.0.1 -n 3 > nul\r\n"
" goto retry\r\n"
")\r\n"
"start \"\" \"{exe}\"\r\n"
"del \"{new}\"\r\n"
"del \"%~f0\"\r\n"
"exit /b 0\r\n"
":fail\r\n"
"del \"{new}\"\r\n"
"del \"%~f0\"\r\n"
"exit /b 1\r\n"
).format(new=tmp_path, exe=exe_path)
with open(bat_path, "w") as f:
f.write(bat_content)
_update_log("Launching swap bat — exiting for update")
subprocess.Popen(
["cmd", "/C", bat_path],
creationflags=subprocess.CREATE_NO_WINDOW if hasattr(subprocess, "CREATE_NO_WINDOW") else 0,
)
return True
except Exception as e:
_update_log("apply_update failed: {}".format(e))
return False
# ── Config helpers ────────────────────────────────────────────────────────────
def _read_config():
"""Read config.json from the appropriate location and return as dict."""
if getattr(sys, "frozen", False):
_appdata = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or ""
config_dir = os.path.join(_appdata, "ThorWatcher")
else:
config_dir = os.path.dirname(os.path.abspath(__file__)) or "."
config_path = os.path.join(config_dir, "config.json")
with open(config_path, "r", encoding="utf-8") as f:
return json.load(f)
# ── Paths ─────────────────────────────────────────────────────────────────────
if getattr(sys, "frozen", False):
HERE = os.path.dirname(os.path.abspath(sys.executable))
else:
HERE = os.path.dirname(os.path.abspath(__file__))
if getattr(sys, "frozen", False):
_appdata = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or HERE
CONFIG_DIR = os.path.join(_appdata, "ThorWatcher")
os.makedirs(CONFIG_DIR, exist_ok=True)
else:
CONFIG_DIR = HERE
CONFIG_PATH = os.path.join(CONFIG_DIR, "config.json")
# ── Icon drawing ──────────────────────────────────────────────────────────────
COLORS = {
"ok": (60, 200, 80), # green
"pending": (230, 180, 0), # amber
"missing": (210, 40, 40), # red
"error": (160, 40, 200), # purple
"starting": (120, 120, 120), # grey
}
ICON_SIZE = 64
def make_icon(status):
"""Draw a plain colored circle for the system tray."""
color = COLORS.get(status, COLORS["starting"])
img = Image.new("RGBA", (ICON_SIZE, ICON_SIZE), (0, 0, 0, 0))
draw = ImageDraw.Draw(img)
margin = 6
draw.ellipse(
[margin, margin, ICON_SIZE - margin, ICON_SIZE - margin],
fill=color,
)
return img
# ── First-run check ───────────────────────────────────────────────────────────
def ensure_config():
"""
If config.json is missing, launch the first-run wizard.
Returns True if config is ready, False if user cancelled.
"""
if os.path.exists(CONFIG_PATH):
return True
from thor_settings_dialog import show_dialog
saved = show_dialog(CONFIG_PATH, wizard=True)
if not saved:
_show_cancel_message()
return False
return True
def _show_cancel_message():
try:
import tkinter as tk
from tkinter import messagebox
root = tk.Tk()
root.withdraw()
messagebox.showwarning(
"Thor Watcher",
"No configuration was saved.\nThe application will now exit.",
)
root.destroy()
except Exception:
pass
# ── Tray app ──────────────────────────────────────────────────────────────────
class WatcherTray:
def __init__(self):
self.state = {}
self.stop_event = threading.Event()
self._watcher_thread = None
self._icon = None
self._menu_lock = threading.Lock()
# ── Watcher thread management ─────────────────────────────────────────────
def _start_watcher(self):
self.stop_event.clear()
self._watcher_thread = threading.Thread(
target=watcher.run_watcher,
args=(self.state, self.stop_event),
daemon=True,
name="watcher",
)
self._watcher_thread.start()
def _stop_watcher(self):
self.stop_event.set()
if self._watcher_thread is not None:
self._watcher_thread.join(timeout=10)
self._watcher_thread = None
def _restart_watcher(self):
self._stop_watcher()
self.stop_event = threading.Event()
self.state["status"] = "starting"
self.state["units"] = []
self.state["last_scan"] = None
self.state["last_error"] = None
self._start_watcher()
# ── Menu callbacks ────────────────────────────────────────────────────────
def _open_settings(self, icon, item):
def _run():
from thor_settings_dialog import show_dialog
saved = show_dialog(CONFIG_PATH, wizard=False)
if saved:
self._restart_watcher()
threading.Thread(target=_run, daemon=True, name="settings-dialog").start()
def _open_logs(self, icon, item):
log_dir = self.state.get("log_dir")
if not log_dir:
log_dir = HERE
if os.path.exists(log_dir):
subprocess.Popen(["explorer", log_dir])
else:
parent = os.path.dirname(log_dir)
if os.path.exists(parent):
subprocess.Popen(["explorer", parent])
else:
subprocess.Popen(["explorer", HERE])
def _exit(self, icon, item):
self.stop_event.set()
icon.stop()
# ── Dynamic menu text ─────────────────────────────────────────────────────
def _status_text(self):
status = self.state.get("status", "starting")
last_err = self.state.get("last_error")
last_scan = self.state.get("last_scan")
api_status = self.state.get("api_status", "disabled")
unit_count = len(self.state.get("units", []))
if status == "error":
return "Error — {}".format(last_err or "unknown")
if status == "starting":
return "Starting..."
if last_scan is not None:
age_secs = int((datetime.now() - last_scan).total_seconds())
age_str = "{}s ago".format(age_secs) if age_secs < 60 else "{}m ago".format(age_secs // 60)
else:
age_str = "never"
if api_status == "ok":
api_str = "API OK"
elif api_status == "fail":
api_str = "API FAIL"
else:
api_str = "API off"
return "Running — {} | {} unit(s) | scan {}".format(api_str, unit_count, age_str)
def _tray_status(self):
status = self.state.get("status", "starting")
if status == "error":
return "error"
if status == "starting":
return "starting"
api_status = self.state.get("api_status", "disabled")
if api_status == "fail":
return "missing" # red — API failing
if api_status == "disabled":
return "pending" # amber — running but not reporting
return "ok" # green — running and API good
def _build_menu(self):
return pystray.Menu(
pystray.MenuItem(lambda item: self._status_text(), None, enabled=False),
pystray.Menu.SEPARATOR,
pystray.MenuItem("Settings...", self._open_settings),
pystray.MenuItem("Open Log Folder", self._open_logs),
pystray.Menu.SEPARATOR,
pystray.MenuItem("Exit", self._exit),
)
# ── Icon/menu update loop ─────────────────────────────────────────────────
def _icon_updater(self):
"""Periodically refresh the tray icon and check for updates."""
last_status = None
update_check_counter = 0 # check every ~5 min (30 × 10s ticks)
while not self.stop_event.is_set():
icon_status = self._tray_status()
if self._icon is not None:
with self._menu_lock:
self._icon.menu = self._build_menu()
if icon_status != last_status:
self._icon.icon = make_icon(icon_status)
self._icon.title = "Thor Watcher — {}".format(self._status_text())
last_status = icon_status
# Terra-View push-triggered update
if self.state.get("update_available"):
self.state["update_available"] = False
self._do_update()
return
# Periodic update check
update_check_counter += 1
if update_check_counter >= 30:
update_check_counter = 0
tag, url = check_for_update()
if tag and url:
self._do_update(url)
return
self.stop_event.wait(timeout=10)
def _do_update(self, download_url=None):
"""Notify tray then apply update. If url is None, fetch it first."""
if download_url is None:
_, download_url = check_for_update()
if not download_url:
return
if self._icon is not None:
self._icon.title = "Thor Watcher — Updating..."
self._icon.icon = make_icon("starting")
success = apply_update(download_url)
if success:
self.stop_event.set()
if self._icon is not None:
self._icon.stop()
# ── Entry point ───────────────────────────────────────────────────────────
def run(self):
self._start_watcher()
icon_img = make_icon("starting")
self._icon = pystray.Icon(
name="thor_watcher",
icon=icon_img,
title="Thor Watcher — Starting...",
menu=self._build_menu(),
)
updater = threading.Thread(
target=self._icon_updater, daemon=True, name="icon-updater"
)
updater.start()
self._icon.run()
# ── Entry point ───────────────────────────────────────────────────────────────
def main():
if not ensure_config():
sys.exit(0)
app = WatcherTray()
app.run()
if __name__ == "__main__":
main()