From 83593f7b33d1f6d2b85ab364024c69ad4b42186f Mon Sep 17 00:00:00 2001 From: serversdwn Date: Wed, 3 Dec 2025 22:16:56 +0000 Subject: [PATCH] update docs --- CHANGELOG.md | 39 +++++++++++- README.md | 168 +++++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 179 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 858c54f..cfe0043 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,39 @@ All notable changes to Seismo Fleet Manager will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.2.1] - 2025-12-03 + +### Added +- `/settings` roster manager page with CSV export/import, live stats, and danger-zone reset controls. +- `/api/settings` router that exposes `export-csv`, `stats`, `roster-units`, `import-csv-replace`, and the clear-* endpoints backing the UI. +- Dedicated HTMX partials/tabs for deployed, benched, retired, and ignored units plus new ignored-table UI to unignore or delete entries. + +### Changed +- Roster and unit detail templates now display device-type specific metadata (calibration windows, modem pairings, IP/phone fields) alongside inline actions. +- Base navigation highlights the new settings workflow and routes retired/ignored buckets through dedicated endpoints + partials. + +### Fixed +- Snapshot summary counts only consider deployed units, preventing dashboard alerts from including benched hardware. +- Snapshot payloads now include address/coordinate metadata so map widgets and CSV exports stay accurate. + +## [0.2.0] - 2025-12-03 + +### Added +- Device-type aware roster schema (seismographs vs modems) with new metadata columns plus `backend/migrate_add_device_types.py` for upgrading existing SQLite files. +- `create_test_db.py` helper that generates a ready-to-use demo database with sample seismographs, modems, and emitter rows. +- Ignore list persistence/API so noisy legacy emitters can be quarantined via `/api/roster/ignore` and surfaced in the UI. +- Roster page enhancements: Add Unit modal, CSV import modal, and HTMX-powered table fed by `/partials/roster-table`. +- Unit detail view rewritten to fetch data via API, expose deployment status, and allow edits to all metadata. + +### Changed +- Snapshot service now merges roster + emitter data into active/benched/retired/unknown buckets and includes device-specific metadata in each record. +- Roster edit endpoints parse date fields, manage modem/seismograph specific attributes, and guarantee records exist when toggling deployed/retired states. +- Dashboard partial endpoints are grouped under `/dashboard/*` so HTMX tabs stay in sync with the consolidated snapshot payload. + +### Fixed +- Toggling deployed/retired flags no longer fails when a unit does not exist because the router now auto-creates placeholder roster rows. +- CSV import applies address/coordinate updates instead of silently dropping unknown columns. + ## [0.1.1] - 2025-12-02 ### Added @@ -59,5 +92,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Photo management per unit - Automated status categorization (OK/Pending/Missing) -[0.1.1]: https://github.com/yourusername/seismo-fleet-manager/compare/v0.1.0...v0.1.1 -[0.1.0]: https://github.com/yourusername/seismo-fleet-manager/releases/tag/v0.1.0 +[0.2.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.2.0...v0.2.1 +[0.2.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.1...v0.2.0 +[0.1.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.0...v0.1.1 +[0.1.0]: https://github.com/serversdwn/seismo-fleet-manager/releases/tag/v0.1.0 diff --git a/README.md b/README.md index 115d204..5bc60c8 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,31 @@ -# Seismo Fleet Manager v0.1.1 +# Seismo Fleet Manager v0.2.1 -Backend API and web interface for managing seismograph fleet status. Track multiple seismographs calling in data from remote deployments, monitor their status, and manage your fleet through a unified database. +Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard. ## Features -- **Web Dashboard**: Modern, responsive UI with dark/light mode support -- **Fleet Monitoring**: Track all seismograph units in one place -- **Roster Management**: Full CRUD operations via API or CSV import *(New in v0.1.1)* -- **Status Management**: Automatically mark units as OK, Pending (>12h), or Missing (>24h) +- **Web Dashboard**: Modern, responsive UI with dark/light mode, live HTMX updates, and integrated fleet map +- **Fleet Monitoring**: Track deployed, benched, retired, and ignored units in separate buckets with unknown-emitter triage +- **Roster Management**: Full CRUD + CSV import/export, device-type aware metadata, and inline actions from the roster tables +- **Settings & Safeguards**: `/settings` page exposes roster stats, exports, replace-all imports, and danger-zone reset tools +- **Device & Modem Metadata**: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit +- **Status Management**: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry - **Data Ingestion**: Accept reports from emitter scripts via REST API - **Photo Management**: Upload and view photos for each unit - **Interactive Maps**: Leaflet-based maps showing unit locations - **SQLite Storage**: Lightweight, file-based database for easy deployment +## Roster Manager & Settings + +Visit [`/settings`](http://localhost:8001/settings) to perform bulk roster operations with guardrails: + +- **CSV export/import**: Download the entire roster, merge updates, or replace all units in one transaction. +- **Live roster table**: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place. +- **Stats at a glance**: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked. +- **Danger zone controls**: Clear specific tables or wipe all fleet data when resetting a lab/demo environment. + +All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts. + ## Tech Stack - **FastAPI**: Modern, fast web framework @@ -76,12 +89,34 @@ The SQLite database and photos are stored in the `./data` directory, which is mo The application will be available at http://localhost:8001 +### Optional: Generate Sample Data + +Need realistic data quickly? Run: + +```bash +python create_test_db.py +cp /tmp/sfm_test.db data/seismo_fleet.db +``` + +The helper script creates a modem/seismograph mix so you can exercise the dashboard, roster tabs, and unit detail screens immediately. + +## Upgrading from v0.1.x + +Versions ≥0.2 introduce new roster columns (device_type, calibration dates, modem metadata, addresses, etc.). Run the migration once per database file before starting the app: + +```bash +python backend/migrate_add_device_types.py +``` + +The script is idempotent—if the new columns already exist it simply exits. + ## API Endpoints ### Web Pages - **GET** `/` - Dashboard home page - **GET** `/roster` - Fleet roster page - **GET** `/unit/{unit_id}` - Unit detail page +- **GET** `/settings` - Roster manager, CSV import/export, and danger-zone utilities ### Fleet Status & Monitoring - **GET** `/api/status-snapshot` - Complete fleet status snapshot @@ -89,34 +124,61 @@ The application will be available at http://localhost:8001 - **GET** `/api/unit/{unit_id}` - Detailed unit information - **GET** `/health` - Health check endpoint -### Roster Management *(New in v0.1.1)* +### Roster Management - **POST** `/api/roster/add` - Add new unit to roster ```bash curl -X POST http://localhost:8001/api/roster/add \ -F "id=BE1234" \ + -F "device_type=seismograph" \ -F "unit_type=series3" \ + -F "project_id=PROJ-001" \ -F "deployed=true" \ -F "note=Main site sensor" ``` +- **GET** `/api/roster/{unit_id}` - Fetch a single roster entry for editing +- **POST** `/api/roster/edit/{unit_id}` - Update all metadata (device type, calibration dates, modem fields, etc.) - **POST** `/api/roster/set-deployed/{unit_id}` - Toggle deployment status - **POST** `/api/roster/set-retired/{unit_id}` - Toggle retired status - **POST** `/api/roster/set-note/{unit_id}` - Update unit notes -- **POST** `/api/roster/import-csv` - Bulk import from CSV +- **DELETE** `/api/roster/{unit_id}` - Remove a roster/emitter pair entirely +- **POST** `/api/roster/import-csv` - Bulk import from CSV (merge/update mode) ```bash curl -X POST http://localhost:8001/api/roster/import-csv \ -F "file=@roster.csv" \ -F "update_existing=true" ``` +- **POST** `/api/roster/ignore/{unit_id}` - Move an unknown emitter to the ignore list +- **DELETE** `/api/roster/ignore/{unit_id}` - Remove a unit from the ignore list +- **GET** `/api/roster/ignored` - List all ignored units with reasons + +### Settings & Data Management +- **GET** `/api/settings/export-csv` - Download the entire roster as CSV +- **GET** `/api/settings/stats` - Counts for roster, emitters, and ignored tables +- **GET** `/api/settings/roster-units` - Raw roster dump for the settings data grid +- **POST** `/api/settings/import-csv-replace` - Replace the entire roster in one atomic transaction +- **POST** `/api/settings/clear-all` - Danger-zone action that wipes roster, emitters, and ignored tables +- **POST** `/api/settings/clear-roster` - Delete only roster entries +- **POST** `/api/settings/clear-emitters` - Delete auto-discovered emitters +- **POST** `/api/settings/clear-ignored` - Reset ignore list ### CSV Import Format -Create a CSV file with the following columns (only `unit_id` is required): -```csv -unit_id,unit_type,deployed,retired,note,project_id,location -BE1234,series3,true,false,Primary sensor,PROJ-001,San Francisco CA -BE5678,series3,true,false,Backup sensor,PROJ-001,Los Angeles CA +Create a CSV file with the following columns (only `unit_id` is required, everything else is optional): + +``` +unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model ``` -See [sample_roster.csv](sample_roster.csv) for a working example. +Boolean columns accept `true/false`, `1/0`, or `yes/no` (case-insensitive). Date columns expect `YYYY-MM-DD`. Use the same schema whether you merge via `/api/roster/import-csv` or replace everything with `/api/settings/import-csv-replace`. + +#### Example + +```csv +unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model +BE1234,series3,seismograph,true,false,Primary sensor,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",2025-01-15,2026-01-15,MDM001,,, +MDM001,modem,modem,true,false,Field modem,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",,,,"192.0.2.10","+1-555-0100","Raven XTV" +``` + +See [sample_roster.csv](sample_roster.csv) for a minimal working example. ### Emitter Reporting - **POST** `/emitters/report` - Submit status report from a seismograph unit @@ -199,30 +261,59 @@ print(response.json()) ## Data Model -### RosterUnit Table (Your Fleet Roster) +### RosterUnit Table (Fleet Roster) + +**Common fields** | Field | Type | Description | |-------|------|-------------| | id | string | Unit identifier (primary key) | -| unit_type | string | Type of seismograph (default: "series3") | -| deployed | boolean | Whether unit is deployed in field | -| retired | boolean | Whether unit is retired from service | +| unit_type | string | Hardware model name (default: `series3`) | +| device_type | string | `seismograph` or `modem` discriminator | +| deployed | boolean | Whether the unit is in the field | +| retired | boolean | Removes the unit from deployments but preserves history | | note | string | Notes about the unit | | project_id | string | Associated project identifier | -| location | string | Deployment location description | +| location | string | Legacy location label | +| address | string | Human-readable address | +| coordinates | string | `lat,lon` pair used by the map | | last_updated | datetime | Last modification timestamp | +**Seismograph-only fields** + +| Field | Type | Description | +|-------|------|-------------| +| last_calibrated | date | Last calibration date | +| next_calibration_due | date | Next calibration date | +| deployed_with_modem_id | string | Which modem is paired during deployment | + +**Modem-only fields** + +| Field | Type | Description | +|-------|------|-------------| +| ip_address | string | Assigned IP (static or DHCP) | +| phone_number | string | Cellular number for the modem | +| hardware_model | string | Modem hardware reference | + ### Emitter Table (Device Check-ins) | Field | Type | Description | |-------|------|-------------| | id | string | Unit identifier (primary key) | -| unit_type | string | Type of seismograph (e.g., "series3") | +| unit_type | string | Reported device type/model | | last_seen | datetime | Last report timestamp | | last_file | string | Last file processed | | status | string | Current status: OK, Pending, Missing | | notes | string | Optional notes (nullable) | +### IgnoredUnit Table (Noise Management) + +| Field | Type | Description | +|-------|------|-------------| +| id | string | Unit identifier (primary key) | +| reason | string | Optional context for ignoring | +| ignored_at | datetime | When the ignore action occurred | + ## Project Structure ``` @@ -230,7 +321,7 @@ seismo-fleet-manager/ ├── backend/ │ ├── main.py # FastAPI app entry point │ ├── database.py # SQLAlchemy database configuration -│ ├── models.py # Database models (RosterUnit, Emitter) +│ ├── models.py # Database models (RosterUnit, Emitter, IgnoredUnit) │ ├── routes.py # Legacy API endpoints │ ├── routers/ # Modular API routers │ │ ├── roster.py # Fleet status endpoints @@ -238,16 +329,24 @@ seismo-fleet-manager/ │ │ ├── units.py # Unit detail endpoints │ │ ├── photos.py # Photo management │ │ ├── dashboard.py # Dashboard partials -│ │ └── dashboard_tabs.py # Dashboard tab endpoints +│ │ ├── dashboard_tabs.py # Dashboard tab endpoints +│ │ └── settings.py # Roster manager/data operations │ ├── services/ │ │ └── snapshot.py # Fleet status snapshot logic +│ ├── migrate_add_device_types.py # SQLite migration for v0.2 schema │ └── static/ # Static assets (CSS, etc.) +├── create_test_db.py # Generate a sample SQLite DB with mixed devices ├── templates/ # Jinja2 HTML templates │ ├── base.html # Base layout with sidebar │ ├── dashboard.html # Main dashboard │ ├── roster.html # Fleet roster table │ ├── unit_detail.html # Unit detail page +│ ├── settings.html # Roster manager UI │ └── partials/ # HTMX partial templates +│ ├── roster_table.html +│ ├── retired_table.html +│ ├── ignored_table.html +│ └── unknown_emitters.html ├── data/ # SQLite database & photos (persisted) ├── requirements.txt # Python dependencies ├── Dockerfile # Docker container definition @@ -299,15 +398,28 @@ docker compose down docker compose down -v ``` -## What's New in v0.1.1 +## Release Highlights +### v0.2.1 — 2025-12-03 +- Added the `/settings` roster manager with CSV export/import, live stats, and danger-zone table reset actions. +- Deployed/Benched/Retired/Ignored tabs now have dedicated HTMX partials, sorting, and inline actions (edit, deploy toggle, ignore, delete). +- Unit detail pages expose device-type specific metadata (calibration windows, modem pairing, IP/phone fields) with a refreshed editing experience. +- Snapshot summary and dashboard counts now focus on deployed units and include address/coordinate data for mapping widgets. + +### v0.2.0 — 2025-12-03 +- Introduced device-type aware roster schema (seismograph vs modem) plus migration + `create_test_db.py` helper for new installs. +- Added Ignore list model/endpoints to quarantine noisy emitters directly from the roster. +- Roster page gained Add Unit + CSV Import modals, HTMX-driven updates, and unknown emitter callouts. +- Snapshot service now returns active/benched/retired/unknown buckets containing richer metadata for the dashboard and roster tabs. + +### v0.1.1 — 2025-12-02 - **Roster Editing API**: Full CRUD operations for managing your fleet roster - **CSV Import**: Bulk upload roster data from CSV files - **Enhanced Data Model**: Added project_id and location fields to roster - **Bug Fixes**: Improved database session management and error handling - **Dashboard Improvements**: Separate views for Active, Benched, and Retired units -See [CHANGELOG.md](CHANGELOG.md) for complete version history. +See [CHANGELOG.md](CHANGELOG.md) for the full release notes. ## Future Enhancements @@ -325,6 +437,10 @@ MIT ## Version -**Current: 0.1.1** - Roster Management & CSV Import (2025-12-02) +**Current: 0.2.1** — Settings & roster manager refresh (2025-12-03) -Previous: 0.1.0 - Initial Release (2024-11-20) +Previous: 0.2.0 — Device-type aware roster + ignore list (2025-12-03) + +0.1.1 — Roster Management & CSV Import (2025-12-02) + +0.1.0 — Initial Release (2024-11-20)