serversdown/terra-view
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects Releases Wiki Activity

v0.4.0 - merge from claude/dev-015sto5mf2MpPCE57TbNKtaF #1

Merged
serversdown merged 32 commits from claude/dev-015sto5mf2MpPCE57TbNKtaF into main 2026-01-02 16:10:54 -05:00
Conversation 0 Commits 32 Files Changed 74 +179 -28
2 changed files with 179 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 83593f7b33 - Show all commits

39
CHANGELOG.md
View File

@@ -5,6 +5,39 @@ All notable changes to Seismo Fleet Manager will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.1] - 2025-12-03
### Added
- `/settings` roster manager page with CSV export/import, live stats, and danger-zone reset controls.
- `/api/settings` router that exposes `export-csv`, `stats`, `roster-units`, `import-csv-replace`, and the clear-* endpoints backing the UI.
- Dedicated HTMX partials/tabs for deployed, benched, retired, and ignored units plus new ignored-table UI to unignore or delete entries.
### Changed
- Roster and unit detail templates now display device-type specific metadata (calibration windows, modem pairings, IP/phone fields) alongside inline actions.
- Base navigation highlights the new settings workflow and routes retired/ignored buckets through dedicated endpoints + partials.
### Fixed
- Snapshot summary counts only consider deployed units, preventing dashboard alerts from including benched hardware.
- Snapshot payloads now include address/coordinate metadata so map widgets and CSV exports stay accurate.
## [0.2.0] - 2025-12-03
### Added
- Device-type aware roster schema (seismographs vs modems) with new metadata columns plus `backend/migrate_add_device_types.py` for upgrading existing SQLite files.
- `create_test_db.py` helper that generates a ready-to-use demo database with sample seismographs, modems, and emitter rows.
- Ignore list persistence/API so noisy legacy emitters can be quarantined via `/api/roster/ignore` and surfaced in the UI.
- Roster page enhancements: Add Unit modal, CSV import modal, and HTMX-powered table fed by `/partials/roster-table`.
- Unit detail view rewritten to fetch data via API, expose deployment status, and allow edits to all metadata.
### Changed
- Snapshot service now merges roster + emitter data into active/benched/retired/unknown buckets and includes device-specific metadata in each record.
- Roster edit endpoints parse date fields, manage modem/seismograph specific attributes, and guarantee records exist when toggling deployed/retired states.
- Dashboard partial endpoints are grouped under `/dashboard/*` so HTMX tabs stay in sync with the consolidated snapshot payload.
### Fixed
- Toggling deployed/retired flags no longer fails when a unit does not exist because the router now auto-creates placeholder roster rows.
- CSV import applies address/coordinate updates instead of silently dropping unknown columns.
## [0.1.1] - 2025-12-02 ## [0.1.1] - 2025-12-02
### Added ### Added
@@ -59,5 +92,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Photo management per unit - Photo management per unit
- Automated status categorization (OK/Pending/Missing) - Automated status categorization (OK/Pending/Missing)
[0.1.1]: https://github.com/yourusername/seismo-fleet-manager/compare/v0.1.0...v0.1.1 [0.2.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.2.0...v0.2.1
[0.1.0]: https://github.com/yourusername/seismo-fleet-manager/releases/tag/v0.1.0 [0.2.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.1...v0.2.0
[0.1.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/serversdwn/seismo-fleet-manager/releases/tag/v0.1.0

168
README.md
View File

@@ -1,18 +1,31 @@
# Seismo Fleet Manager v0.1.1 # Seismo Fleet Manager v0.2.1
Backend API and web interface for managing seismograph fleet status. Track multiple seismographs calling in data from remote deployments, monitor their status, and manage your fleet through a unified database. Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.
## Features ## Features
- **Web Dashboard**: Modern, responsive UI with dark/light mode support - **Web Dashboard**: Modern, responsive UI with dark/light mode, live HTMX updates, and integrated fleet map
- **Fleet Monitoring**: Track all seismograph units in one place - **Fleet Monitoring**: Track deployed, benched, retired, and ignored units in separate buckets with unknown-emitter triage
- **Roster Management**: Full CRUD operations via API or CSV import *(New in v0.1.1)* - **Roster Management**: Full CRUD + CSV import/export, device-type aware metadata, and inline actions from the roster tables
- **Status Management**: Automatically mark units as OK, Pending (>12h), or Missing (>24h) - **Settings & Safeguards**: `/settings` page exposes roster stats, exports, replace-all imports, and danger-zone reset tools
- **Device & Modem Metadata**: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit
- **Status Management**: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry
- **Data Ingestion**: Accept reports from emitter scripts via REST API - **Data Ingestion**: Accept reports from emitter scripts via REST API
- **Photo Management**: Upload and view photos for each unit - **Photo Management**: Upload and view photos for each unit
- **Interactive Maps**: Leaflet-based maps showing unit locations - **Interactive Maps**: Leaflet-based maps showing unit locations
- **SQLite Storage**: Lightweight, file-based database for easy deployment - **SQLite Storage**: Lightweight, file-based database for easy deployment
## Roster Manager & Settings
Visit [`/settings`](http://localhost:8001/settings) to perform bulk roster operations with guardrails:
- **CSV export/import**: Download the entire roster, merge updates, or replace all units in one transaction.
- **Live roster table**: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place.
- **Stats at a glance**: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked.
- **Danger zone controls**: Clear specific tables or wipe all fleet data when resetting a lab/demo environment.
All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts.
## Tech Stack ## Tech Stack
- **FastAPI**: Modern, fast web framework - **FastAPI**: Modern, fast web framework
@@ -76,12 +89,34 @@ The SQLite database and photos are stored in the `./data` directory, which is mo
The application will be available at http://localhost:8001 The application will be available at http://localhost:8001
### Optional: Generate Sample Data
Need realistic data quickly? Run:
```bash
python create_test_db.py
cp /tmp/sfm_test.db data/seismo_fleet.db
```
The helper script creates a modem/seismograph mix so you can exercise the dashboard, roster tabs, and unit detail screens immediately.
## Upgrading from v0.1.x
Versions ≥0.2 introduce new roster columns (device_type, calibration dates, modem metadata, addresses, etc.). Run the migration once per database file before starting the app:
```bash
python backend/migrate_add_device_types.py
```
The script is idempotent—if the new columns already exist it simply exits.
## API Endpoints ## API Endpoints
### Web Pages ### Web Pages
- **GET** `/` - Dashboard home page - **GET** `/` - Dashboard home page
- **GET** `/roster` - Fleet roster page - **GET** `/roster` - Fleet roster page
- **GET** `/unit/{unit_id}` - Unit detail page - **GET** `/unit/{unit_id}` - Unit detail page
- **GET** `/settings` - Roster manager, CSV import/export, and danger-zone utilities
### Fleet Status & Monitoring ### Fleet Status & Monitoring
- **GET** `/api/status-snapshot` - Complete fleet status snapshot - **GET** `/api/status-snapshot` - Complete fleet status snapshot
@@ -89,34 +124,61 @@ The application will be available at http://localhost:8001
- **GET** `/api/unit/{unit_id}` - Detailed unit information - **GET** `/api/unit/{unit_id}` - Detailed unit information
- **GET** `/health` - Health check endpoint - **GET** `/health` - Health check endpoint
### Roster Management *(New in v0.1.1)* ### Roster Management
- **POST** `/api/roster/add` - Add new unit to roster - **POST** `/api/roster/add` - Add new unit to roster
```bash ```bash
curl -X POST http://localhost:8001/api/roster/add \ curl -X POST http://localhost:8001/api/roster/add \
-F "id=BE1234" \ -F "id=BE1234" \
-F "device_type=seismograph" \
-F "unit_type=series3" \ -F "unit_type=series3" \
-F "project_id=PROJ-001" \
-F "deployed=true" \ -F "deployed=true" \
-F "note=Main site sensor" -F "note=Main site sensor"
``` ```
- **GET** `/api/roster/{unit_id}` - Fetch a single roster entry for editing
- **POST** `/api/roster/edit/{unit_id}` - Update all metadata (device type, calibration dates, modem fields, etc.)
- **POST** `/api/roster/set-deployed/{unit_id}` - Toggle deployment status - **POST** `/api/roster/set-deployed/{unit_id}` - Toggle deployment status
- **POST** `/api/roster/set-retired/{unit_id}` - Toggle retired status - **POST** `/api/roster/set-retired/{unit_id}` - Toggle retired status
- **POST** `/api/roster/set-note/{unit_id}` - Update unit notes - **POST** `/api/roster/set-note/{unit_id}` - Update unit notes
- **POST** `/api/roster/import-csv` - Bulk import from CSV - **DELETE** `/api/roster/{unit_id}` - Remove a roster/emitter pair entirely
- **POST** `/api/roster/import-csv` - Bulk import from CSV (merge/update mode)
```bash ```bash
curl -X POST http://localhost:8001/api/roster/import-csv \ curl -X POST http://localhost:8001/api/roster/import-csv \
-F "file=@roster.csv" \ -F "file=@roster.csv" \
-F "update_existing=true" -F "update_existing=true"
``` ```
- **POST** `/api/roster/ignore/{unit_id}` - Move an unknown emitter to the ignore list
- **DELETE** `/api/roster/ignore/{unit_id}` - Remove a unit from the ignore list
- **GET** `/api/roster/ignored` - List all ignored units with reasons
### Settings & Data Management
- **GET** `/api/settings/export-csv` - Download the entire roster as CSV
- **GET** `/api/settings/stats` - Counts for roster, emitters, and ignored tables
- **GET** `/api/settings/roster-units` - Raw roster dump for the settings data grid
- **POST** `/api/settings/import-csv-replace` - Replace the entire roster in one atomic transaction
- **POST** `/api/settings/clear-all` - Danger-zone action that wipes roster, emitters, and ignored tables
- **POST** `/api/settings/clear-roster` - Delete only roster entries
- **POST** `/api/settings/clear-emitters` - Delete auto-discovered emitters
- **POST** `/api/settings/clear-ignored` - Reset ignore list
### CSV Import Format ### CSV Import Format
Create a CSV file with the following columns (only `unit_id` is required): Create a CSV file with the following columns (only `unit_id` is required, everything else is optional):
```csv
unit_id,unit_type,deployed,retired,note,project_id,location ```
BE1234,series3,true,false,Primary sensor,PROJ-001,San Francisco CA unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
BE5678,series3,true,false,Backup sensor,PROJ-001,Los Angeles CA
``` ```
See [sample_roster.csv](sample_roster.csv) for a working example. Boolean columns accept `true/false`, `1/0`, or `yes/no` (case-insensitive). Date columns expect `YYYY-MM-DD`. Use the same schema whether you merge via `/api/roster/import-csv` or replace everything with `/api/settings/import-csv-replace`.
#### Example
```csv
unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
BE1234,series3,seismograph,true,false,Primary sensor,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",2025-01-15,2026-01-15,MDM001,,,
MDM001,modem,modem,true,false,Field modem,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",,,,"192.0.2.10","+1-555-0100","Raven XTV"
```
See [sample_roster.csv](sample_roster.csv) for a minimal working example.
### Emitter Reporting ### Emitter Reporting
- **POST** `/emitters/report` - Submit status report from a seismograph unit - **POST** `/emitters/report` - Submit status report from a seismograph unit
@@ -199,30 +261,59 @@ print(response.json())
## Data Model ## Data Model
### RosterUnit Table (Your Fleet Roster) ### RosterUnit Table (Fleet Roster)
**Common fields**
| Field | Type | Description | | Field | Type | Description |
|-------|------|-------------| |-------|------|-------------|
| id | string | Unit identifier (primary key) | | id | string | Unit identifier (primary key) |
| unit_type | string | Type of seismograph (default: "series3") | | unit_type | string | Hardware model name (default: `series3`) |
| deployed | boolean | Whether unit is deployed in field | | device_type | string | `seismograph` or `modem` discriminator |
| retired | boolean | Whether unit is retired from service | | deployed | boolean | Whether the unit is in the field |
| retired | boolean | Removes the unit from deployments but preserves history |
| note | string | Notes about the unit | | note | string | Notes about the unit |
| project_id | string | Associated project identifier | | project_id | string | Associated project identifier |
| location | string | Deployment location description | | location | string | Legacy location label |
| address | string | Human-readable address |
| coordinates | string | `lat,lon` pair used by the map |
| last_updated | datetime | Last modification timestamp | | last_updated | datetime | Last modification timestamp |
**Seismograph-only fields**
| Field | Type | Description |
|-------|------|-------------|
| last_calibrated | date | Last calibration date |
| next_calibration_due | date | Next calibration date |
| deployed_with_modem_id | string | Which modem is paired during deployment |
**Modem-only fields**
| Field | Type | Description |
|-------|------|-------------|
| ip_address | string | Assigned IP (static or DHCP) |
| phone_number | string | Cellular number for the modem |
| hardware_model | string | Modem hardware reference |
### Emitter Table (Device Check-ins) ### Emitter Table (Device Check-ins)
| Field | Type | Description | | Field | Type | Description |
|-------|------|-------------| |-------|------|-------------|
| id | string | Unit identifier (primary key) | | id | string | Unit identifier (primary key) |
| unit_type | string | Type of seismograph (e.g., "series3") | | unit_type | string | Reported device type/model |
| last_seen | datetime | Last report timestamp | | last_seen | datetime | Last report timestamp |
| last_file | string | Last file processed | | last_file | string | Last file processed |
| status | string | Current status: OK, Pending, Missing | | status | string | Current status: OK, Pending, Missing |
| notes | string | Optional notes (nullable) | | notes | string | Optional notes (nullable) |
### IgnoredUnit Table (Noise Management)
| Field | Type | Description |
|-------|------|-------------|
| id | string | Unit identifier (primary key) |
| reason | string | Optional context for ignoring |
| ignored_at | datetime | When the ignore action occurred |
## Project Structure ## Project Structure
``` ```
@@ -230,7 +321,7 @@ seismo-fleet-manager/
├── backend/ ├── backend/
│ ├── main.py # FastAPI app entry point │ ├── main.py # FastAPI app entry point
│ ├── database.py # SQLAlchemy database configuration │ ├── database.py # SQLAlchemy database configuration
│ ├── models.py # Database models (RosterUnit, Emitter) │ ├── models.py # Database models (RosterUnit, Emitter, IgnoredUnit)
│ ├── routes.py # Legacy API endpoints │ ├── routes.py # Legacy API endpoints
│ ├── routers/ # Modular API routers │ ├── routers/ # Modular API routers
│ │ ├── roster.py # Fleet status endpoints │ │ ├── roster.py # Fleet status endpoints
@@ -238,16 +329,24 @@ seismo-fleet-manager/
│ │ ├── units.py # Unit detail endpoints │ │ ├── units.py # Unit detail endpoints
│ │ ├── photos.py # Photo management │ │ ├── photos.py # Photo management
│ │ ├── dashboard.py # Dashboard partials │ │ ├── dashboard.py # Dashboard partials
│ │ └── dashboard_tabs.py # Dashboard tab endpoints │ │ ├── dashboard_tabs.py # Dashboard tab endpoints
│ │ └── settings.py # Roster manager/data operations
│ ├── services/ │ ├── services/
│ │ └── snapshot.py # Fleet status snapshot logic │ │ └── snapshot.py # Fleet status snapshot logic
│ ├── migrate_add_device_types.py # SQLite migration for v0.2 schema
│ └── static/ # Static assets (CSS, etc.) │ └── static/ # Static assets (CSS, etc.)
├── create_test_db.py # Generate a sample SQLite DB with mixed devices
├── templates/ # Jinja2 HTML templates ├── templates/ # Jinja2 HTML templates
│ ├── base.html # Base layout with sidebar │ ├── base.html # Base layout with sidebar
│ ├── dashboard.html # Main dashboard │ ├── dashboard.html # Main dashboard
│ ├── roster.html # Fleet roster table │ ├── roster.html # Fleet roster table
│ ├── unit_detail.html # Unit detail page │ ├── unit_detail.html # Unit detail page
│ ├── settings.html # Roster manager UI
│ └── partials/ # HTMX partial templates │ └── partials/ # HTMX partial templates
│ ├── roster_table.html
│ ├── retired_table.html
│ ├── ignored_table.html
│ └── unknown_emitters.html
├── data/ # SQLite database & photos (persisted) ├── data/ # SQLite database & photos (persisted)
├── requirements.txt # Python dependencies ├── requirements.txt # Python dependencies
├── Dockerfile # Docker container definition ├── Dockerfile # Docker container definition
@@ -299,15 +398,28 @@ docker compose down
docker compose down -v docker compose down -v
``` ```
## What's New in v0.1.1 ## Release Highlights
### v0.2.1 — 2025-12-03
- Added the `/settings` roster manager with CSV export/import, live stats, and danger-zone table reset actions.
- Deployed/Benched/Retired/Ignored tabs now have dedicated HTMX partials, sorting, and inline actions (edit, deploy toggle, ignore, delete).
- Unit detail pages expose device-type specific metadata (calibration windows, modem pairing, IP/phone fields) with a refreshed editing experience.
- Snapshot summary and dashboard counts now focus on deployed units and include address/coordinate data for mapping widgets.
### v0.2.0 — 2025-12-03
- Introduced device-type aware roster schema (seismograph vs modem) plus migration + `create_test_db.py` helper for new installs.
- Added Ignore list model/endpoints to quarantine noisy emitters directly from the roster.
- Roster page gained Add Unit + CSV Import modals, HTMX-driven updates, and unknown emitter callouts.
- Snapshot service now returns active/benched/retired/unknown buckets containing richer metadata for the dashboard and roster tabs.
### v0.1.1 — 2025-12-02
- **Roster Editing API**: Full CRUD operations for managing your fleet roster - **Roster Editing API**: Full CRUD operations for managing your fleet roster
- **CSV Import**: Bulk upload roster data from CSV files - **CSV Import**: Bulk upload roster data from CSV files
- **Enhanced Data Model**: Added project_id and location fields to roster - **Enhanced Data Model**: Added project_id and location fields to roster
- **Bug Fixes**: Improved database session management and error handling - **Bug Fixes**: Improved database session management and error handling
- **Dashboard Improvements**: Separate views for Active, Benched, and Retired units - **Dashboard Improvements**: Separate views for Active, Benched, and Retired units
See [CHANGELOG.md](CHANGELOG.md) for complete version history. See [CHANGELOG.md](CHANGELOG.md) for the full release notes.
## Future Enhancements ## Future Enhancements
@@ -325,6 +437,10 @@ MIT
## Version ## Version
**Current: 0.1.1** - Roster Management & CSV Import (2025-12-02) **Current: 0.2.1** — Settings & roster manager refresh (2025-12-03)
Previous: 0.1.0 - Initial Release (2024-11-20) Previous: 0.2.0 — Device-type aware roster + ignore list (2025-12-03)
0.1.1 — Roster Management & CSV Import (2025-12-02)
0.1.0 — Initial Release (2024-11-20)