diff --git a/CHANGELOG.md b/CHANGELOG.md index 1eb567f..99ff90c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,68 @@ All notable changes to Seismo Fleet Manager will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.4.0] - 2025-12-16 + +### Added +- **Database Management System**: Comprehensive backup and restore capabilities + - **Manual Snapshots**: Create on-demand backups of the entire database with optional descriptions + - **Restore from Snapshot**: Restore database from any snapshot with automatic safety backup + - **Upload/Download Snapshots**: Transfer database snapshots to/from the server + - **Database Tab**: New dedicated tab in Settings for all database management operations + - **Database Statistics**: View database size, row counts by table, and last modified time + - **Snapshot Metadata**: Each snapshot includes creation time, description, size, and type (manual/automatic) + - **Safety Backups**: Automatic backup created before any restore operation +- **Remote Database Cloning**: Dev tools for cloning production database to remote development servers + - **Clone Script**: `scripts/clone_db_to_dev.py` for copying database over WAN + - **Network Upload**: Upload snapshots via HTTP to remote servers + - **Auto-restore**: Automatically restore uploaded database on target server + - **Authentication Support**: Optional token-based authentication for secure transfers +- **Automatic Backup Scheduler**: Background service for automated database backups + - **Configurable Intervals**: Set backup frequency (default: 24 hours) + - **Retention Management**: Automatically delete old backups (configurable keep count) + - **Manual Trigger**: Force immediate backup via API + - **Status Monitoring**: Check scheduler status and next scheduled run time + - **Background Thread**: Non-blocking operation using Python threading +- **Settings Reorganization**: Improved tab structure for better organization + - Renamed "Data Management" tab to "Roster Management" + - Moved CSV Replace Mode from Advanced tab to Roster Management tab + - Created dedicated Database tab for all backup/restore operations +- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` guide covering: + - Manual snapshot creation and restoration workflows + - Download/upload procedures for off-site backups + - Remote database cloning setup and usage + - Automatic backup configuration and integration + - API reference for all database endpoints + - Best practices and troubleshooting guide + +### Changed +- **Settings Tab Organization**: Restructured for better logical grouping + - **General**: Display preferences (timezone, theme, auto-refresh) + - **Roster Management**: CSV operations and roster table (now includes Replace Mode) + - **Database**: All backup/restore operations (NEW) + - **Advanced**: Power user settings (calibration, thresholds) + - **Danger Zone**: Destructive operations +- CSV Replace Mode warnings enhanced and moved to Roster Management context + +### Technical Details +- **SQLite Backup API**: Uses native SQLite backup API for concurrent-safe snapshots +- **Metadata Tracking**: JSON sidecar files store snapshot metadata alongside database files +- **Atomic Operations**: Database restoration is atomic with automatic rollback on failure +- **File Structure**: Snapshots stored in `./data/backups/` with timestamped filenames +- **API Endpoints**: 7 new endpoints for database management operations +- **Backup Service**: `backend/services/database_backup.py` - Core backup/restore logic +- **Scheduler Service**: `backend/services/backup_scheduler.py` - Automatic backup automation +- **Clone Utility**: `scripts/clone_db_to_dev.py` - Remote database synchronization tool + +### Security Considerations +- Snapshots contain full database data and should be secured appropriately +- Remote cloning supports optional authentication tokens +- Restore operations require safety backup creation by default +- All destructive operations remain in Danger Zone with warnings + +### Migration Notes +No database migration required for v0.4.0. All new features use existing database structure and add new backup management capabilities without modifying the core schema. + ## [0.3.3] - 2025-12-12 ### Changed @@ -231,6 +293,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Photo management per unit - Automated status categorization (OK/Pending/Missing) +[0.4.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.3...v0.4.0 [0.3.3]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.2...v0.3.3 [0.3.2]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.1...v0.3.2 [0.3.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.0...v0.3.1 diff --git a/README.md b/README.md index e34142d..3451713 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Seismo Fleet Manager v0.3.3 +# Seismo Fleet Manager v0.4.0 Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard. ## Features @@ -19,6 +19,12 @@ Backend API and HTMX-powered web interface for managing a mixed fleet of seismog - **Photo Management**: Upload and view photos for each unit - **Interactive Maps**: Leaflet-based maps showing unit locations with tap-to-navigate for mobile - **SQLite Storage**: Lightweight, file-based database for easy deployment +- **Database Management**: Comprehensive backup and restore system + - **Manual Snapshots**: Create on-demand backups with descriptions + - **Restore from Snapshot**: Restore database with automatic safety backups + - **Upload/Download**: Transfer database snapshots for off-site storage + - **Remote Cloning**: Copy production database to remote dev servers over WAN + - **Automatic Backups**: Scheduled background backups with configurable retention ## Roster Manager & Settings @@ -26,10 +32,12 @@ Visit [`/settings`](http://localhost:8001/settings) to perform bulk roster opera - **CSV export/import**: Download the entire roster, merge updates, or replace all units in one transaction. - **Live roster table**: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place. +- **Database backups**: Create snapshots, restore from backups, upload/download database files, view database statistics. +- **Remote cloning**: Clone production database to remote development servers over the network (see `scripts/clone_db_to_dev.py`). - **Stats at a glance**: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked. - **Danger zone controls**: Clear specific tables or wipe all fleet data when resetting a lab/demo environment. -All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts. +All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts. See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for comprehensive database backup and restore documentation. ## Tech Stack @@ -180,6 +188,17 @@ Both migration scripts are idempotent—if the columns/tables already exist, the - **POST** `/api/settings/clear-emitters` - Delete auto-discovered emitters - **POST** `/api/settings/clear-ignored` - Reset ignore list +### Database Management +- **GET** `/api/settings/database/stats` - Database size, row counts, and last modified time +- **POST** `/api/settings/database/snapshot` - Create manual database snapshot with optional description +- **GET** `/api/settings/database/snapshots` - List all available snapshots with metadata +- **GET** `/api/settings/database/snapshot/{filename}` - Download a specific snapshot file +- **DELETE** `/api/settings/database/snapshot/{filename}` - Delete a snapshot +- **POST** `/api/settings/database/restore` - Restore database from snapshot (creates safety backup) +- **POST** `/api/settings/database/upload-snapshot` - Upload snapshot file to server + +See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for detailed documentation and examples. + ### CSV Import Format Create a CSV file with the following columns (only `unit_id` is required, everything else is optional): @@ -368,7 +387,9 @@ seismo-fleet-manager/ │ │ ├── dashboard_tabs.py # Dashboard tab endpoints │ │ └── settings.py # Settings, preferences, and data management │ ├── services/ -│ │ └── snapshot.py # Fleet status snapshot logic +│ │ ├── snapshot.py # Fleet status snapshot logic +│ │ ├── database_backup.py # Database backup and restore service +│ │ └── backup_scheduler.py # Automatic backup scheduler │ ├── migrate_add_device_types.py # SQLite migration for v0.2 schema │ ├── migrate_add_user_preferences.py # SQLite migration for v0.3 schema │ └── static/ # Static assets (CSS, etc.) @@ -385,6 +406,11 @@ seismo-fleet-manager/ │ ├── ignored_table.html │ └── unknown_emitters.html ├── data/ # SQLite database & photos (persisted) +│ └── backups/ # Database snapshots directory +├── scripts/ +│ └── clone_db_to_dev.py # Remote database cloning utility +├── docs/ +│ └── DATABASE_MANAGEMENT.md # Database backup/restore guide ├── requirements.txt # Python dependencies ├── Dockerfile # Docker container definition ├── docker-compose.yml # Docker Compose configuration @@ -437,6 +463,14 @@ docker compose down -v ## Release Highlights +### v0.4.0 — 2025-12-16 +- **Database Management System**: Complete backup and restore functionality with manual snapshots, restore operations, and upload/download capabilities +- **Remote Database Cloning**: New `clone_db_to_dev.py` script for copying production database to remote dev servers over WAN +- **Automatic Backup Scheduler**: Background service for scheduled backups with configurable retention management +- **Database Tab**: New dedicated tab in Settings for all database operations with real-time statistics +- **Settings Reorganization**: Improved tab structure - renamed "Data Management" to "Roster Management", moved CSV Replace Mode, created Database tab +- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` with complete guide to backup/restore workflows, API reference, and best practices + ### v0.3.3 — 2025-12-12 - **Improved Mobile Navigation**: Hamburger menu moved to bottom nav bar (no more floating button covering content) - **Better Status Visibility**: Larger status dots (16px) in dashboard fleet overview for easier at-a-glance status checks @@ -491,7 +525,6 @@ See [CHANGELOG.md](CHANGELOG.md) for the full release notes. - PostgreSQL support for larger deployments - Advanced filtering and search - Export roster to various formats -- Automated backup and restore ## License @@ -499,9 +532,11 @@ MIT ## Version -**Current: 0.3.3** — Mobile navigation improvements and better status visibility (2025-12-12) +**Current: 0.4.0** — Database management system with backup/restore and remote cloning (2025-12-16) -Previous: 0.3.2 — Progressive Web App with mobile optimization (2025-12-12) +Previous: 0.3.3 — Mobile navigation improvements and better status visibility (2025-12-12) + +0.3.2 — Progressive Web App with mobile optimization (2025-12-12) 0.3.1 — Dashboard alerts and status fixes (2025-12-12) diff --git a/backend/main.py b/backend/main.py index 1c95686..575f7dc 100644 --- a/backend/main.py +++ b/backend/main.py @@ -20,7 +20,7 @@ Base.metadata.create_all(bind=engine) ENVIRONMENT = os.getenv("ENVIRONMENT", "production") # Initialize FastAPI app -VERSION = "0.3.3" +VERSION = "0.4.0" app = FastAPI( title="Seismo Fleet Manager", description="Backend API for managing seismograph fleet status", diff --git a/templates/base.html b/templates/base.html index a741902..90ce252 100644 --- a/templates/base.html +++ b/templates/base.html @@ -360,10 +360,10 @@ - + - + {% block extra_scripts %}{% endblock %}