docs updated to v0.4.0

CHANGELOG.md

@@ -5,6 +5,68 @@ All notable changes to Seismo Fleet Manager will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.0] - 2025-12-16
+
+### Added
+- **Database Management System**: Comprehensive backup and restore capabilities
+  - **Manual Snapshots**: Create on-demand backups of the entire database with optional descriptions
+  - **Restore from Snapshot**: Restore database from any snapshot with automatic safety backup
+  - **Upload/Download Snapshots**: Transfer database snapshots to/from the server
+  - **Database Tab**: New dedicated tab in Settings for all database management operations
+  - **Database Statistics**: View database size, row counts by table, and last modified time
+  - **Snapshot Metadata**: Each snapshot includes creation time, description, size, and type (manual/automatic)
+  - **Safety Backups**: Automatic backup created before any restore operation
+- **Remote Database Cloning**: Dev tools for cloning production database to remote development servers
+  - **Clone Script**: `scripts/clone_db_to_dev.py` for copying database over WAN
+  - **Network Upload**: Upload snapshots via HTTP to remote servers
+  - **Auto-restore**: Automatically restore uploaded database on target server
+  - **Authentication Support**: Optional token-based authentication for secure transfers
+- **Automatic Backup Scheduler**: Background service for automated database backups
+  - **Configurable Intervals**: Set backup frequency (default: 24 hours)
+  - **Retention Management**: Automatically delete old backups (configurable keep count)
+  - **Manual Trigger**: Force immediate backup via API
+  - **Status Monitoring**: Check scheduler status and next scheduled run time
+  - **Background Thread**: Non-blocking operation using Python threading
+- **Settings Reorganization**: Improved tab structure for better organization
+  - Renamed "Data Management" tab to "Roster Management"
+  - Moved CSV Replace Mode from Advanced tab to Roster Management tab
+  - Created dedicated Database tab for all backup/restore operations
+- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` guide covering:
+  - Manual snapshot creation and restoration workflows
+  - Download/upload procedures for off-site backups
+  - Remote database cloning setup and usage
+  - Automatic backup configuration and integration
+  - API reference for all database endpoints
+  - Best practices and troubleshooting guide
+
+### Changed
+- **Settings Tab Organization**: Restructured for better logical grouping
+  - **General**: Display preferences (timezone, theme, auto-refresh)
+  - **Roster Management**: CSV operations and roster table (now includes Replace Mode)
+  - **Database**: All backup/restore operations (NEW)
+  - **Advanced**: Power user settings (calibration, thresholds)
+  - **Danger Zone**: Destructive operations
+- CSV Replace Mode warnings enhanced and moved to Roster Management context
+
+### Technical Details
+- **SQLite Backup API**: Uses native SQLite backup API for concurrent-safe snapshots
+- **Metadata Tracking**: JSON sidecar files store snapshot metadata alongside database files
+- **Atomic Operations**: Database restoration is atomic with automatic rollback on failure
+- **File Structure**: Snapshots stored in `./data/backups/` with timestamped filenames
+- **API Endpoints**: 7 new endpoints for database management operations
+- **Backup Service**: `backend/services/database_backup.py` - Core backup/restore logic
+- **Scheduler Service**: `backend/services/backup_scheduler.py` - Automatic backup automation
+- **Clone Utility**: `scripts/clone_db_to_dev.py` - Remote database synchronization tool
+
+### Security Considerations
+- Snapshots contain full database data and should be secured appropriately
+- Remote cloning supports optional authentication tokens
+- Restore operations require safety backup creation by default
+- All destructive operations remain in Danger Zone with warnings
+
+### Migration Notes
+No database migration required for v0.4.0. All new features use existing database structure and add new backup management capabilities without modifying the core schema.
+
 ## [0.3.3] - 2025-12-12
 
 ### Changed
@@ -231,6 +293,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Photo management per unit
 - Automated status categorization (OK/Pending/Missing)
 
+[0.4.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.3...v0.4.0
 [0.3.3]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.2...v0.3.3
 [0.3.2]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.1...v0.3.2
 [0.3.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.0...v0.3.1

README.md

@@ -1,4 +1,4 @@
-# Seismo Fleet Manager v0.3.3
+# Seismo Fleet Manager v0.4.0
 Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.
 
 ## Features
@@ -19,6 +19,12 @@ Backend API and HTMX-powered web interface for managing a mixed fleet of seismog
 - **Photo Management**: Upload and view photos for each unit
 - **Interactive Maps**: Leaflet-based maps showing unit locations with tap-to-navigate for mobile
 - **SQLite Storage**: Lightweight, file-based database for easy deployment
+- **Database Management**: Comprehensive backup and restore system
+  - **Manual Snapshots**: Create on-demand backups with descriptions
+  - **Restore from Snapshot**: Restore database with automatic safety backups
+  - **Upload/Download**: Transfer database snapshots for off-site storage
+  - **Remote Cloning**: Copy production database to remote dev servers over WAN
+  - **Automatic Backups**: Scheduled background backups with configurable retention
 
 ## Roster Manager & Settings
 
@@ -26,10 +32,12 @@ Visit [`/settings`](http://localhost:8001/settings) to perform bulk roster opera
 
 - **CSV export/import**: Download the entire roster, merge updates, or replace all units in one transaction.
 - **Live roster table**: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place.
+- **Database backups**: Create snapshots, restore from backups, upload/download database files, view database statistics.
+- **Remote cloning**: Clone production database to remote development servers over the network (see `scripts/clone_db_to_dev.py`).
 - **Stats at a glance**: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked.
 - **Danger zone controls**: Clear specific tables or wipe all fleet data when resetting a lab/demo environment.
 
-All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts.
+All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts. See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for comprehensive database backup and restore documentation.
 
 ## Tech Stack
 
@@ -180,6 +188,17 @@ Both migration scripts are idempotent—if the columns/tables already exist, the
 - **POST** `/api/settings/clear-emitters` - Delete auto-discovered emitters
 - **POST** `/api/settings/clear-ignored` - Reset ignore list
 
+### Database Management
+- **GET** `/api/settings/database/stats` - Database size, row counts, and last modified time
+- **POST** `/api/settings/database/snapshot` - Create manual database snapshot with optional description
+- **GET** `/api/settings/database/snapshots` - List all available snapshots with metadata
+- **GET** `/api/settings/database/snapshot/{filename}` - Download a specific snapshot file
+- **DELETE** `/api/settings/database/snapshot/{filename}` - Delete a snapshot
+- **POST** `/api/settings/database/restore` - Restore database from snapshot (creates safety backup)
+- **POST** `/api/settings/database/upload-snapshot` - Upload snapshot file to server
+
+See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for detailed documentation and examples.
+
 ### CSV Import Format
 Create a CSV file with the following columns (only `unit_id` is required, everything else is optional):
 
@@ -368,7 +387,9 @@ seismo-fleet-manager/
 │   │   ├── dashboard_tabs.py # Dashboard tab endpoints
 │   │   └── settings.py      # Settings, preferences, and data management
 │   ├── services/
-│   │   └── snapshot.py      # Fleet status snapshot logic
+│   │   ├── snapshot.py      # Fleet status snapshot logic
+│   │   ├── database_backup.py # Database backup and restore service
+│   │   └── backup_scheduler.py # Automatic backup scheduler
 │   ├── migrate_add_device_types.py # SQLite migration for v0.2 schema
 │   ├── migrate_add_user_preferences.py # SQLite migration for v0.3 schema
 │   └── static/              # Static assets (CSS, etc.)
@@ -385,6 +406,11 @@ seismo-fleet-manager/
 │       ├── ignored_table.html
 │       └── unknown_emitters.html
 ├── data/                    # SQLite database & photos (persisted)
+│   └── backups/             # Database snapshots directory
+├── scripts/
+│   └── clone_db_to_dev.py   # Remote database cloning utility
+├── docs/
+│   └── DATABASE_MANAGEMENT.md # Database backup/restore guide
 ├── requirements.txt         # Python dependencies
 ├── Dockerfile               # Docker container definition
 ├── docker-compose.yml       # Docker Compose configuration
@@ -437,6 +463,14 @@ docker compose down -v
 
 ## Release Highlights
 
+### v0.4.0 — 2025-12-16
+- **Database Management System**: Complete backup and restore functionality with manual snapshots, restore operations, and upload/download capabilities
+- **Remote Database Cloning**: New `clone_db_to_dev.py` script for copying production database to remote dev servers over WAN
+- **Automatic Backup Scheduler**: Background service for scheduled backups with configurable retention management
+- **Database Tab**: New dedicated tab in Settings for all database operations with real-time statistics
+- **Settings Reorganization**: Improved tab structure - renamed "Data Management" to "Roster Management", moved CSV Replace Mode, created Database tab
+- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` with complete guide to backup/restore workflows, API reference, and best practices
+
 ### v0.3.3 — 2025-12-12
 - **Improved Mobile Navigation**: Hamburger menu moved to bottom nav bar (no more floating button covering content)
 - **Better Status Visibility**: Larger status dots (16px) in dashboard fleet overview for easier at-a-glance status checks
@@ -491,7 +525,6 @@ See [CHANGELOG.md](CHANGELOG.md) for the full release notes.
 - PostgreSQL support for larger deployments
 - Advanced filtering and search
 - Export roster to various formats
-- Automated backup and restore
 
 ## License
 
@@ -499,9 +532,11 @@ MIT
 
 ## Version
 
-**Current: 0.3.3** — Mobile navigation improvements and better status visibility (2025-12-12)
+**Current: 0.4.0** — Database management system with backup/restore and remote cloning (2025-12-16)
 
-Previous: 0.3.2 — Progressive Web App with mobile optimization (2025-12-12)
+Previous: 0.3.3 — Mobile navigation improvements and better status visibility (2025-12-12)
+
+0.3.2 — Progressive Web App with mobile optimization (2025-12-12)
 
 0.3.1 — Dashboard alerts and status fixes (2025-12-12)
 

backend/main.py

@@ -20,7 +20,7 @@ Base.metadata.create_all(bind=engine)
 ENVIRONMENT = os.getenv("ENVIRONMENT", "production")
 
 # Initialize FastAPI app
-VERSION = "0.3.3"
+VERSION = "0.4.0"
 app = FastAPI(
     title="Seismo Fleet Manager",
     description="Backend API for managing seismograph fleet status",

templates/base.html

@@ -360,10 +360,10 @@
     </script>
 
     <!-- Offline Database -->
-    <script src="/static/offline-db.js?v=0.3.3"></script>
+    <script src="/static/offline-db.js?v=0.4.0"></script>
 
     <!-- Mobile JavaScript -->
-    <script src="/static/mobile.js?v=0.3.3"></script>
+    <script src="/static/mobile.js?v=0.4.0"></script>
 
     {% block extra_scripts %}{% endblock %}
 </body>