Backend API and HTMX-powered web interface for Terra-View - a unified fleet management system. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard. Terra-View supports seismographs (SFM module), sound level meters, field modems, and other monitoring devices.

Features

Progressive Web App (PWA): Mobile-first responsive design optimized for field deployment operations
- Install as App: Add to home screen on iOS/Android for native app experience
- Offline Capable: Service worker caching with IndexedDB storage for offline operation
- Touch Optimized: 44x44px minimum touch targets, hamburger menu, bottom navigation bar
- Mobile Card View: Compact unit cards with status dots, tap-to-navigate locations, and detail modals
- Background Sync: Queue edits while offline and automatically sync when connection returns
Web Dashboard: Modern, responsive UI with dark/light mode, live HTMX updates, and integrated fleet map
Fleet Monitoring: Track deployed, benched, retired, and ignored units in separate buckets with unknown-emitter triage
Roster Management: Full CRUD + CSV import/export, device-type aware metadata, and inline actions from the roster tables
Settings & Safeguards: /settings page exposes roster stats, exports, replace-all imports, and danger-zone reset tools
Device & Modem Metadata: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit
Status Management: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry
Data Ingestion: Accept reports from emitter scripts via REST API
Photo Management: Upload and view photos for each unit
Interactive Maps: Leaflet-based maps showing unit locations with tap-to-navigate for mobile
SQLite Storage: Lightweight, file-based database for easy deployment
Database Management: Comprehensive backup and restore system
- Manual Snapshots: Create on-demand backups with descriptions
- Restore from Snapshot: Restore database with automatic safety backups
- Upload/Download: Transfer database snapshots for off-site storage
- Remote Cloning: Copy production database to remote dev servers over WAN
- Automatic Backups: Scheduled background backups with configurable retention

Roster Manager & Settings

Visit /settings to perform bulk roster operations with guardrails:

CSV export/import: Download the entire roster, merge updates, or replace all units in one transaction.
Live roster table: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place.
Database backups: Create snapshots, restore from backups, upload/download database files, view database statistics.
Remote cloning: Clone production database to remote development servers over the network (see scripts/clone_db_to_dev.py).
Stats at a glance: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked.
Danger zone controls: Clear specific tables or wipe all fleet data when resetting a lab/demo environment.

All UI actions call GET/POST /api/settings/* endpoints so you can automate the same workflows from scripts. See docs/DATABASE_MANAGEMENT.md for comprehensive database backup and restore documentation.

Tech Stack

FastAPI: Modern, fast web framework
SQLAlchemy: SQL toolkit and ORM
SQLite: Lightweight database
HTMX: Dynamic updates without heavy JavaScript frameworks
TailwindCSS: Utility-first CSS framework
Leaflet: Interactive maps
Jinja2: Server-side templating
uvicorn: ASGI server
Docker: Containerization for easy deployment

Quick Start with Docker Compose (Recommended)

Prerequisites

Docker and Docker Compose installed

Running the Application

Start the service:
```
docker compose up -d
```
Check logs:
```
docker compose logs -f
```
Stop the service:
```
docker compose down
```

The application will be available at:

Web Interface: http://localhost:8001
API Documentation: http://localhost:8001/docs
Health Check: http://localhost:8001/health

Data Persistence

The SQLite database and photos are stored in the ./data directory, which is mounted as a volume. Your data will persist even if you restart or rebuild the container.

Local Development (Without Docker)

Prerequisites

Python 3.11+
pip

Setup

Install dependencies:
```
pip install -r requirements.txt
```

Run the server:

uvicorn backend.main:app --host 0.0.0.0 --port 8001 --reload

The application will be available at http://localhost:8001

Optional: Generate Sample Data

Need realistic data quickly? Run:

python create_test_db.py
cp /tmp/sfm_test.db data/seismo_fleet.db

The helper script creates a modem/seismograph mix so you can exercise the dashboard, roster tabs, and unit detail screens immediately.

Upgrading from Previous Versions

From v0.2.x to v0.3.0

Version 0.3.0 introduces user preferences storage. Run the migration once per database file:

python backend/migrate_add_user_preferences.py

This creates the user_preferences table for persistent settings storage (timezone, theme, auto-refresh interval, calibration defaults, status thresholds).

From v0.1.x to v0.2.x or later

Versions ≥0.2 introduce new roster columns (device_type, calibration dates, modem metadata, addresses, etc.). Run the migration once per database file:

python backend/migrate_add_device_types.py

Both migration scripts are idempotent—if the columns/tables already exist, they simply exit.

API Endpoints

Web Pages

GET / - Dashboard home page
GET /roster - Fleet roster page
GET /unit/{unit_id} - Unit detail page
GET /settings - Roster manager, CSV import/export, and danger-zone utilities

Fleet Status & Monitoring

GET /api/status-snapshot - Complete fleet status snapshot
GET /api/roster - List of all units with metadata
GET /api/unit/{unit_id} - Detailed unit information
GET /health - Health check endpoint

Roster Management

POST /api/roster/add - Add new unit to roster

curl -X POST http://localhost:8001/api/roster/add \
  -F "id=BE1234" \
  -F "device_type=seismograph" \
  -F "unit_type=series3" \
  -F "project_id=PROJ-001" \
  -F "deployed=true" \
  -F "note=Main site sensor"

GET /api/roster/{unit_id} - Fetch a single roster entry for editing
POST /api/roster/edit/{unit_id} - Update all metadata (device type, calibration dates, modem fields, etc.)
POST /api/roster/set-deployed/{unit_id} - Toggle deployment status
POST /api/roster/set-retired/{unit_id} - Toggle retired status
POST /api/roster/set-note/{unit_id} - Update unit notes
DELETE /api/roster/{unit_id} - Remove a roster/emitter pair entirely

POST /api/roster/import-csv - Bulk import from CSV (merge/update mode)

curl -X POST http://localhost:8001/api/roster/import-csv \
  -F "file=@roster.csv" \
  -F "update_existing=true"

POST /api/roster/ignore/{unit_id} - Move an unknown emitter to the ignore list
DELETE /api/roster/ignore/{unit_id} - Remove a unit from the ignore list
GET /api/roster/ignored - List all ignored units with reasons

Settings & Data Management

GET /api/settings/export-csv - Download the entire roster as CSV
GET /api/settings/stats - Counts for roster, emitters, and ignored tables
GET /api/settings/roster-units - Raw roster dump for the settings data grid
POST /api/settings/import-csv-replace - Replace the entire roster in one atomic transaction
GET /api/settings/preferences - Get user preferences (timezone, theme, calibration defaults, etc.)
PUT /api/settings/preferences - Update user preferences (supports partial updates)
POST /api/settings/clear-all - Danger-zone action that wipes roster, emitters, and ignored tables
POST /api/settings/clear-roster - Delete only roster entries
POST /api/settings/clear-emitters - Delete auto-discovered emitters
POST /api/settings/clear-ignored - Reset ignore list

Database Management

GET /api/settings/database/stats - Database size, row counts, and last modified time
POST /api/settings/database/snapshot - Create manual database snapshot with optional description
GET /api/settings/database/snapshots - List all available snapshots with metadata
GET /api/settings/database/snapshot/{filename} - Download a specific snapshot file
DELETE /api/settings/database/snapshot/{filename} - Delete a snapshot
POST /api/settings/database/restore - Restore database from snapshot (creates safety backup)
POST /api/settings/database/upload-snapshot - Upload snapshot file to server

See docs/DATABASE_MANAGEMENT.md for detailed documentation and examples.

CSV Import Format

Create a CSV file with the following columns (only unit_id is required, everything else is optional):

unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model

Boolean columns accept true/false, 1/0, or yes/no (case-insensitive). Date columns expect YYYY-MM-DD. Use the same schema whether you merge via /api/roster/import-csv or replace everything with /api/settings/import-csv-replace.

Example

unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
BE1234,series3,seismograph,true,false,Primary sensor,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",2025-01-15,2026-01-15,MDM001,,,
MDM001,modem,modem,true,false,Field modem,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",,,,"192.0.2.10","+1-555-0100","Raven XTV"

See sample_roster.csv for a minimal working example.

Emitter Reporting

POST /emitters/report - Submit status report from a seismograph unit
POST /api/series3/heartbeat - Series 3 multi-unit telemetry payload
POST /api/series4/heartbeat - Series 4 (Micromate) multi-unit telemetry payload
GET /fleet/status - Retrieve status of all seismograph units (legacy)

Photo Management

GET /api/unit/{unit_id}/photos - List photos for a unit
GET /api/unit/{unit_id}/photo/{filename} - Serve specific photo file

API Documentation

Once running, interactive API documentation is available at:

Swagger UI: http://localhost:8001/docs
ReDoc: http://localhost:8001/redoc

Testing the API

Using curl

Submit a report:

curl -X POST http://localhost:8001/emitters/report \
  -H "Content-Type: application/json" \
  -d '{
    "unit": "SEISMO-001",
    "unit_type": "series3",
    "timestamp": "2025-11-20T10:30:00",
    "file": "event_20251120_103000.dat",
    "status": "OK"
  }'

Get fleet status:

curl http://localhost:8001/api/roster

Import roster from CSV:

curl -X POST http://localhost:8001/api/roster/import-csv \
  -F "file=@sample_roster.csv" \
  -F "update_existing=true"

Using Python

import requests
from datetime import datetime

# Submit report
response = requests.post(
    "http://localhost:8001/emitters/report",
    json={
        "unit": "SEISMO-001",
        "unit_type": "series3",
        "timestamp": datetime.utcnow().isoformat(),
        "file": "event_20251120_103000.dat",
        "status": "OK"
    }
)
print(response.json())

# Get fleet status
response = requests.get("http://localhost:8001/api/roster")
print(response.json())

# Import CSV
with open('roster.csv', 'rb') as f:
    files = {'file': f}
    data = {'update_existing': 'true'}
    response = requests.post(
        "http://localhost:8001/api/roster/import-csv",
        files=files,
        data=data
    )
print(response.json())

Data Model

RosterUnit Table (Fleet Roster)

Common fields

Field	Type	Description
id	string	Unit identifier (primary key)
unit_type	string	Hardware model name (default: `series3`)
device_type	string	`seismograph` or `modem` discriminator
deployed	boolean	Whether the unit is in the field
retired	boolean	Removes the unit from deployments but preserves history
note	string	Notes about the unit
project_id	string	Associated project identifier
location	string	Legacy location label
address	string	Human-readable address
coordinates	string	`lat,lon` pair used by the map
last_updated	datetime	Last modification timestamp

Seismograph-only fields

Field	Type	Description
last_calibrated	date	Last calibration date
next_calibration_due	date	Next calibration date
deployed_with_modem_id	string	Which modem is paired during deployment

Modem-only fields

Field	Type	Description
ip_address	string	Assigned IP (static or DHCP)
phone_number	string	Cellular number for the modem
hardware_model	string	Modem hardware reference

Emitter Table (Device Check-ins)

Field	Type	Description
id	string	Unit identifier (primary key)
unit_type	string	Reported device type/model
last_seen	datetime	Last report timestamp
last_file	string	Last file processed
status	string	Current status: OK, Pending, Missing
notes	string	Optional notes (nullable)

IgnoredUnit Table (Noise Management)

Field	Type	Description
id	string	Unit identifier (primary key)
reason	string	Optional context for ignoring
ignored_at	datetime	When the ignore action occurred

UserPreferences Table (Settings Storage)

Field	Type	Description
id	integer	Always 1 (single-row table)
timezone	string	Display timezone (default: America/New_York)
theme	string	UI theme: auto, light, or dark
auto_refresh_interval	integer	Dashboard refresh interval in seconds
date_format	string	Date format preference
table_rows_per_page	integer	Default pagination size
calibration_interval_days	integer	Default days between calibrations
calibration_warning_days	integer	Warning threshold before calibration due
status_ok_threshold_hours	integer	Hours for OK status threshold
status_pending_threshold_hours	integer	Hours for Pending status threshold
updated_at	datetime	Last preference update timestamp

Project Structure

seismo-fleet-manager/
├── backend/
│   ├── main.py              # FastAPI app entry point
│   ├── database.py          # SQLAlchemy database configuration
│   ├── models.py            # Database models (RosterUnit, Emitter, IgnoredUnit, UserPreferences)
│   ├── routes.py            # Legacy API endpoints + Series 3/4 heartbeat endpoints
│   ├── routers/             # Modular API routers
│   │   ├── roster.py        # Fleet status endpoints
│   │   ├── roster_edit.py   # Roster management & CSV import
│   │   ├── units.py         # Unit detail endpoints
│   │   ├── photos.py        # Photo management
│   │   ├── dashboard.py     # Dashboard partials
│   │   ├── dashboard_tabs.py # Dashboard tab endpoints
│   │   └── settings.py      # Settings, preferences, and data management
│   ├── services/
│   │   ├── snapshot.py      # Fleet status snapshot logic
│   │   ├── database_backup.py # Database backup and restore service
│   │   └── backup_scheduler.py # Automatic backup scheduler
│   ├── migrate_add_device_types.py # SQLite migration for v0.2 schema
│   ├── migrate_add_user_preferences.py # SQLite migration for v0.3 schema
│   └── static/              # Static assets (CSS, etc.)
├── create_test_db.py        # Generate a sample SQLite DB with mixed devices
├── templates/               # Jinja2 HTML templates
│   ├── base.html            # Base layout with sidebar
│   ├── dashboard.html       # Main dashboard
│   ├── roster.html          # Fleet roster table
│   ├── unit_detail.html     # Unit detail page
│   ├── settings.html        # Roster manager UI
│   └── partials/            # HTMX partial templates
│       ├── roster_table.html
│       ├── retired_table.html
│       ├── ignored_table.html
│       └── unknown_emitters.html
├── data/                    # SQLite database & photos (persisted)
│   └── backups/             # Database snapshots directory
├── scripts/
│   └── clone_db_to_dev.py   # Remote database cloning utility
├── docs/
│   └── DATABASE_MANAGEMENT.md # Database backup/restore guide
├── requirements.txt         # Python dependencies
├── Dockerfile               # Docker container definition
├── docker-compose.yml       # Docker Compose configuration
├── CHANGELOG.md             # Version history
├── FRONTEND_README.md       # Frontend documentation
└── README.md                # This file

Docker Commands

Build the image:

docker compose build

Start in foreground:

docker compose up

Start in background:

docker compose up -d

View logs:

docker compose logs -f seismo-backend

Restart service:

docker compose restart

Rebuild and restart:

docker compose up -d --build

Stop and remove containers:

docker compose down

Remove containers and volumes:

docker compose down -v

Release Highlights

v0.4.0 — 2025-12-16

Database Management System: Complete backup and restore functionality with manual snapshots, restore operations, and upload/download capabilities
Remote Database Cloning: New clone_db_to_dev.py script for copying production database to remote dev servers over WAN
Automatic Backup Scheduler: Background service for scheduled backups with configurable retention management
Database Tab: New dedicated tab in Settings for all database operations with real-time statistics
Settings Reorganization: Improved tab structure - renamed "Data Management" to "Roster Management", moved CSV Replace Mode, created Database tab
Comprehensive Documentation: New docs/DATABASE_MANAGEMENT.md with complete guide to backup/restore workflows, API reference, and best practices

v0.3.3 — 2025-12-12

Improved Mobile Navigation: Hamburger menu moved to bottom nav bar (no more floating button covering content)
Better Status Visibility: Larger status dots (16px) in dashboard fleet overview for easier at-a-glance status checks
Cleaner Roster Cards: Location navigation links moved to detail modal only, reducing clutter in card view

v0.3.2 — 2025-12-12

Progressive Web App (PWA): Complete mobile optimization with offline support, installable as standalone app
Mobile-First UI: Hamburger menu, bottom navigation bar, card-based roster view optimized for touch
Tap-to-Navigate: Location links open in user's preferred navigation app (Google Maps, Apple Maps, Waze)
Offline Editing: Service worker + IndexedDB for offline operation with automatic sync when online
Unit Detail Modals: Bottom sheet modals for quick unit info access with full edit capabilities
Hard Reload Utility: "Clear Cache & Reload" button to force fresh assets (helpful for development)

v0.3.1 — 2025-12-12

Dashboard Alerts: Only Missing units show in notifications (Pending units no longer alert)
Status Fixes: Fixed "Unknown" status issues in mobile card views and detail modals
Backend Improvements: Safer data access with .get() defaults to prevent errors

v0.3.0 — 2025-12-09

Series 4 Support: New /api/series4/heartbeat endpoint with auto-detection for Micromate units (UM##### pattern)
Settings Redesign: Completely redesigned Settings page with 4-tab interface (General, Data Management, Advanced, Danger Zone)
User Preferences: Backend storage for timezone, theme, auto-refresh interval, calibration defaults, and status thresholds
Development Labels: Visual indicators to distinguish dev from production environments
Timezone Support: Comprehensive timezone handling with human-readable timestamps site-wide
Quality of Life: Relative timestamps, status icons for accessibility, breadcrumb navigation, copy-to-clipboard, search functionality

v0.2.1 — 2025-12-03

Added the /settings roster manager with CSV export/import, live stats, and danger-zone table reset actions
Deployed/Benched/Retired/Ignored tabs now have dedicated HTMX partials, sorting, and inline actions
Unit detail pages expose device-type specific metadata (calibration windows, modem pairing, IP/phone fields)
Snapshot summary and dashboard counts now focus on deployed units and include address/coordinate data

v0.2.0 — 2025-12-03

Introduced device-type aware roster schema (seismograph vs modem) plus migration + create_test_db.py helper
Added Ignore list model/endpoints to quarantine noisy emitters directly from the roster
Roster page gained Add Unit + CSV Import modals, HTMX-driven updates, and unknown emitter callouts
Snapshot service now returns active/benched/retired/unknown buckets containing richer metadata

v0.1.1 — 2025-12-02

Roster Editing API: Full CRUD operations for managing your fleet roster
CSV Import: Bulk upload roster data from CSV files
Enhanced Data Model: Added project_id and location fields to roster
Bug Fixes: Improved database session management and error handling

See CHANGELOG.md for the full release notes.

Future Enhancements

Email/SMS alerts for missing units
Historical data tracking and reporting
Multi-user authentication
PostgreSQL support for larger deployments
Advanced filtering and search
Export roster to various formats

License

MIT

Version

Current: 0.4.0 — Database management system with backup/restore and remote cloning (2025-12-16)

Previous: 0.3.3 — Mobile navigation improvements and better status visibility (2025-12-12)

0.3.2 — Progressive Web App with mobile optimization (2025-12-12)

0.3.1 — Dashboard alerts and status fixes (2025-12-12)

0.3.0 — Series 4 support, settings redesign, user preferences (2025-12-09)

0.2.1 — Settings & roster manager refresh (2025-12-03)

0.2.0 — Device-type aware roster + ignore list (2025-12-03)

0.1.1 — Roster Management & CSV Import (2025-12-02)

0.1.0 — Initial Release (2024-11-20)