Seismo Fleet Manager v0.4.3

Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.

Features

• Progressive Web App (PWA): Mobile-first responsive design optimized for field deployment operations
  • Install as App: Add to home screen on iOS/Android for native app experience
  • Offline Capable: Service worker caching with IndexedDB storage for offline operation
  • Touch Optimized: 44x44px minimum touch targets, hamburger menu, bottom navigation bar
  • Mobile Card View: Compact unit cards with status dots, tap-to-navigate locations, and detail modals
  • Background Sync: Queue edits while offline and automatically sync when connection returns
• Web Dashboard: Modern, responsive UI with dark/light mode, live HTMX updates, and integrated fleet map
• Fleet Monitoring: Track deployed, benched, retired, and ignored units in separate buckets with unknown-emitter triage
• Roster Management: Full CRUD + CSV import/export, device-type aware metadata, and inline actions from the roster tables
• Settings & Safeguards: /settings page exposes roster stats, exports, replace-all imports, and danger-zone reset tools
• Device & Modem Metadata: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit
• Status Management: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry
• Data Ingestion: Accept reports from emitter scripts via REST API
• Photo Management: Upload and view photos for each unit
• Interactive Maps: Leaflet-based maps showing unit locations with tap-to-navigate for mobile
• SQLite Storage: Lightweight, file-based database for easy deployment
• Database Management: Comprehensive backup and restore system
  • Manual Snapshots: Create on-demand backups with descriptions
  • Restore from Snapshot: Restore database with automatic safety backups
  • Upload/Download: Transfer database snapshots for off-site storage
  • Remote Cloning: Copy production database to remote dev servers over WAN
  • Automatic Backups: Scheduled background backups with configurable retention

Roster Manager & Settings

Visit /settings to perform bulk roster operations with guardrails:

• CSV export/import: Download the entire roster, merge updates, or replace all units in one transaction.
• Live roster table: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place.
• Database backups: Create snapshots, restore from backups, upload/download database files, view database statistics.
• Remote cloning: Clone production database to remote development servers over the network (see scripts/clone_db_to_dev.py).
• Stats at a glance: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked.
• Danger zone controls: Clear specific tables or wipe all fleet data when resetting a lab/demo environment.

All UI actions call GET/POST /api/settings/* endpoints so you can automate the same workflows from scripts. See docs/DATABASE_MANAGEMENT.md for comprehensive database backup and restore documentation.

Tech Stack

• FastAPI: Modern, fast web framework
• SQLAlchemy: SQL toolkit and ORM
• SQLite: Lightweight database
• HTMX: Dynamic updates without heavy JavaScript frameworks
• TailwindCSS: Utility-first CSS framework
• Leaflet: Interactive maps
• Jinja2: Server-side templating
• uvicorn: ASGI server
• Docker: Containerization for easy deployment

Quick Start with Docker Compose (Recommended)

Prerequisites

• Docker and Docker Compose installed

Running the Application

1. Start the service:

   docker compose up -d
   

2. Check logs:

   docker compose logs -f
   

3. Stop the service:

   docker compose down
   

The application will be available at:

• Web Interface: http://localhost:8001
• API Documentation: http://localhost:8001/docs
• Health Check: http://localhost:8001/health

Data Persistence

The SQLite database and photos are stored in the ./data directory, which is mounted as a volume. Your data will persist even if you restart or rebuild the container.

Local Development (Without Docker)

Prerequisites

• Python 3.11+
• pip

Setup

1. Install dependencies:

   pip install -r requirements.txt
   

2. Run the server:

   uvicorn backend.main:app --host 0.0.0.0 --port 8001 --reload
   

The application will be available at http://localhost:8001

Optional: Generate Sample Data

Need realistic data quickly? Run:

python create_test_db.py
cp /tmp/sfm_test.db data/seismo_fleet.db

The helper script creates a modem/seismograph mix so you can exercise the dashboard, roster tabs, and unit detail screens immediately.

Upgrading from Previous Versions

From v0.2.x to v0.3.0

Version 0.3.0 introduces user preferences storage. Run the migration once per database file:

python backend/migrate_add_user_preferences.py

This creates the user_preferences table for persistent settings storage (timezone, theme, auto-refresh interval, calibration defaults, status thresholds).

From v0.1.x to v0.2.x or later

Versions ≥0.2 introduce new roster columns (device_type, calibration dates, modem metadata, addresses, etc.). Run the migration once per database file:

python backend/migrate_add_device_types.py

Both migration scripts are idempotent—if the columns/tables already exist, they simply exit.

API Endpoints

Web Pages

• GET / - Dashboard home page
• GET /roster - Fleet roster page
• GET /unit/{unit_id} - Unit detail page
• GET /settings - Roster manager, CSV import/export, and danger-zone utilities

Fleet Status & Monitoring

• GET /api/status-snapshot - Complete fleet status snapshot
• GET /api/roster - List of all units with metadata
• GET /api/unit/{unit_id} - Detailed unit information
• GET /health - Health check endpoint

Roster Management

• POST /api/roster/add - Add new unit to roster

  curl -X POST http://localhost:8001/api/roster/add \
    -F "id=BE1234" \
    -F "device_type=seismograph" \
    -F "unit_type=series3" \
    -F "project_id=PROJ-001" \
    -F "deployed=true" \
    -F "note=Main site sensor"
  

• GET /api/roster/{unit_id} - Fetch a single roster entry for editing
• POST /api/roster/edit/{unit_id} - Update all metadata (device type, calibration dates, modem fields, etc.)
• POST /api/roster/set-deployed/{unit_id} - Toggle deployment status
• POST /api/roster/set-retired/{unit_id} - Toggle retired status
• POST /api/roster/set-note/{unit_id} - Update unit notes
• DELETE /api/roster/{unit_id} - Remove a roster/emitter pair entirely
• POST /api/roster/import-csv - Bulk import from CSV (merge/update mode)

  curl -X POST http://localhost:8001/api/roster/import-csv \
    -F "file=@roster.csv" \
    -F "update_existing=true"
  

• POST /api/roster/ignore/{unit_id} - Move an unknown emitter to the ignore list
• DELETE /api/roster/ignore/{unit_id} - Remove a unit from the ignore list
• GET /api/roster/ignored - List all ignored units with reasons

Settings & Data Management

• GET /api/settings/export-csv - Download the entire roster as CSV
• GET /api/settings/stats - Counts for roster, emitters, and ignored tables
• GET /api/settings/roster-units - Raw roster dump for the settings data grid
• POST /api/settings/import-csv-replace - Replace the entire roster in one atomic transaction
• GET /api/settings/preferences - Get user preferences (timezone, theme, calibration defaults, etc.)
• PUT /api/settings/preferences - Update user preferences (supports partial updates)
• POST /api/settings/clear-all - Danger-zone action that wipes roster, emitters, and ignored tables
• POST /api/settings/clear-roster - Delete only roster entries
• POST /api/settings/clear-emitters - Delete auto-discovered emitters
• POST /api/settings/clear-ignored - Reset ignore list

Database Management

• GET /api/settings/database/stats - Database size, row counts, and last modified time
• POST /api/settings/database/snapshot - Create manual database snapshot with optional description
• GET /api/settings/database/snapshots - List all available snapshots with metadata
• GET /api/settings/database/snapshot/{filename} - Download a specific snapshot file
• DELETE /api/settings/database/snapshot/{filename} - Delete a snapshot
• POST /api/settings/database/restore - Restore database from snapshot (creates safety backup)
• POST /api/settings/database/upload-snapshot - Upload snapshot file to server

See docs/DATABASE_MANAGEMENT.md for detailed documentation and examples.

CSV Import Format

Create a CSV file with the following columns (only unit_id is required, everything else is optional):

unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model

Boolean columns accept true/false, 1/0, or yes/no (case-insensitive). Date columns expect YYYY-MM-DD. Use the same schema whether you merge via /api/roster/import-csv or replace everything with /api/settings/import-csv-replace.

Example

unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
BE1234,series3,seismograph,true,false,Primary sensor,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",2025-01-15,2026-01-15,MDM001,,,
MDM001,modem,modem,true,false,Field modem,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",,,,"192.0.2.10","+1-555-0100","Raven XTV"

See sample_roster.csv for a minimal working example.

Emitter Reporting

• POST /emitters/report - Submit status report from a seismograph unit
• POST /api/series3/heartbeat - Series 3 multi-unit telemetry payload
• POST /api/series4/heartbeat - Series 4 (Micromate) multi-unit telemetry payload
• GET /fleet/status - Retrieve status of all seismograph units (legacy)

Photo Management

• GET /api/unit/{unit_id}/photos - List photos for a unit
• GET /api/unit/{unit_id}/photo/{filename} - Serve specific photo file

API Documentation

Once running, interactive API documentation is available at:

• Swagger UI: http://localhost:8001/docs
• ReDoc: http://localhost:8001/redoc

Testing the API

Using curl

Submit a report:

curl -X POST http://localhost:8001/emitters/report \
  -H "Content-Type: application/json" \
  -d '{
    "unit": "SEISMO-001",
    "unit_type": "series3",
    "timestamp": "2025-11-20T10:30:00",
    "file": "event_20251120_103000.dat",
    "status": "OK"
  }'

Get fleet status:

curl http://localhost:8001/api/roster

Import roster from CSV:

curl -X POST http://localhost:8001/api/roster/import-csv \
  -F "file=@sample_roster.csv" \
  -F "update_existing=true"

Using Python

import requests
from datetime import datetime

# Submit report
response = requests.post(
    "http://localhost:8001/emitters/report",
    json={
        "unit": "SEISMO-001",
        "unit_type": "series3",
        "timestamp": datetime.utcnow().isoformat(),
        "file": "event_20251120_103000.dat",
        "status": "OK"
    }
)
print(response.json())

# Get fleet status
response = requests.get("http://localhost:8001/api/roster")
print(response.json())

# Import CSV
with open('roster.csv', 'rb') as f:
    files = {'file': f}
    data = {'update_existing': 'true'}
    response = requests.post(
        "http://localhost:8001/api/roster/import-csv",
        files=files,
        data=data
    )
print(response.json())

Data Model

RosterUnit Table (Fleet Roster)

Common fields

Field	Type	Description
id	string	Unit identifier (primary key)
unit_type	string	Hardware model name (default: series3)
device_type	string	Device type: "seismograph", "modem", or "slm" (sound level meter)
deployed	boolean	Whether the unit is in the field
retired	boolean	Removes the unit from deployments but preserves history
note	string	Notes about the unit
project_id	string	Associated project identifier
location	string	Legacy location label
address	string	Human-readable address
coordinates	string	lat,lon pair used by the map
last_updated	datetime	Last modification timestamp

Seismograph-only fields

Field	Type	Description
last_calibrated	date	Last calibration date
next_calibration_due	date	Next calibration date
deployed_with_modem_id	string	Which modem is paired during deployment

Modem-only fields

Field	Type	Description
ip_address	string	Assigned IP (static or DHCP)
phone_number	string	Cellular number for the modem
hardware_model	string	Modem hardware reference

Sound Level Meter (SLM) fields

Field	Type	Description
slm_host	string	Direct IP address for SLM (if not using modem)
slm_tcp_port	integer	TCP control port (default: 2255)
slm_ftp_port	integer	FTP file transfer port (default: 21)
slm_model	string	Device model (NL-43, NL-53)
slm_serial_number	string	Manufacturer serial number
slm_frequency_weighting	string	Frequency weighting setting (A, C, Z)
slm_time_weighting	string	Time weighting setting (F=Fast, S=Slow)
slm_measurement_range	string	Measurement range setting
slm_last_check	datetime	Last status check timestamp
deployed_with_modem_id	string	Modem pairing (shared with seismographs)

Device Type Schema

Terra-View supports three device types with the following standardized device_type values:

• "seismograph" (default) - Seismic monitoring devices (Series 3, Series 4, Micromate)

  • Uses: calibration dates, modem pairing
  • Examples: BE1234, UM12345 (Series 3/4 units)

• "modem" - Field modems and network equipment

  • Uses: IP address, phone number, hardware model
  • Examples: MDM001, MODEM-2025-01

• "slm" - Sound level meters (Rion NL-43/NL-53)

  • Uses: TCP/FTP configuration, measurement settings, modem pairing
  • Examples: SLM-43-01, NL43-001

Important: All device_type values must be lowercase. The legacy value "sound_level_meter" has been deprecated in favor of the shorter "slm". Run backend/migrate_standardize_device_types.py to update existing databases.

Emitter Table (Device Check-ins)

Field	Type	Description
id	string	Unit identifier (primary key)
unit_type	string	Reported device type/model
last_seen	datetime	Last report timestamp
last_file	string	Last file processed
status	string	Current status: OK, Pending, Missing
notes	string	Optional notes (nullable)

IgnoredUnit Table (Noise Management)

Field	Type	Description
id	string	Unit identifier (primary key)
reason	string	Optional context for ignoring
ignored_at	datetime	When the ignore action occurred

UserPreferences Table (Settings Storage)

Field	Type	Description
id	integer	Always 1 (single-row table)
timezone	string	Display timezone (default: America/New_York)
theme	string	UI theme: auto, light, or dark
auto_refresh_interval	integer	Dashboard refresh interval in seconds
date_format	string	Date format preference
table_rows_per_page	integer	Default pagination size
calibration_interval_days	integer	Default days between calibrations
calibration_warning_days	integer	Warning threshold before calibration due
status_ok_threshold_hours	integer	Hours for OK status threshold
status_pending_threshold_hours	integer	Hours for Pending status threshold
updated_at	datetime	Last preference update timestamp

Project Structure

seismo-fleet-manager/
├── backend/
│   ├── main.py              # FastAPI app entry point
│   ├── database.py          # SQLAlchemy database configuration
│   ├── models.py            # Database models (RosterUnit, Emitter, IgnoredUnit, UserPreferences)
│   ├── routes.py            # Legacy API endpoints + Series 3/4 heartbeat endpoints
│   ├── routers/             # Modular API routers
│   │   ├── roster.py        # Fleet status endpoints
│   │   ├── roster_edit.py   # Roster management & CSV import
│   │   ├── units.py         # Unit detail endpoints
│   │   ├── photos.py        # Photo management
│   │   ├── dashboard.py     # Dashboard partials
│   │   ├── dashboard_tabs.py # Dashboard tab endpoints
│   │   └── settings.py      # Settings, preferences, and data management
│   ├── services/
│   │   ├── snapshot.py      # Fleet status snapshot logic
│   │   ├── database_backup.py # Database backup and restore service
│   │   └── backup_scheduler.py # Automatic backup scheduler
│   ├── migrate_add_device_types.py # SQLite migration for v0.2 schema
│   ├── migrate_add_user_preferences.py # SQLite migration for v0.3 schema
│   └── static/              # Static assets (CSS, etc.)
├── create_test_db.py        # Generate a sample SQLite DB with mixed devices
├── templates/               # Jinja2 HTML templates
│   ├── base.html            # Base layout with sidebar
│   ├── dashboard.html       # Main dashboard
│   ├── roster.html          # Fleet roster table
│   ├── unit_detail.html     # Unit detail page
│   ├── settings.html        # Roster manager UI
│   └── partials/            # HTMX partial templates
│       ├── roster_table.html
│       ├── retired_table.html
│       ├── ignored_table.html
│       └── unknown_emitters.html
├── data/                    # SQLite database & photos (persisted)
│   └── backups/             # Database snapshots directory
├── scripts/
│   └── clone_db_to_dev.py   # Remote database cloning utility
├── docs/
│   └── DATABASE_MANAGEMENT.md # Database backup/restore guide
├── requirements.txt         # Python dependencies
├── Dockerfile               # Docker container definition
├── docker-compose.yml       # Docker Compose configuration
├── CHANGELOG.md             # Version history
├── FRONTEND_README.md       # Frontend documentation
└── README.md                # This file

Docker Commands

Build the image:

docker compose build

Start in foreground:

docker compose up

Start in background:

docker compose up -d

View logs:

docker compose logs -f seismo-backend

Restart service:

docker compose restart

Rebuild and restart:

docker compose up -d --build

Stop and remove containers:

docker compose down

Remove containers and volumes:

docker compose down -v

Release Highlights

v0.4.3 — 2026-01-14

• Sound Level Meter workflow: Roster manager surfaces SLM metadata, supports rename actions, and adds return-to-project navigation plus schedule/unit templates for project planning.
• Project insight panels: Project dashboards now expose file and session lists so teams can see what each project stores before diving into units.
• Project view polish: FTP browser supports folder downloads, the timer display was reimplemented, and the project/device templates gained edit modals for projects and locations to streamline navigation.
• SLM sync & accuracy: Configuration edits now propagate to SLMM (which caches configs for faster responses) and the live view uses the correct DRD fields so telemetry aligns with the control center.

v0.4.0 — 2025-12-16

• Database Management System: Complete backup and restore functionality with manual snapshots, restore operations, and upload/download capabilities
• Remote Database Cloning: New clone_db_to_dev.py script for copying production database to remote dev servers over WAN
• Automatic Backup Scheduler: Background service for scheduled backups with configurable retention management
• Database Tab: New dedicated tab in Settings for all database operations with real-time statistics
• Settings Reorganization: Improved tab structure - renamed "Data Management" to "Roster Management", moved CSV Replace Mode, created Database tab
• Comprehensive Documentation: New docs/DATABASE_MANAGEMENT.md with complete guide to backup/restore workflows, API reference, and best practices

v0.3.3 — 2025-12-12

• Improved Mobile Navigation: Hamburger menu moved to bottom nav bar (no more floating button covering content)
• Better Status Visibility: Larger status dots (16px) in dashboard fleet overview for easier at-a-glance status checks
• Cleaner Roster Cards: Location navigation links moved to detail modal only, reducing clutter in card view

v0.3.2 — 2025-12-12

• Progressive Web App (PWA): Complete mobile optimization with offline support, installable as standalone app
• Mobile-First UI: Hamburger menu, bottom navigation bar, card-based roster view optimized for touch
• Tap-to-Navigate: Location links open in user's preferred navigation app (Google Maps, Apple Maps, Waze)
• Offline Editing: Service worker + IndexedDB for offline operation with automatic sync when online
• Unit Detail Modals: Bottom sheet modals for quick unit info access with full edit capabilities
• Hard Reload Utility: "Clear Cache & Reload" button to force fresh assets (helpful for development)

v0.3.1 — 2025-12-12

• Dashboard Alerts: Only Missing units show in notifications (Pending units no longer alert)
• Status Fixes: Fixed "Unknown" status issues in mobile card views and detail modals
• Backend Improvements: Safer data access with .get() defaults to prevent errors

v0.3.0 — 2025-12-09

• Series 4 Support: New /api/series4/heartbeat endpoint with auto-detection for Micromate units (UM##### pattern)
• Settings Redesign: Completely redesigned Settings page with 4-tab interface (General, Data Management, Advanced, Danger Zone)
• User Preferences: Backend storage for timezone, theme, auto-refresh interval, calibration defaults, and status thresholds
• Development Labels: Visual indicators to distinguish dev from production environments
• Timezone Support: Comprehensive timezone handling with human-readable timestamps site-wide
• Quality of Life: Relative timestamps, status icons for accessibility, breadcrumb navigation, copy-to-clipboard, search functionality

v0.2.1 — 2025-12-03

• Added the /settings roster manager with CSV export/import, live stats, and danger-zone table reset actions
• Deployed/Benched/Retired/Ignored tabs now have dedicated HTMX partials, sorting, and inline actions
• Unit detail pages expose device-type specific metadata (calibration windows, modem pairing, IP/phone fields)
• Snapshot summary and dashboard counts now focus on deployed units and include address/coordinate data

v0.2.0 — 2025-12-03

• Introduced device-type aware roster schema (seismograph vs modem) plus migration + create_test_db.py helper
• Added Ignore list model/endpoints to quarantine noisy emitters directly from the roster
• Roster page gained Add Unit + CSV Import modals, HTMX-driven updates, and unknown emitter callouts
• Snapshot service now returns active/benched/retired/unknown buckets containing richer metadata

v0.1.1 — 2025-12-02

• Roster Editing API: Full CRUD operations for managing your fleet roster
• CSV Import: Bulk upload roster data from CSV files
• Enhanced Data Model: Added project_id and location fields to roster
• Bug Fixes: Improved database session management and error handling

See CHANGELOG.md for the full release notes.

Future Enhancements

• Email/SMS alerts for missing units
• Historical data tracking and reporting
• Multi-user authentication
• PostgreSQL support for larger deployments
• Advanced filtering and search
• Export roster to various formats

License

MIT

Version

Current: 0.4.3 — SLM roster/project view refresh, project insight panels, FTP browser folder downloads, and SLMM sync (2026-01-14)

Previous: 0.4.2 — SLM configuration interface with TCP/FTP controls, modem diagnostics, and dashboard endpoints for Sound Level Meters (2026-01-05)

0.4.1 — Sound Level Meter integration with full management UI for SLM units (2026-01-05)

0.4.0 — Database management system with backup/restore and remote cloning (2025-12-16)

0.3.3 — Mobile navigation improvements and better status visibility (2025-12-12)

0.3.2 — Progressive Web App with mobile optimization (2025-12-12)

0.3.1 — Dashboard alerts and status fixes (2025-12-12)

0.3.0 — Series 4 support, settings redesign, user preferences (2025-12-09)

0.2.1 — Settings & roster manager refresh (2025-12-03)

0.2.0 — Device-type aware roster + ignore list (2025-12-03)

0.1.1 — Roster Management & CSV Import (2025-12-02)

0.1.0 — Initial Release (2024-11-20)