# Terra-View v0.14.0
Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs, sound level meters, and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.

## Features

- **Progressive Web App (PWA)**: Mobile-first responsive design optimized for field deployment operations
  - **Install as App**: Add to home screen on iOS/Android for native app experience
  - **Offline Capable**: Service worker caching with IndexedDB storage for offline operation
  - **Touch Optimized**: 44x44px minimum touch targets, hamburger menu, bottom navigation bar
  - **Mobile Card View**: Compact unit cards with status dots, tap-to-navigate locations, and detail modals
  - **Background Sync**: Queue edits while offline and automatically sync when connection returns
- **Field-Deployment Workflow**: One-photo mobile capture at `/deploy` → desk-side classification at `/tools/pending-deployments` → automatic UnitAssignment creation with EXIF GPS
- **Unit Swap Wizard** (`/tools/unit-swap`): mobile-first 4-step flow for swapping a vibration unit (and optionally its modem) at a monitoring location.  Surfaces benched-fleet candidates as eligible incoming units; cleans up stale modem back-references on swap
- **Editable Deployment Timeline** on every unit detail page: inline edit / delete each assignment, plus an "Add deployment record" button for backfilling historical windows.  Frees-up previously-orphaned events when their timestamp now falls inside an assignment
- **Web Dashboard**: Modern, responsive UI with dark/light mode, live HTMX updates, and integrated fleet map
- **Fleet Monitoring**: Track deployed, benched, retired, and ignored units in separate buckets with unknown-emitter triage
- **Roster Management**: Full CRUD + CSV import/export, device-type aware metadata, and inline actions from the roster tables
- **Settings & Safeguards**: `/settings` page exposes roster stats, exports, replace-all imports, and danger-zone reset tools
- **Device & Modem Metadata**: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit
- **Status Management**: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry
- **Sound Level Meter Monitoring**: Live per-device monitoring through SLMM's shared, cached feed — multiple viewers without contending for the NL-43's single connection — with L1/L10 percentile lines, a measuring/freshness indicator, and on-demand refresh
- **Automated Night Reports**: Daily per-location noise report (last night vs a baseline) for 24/7 remote sound jobs — pulls the meter's 15-minute Leq over FTP and emails an HTML summary + Excel; the meter is auto-cycled (stop → download → ingest → restart, with restart verification) each morning
- **Client Portal** (`/portal/*`): scoped, read-only, client-facing live view of *their* locations only, gated by a per-project link + shared password (argon2-hashed)
- **SFM Event DB Manager** (`/admin/events`): cross-unit event browser with bulk false-trigger flagging and admin-only hard-delete (cleans on-disk binaries + sidecars too) for purging bogus events from misbehaving units
- **Deployment-History Calendar + Gantt** (`/tools/deployment-history`): fleet-wide 12-month calendar with side-panel day drill-down, plus "Gantt by Project" / "Gantt by Unit" tabs
- **Photo Management**: Upload and view photos for each unit
- **Interactive Maps**: Leaflet-based maps showing unit locations with tap-to-navigate for mobile (reusable location-map partial across project overview + Vibration tab)
- **Timezone-Aware Timeline**: deployment assignments display and edit in the user's configured local timezone; UTC stays canonical on disk
- **SQLite Storage**: Lightweight, file-based database for easy deployment
- **Database Management**: Comprehensive backup and restore system
  - **Manual Snapshots**: Create on-demand backups with descriptions
  - **Restore from Snapshot**: Restore database with automatic safety backups
  - **Upload/Download**: Transfer database snapshots for off-site storage
  - **Remote Cloning**: Copy production database to remote dev servers over WAN
  - **Automatic Backups**: Scheduled background backups with configurable retention

## Roster Manager & Settings

Visit [`/settings`](http://localhost:8001/settings) to perform bulk roster operations with guardrails:

- **CSV export/import**: Download the entire roster, merge updates, or replace all units in one transaction.
- **Live roster table**: Fetch every unit via HTMX, edit metadata, toggle deployed/retired states, move emitters to the ignore list, or delete records in-place.
- **Database backups**: Create snapshots, restore from backups, upload/download database files, view database statistics.
- **Remote cloning**: Clone production database to remote development servers over the network (see `scripts/clone_db_to_dev.py`).
- **Stats at a glance**: View counts for the roster, emitters, and ignored units to confirm import/cleanup operations worked.
- **Danger zone controls**: Clear specific tables or wipe all fleet data when resetting a lab/demo environment.

All UI actions call `GET/POST /api/settings/*` endpoints so you can automate the same workflows from scripts. See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for comprehensive database backup and restore documentation.

## Tech Stack

- **FastAPI**: Modern, fast web framework
- **SQLAlchemy**: SQL toolkit and ORM
- **SQLite**: Lightweight database
- **HTMX**: Dynamic updates without heavy JavaScript frameworks
- **TailwindCSS**: Utility-first CSS framework
- **Leaflet**: Interactive maps
- **Jinja2**: Server-side templating
- **uvicorn**: ASGI server
- **Docker**: Containerization for easy deployment

## Quick Start with Docker Compose (Recommended)

### Prerequisites
- Docker and Docker Compose installed

### Running the Application

1. **Start the service:**
   ```bash
   docker compose up -d
   ```

2. **Check logs:**
   ```bash
   docker compose logs -f
   ```

3. **Stop the service:**
   ```bash
   docker compose down
   ```

The application will be available at:
- **Web Interface**: http://localhost:8001
- **API Documentation**: http://localhost:8001/docs
- **Health Check**: http://localhost:8001/health

### Data Persistence

The SQLite database and photos are stored in the `./data` directory, which is mounted as a volume. Your data will persist even if you restart or rebuild the container.

## Local Development (Without Docker)

### Prerequisites
- Python 3.11+
- pip

### Setup

1. **Install dependencies:**
   ```bash
   pip install -r requirements.txt
   ```

2. **Run the server:**
   ```bash
   uvicorn backend.main:app --host 0.0.0.0 --port 8001 --reload
   ```

The application will be available at http://localhost:8001

### Optional: Generate Sample Data

Need realistic data quickly? Run:

```bash
python create_test_db.py
cp /tmp/sfm_test.db data/seismo_fleet.db
```

The helper script creates a modem/seismograph mix so you can exercise the dashboard, roster tabs, and unit detail screens immediately.

## Upgrading from Previous Versions

### From v0.2.x to v0.3.0

Version 0.3.0 introduces user preferences storage. Run the migration once per database file:

```bash
python backend/migrate_add_user_preferences.py
```

This creates the `user_preferences` table for persistent settings storage (timezone, theme, auto-refresh interval, calibration defaults, status thresholds).

### From v0.1.x to v0.2.x or later

Versions ≥0.2 introduce new roster columns (device_type, calibration dates, modem metadata, addresses, etc.). Run the migration once per database file:

```bash
python backend/migrate_add_device_types.py
```

Both migration scripts are idempotent—if the columns/tables already exist, they simply exit.

## API Endpoints

### Web Pages
- **GET** `/` - Dashboard home page
- **GET** `/roster` - Fleet roster page
- **GET** `/unit/{unit_id}` - Unit detail page
- **GET** `/settings` - Roster manager, CSV import/export, and danger-zone utilities

### Fleet Status & Monitoring
- **GET** `/api/status-snapshot` - Complete fleet status snapshot
- **GET** `/api/roster` - List of all units with metadata
- **GET** `/api/unit/{unit_id}` - Detailed unit information
- **GET** `/health` - Health check endpoint

### Roster Management
- **POST** `/api/roster/add` - Add new unit to roster
  ```bash
  curl -X POST http://localhost:8001/api/roster/add \
    -F "id=BE1234" \
    -F "device_type=seismograph" \
    -F "unit_type=series3" \
    -F "project_id=PROJ-001" \
    -F "deployed=true" \
    -F "note=Main site sensor"
  ```
- **GET** `/api/roster/{unit_id}` - Fetch a single roster entry for editing
- **POST** `/api/roster/edit/{unit_id}` - Update all metadata (device type, calibration dates, modem fields, etc.)
- **POST** `/api/roster/set-deployed/{unit_id}` - Toggle deployment status
- **POST** `/api/roster/set-retired/{unit_id}` - Toggle retired status
- **POST** `/api/roster/set-note/{unit_id}` - Update unit notes
- **DELETE** `/api/roster/{unit_id}` - Remove a roster/emitter pair entirely
- **POST** `/api/roster/import-csv` - Bulk import from CSV (merge/update mode)
  ```bash
  curl -X POST http://localhost:8001/api/roster/import-csv \
    -F "file=@roster.csv" \
    -F "update_existing=true"
  ```
- **POST** `/api/roster/ignore/{unit_id}` - Move an unknown emitter to the ignore list
- **DELETE** `/api/roster/ignore/{unit_id}` - Remove a unit from the ignore list
- **GET** `/api/roster/ignored` - List all ignored units with reasons

### Settings & Data Management
- **GET** `/api/settings/export-csv` - Download the entire roster as CSV
- **GET** `/api/settings/stats` - Counts for roster, emitters, and ignored tables
- **GET** `/api/settings/roster-units` - Raw roster dump for the settings data grid
- **POST** `/api/settings/import-csv-replace` - Replace the entire roster in one atomic transaction
- **GET** `/api/settings/preferences` - Get user preferences (timezone, theme, calibration defaults, etc.)
- **PUT** `/api/settings/preferences` - Update user preferences (supports partial updates)
- **POST** `/api/settings/clear-all` - Danger-zone action that wipes roster, emitters, and ignored tables
- **POST** `/api/settings/clear-roster` - Delete only roster entries
- **POST** `/api/settings/clear-emitters` - Delete auto-discovered emitters
- **POST** `/api/settings/clear-ignored` - Reset ignore list

### Database Management
- **GET** `/api/settings/database/stats` - Database size, row counts, and last modified time
- **POST** `/api/settings/database/snapshot` - Create manual database snapshot with optional description
- **GET** `/api/settings/database/snapshots` - List all available snapshots with metadata
- **GET** `/api/settings/database/snapshot/{filename}` - Download a specific snapshot file
- **DELETE** `/api/settings/database/snapshot/{filename}` - Delete a snapshot
- **POST** `/api/settings/database/restore` - Restore database from snapshot (creates safety backup)
- **POST** `/api/settings/database/upload-snapshot` - Upload snapshot file to server

See [docs/DATABASE_MANAGEMENT.md](docs/DATABASE_MANAGEMENT.md) for detailed documentation and examples.

### CSV Import Format
Create a CSV file with the following columns (only `unit_id` is required, everything else is optional):

```
unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
```

Boolean columns accept `true/false`, `1/0`, or `yes/no` (case-insensitive). Date columns expect `YYYY-MM-DD`. Use the same schema whether you merge via `/api/roster/import-csv` or replace everything with `/api/settings/import-csv-replace`.

#### Example

```csv
unit_id,unit_type,device_type,deployed,retired,note,project_id,location,address,coordinates,last_calibrated,next_calibration_due,deployed_with_modem_id,ip_address,phone_number,hardware_model
BE1234,series3,seismograph,true,false,Primary sensor,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",2025-01-15,2026-01-15,MDM001,,,
MDM001,modem,modem,true,false,Field modem,PROJ-001,"Station A","123 Market St, San Francisco, CA","37.7937,-122.3965",,,,"192.0.2.10","+1-555-0100","Raven XTV"
```

See [sample_roster.csv](sample_roster.csv) for a minimal working example.

### Emitter Reporting
- **POST** `/emitters/report` - Submit status report from a seismograph unit
- **POST** `/api/series3/heartbeat` - Series 3 multi-unit telemetry payload
- **POST** `/api/series4/heartbeat` - Series 4 (Micromate) multi-unit telemetry payload
- **GET** `/fleet/status` - Retrieve status of all seismograph units (legacy)

### Photo Management
- **GET** `/api/unit/{unit_id}/photos` - List photos for a unit
- **GET** `/api/unit/{unit_id}/photo/{filename}` - Serve specific photo file

## API Documentation

Once running, interactive API documentation is available at:
- **Swagger UI**: http://localhost:8001/docs
- **ReDoc**: http://localhost:8001/redoc

## Testing the API

### Using curl

**Submit a report:**
```bash
curl -X POST http://localhost:8001/emitters/report \
  -H "Content-Type: application/json" \
  -d '{
    "unit": "SEISMO-001",
    "unit_type": "series3",
    "timestamp": "2025-11-20T10:30:00",
    "file": "event_20251120_103000.dat",
    "status": "OK"
  }'
```

**Get fleet status:**
```bash
curl http://localhost:8001/api/roster
```

**Import roster from CSV:**
```bash
curl -X POST http://localhost:8001/api/roster/import-csv \
  -F "file=@sample_roster.csv" \
  -F "update_existing=true"
```

### Using Python

```python
import requests
from datetime import datetime

# Submit report
response = requests.post(
    "http://localhost:8001/emitters/report",
    json={
        "unit": "SEISMO-001",
        "unit_type": "series3",
        "timestamp": datetime.utcnow().isoformat(),
        "file": "event_20251120_103000.dat",
        "status": "OK"
    }
)
print(response.json())

# Get fleet status
response = requests.get("http://localhost:8001/api/roster")
print(response.json())

# Import CSV
with open('roster.csv', 'rb') as f:
    files = {'file': f}
    data = {'update_existing': 'true'}
    response = requests.post(
        "http://localhost:8001/api/roster/import-csv",
        files=files,
        data=data
    )
print(response.json())
```

## Data Model

### RosterUnit Table (Fleet Roster)

**Common fields**

| Field | Type | Description |
|-------|------|-------------|
| id | string | Unit identifier (primary key) |
| unit_type | string | Hardware model name (default: `series3`) |
| device_type | string | Device type: `"seismograph"`, `"modem"`, or `"slm"` (sound level meter) |
| deployed | boolean | Whether the unit is in the field |
| retired | boolean | Removes the unit from deployments but preserves history |
| note | string | Notes about the unit |
| project_id | string | Associated project identifier |
| location | string | Legacy location label |
| address | string | Human-readable address |
| coordinates | string | `lat,lon` pair used by the map |
| last_updated | datetime | Last modification timestamp |

**Seismograph-only fields**

| Field | Type | Description |
|-------|------|-------------|
| last_calibrated | date | Last calibration date |
| next_calibration_due | date | Next calibration date |
| deployed_with_modem_id | string | Which modem is paired during deployment |

**Modem-only fields**

| Field | Type | Description |
|-------|------|-------------|
| ip_address | string | Assigned IP (static or DHCP) |
| phone_number | string | Cellular number for the modem |
| hardware_model | string | Modem hardware reference |

**Sound Level Meter (SLM) fields**

| Field | Type | Description |
|-------|------|-------------|
| slm_host | string | Direct IP address for SLM (if not using modem) |
| slm_tcp_port | integer | TCP control port (default: 2255) |
| slm_ftp_port | integer | FTP file transfer port (default: 21) |
| slm_model | string | Device model (NL-43, NL-53) |
| slm_serial_number | string | Manufacturer serial number |
| slm_frequency_weighting | string | Frequency weighting setting (A, C, Z) |
| slm_time_weighting | string | Time weighting setting (F=Fast, S=Slow) |
| slm_measurement_range | string | Measurement range setting |
| slm_last_check | datetime | Last status check timestamp |
| deployed_with_modem_id | string | Modem pairing (shared with seismographs) |

### Device Type Schema

Terra-View supports three device types with the following standardized `device_type` values:

- **`"seismograph"`** (default) - Seismic monitoring devices (Series 3, Series 4, Micromate)
  - Uses: calibration dates, modem pairing
  - Examples: BE1234, UM12345 (Series 3/4 units)

- **`"modem"`** - Field modems and network equipment
  - Uses: IP address, phone number, hardware model
  - Examples: MDM001, MODEM-2025-01

- **`"slm"`** - Sound level meters (Rion NL-43/NL-53)
  - Uses: TCP/FTP configuration, measurement settings, modem pairing
  - Examples: SLM-43-01, NL43-001

**Important**: All `device_type` values must be lowercase. The legacy value `"sound_level_meter"` has been deprecated in favor of the shorter `"slm"`. Run `backend/migrate_standardize_device_types.py` to update existing databases.

### Emitter Table (Device Check-ins)

| Field | Type | Description |
|-------|------|-------------|
| id | string | Unit identifier (primary key) |
| unit_type | string | Reported device type/model |
| last_seen | datetime | Last report timestamp |
| last_file | string | Last file processed |
| status | string | Current status: OK, Pending, Missing |
| notes | string | Optional notes (nullable) |

### IgnoredUnit Table (Noise Management)

| Field | Type | Description |
|-------|------|-------------|
| id | string | Unit identifier (primary key) |
| reason | string | Optional context for ignoring |
| ignored_at | datetime | When the ignore action occurred |

### UserPreferences Table (Settings Storage)

| Field | Type | Description |
|-------|------|-------------|
| id | integer | Always 1 (single-row table) |
| timezone | string | Display timezone (default: America/New_York) |
| theme | string | UI theme: auto, light, or dark |
| auto_refresh_interval | integer | Dashboard refresh interval in seconds |
| date_format | string | Date format preference |
| table_rows_per_page | integer | Default pagination size |
| calibration_interval_days | integer | Default days between calibrations |
| calibration_warning_days | integer | Warning threshold before calibration due |
| status_ok_threshold_hours | integer | Hours for OK status threshold |
| status_pending_threshold_hours | integer | Hours for Pending status threshold |
| updated_at | datetime | Last preference update timestamp |

## Project Structure

```
seismo-fleet-manager/
├── backend/
│   ├── main.py              # FastAPI app entry point
│   ├── database.py          # SQLAlchemy database configuration
│   ├── models.py            # Database models (RosterUnit, Emitter, IgnoredUnit, UserPreferences)
│   ├── routes.py            # Legacy API endpoints + Series 3/4 heartbeat endpoints
│   ├── routers/             # Modular API routers
│   │   ├── roster.py        # Fleet status endpoints
│   │   ├── roster_edit.py   # Roster management & CSV import
│   │   ├── units.py         # Unit detail endpoints
│   │   ├── photos.py        # Photo management
│   │   ├── dashboard.py     # Dashboard partials
│   │   ├── dashboard_tabs.py # Dashboard tab endpoints
│   │   └── settings.py      # Settings, preferences, and data management
│   ├── services/
│   │   ├── snapshot.py      # Fleet status snapshot logic
│   │   ├── database_backup.py # Database backup and restore service
│   │   └── backup_scheduler.py # Automatic backup scheduler
│   ├── migrate_add_device_types.py # SQLite migration for v0.2 schema
│   ├── migrate_add_user_preferences.py # SQLite migration for v0.3 schema
│   └── static/              # Static assets (CSS, etc.)
├── create_test_db.py        # Generate a sample SQLite DB with mixed devices
├── templates/               # Jinja2 HTML templates
│   ├── base.html            # Base layout with sidebar
│   ├── dashboard.html       # Main dashboard
│   ├── roster.html          # Fleet roster table
│   ├── unit_detail.html     # Unit detail page
│   ├── settings.html        # Roster manager UI
│   └── partials/            # HTMX partial templates
│       ├── roster_table.html
│       ├── retired_table.html
│       ├── ignored_table.html
│       └── unknown_emitters.html
├── data/                    # SQLite database & photos (persisted)
│   └── backups/             # Database snapshots directory
├── scripts/
│   └── clone_db_to_dev.py   # Remote database cloning utility
├── docs/
│   └── DATABASE_MANAGEMENT.md # Database backup/restore guide
├── requirements.txt         # Python dependencies
├── Dockerfile               # Docker container definition
├── docker-compose.yml       # Docker Compose configuration
├── CHANGELOG.md             # Version history
├── FRONTEND_README.md       # Frontend documentation
└── README.md                # This file
```

## Docker Commands

**Build the image:**
```bash
docker compose build
```

**Start in foreground:**
```bash
docker compose up
```

**Start in background:**
```bash
docker compose up -d
```

**View logs:**
```bash
docker compose logs -f seismo-backend
```

**Restart service:**
```bash
docker compose restart
```

**Rebuild and restart:**
```bash
docker compose up -d --build
```

**Stop and remove containers:**
```bash
docker compose down
```

**Remove containers and volumes:**
```bash
docker compose down -v
```

## Release Highlights

### v0.11.0 — 2026-05-15
- **Soft-Remove Monitoring Locations**: Mark a location as no longer actively monitored without destroying history. Closes active unit assignments and cancels pending scheduled actions; historical events stay attributed. Restore brings it back. Surfaces as a Removed Locations collapsed section on the project page.
- **Per-Unit Deployment Gantt**: Visual timeline above the deployment history list on each unit detail page. Color-coded bars per location, today marker, mergeable-group dashed underlines, click a bar to scroll its detail row into view.
- **Merge Consecutive Deployments**: Auto-detects runs of same-location assignments within a 7-day gap and offers a one-click "Merge into one" button. Preserves notes, ingest source, and writes an `assignment_merged` audit entry.
- **Delete Assignment for Mis-Clicks**: Trash icon on each row of the location's Deployment History panel. Hard-deletes the assignment with a safety check that refuses if real MonitoringSessions sit inside the window (those should go through Unassign instead).
- **Drag-to-Reorder Location Cards**: Six-dot drag handle on each card; drop order persists via a new `/locations/reorder` endpoint. Removed locations stay sorted by removal date (their order is historical).
- **Three-Dot Kebab Menu**: Replaces the inline Unassign / Edit / Remove / Delete pill row with a single ⋮ menu. Much cleaner card layout, especially for projects with many locations.
- **Event Count on Vibration Cards**: Vibration locations now show "{N} events" instead of "Sessions: 0" (sessions don't exist under the watcher-forward pipeline). Sound locations are unchanged.
- **Project Location Map**: Right column of the project overview is now a Leaflet map with a pin per location. Click pin → scrolls + flashes the matching card. Replaces the lightly-used Upcoming Actions panel (still discoverable via a link to the Schedules tab when actions exist).
- **Stricter Location Fuzzy Matching**: Metadata-backfill no longer suggests obviously-wrong matches. WRatio was over-confident on location names ("Area 2 - Brookville Dam - Loc 2" vs "Area 1 - Loc 1 - 87 Jenks" used to score 86%); now uses `token_set_ratio` + a multi-digit penalty so disjoint address numbers correctly demote the score.
- **Fixed: Multiple typeahead dropdowns weren't clickable**: Same JSON.stringify quote-collision bug surfaced in three places (location Remove button, backfill typeahead, project-merge dropdown). All three fixed by switching to `data-*` attributes + trampoline functions.
- **Fixed: Merge-project modal had to be scrolled to see options**: Modal body's `flex-1 overflow-y-auto` collapsed too tight; added `min-height` so the dropdown has room to render below the input.

### v0.10.0 — 2026-05-14
- **SFM Integration**: terra-view now consumes events from the SFM (Seismograph Field Module) backend in real time, with a fleet-wide events page at `/sfm`, per-unit attribution against project assignment windows, and a project-level vibration roll-up that uses SFM data as the single source of truth.
- **SFM-Primary Seismograph Status**: Deployed seismograph status (OK/Pending/Missing) now flows from SFM event forwards first; the watcher heartbeat stays as a transparent backup. Each unit's active table row shows a small `SFM` or `HB` badge so operators can see at a glance which signal is currently driving the status.
- **Dashboard Rework**: Top row reordered to Recent Alerts → Recent Call-Ins (double-wide) → Fleet Summary. Today's Schedule moves to a horizontal collapsible card below the Fleet Map, auto-expanding only when there's a pending action. Recent Call-Ins now sources from SFM event forwards instead of the legacy watcher-heartbeat endpoint.
- **Event Detail Modal**: Click any event anywhere in the app to open a rich detail modal showing peak particle velocity per channel, microphone dB(L), sensor self-check results, device/recording metadata, and download buttons for the original Blastware binary and sidecar JSON. Includes an inline JSON viewer with one-click copy.
- **Sortable Events Tables**: Every events table (project events, unit-detail events, fleet-wide /sfm) now supports clickable column-header sorting with directional indicators. Defaults to newest-first.
- **Events Attribution & Backfill**: Each SFM event is automatically attributed to a project/location based on `UnitAssignment` time windows. Unattributed events get a diagnostic showing the nearest assignment and a delta-days gap. The metadata-backfill tool in `/tools` scans operator-typed project/sensor-location strings in event sidecars and clusters them via fuzzy matching to propose new assignment retroactives.
- **Projects Tools**: New `/tools` workflow hub consolidates Pair Devices, Project Tidy (fuzzy-detect + merge duplicate projects), Metadata Backfill, Reports, and Swap Detection (placeholder).
- **Sidebar Reorganization**: Devices → Projects → Events → Tools → Job Planner → Settings. Devices is now a single entry with internal tabs (All Devices / Seismographs / Sound Level Meters / Modems / Pair Devices).
- **Developer → SFM Admin**: New `/admin/sfm` page surfacing SFM health, per-unit roll-up from `/db/units`, recent-events table with forwarding latency (so operators can spot stale watcher forwards), stale-table counts, and a raw API tester. Companion `/admin/slmm` page covers SLMM health + raw API.
- **"Overall Peak" excludes False Triggers**: The project-level Overall Peak KPI tile now excludes events flagged as false triggers — operators see the highest real event, not the biggest sensor glitch.
- **Synology Deployment Doc**: New `docs/SYNOLOGY_DEPLOYMENT.md` covers migrating the stack to an always-on office NAS, including phased rollout, data rsync, watcher repoint, external-access (Tailscale or reverse-proxy), and rollback plan.

### v0.8.0 — 2026-03-18
- **Watcher Manager**: Admin page for monitoring field watcher agents with live status cards, log tails, and one-click update triggering
- **Watcher Status Fix**: Agent status now reflects heartbeat connectivity (missing if not heard from in >60 min) rather than unit-level data staleness
- **Live Refresh**: Watcher Manager surgically patches status, last-seen, and pending indicators every 30s without a full page reload

### v0.7.0 — 2026-03-07
- **Project Status Management**: On-hold and archived project states with automatic cancellation of pending actions
- **Manual SD Card Upload**: Upload offline NRL/SLM data directly from SD card (ZIP or multi-file); auto-creates monitoring sessions from `.rnh` metadata
- **Combined Report Wizard**: Multi-session Excel report generation with location grouping, period type filtering, and ZIP download
- **NL32 Support**: Report generator and web viewer now handle NL32 measurement data
- **Chart Preview**: Live chart preview in the report generator matching final output styling
- **Standalone SLM Mode**: SLMs can now be configured without a paired modem (direct IP)
- **Vibration Project Isolation**: Vibration project views no longer show SLM-specific tabs
- **MonitoringSession Rename**: `RecordingSession` renamed to `MonitoringSession` throughout; run migration before deploying

### v0.6.1 — 2026-02-16
- **One-Off Recording Schedules**: Schedule single recordings with specific start/end datetimes
- **Bidirectional Pairing Sync**: Device-modem pairing now updates both sides automatically
- **Scheduler Timezone Fix**: One-off schedule times use local time instead of UTC

### v0.6.0 — 2026-02-06
- **Calendar & Reservation Mode**: Fleet calendar view with device deployment scheduling and reservation system
- **Device Pairing Interface**: New `/pair-devices` page with two-column layout for linking recorders with modems, fuzzy-search, and visual pairing workflow
- **Calibration UX Overhaul**: Users now set date of previous calibration (not expiry); seismograph list enhanced with color-coded calibration status, filtering, and sorting
- **Modem Dashboard**: Model number as dedicated config, modem login links, list view format, and pairing options accessible from modem page
- **SLMM Improvements**: Device control lock prevents command flooding, fixed polling intervals and scheduled downloads
- **UI Polish**: Tab state persists in URL hash, settings save without reload, scheduler changes cascade to events, fixed mobile type display

### v0.4.3 — 2026-01-14
- **Sound Level Meter workflow**: Roster manager surfaces SLM metadata, supports rename actions, and adds return-to-project navigation plus schedule/unit templates for project planning.
- **Project insight panels**: Project dashboards now expose file and session lists so teams can see what each project stores before diving into units.
- **Project view polish**: FTP browser supports folder downloads, the timer display was reimplemented, and the project/device templates gained edit modals for projects and locations to streamline navigation.
- **SLM sync & accuracy**: Configuration edits now propagate to SLMM (which caches configs for faster responses) and the live view uses the correct DRD fields so telemetry aligns with the control center.

### v0.4.0 — 2025-12-16
- **Database Management System**: Complete backup and restore functionality with manual snapshots, restore operations, and upload/download capabilities
- **Remote Database Cloning**: New `clone_db_to_dev.py` script for copying production database to remote dev servers over WAN
- **Automatic Backup Scheduler**: Background service for scheduled backups with configurable retention management
- **Database Tab**: New dedicated tab in Settings for all database operations with real-time statistics
- **Settings Reorganization**: Improved tab structure - renamed "Data Management" to "Roster Management", moved CSV Replace Mode, created Database tab
- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` with complete guide to backup/restore workflows, API reference, and best practices

### v0.3.3 — 2025-12-12
- **Improved Mobile Navigation**: Hamburger menu moved to bottom nav bar (no more floating button covering content)
- **Better Status Visibility**: Larger status dots (16px) in dashboard fleet overview for easier at-a-glance status checks
- **Cleaner Roster Cards**: Location navigation links moved to detail modal only, reducing clutter in card view

### v0.3.2 — 2025-12-12
- **Progressive Web App (PWA)**: Complete mobile optimization with offline support, installable as standalone app
- **Mobile-First UI**: Hamburger menu, bottom navigation bar, card-based roster view optimized for touch
- **Tap-to-Navigate**: Location links open in user's preferred navigation app (Google Maps, Apple Maps, Waze)
- **Offline Editing**: Service worker + IndexedDB for offline operation with automatic sync when online
- **Unit Detail Modals**: Bottom sheet modals for quick unit info access with full edit capabilities
- **Hard Reload Utility**: "Clear Cache & Reload" button to force fresh assets (helpful for development)

### v0.3.1 — 2025-12-12
- **Dashboard Alerts**: Only Missing units show in notifications (Pending units no longer alert)
- **Status Fixes**: Fixed "Unknown" status issues in mobile card views and detail modals
- **Backend Improvements**: Safer data access with `.get()` defaults to prevent errors

### v0.3.0 — 2025-12-09
- **Series 4 Support**: New `/api/series4/heartbeat` endpoint with auto-detection for Micromate units (UM##### pattern)
- **Settings Redesign**: Completely redesigned Settings page with 4-tab interface (General, Data Management, Advanced, Danger Zone)
- **User Preferences**: Backend storage for timezone, theme, auto-refresh interval, calibration defaults, and status thresholds
- **Development Labels**: Visual indicators to distinguish dev from production environments
- **Timezone Support**: Comprehensive timezone handling with human-readable timestamps site-wide
- **Quality of Life**: Relative timestamps, status icons for accessibility, breadcrumb navigation, copy-to-clipboard, search functionality

### v0.2.1 — 2025-12-03
- Added the `/settings` roster manager with CSV export/import, live stats, and danger-zone table reset actions
- Deployed/Benched/Retired/Ignored tabs now have dedicated HTMX partials, sorting, and inline actions
- Unit detail pages expose device-type specific metadata (calibration windows, modem pairing, IP/phone fields)
- Snapshot summary and dashboard counts now focus on deployed units and include address/coordinate data

### v0.2.0 — 2025-12-03
- Introduced device-type aware roster schema (seismograph vs modem) plus migration + `create_test_db.py` helper
- Added Ignore list model/endpoints to quarantine noisy emitters directly from the roster
- Roster page gained Add Unit + CSV Import modals, HTMX-driven updates, and unknown emitter callouts
- Snapshot service now returns active/benched/retired/unknown buckets containing richer metadata

### v0.1.1 — 2025-12-02
- **Roster Editing API**: Full CRUD operations for managing your fleet roster
- **CSV Import**: Bulk upload roster data from CSV files
- **Enhanced Data Model**: Added project_id and location fields to roster
- **Bug Fixes**: Improved database session management and error handling

See [CHANGELOG.md](CHANGELOG.md) for the full release notes.

## Future Enhancements

- Email/SMS alerts for missing units
- Historical data tracking and reporting
- Multi-user authentication
- PostgreSQL support for larger deployments
- Advanced filtering and search
- Export roster to various formats

## License

MIT

## Version

**Current: 0.11.0** — Soft-remove locations, per-unit Gantt, merge/delete assignments, drag-to-reorder, three-dot kebab menu, event count on vibration cards, project location map, stricter backfill fuzzy match, modal/typeahead bug fixes (2026-05-15)

Previous: 0.10.0 — SFM integration, SFM-primary seismograph status, dashboard rework, sortable events tables, event detail modal, /admin/sfm + /admin/slmm diagnostic pages, Tools workflow hub (2026-05-14)

0.9.4 — Modular project types, deleted project management, swap modal search, roster auto-refresh fix (2026-04-06)

0.9.3 — Monitoring session detail page, configurable period windows, vibration project redesign, modem assignment on locations (2026-03-28)

0.9.2 — Deployment records, allocated status, quick-info unit modal, inline seismograph editing (2026-03-27)

0.9.1 — Fix location slots not persisting on save/reload (2026-03-20)

0.9.0 — Job Planner redesign, monitoring locations, estimated units, smart color picker, calendar bar tooltips, toast notifications (2026-03-19)

0.8.0 — Watcher Manager admin page, live agent status refresh, watcher connectivity-based status (2026-03-18)

0.7.1 — Out-for-calibration status, reservation modal, migration fixes (2026-03-12)

0.7.0 — Project status management, manual SD card upload, combined report wizard, NL32 support, MonitoringSession rename (2026-03-07)

0.6.1 — One-off recording schedules, bidirectional pairing sync, scheduler timezone fix (2026-02-16)

0.6.0 — Calendar & reservation mode, device pairing interface, calibration UX overhaul, modem dashboard enhancements (2026-02-06)

0.5.1 — Dashboard schedule view with today's actions panel, new Terra-View branding and logo rework (2026-01-27)

0.4.4 — Recurring schedules, alerting UI, report templates + RND viewer, and SLM workflow polish (2026-01-23)

0.4.3 — SLM roster/project view refresh, project insight panels, FTP browser folder downloads, and SLMM sync (2026-01-14)

0.4.2 — SLM configuration interface with TCP/FTP controls, modem diagnostics, and dashboard endpoints for Sound Level Meters (2026-01-05)

0.4.1 — Sound Level Meter integration with full management UI for SLM units (2026-01-05)

0.4.0 — Database management system with backup/restore and remote cloning (2025-12-16)

0.3.3 — Mobile navigation improvements and better status visibility (2025-12-12)

0.3.2 — Progressive Web App with mobile optimization (2025-12-12)

0.3.1 — Dashboard alerts and status fixes (2025-12-12)

0.3.0 — Series 4 support, settings redesign, user preferences (2025-12-09)

0.2.1 — Settings & roster manager refresh (2025-12-03)

0.2.0 — Device-type aware roster + ignore list (2025-12-03)

0.1.1 — Roster Management & CSV Import (2025-12-02)

0.1.0 — Initial Release (2024-11-20)
