# Changelog

All notable changes to Terra-View will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.10.0] - 2026-05-14

This release brings terra-view onto the SFM (Seismograph Field Module) event pipeline. Triggered events forwarded by series3-watcher now land in SFM, and terra-view reads from that store as the authoritative source for vibration data. The watcher heartbeat is preserved as a transparent fallback signal.

### Added
- **SFM Integration**: New fleet-wide events page at `/sfm` listing every event ingested by SFM, with filters for serial, date range, false-trigger flag, and limit. Unit detail pages and project-location pages show their own attributed subsets of the same event stream.
- **Event Detail Modal**: Shared across `/sfm`, unit detail, and project-location pages — clicking any event opens a rich modal showing peaks per channel (PVS color-coded by magnitude), microphone dB(L) + ZC frequency + time of peak, sensor self-check table with pass/fail per channel, device/recording metadata (firmware, battery, calibration date, geo range), and download buttons for the original Blastware binary and the sidecar JSON. Includes an inline pretty-printed JSON viewer with copy-to-clipboard.
- **Events Attribution Engine** (`backend/services/sfm_events.py`): Per-event attribution against `UnitAssignment` time windows. Events outside any assignment window surface in an "Unattributed" bucket with the nearest-assignment diagnostic (which location, signed delta in days).
- **Metadata Backfill Tool** (`/tools` → Backfill from event metadata): Scans operator-typed `project` and `sensor_location` strings in event sidecars, fuzzy-clusters them via `rapidfuzz.WRatio`, and proposes retroactive `UnitAssignment` records to attribute orphan events. Tracks operator decisions per cluster across re-scans.
- **Project Tidy Tool** (`/tools` → Project Tidy): Fuzzy-detect duplicate projects and bulk-merge them with a single click. Source projects soft-deleted with full audit trail.
- **Vibration Summary on Project Pages**: New roll-up card on vibration project detail pages showing per-location event counts, the project's "Overall Peak" PVS (false triggers excluded), last event timestamp, and a Top Locations by Activity list.
- **SFM-Primary Seismograph Status**: `emit_status_snapshot()` now consults SFM's `/db/units` (cached 15s) before falling back to `Emitter.last_seen` for each seismograph. The fresher signal wins; the choice is recorded in a new per-unit `last_seen_source` field. A small `SFM` (orange) or `HB` (gray) badge on each unit's active-table row shows which path is currently driving the status.
- **Dashboard Rework**: Top row reordered to Recent Alerts → Recent Call-Ins (double-wide) → Fleet Summary. Today's Schedule moved to a horizontal collapsible card below the Fleet Map, auto-expanding only when pending actions exist. Recent Call-Ins now sources from a new `/api/recent-event-callins` endpoint backed by SFM event forwards instead of the watcher-heartbeat endpoint.
- **Sortable Events Tables**: `/sfm` and unit-detail SFM Events tables now have clickable column headers with ↕/↓/↑ indicators. Default sort is Timestamp DESC. Click same column to toggle direction; click different column to switch and reset to DESC. Pure client-side over cached rows — no re-fetches.
- **Developer → SFM Admin** (`/admin/sfm`): Health banner with reachability indicator, terra-view↔SFM connection panel, 4 KPI tiles (known units, total events, stale `monitor_log` rows, stale `ach_sessions` rows), per-unit roll-up table, recent-events table with color-coded forwarding latency (so stale watcher forwards stand out), and a raw API tester for any `/api/sfm/*` path.
- **Developer → SLMM Admin** (`/admin/slmm`): Stripped-down companion page — health, connection info, raw API tester.
- **Tools Workflow Hub** (`/tools`): New top-level sidebar entry consolidating Pair Devices, Project Tidy, Metadata Backfill, Reports (info card), and Swap Detection (placeholder).
- **Sidebar Reorganization**: Devices → Projects → Events → Tools → Job Planner → Settings. Devices is now a single entry with internal tabs (All Devices / Seismographs / Sound Level Meters / Modems / Pair Devices) replacing five separate sidebar items.
- **Synology Deployment Doc** (`docs/SYNOLOGY_DEPLOYMENT.md`): End-to-end playbook for migrating the stack to an always-on office NAS — phased rollout (pre-stage, data rsync, watcher repoint, external access, decommission), Tailscale vs reverse-proxy options, rollback plan, and gotchas.

### Changed
- **Overall Peak excludes false triggers**: The project-level "Overall Peak" KPI tile (and the underlying `_compute_stats()` function in `sfm_events.py`) now skip events flagged as false triggers when computing the highest PVS, so operators see the highest real event rather than the biggest sensor glitch. `false_trigger_count` still includes flagged events so operators can see how many were filtered out.
- **`RosterUnit.note` Editing**: Inline edit on seismograph cards is more forgiving and now auto-saves on blur.
- **Sidebar Nav Renamed**: Old "Fleet" sidebar entry → "Devices" (renamed because it always meant the device list, not the broader fleet view).

### Fixed
- **Status drift between watcher heartbeat and actual event arrivals**: Seismographs are now reported with whichever signal is more recent — eliminates the case where a unit had recent SFM events but a stale heartbeat (or vice-versa) showed the wrong status.
- **Event modal: Record Type always showed "Waveform"**: Workaround client-side — Record Type now derived from the Blastware filename's last-char code (`H`=Histogram, `W`=Waveform, `M`=Manual, `E`=Event, `C`=Combo). The proper fix lives in SFM's sidecar parser; tracked separately.
- **Event modal: Mic PSI tile removed**: Operators only care about dB(L); the redundant PSI tile was dropped.

### Migration Notes
Run on each database before deploying. Every migration is idempotent.

```bash
# Cleanest: re-run all migrations in chronological order.
# Already-applied migrations no-op safely.
for f in backend/migrate_*.py; do
  docker exec terra-view-terra-view-1 python3 "/app/backend/$(basename $f)"
done
```

Migrations new in this release:
- `migrate_add_metadata_backfill.py` — adds `unit_assignments.source` column and `metadata_backfill_decisions` table for the Metadata Backfill tool

### Deployment Notes
- **`SFM_BASE_URL`**: Confirm prod's `docker-compose.yml` sets this for the terra-view service (typically `http://sfm:8200` for the in-stack SFM container, or an external URL if SFM lives elsewhere).
- **Watcher repoint**: series3-watcher's `sfm_forward_url` should point at `https://<your-terra-view-host>/api/sfm` (proxy-based — no second port forward needed). Watcher composes the full path `/db/import/blastware_file` itself.

---

## [0.9.4] - 2026-04-06

### Added
- **Modular Project Types**: Projects now support optional modules (Sound Monitoring, Vibration Monitoring) selectable at creation time. The project header and dashboard dynamically show/hide tabs and actions based on which modules are enabled, and modules can be added or removed after creation.
- **Deleted Project Management**: Settings page now includes a section for soft-deleted projects with options to restore or permanently delete each one. Deleted projects load automatically when the Data tab is opened.

### Changed
- **Swap Modal Search**: The unit/modem swap modal on vibration location detail pages now includes live search filtering for both seismographs and modems, making it easier to find the right unit in large fleets.

### Fixed
- **Roster Auto-Refresh No Longer Disrupts Scroll/Sort**: The roster page's 30-second background refresh now updates status, age, and last-seen values in-place via a lightweight JSON poll instead of replacing the entire table HTML. Sort order, scroll position, and active filters are all preserved across refreshes.

### Migration Notes
Run on each database before deploying:
```bash
docker compose exec terra-view python3 backend/migrate_add_project_modules.py
```

---

## [0.9.3] - 2026-03-28

### Added
- **Monitoring Session Detail Page**: New dedicated page for each session showing session info, data files (with View/Report/Download actions), an editable session panel, and report actions.
- **Session Calendar with Gantt Bars**: Monthly calendar view below the session list, showing each session as a Gantt-style bar. The dim bar represents the full device on/off window; the bright bar highlights the effective recording window. Bars extend edge-to-edge across day cells for sessions spanning midnight.
- **Configurable Period Windows**: Sessions now store `period_start_hour` and `period_end_hour` to define the exact hours that count toward reports, replacing hardcoded day/night defaults. The session edit panel shows a "Required Recording Window" section with a live preview (e.g. "7:00 AM → 7:00 PM") and a Defaults button that auto-fills based on period type.
- **Report Date Field**: Sessions can now store an explicit `report_date` to override the automatic target-date heuristic — useful when a device ran across multiple days but only one specific day's data is needed for the report.
- **Effective Window on Session Info**: Session detail and session cards now show an "Effective" row displaying the computed recording window dates and times in local time.
- **Vibration Project Redesign**: Vibration project detail page is stripped back to project details and monitoring locations only. Each location supports assigning a seismograph and optional modem. Sound-specific tabs (Schedules, Sessions, Data Files, Assigned Units) are hidden for vibration projects.
- **Modem Assignment on Locations**: Vibration monitoring locations now support an optional paired modem alongside the seismograph. The swap endpoint handles both assignments atomically, updating bidirectional pairing fields on both units.
- **Available Modems Endpoint**: New `GET /api/projects/{project_id}/available-modems` endpoint returning all deployed, non-retired modems for use in assignment dropdowns.

### Fixed
- **Active Assignment Checks**: Unified all `UnitAssignment` "active" checks from `status == "active"` to `assigned_until IS NULL` throughout `project_locations.py` and `projects.py` for consistency with the canonical active definition.

### Changed
- **Sound-Only Endpoint Guards**: FTP browser, RND viewer, Excel report generation, combined report wizard, and data upload endpoints now return HTTP 400 if called on a non-sound-monitoring project.

### Migration Notes
Run on each database before deploying:
```bash
docker compose exec terra-view python3 backend/migrate_add_session_period_hours.py
docker compose exec terra-view python3 backend/migrate_add_session_report_date.py
```

---

## [0.9.2] - 2026-03-27

### Added
- **Deployment Records**: Seismographs now track a full deployment history (location, project, dates). Each deployment is logged on the unit detail page with start/end dates, and the fleet calendar service uses this history for availability calculations.
- **Allocated Unit Status**: New `allocated` status for units reserved for an upcoming job but not yet deployed. Allocated units appear in the dashboard summary, roster filters, and devices table with visual indicators.
- **Project Allocation**: Units can be linked to a project via `allocated_to_project_id`. Allocation is shown on the unit detail page and in a new quick-info modal accessible from the fleet calendar and roster.
- **Quick-Info Unit Modal**: Click any unit in the fleet calendar or roster to open a modal showing cal status, project allocation, upcoming jobs, and deployment state — without leaving the page.
- **Cal Date in Planner**: When a unit is selected for a monitoring location slot in the Job Planner, its calibration expiry date is now shown inline so you can spot near-expiry units before committing.
- **Inline Seismograph Editing**: Unit rows in the seismograph dashboard now support inline editing of cal date, notes, and deployment status without navigating to the full detail page.

### Migration Notes
Run on each database before deploying:
```bash
docker compose exec terra-view python3 backend/migrate_add_allocated.py
docker compose exec terra-view python3 backend/migrate_add_deployment_records.py
```

---

## [0.9.1] - 2026-03-20

### Fixed
- **Location slots not persisting**: Empty monitoring location slots (no unit assigned yet) were lost on save/reload. Added `location_slots` JSON column to `job_reservations` to store the full slot list including empty slots.
- **Modems in Recent Alerts**: Modems no longer appear in the dashboard Recent Alerts panel — alerts are for seismographs and SLMs only. Modem status is still tracked internally via paired device inheritance.
- **Series 4 heartbeat `source_id`**: Updated heartbeat endpoint to accept the new `source_id` field from Series 4 units with fallback to the legacy field for backwards compatibility.

### Migration Notes
Run on each database before deploying:
```bash
docker compose exec terra-view python3 backend/migrate_add_location_slots.py
```

---

## [0.9.0] - 2026-03-19

### Added
- **Job Planner**: Full redesign of the Fleet Calendar into a two-tab Job Planner / Calendar interface
  - **Planner tab**: Create and manage job reservations with name, device type, dates, color, estimated units, and monitoring locations
  - **Calendar tab**: 12-month rolling heatmap with colored job bars per day; confirmed jobs solid, planned jobs dashed
  - **Monitoring Locations**: Each job has named location slots (filled = unit assigned, empty = needs a unit); progress shown as `2/5` with colored squares that fill as units are assigned
  - **Estimated Units**: Separate planning number independent of actual location count; shown prominently on job cards
  - **Fleet Summary panel**: Unit counts as clickable filter buttons; unit list shows reservation badges with job name, dates, and color
  - **Available Units panel**: Shows units available for the job's date range when assigning
  - **Smart color picker**: 18-swatch palette + custom color wheel; new jobs auto-pick a color maximally distant in hue from existing jobs
  - **Job card progress**: `est. N · X/Y (Z more)` with filled/empty squares; amber → green when fully assigned
  - **Promote to Project**: Promote a planned job to a tracked project directly from the planner form
  - **Collapsible job details**: Name, dates, device type, color, project link, and estimated units collapse into a summary header
  - **Calendar bar tooltips**: Hover any job bar to see job name and date range
  - **Hash-based tab persistence**: `#cal` in URL restores Calendar tab on refresh; device type toggle preserves active tab
  - **Auto-scroll to today**: Switching to Calendar tab smooth-scrolls to the current month
  - **Upcoming project status**: New `upcoming` status for projects promoted from reservations
- **Job device type**: Reservations carry a device type so they only appear on the correct calendar
- **Project filtering by device type**: Projects only appear on the calendar matching their type (vibration → seismograph, sound → SLM, combined → both)
- **Confirmed/Planned toggles**: Independent show/hide toggles for job bar layers on the calendar
- **Cal expire dots toggle**: Calibration expiry dots off by default, togglable

### Changed
- **Renamed**: "Fleet Calendar" / "Reservation Planner" → **"Job Planner"** throughout UI and sidebar
- **Project status dropdown**: Inline `<select>` in project header for quick status changes
- **"All Projects" tab**: Shows everything except deleted; default view excludes archived/completed
- **Toast notifications**: All `alert()` dialogs replaced with non-blocking toasts (green = success, red = error)

### Migration Notes
Run on each database before deploying:
```bash
docker compose exec terra-view python3 -c "
import sqlite3
conn = sqlite3.connect('/app/data/seismo_fleet.db')
conn.execute('ALTER TABLE job_reservations ADD COLUMN estimated_units INTEGER')
conn.commit()
conn.close()
"
```

---

## [0.8.0] - 2026-03-18

### Added
- **Watcher Manager**: New admin page (`/admin/watchers`) for monitoring field watcher agents
  - Live status cards per agent showing connectivity, version, IP, last-seen age, and log tail
  - Trigger Update button to queue a self-update on the agent's next heartbeat
  - Expand/collapse log tail with full-log expand mode
  - Live surgical refresh every 30 seconds via `/api/admin/watchers` — no full page reload, open logs stay open

### Changed
- **Watcher status logic**: Agent status now reflects whether Terra-View is hearing from the watcher (ok if seen within 60 minutes, missing otherwise) — previously reflected the worst unit status from the last heartbeat payload, which caused false alarms when units went missing

### Fixed
- **Watcher Manager meta row**: Dark mode background was white due to invalid `dark:bg-slate-850` Tailwind class; corrected to `dark:bg-slate-800`

---

## [0.7.1] - 2026-03-12

### Added
- **"Out for Calibration" Unit Status**: New `out_for_cal` status for units currently away for calibration, with visual indicators in the roster, unit list, and seismograph stats panel
- **Reservation Modal**: Fleet calendar reservation modal is now fully functional for creating and managing device reservations

### Changed
- **Retire Unit Button**: Redesigned to be more visually prominent/destructive to reduce accidental clicks

### Fixed
- **Migration Scripts**: Fixed database path references in several migration scripts
- **Docker Compose**: Removed dev override file from the repository; dev environment config kept separate

### Migration Notes
Run the following migration script once per database before deploying:
```bash
python backend/migrate_add_out_for_calibration.py
```

---

## [0.7.0] - 2026-03-07

### Added
- **Project Status Management**: Projects can now be placed `on_hold` or `archived`, with automatic cancellation of pending scheduled actions
- **Hard Delete Projects**: Support for permanently deleting projects, in addition to soft-delete with auto-pruning
- **Vibration Location Detail**: New dedicated template for vibration project location detail views
- **Vibration Project Isolation**: Vibration projects no longer show SLM-specific project tabs
- **Manual SD Card Data Upload**: Upload offline NRL data directly from SD card via ZIP or multi-file select
  - Accepts `.rnd`/`.rnh` files; parses `.rnh` metadata for session start/stop times, serial number, and store name
  - Creates `MonitoringSession` and `DataFile` records automatically; no unit assignment required
  - Upload panel on NRL detail Data Files tab with inline feedback and auto-refresh via HTMX
- **Standalone SLM Type**: New SLM device mode that operates without a modem (direct IP connection)
- **NL32 Data Support**: Report generator and web viewer now support NL32 measurement data format
- **Combined Report Wizard**: Multi-session combined Excel report generation tool
  - Wizard UI grouped by location with period type badges (day/night)
  - Each selected session produces one `.xlsx` in a ZIP archive
  - Period type filtering: day sessions keep last calendar date (7AM–6:59PM); night sessions span both days (7PM–6:59AM)
- **Combined Report Preview**: Interactive spreadsheet-style preview before generating combined reports
- **Chart Preview**: Live chart preview in the report generator matching final report styling
- **SLM Model Schemas**: Per-model configuration schemas for NL32, NL43, NL53 devices
- **Data Collection Mode**: Projects now store a data collection mode field with UI controls and migration

### Changed
- **MonitoringSession rename**: `RecordingSession` renamed to `MonitoringSession` throughout codebase; DB table renamed from `recording_sessions` to `monitoring_sessions`
  - Migration: `backend/migrate_rename_recording_to_monitoring_sessions.py`
- **Combined Report Split Logic**: Separate days now generate separate `.xlsx` files; NRLs remain one per sheet
- **Mass Upload Parsing**: Smarter file filtering — no longer imports unneeded Lp files or `.xlsx` files
- **SLM Start Time Grace Period**: 15-minute grace window added so data starting at session start time is included
- **NL32 Date Parsing**: Date now read from `start_time` field instead of file metadata
- **Project Data Labels**: Improved Jinja filters and UI label clarity for project data views

### Fixed
- **Dev/Prod Separation**: Dev server now uses Docker Compose override; production deployment no longer affected by dev config
- **SLM Modal**: Bench/deploy toggle now correctly shown in SLM unit modal
- **Auto-Downloaded Files**: Files downloaded by scheduler now appear in project file listings
- **Duplicate Download**: Removed duplicate file download that occurred following a scheduled stop
- **SLMM Environment Variables**: `TCP_IDLE_TTL` and `TCP_MAX_AGE` now correctly passed to SLMM service via docker-compose

### Technical Details
- `session_label` and `period_type` stored on `monitoring_sessions` table (migration: `migrate_add_session_period_type.py`)
- `device_model` stored on `monitoring_sessions` table (migration: `migrate_add_session_device_model.py`)
- Upload endpoint: `POST /api/projects/{project_id}/nrl/{location_id}/upload-data`
- ZIP filename format: `{session_label}_{project_name}_report.xlsx` (label first)

### Migration Notes
Run the following migration scripts once per database before deploying:
```bash
python backend/migrate_rename_recording_to_monitoring_sessions.py
python backend/migrate_add_session_period_type.py
python backend/migrate_add_session_device_model.py
```

---

## [0.6.1] - 2026-02-16

### Added
- **One-Off Recording Schedules**: Support for scheduling single recordings with specific start and end datetimes
- **Bidirectional Pairing Sync**: Pairing a device with a modem now automatically updates both sides, clearing stale pairings when reassigned
- **Auto-Fill Notes from Modem**: Notes are now copied from modem to paired device when fields are empty
- **SLMM Download Requests**: New `_download_request` method in SLMM client for binary file downloads with local save

### Fixed
- **Scheduler Timezone**: One-off scheduler times now use local time instead of UTC
- **Pairing Consistency**: Old device references are properly cleared when a modem is re-paired to a new device

## [0.6.0] - 2026-02-06

### Added
- **Calendar & Reservation Mode**: Fleet calendar view with reservation system for scheduling device deployments
- **Device Pairing Interface**: New two-column pairing page (`/pair-devices`) for linking recorders (seismographs/SLMs) with modems
  - Visual pairing interface with drag-and-drop style interactions
  - Fuzzy-search modem pairing for SLMs
  - Pairing options now accessible from modem page
  - Improved pair status sharing across views
- **Modem Dashboard Enhancements**:
  - Modem model number now a dedicated configuration field with per-model options
  - Direct link to modem login page from unit detail view
  - Modem view converted to list format
- **Seismograph List Improvements**:
  - Enhanced visibility with better filtering and sorting
  - Calibration dates now color-coded for quick status assessment
  - User sets date of previous calibration (not expiry) for clearer workflow
- **SLMM Device Control Lock**: Prevents command flooding to NL-43 devices

### Changed
- **Calibration Date UX**: Users now set the date of the previous calibration rather than upcoming expiry dates - more intuitive workflow
- **Settings Persistence**: Settings save no longer reloads the page
- **Tab State**: Tab state now persists in URL hash for better navigation
- **Scheduler Management**: Schedule changes now cascade to individual events
- **Dashboard Filtering**: Enhanced dashboard with additional filtering options and SLM status sync
- **SLMM Polling Intervals**: Fixed and improved polling intervals for better responsiveness
- **24-Hour Scheduler Cycle**: Improved cycle handling to prevent issues with scheduled downloads

### Fixed
- **SLM Modal Fields**: Modal now only contains correct device-specific fields
- **IP Address Handling**: IP address correctly passed via modem pairing
- **Mobile Type Display**: Fixed incorrect device type display in roster and device tables
- **SLMM Scheduled Downloads**: Fixed issues with scheduled download operations

## [0.5.1] - 2026-01-27

### Added
- **Dashboard Schedule View**: Today's scheduled actions now display directly on the main dashboard
  - New "Today's Actions" panel showing upcoming and past scheduled events
  - Schedule list partial for project-specific schedule views
  - API endpoint for fetching today's schedule data
- **New Branding Assets**: Complete logo rework for Terra-View
  - New Terra-View logos for light and dark themes
  - Retina-ready (@2x) logo variants
  - Updated favicons (16px and 32px)
  - Refreshed PWA icons (72px through 512px)

### Changed
- **Dashboard Layout**: Reorganized to include schedule information panel
- **Base Template**: Updated to use new Terra-View logos with theme-aware switching

## [0.5.0] - 2026-01-23

_Note: This version was not formally released; changes were included in v0.5.1._

## [0.4.4] - 2026-01-23

### Added
- **Recurring schedules**: New scheduler service, recurring schedule APIs, and schedule templates (calendar/interval/list).
- **Alerts UI + backend**: Alerting service plus dropdown/list templates for surfacing notifications.
- **Report templates + viewers**: CRUD API for report templates, report preview screen, and RND file viewer.
- **SLM tooling**: SLM settings modal and SLM project report generator workflow.

### Changed
- **Project data management**: Unified files view, refreshed FTP browser, and new project header/templates for file/session/unit/assignment lists.
- **Device/SLM sync**: Standardized SLM device types and tightened SLMM sync paths.
- **Docs/scripts**: Cleanup pass and expanded device-type documentation.

### Fixed
- **Scheduler actions**: Strict command definitions so actions run reliably.
- **Project view title**: Resolved JSON string rendering in project headers.

## [0.4.3] - 2026-01-14

### Added
- **Sound Level Meter roster tooling**: Roster manager surfaces SLM metadata, supports rename unit flows, and adds return-to-project navigation to keep SLM dashboard users oriented.
- **Project management templates**: New schedule and unit list templates plus file/session lists show what each project stores before teams dive into deployments.

### Changed
- **Project view refresh**: FTP browser now downloads folders locally, the countdown timer was rebuilt, and project/device templates gained edit modals for projects and locations so navigation feels smoother.
- **SLM control sync & accuracy**: Control center groundwork now runs inside the dev UI, configuration edits propagate to SLMM (which caches configs for faster responses), and the SLM live view reads the correct DRD fields after the refactor.

### Fixed
- **SLM UI syntax bug**: Resolved the unexpected token error that appeared in the refreshed SLM components.

## [0.4.2] - 2026-01-05

### Added
- **SLM Configuration Interface**: Sound Level Meters can now be configured directly from the SLM dashboard
  - Configuration modal with comprehensive SLM parameter editing
  - TCP port configuration for SLM control connections (default: 2255)
  - FTP port configuration for SLM data retrieval (default: 21)
  - Modem assignment for network access or direct IP connection support
  - Test Modem button with ping-based connectivity verification (shows IP and response time)
  - Test SLM Connection button for end-to-end connectivity validation
  - Dynamic form fields that hide/show based on modem selection
- **SLM Dashboard Endpoints**: New API routes for SLM management
  - `GET /api/slm-dashboard/config/{unit_id}` - Load SLM configuration form
  - `POST /api/slm-dashboard/config/{unit_id}` - Save SLM configuration
  - `GET /api/slm-dashboard/test-modem/{modem_id}` - Ping modem for connectivity test
- **Database Schema Updates**: Added `slm_ftp_port` column to roster table
  - Migration script: `scripts/add_slm_ftp_port.py`
  - Supports both TCP (control) and FTP (data) port configuration per SLM unit
- **Docker Environment Enhancements**:
  - Added `iputils-ping` and `curl` packages to Docker image for network diagnostics
  - Health check endpoint support via curl

### Fixed
- **Form Validation**: Fixed 400 Bad Request error when adding modem units
  - Form fields for device-specific parameters now properly disabled when hidden
  - Empty string values for integer fields no longer cause validation failures
  - JavaScript now disables hidden form sections to prevent unwanted data submission
- **Unit Status Accuracy**: Fixed issue where unit status was loading from a saved cache instead of actual last-heard time
  - Unit status now accurately reflects real-time connectivity
  - Status determination based on actual `slm_last_check` timestamp

### Changed
- **Roster Form Behavior**: Device-specific form fields are now disabled (not just hidden) when not applicable
  - Prevents SLM fields from submitting when adding modems
  - Prevents modem fields from submitting when adding SLMs
  - Cleaner form submissions with only relevant data
- **Port Field Handling**: Backend now accepts port fields as strings and converts to integers
  - Handles empty string values gracefully
  - Proper type conversion with None fallback for empty values

### Technical Details
- Added `setFieldsDisabled()` helper function for managing form field state
- Updated `toggleDeviceFields()` and `toggleEditDeviceFields()` to disable/enable fields
- Backend type conversion: `slm_tcp_port` and `slm_ftp_port` accept strings, convert to int with empty string handling
- Modem ping uses subprocess with 1 packet, 2-second timeout, returns response time in milliseconds
- Configuration form uses 3-column grid layout for TCP Port, FTP Port, and Direct IP fields

## [0.4.1] - 2026-01-05
### Added
- **SLM Integration**: Sound Level Meters are now manageable in SFM

### Fixed
- Fixed an issue where unit status was loading from a saved cache and not based on when it was actually heard from last. Unit status is now accurate.


## [0.4.0] - 2025-12-16

### Added
- **Database Management System**: Comprehensive backup and restore capabilities
  - **Manual Snapshots**: Create on-demand backups of the entire database with optional descriptions
  - **Restore from Snapshot**: Restore database from any snapshot with automatic safety backup
  - **Upload/Download Snapshots**: Transfer database snapshots to/from the server
  - **Database Tab**: New dedicated tab in Settings for all database management operations
  - **Database Statistics**: View database size, row counts by table, and last modified time
  - **Snapshot Metadata**: Each snapshot includes creation time, description, size, and type (manual/automatic)
  - **Safety Backups**: Automatic backup created before any restore operation
- **Remote Database Cloning**: Dev tools for cloning production database to remote development servers
  - **Clone Script**: `scripts/clone_db_to_dev.py` for copying database over WAN
  - **Network Upload**: Upload snapshots via HTTP to remote servers
  - **Auto-restore**: Automatically restore uploaded database on target server
  - **Authentication Support**: Optional token-based authentication for secure transfers
- **Automatic Backup Scheduler**: Background service for automated database backups
  - **Configurable Intervals**: Set backup frequency (default: 24 hours)
  - **Retention Management**: Automatically delete old backups (configurable keep count)
  - **Manual Trigger**: Force immediate backup via API
  - **Status Monitoring**: Check scheduler status and next scheduled run time
  - **Background Thread**: Non-blocking operation using Python threading
- **Settings Reorganization**: Improved tab structure for better organization
  - Renamed "Data Management" tab to "Roster Management"
  - Moved CSV Replace Mode from Advanced tab to Roster Management tab
  - Created dedicated Database tab for all backup/restore operations
- **Comprehensive Documentation**: New `docs/DATABASE_MANAGEMENT.md` guide covering:
  - Manual snapshot creation and restoration workflows
  - Download/upload procedures for off-site backups
  - Remote database cloning setup and usage
  - Automatic backup configuration and integration
  - API reference for all database endpoints
  - Best practices and troubleshooting guide

### Changed
- **Settings Tab Organization**: Restructured for better logical grouping
  - **General**: Display preferences (timezone, theme, auto-refresh)
  - **Roster Management**: CSV operations and roster table (now includes Replace Mode)
  - **Database**: All backup/restore operations (NEW)
  - **Advanced**: Power user settings (calibration, thresholds)
  - **Danger Zone**: Destructive operations
- CSV Replace Mode warnings enhanced and moved to Roster Management context

### Technical Details
- **SQLite Backup API**: Uses native SQLite backup API for concurrent-safe snapshots
- **Metadata Tracking**: JSON sidecar files store snapshot metadata alongside database files
- **Atomic Operations**: Database restoration is atomic with automatic rollback on failure
- **File Structure**: Snapshots stored in `./data/backups/` with timestamped filenames
- **API Endpoints**: 7 new endpoints for database management operations
- **Backup Service**: `backend/services/database_backup.py` - Core backup/restore logic
- **Scheduler Service**: `backend/services/backup_scheduler.py` - Automatic backup automation
- **Clone Utility**: `scripts/clone_db_to_dev.py` - Remote database synchronization tool

### Security Considerations
- Snapshots contain full database data and should be secured appropriately
- Remote cloning supports optional authentication tokens
- Restore operations require safety backup creation by default
- All destructive operations remain in Danger Zone with warnings

### Migration Notes
No database migration required for v0.4.0. All new features use existing database structure and add new backup management capabilities without modifying the core schema.

## [0.3.3] - 2025-12-12

### Changed
- **Mobile Navigation**: Moved hamburger menu button from floating top-right to bottom navigation bar
  - Bottom nav now shows: Menu (hamburger), Dashboard, Roster, Settings
  - Removed "Add Unit" from bottom nav (still accessible via sidebar menu)
  - Hamburger no longer floats over content on mobile
- **Status Dot Visibility**: Increased status dot size from 12px to 16px (w-3/h-3 → w-4/h-4) in dashboard fleet overview for better at-a-glance visibility
  - Affects both Active and Benched tabs in dashboard
  - Makes status colors (green/yellow/red) easier to spot during quick scroll

### Fixed
- **Location Navigation**: Moved tap-to-navigate functionality from roster card view to unit detail modal only
  - Roster cards now show simple location text with pin emoji
  - Navigation links (opening Maps app) only appear in the modal when tapping a unit
  - Reduces visual clutter and accidental navigation triggers

### Technical Details
- Bottom navigation remains at 4 buttons, first button now triggers sidebar menu
- Removed standalone hamburger button element and associated CSS
- Modal already had navigation links, no changes needed there

## [0.3.2] - 2025-12-12

### Added
- **Progressive Web App (PWA) Mobile Optimization**: Complete mobile-first redesign for field deployment usage
  - **Responsive Navigation**: Hamburger menu with slide-in sidebar for mobile, always-visible sidebar for desktop
  - **Bottom Navigation Bar**: Quick access to Dashboard, Roster, Add Unit, and Settings (mobile only)
  - **Mobile Card View**: Compact card layout for roster units with status dots, location, and project ID
  - **Tap-to-Navigate**: Location addresses and coordinates are clickable and open in user's default navigation app (Google Maps, Apple Maps, Waze, etc.)
  - **Unit Detail Modal**: Bottom sheet modal showing full unit details with edit capabilities (tap any unit card to open)
  - **Touch Optimization**: 44x44px minimum button targets following iOS/Android accessibility guidelines
  - **Service Worker**: Network-first caching strategy for offline-capable operation
  - **IndexedDB Storage**: Offline data persistence for unit information and pending edits
  - **Background Sync**: Queues edits made while offline and syncs automatically when connection returns
  - **Offline Indicator**: Visual banner showing offline status with manual sync button
  - **PWA Manifest**: Installable as a standalone app on mobile devices with custom icons
  - **Hard Reload Button**: "Clear Cache & Reload" utility in sidebar menu to force fresh JavaScript/CSS
- **Mobile-Specific Files**:
  - `backend/static/mobile.css` - Mobile UI styles, hamburger menu, bottom nav, cards, modals
  - `backend/static/mobile.js` - Mobile interactions, offline sync, modal management
  - `backend/static/sw.js` - Service worker for PWA functionality
  - `backend/static/offline-db.js` - IndexedDB wrapper for offline storage
  - `backend/static/manifest.json` - PWA configuration
  - `backend/static/icons/` - 8 PWA icon sizes (72px-512px)

### Changed
- **Dashboard Alerts**: Only show Missing units in notifications (Pending units no longer appear in alerts)
- **Roster Template**: Mobile card view shows status from server-side render instead of fetching separately
- **Mobile Status Display**: Benched units show "Benched" label instead of "Unknown" or "N/A"
- **Base Template**: Added cache-busting query parameters to JavaScript files (e.g., `mobile.js?v=0.3.2`)
- **Sidebar Menu**: Added utility section with "Toggle theme" and "Clear Cache & Reload" buttons

### Fixed
- **Modal Status Display**: Fixed unit detail modal showing "Unknown" status by passing status data from card to modal
- **Mobile Card Status**: Fixed grey dot with "Unknown" label for benched units - now properly shows deployment state
- **Status Data Passing**: Roster cards now pass status and age to modal via function parameters and global status map
- **Service Worker Caching**: Aggressive browser caching issue resolved with version query parameters and hard reload function

### Technical Details
- Mobile breakpoint at 768px (`md:` prefix in TailwindCSS)
- PWA installable via Add to Home Screen on iOS/Android
- Service worker caches all static assets with network-first strategy
- Google Maps search API used for universal navigation links (works across all map apps)
- Status map stored in `window.rosterStatusMap` from server-side rendered data
- Hard reload function clears service worker caches, unregisters workers, and deletes IndexedDB

## [0.3.1] - 2025-12-12

### Fixed
- **Dashboard Notifications**: Removed Pending units from alert list - only Missing units now trigger notifications
- **Status Dots**: Verified deployed units display correct status dots (OK=green, Pending=yellow, Missing=red) in both active and benched tables
- **Mobile Card View**: Fixed roster cards showing "Unknown" status by using `.get()` with defaults in backend routes
- **Backend Status Handling**: Added default values for status, age, last_seen fields to prevent KeyError exceptions

### Changed
- Backend roster partial routes (`/partials/roster-deployed`, `/partials/roster-benched`) now use `.get()` method with sensible defaults
- Deployed units default to "Unknown" status when data unavailable
- Benched units default to "N/A" status when data unavailable

## [0.3.0] - 2025-12-09

### Added
- **Series 4 (Micromate) Support**: New `/api/series4/heartbeat` endpoint for receiving telemetry from Series 4 Micromate units
  - Auto-detection of Series 4 units via UM##### ID pattern
  - Stores project hints from emitter payload in unit notes
  - Automatic unit type classification across both Series 3 and Series 4 endpoints
- **Development Environment Labels**: Visual indicators to distinguish dev from production deployments
  - Yellow "DEV" badge in sidebar navigation
  - "[DEV]" prefix in browser title
  - Yellow banner on dashboard when running in development mode
  - Environment variable support in docker-compose.yml (ENVIRONMENT=production|development)
- **Quality of Life Improvements**:
  - Human-readable relative timestamps (e.g., "2h 15m ago", "3d ago") with full date in tooltips
  - "Last Updated" timestamp indicator on dashboard
  - Status icons for colorblind accessibility (checkmark for OK, clock for Pending, X for Missing)
  - Breadcrumb navigation on unit detail pages
  - Copy-to-clipboard buttons for unit IDs
  - Search/filter functionality for fleet roster table
  - Improved empty state messages with icons
- **Timezone Support**: Comprehensive timezone handling across the application
  - Timezone selector in Settings (defaults to America/New_York EST)
  - Human-readable timestamp format (e.g., "9/10/2020 8:00 AM EST")
  - Timezone-aware display for all timestamps site-wide
  - Settings stored in localStorage for immediate effect
- **Settings Page Redesign**: Complete overhaul with tabbed interface and persistent preferences
  - **General Tab**: Display preferences (timezone, theme, auto-refresh interval)
  - **Data Management Tab**: Safe operations (CSV export, merge import, roster table)
  - **Advanced Tab**: Power user settings (replace mode import, calibration defaults, status thresholds)
  - **Danger Zone Tab**: Destructive operations isolated with enhanced warnings
  - Backend preferences storage via new UserPreferences model
  - Tab state persistence in localStorage
  - Smooth animations and consistent styling with existing pages
- **User Preferences API**: New backend endpoints for persistent settings storage
  - `GET /api/settings/preferences` - Retrieve all user preferences
  - `PUT /api/settings/preferences` - Update preferences (supports partial updates)
  - Database-backed storage for cross-device preference sync
  - Migration script: `backend/migrate_add_user_preferences.py`

### Changed
- Timestamps now display in user-selected timezone with human-readable format throughout the application
- Settings page reorganized from 644-line flat layout to clean 4-tab interface
- CSV Replace Mode moved from Data Management to Advanced tab with additional warnings
- Import operations separated: safe merge in Data Management tab, destructive replace in Advanced tab
- Page title changed from "Roster Manager" to "Settings" for better clarity
- All preferences now persist to backend database instead of relying solely on localStorage

### Fixed
- Unit type classification now consistent across Series 3 and Series 4 heartbeat endpoints
- Auto-correction of misclassified unit types when they report to wrong endpoint

### Technical Details
- New `detect_unit_type()` helper function for pattern-based unit classification
- UserPreferences model with single-row table pattern (id=1) for global settings
- Series 4 units identified by UM prefix followed by digits (e.g., UM11719)
- JavaScript Intl API used for client-side timezone conversion
- Pydantic schema for partial preference updates (PreferencesUpdate model)
- Environment context injection via custom FastAPI template response wrapper

## [0.2.1] - 2025-12-03

### Added
- `/settings` roster manager page with CSV export/import, live stats, and danger-zone reset controls.
- `/api/settings` router that exposes `export-csv`, `stats`, `roster-units`, `import-csv-replace`, and the clear-* endpoints backing the UI.
- Dedicated HTMX partials/tabs for deployed, benched, retired, and ignored units plus new ignored-table UI to unignore or delete entries.

### Changed
- Roster and unit detail templates now display device-type specific metadata (calibration windows, modem pairings, IP/phone fields) alongside inline actions.
- Base navigation highlights the new settings workflow and routes retired/ignored buckets through dedicated endpoints + partials.

### Fixed
- Snapshot summary counts only consider deployed units, preventing dashboard alerts from including benched hardware.
- Snapshot payloads now include address/coordinate metadata so map widgets and CSV exports stay accurate.

## [0.2.0] - 2025-12-03

### Added
- Device-type aware roster schema (seismographs vs modems) with new metadata columns plus `backend/migrate_add_device_types.py` for upgrading existing SQLite files.
- `create_test_db.py` helper that generates a ready-to-use demo database with sample seismographs, modems, and emitter rows.
- Ignore list persistence/API so noisy legacy emitters can be quarantined via `/api/roster/ignore` and surfaced in the UI.
- Roster page enhancements: Add Unit modal, CSV import modal, and HTMX-powered table fed by `/partials/roster-table`.
- Unit detail view rewritten to fetch data via API, expose deployment status, and allow edits to all metadata.

### Changed
- Snapshot service now merges roster + emitter data into active/benched/retired/unknown buckets and includes device-specific metadata in each record.
- Roster edit endpoints parse date fields, manage modem/seismograph specific attributes, and guarantee records exist when toggling deployed/retired states.
- Dashboard partial endpoints are grouped under `/dashboard/*` so HTMX tabs stay in sync with the consolidated snapshot payload.

### Fixed
- Toggling deployed/retired flags no longer fails when a unit does not exist because the router now auto-creates placeholder roster rows.
- CSV import applies address/coordinate updates instead of silently dropping unknown columns.

## [0.1.1] - 2025-12-02

### Added
- **Roster Editing API**: Full CRUD operations for roster management
  - `POST /api/roster/add` - Add new units to roster
  - `POST /api/roster/set-deployed/{unit_id}` - Toggle deployment status
  - `POST /api/roster/set-retired/{unit_id}` - Toggle retired status
  - `POST /api/roster/set-note/{unit_id}` - Update unit notes
- **CSV Import**: Bulk roster import functionality
  - `POST /api/roster/import-csv` - Import units from CSV file
  - Support for all roster fields: unit_id, unit_type, deployed, retired, note, project_id, location
  - Optional update_existing parameter to control duplicate handling
  - Detailed import summary with added/updated/skipped/error counts
- **Enhanced Database Models**:
  - Added `project_id` field to RosterUnit model
  - Added `location` field to RosterUnit model
  - Added `last_updated` timestamp tracking
- **Dashboard Enhancements**:
  - Separate views for Active, Benched, and Retired units
  - New endpoints: `/dashboard/active` and `/dashboard/benched`

### Fixed
- Database session management bug in `emit_status_snapshot()`
  - Added `get_db_session()` helper function for direct session access
  - Implemented proper session cleanup with try/finally blocks
- Database schema synchronization issues
  - Database now properly recreates when model changes are detected

### Changed
- Updated RosterUnit model to include additional metadata fields
- Improved error handling in CSV import with row-level error reporting
- Enhanced snapshot service to properly manage database connections

### Technical Details
- All roster editing endpoints use Form data for better HTML form compatibility
- CSV import uses multipart/form-data for file uploads
- Boolean fields in CSV accept: 'true', '1', 'yes' (case-insensitive)
- Database sessions now properly closed to prevent connection leaks

## [0.1.0] - 2024-11-20

### Added
- Initial release of Seismo Fleet Manager
- FastAPI-based REST API for fleet management
- SQLite database with SQLAlchemy ORM
- Emitter reporting endpoints
- Basic fleet status monitoring
- Docker and Docker Compose support
- Web-based dashboard with HTMX
- Dark/light mode toggle
- Interactive maps with Leaflet
- Photo management per unit
- Automated status categorization (OK/Pending/Missing)

[0.7.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.6.1...v0.7.0
[0.6.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.5.1...v0.6.0
[0.5.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.5.0...v0.5.1
[0.5.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.4.4...v0.5.0
[0.4.4]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.4.3...v0.4.4
[0.4.3]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.4.2...v0.4.3
[0.4.2]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.4.1...v0.4.2
[0.4.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.4.0...v0.4.1
[0.4.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.3...v0.4.0
[0.3.3]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.2...v0.3.3
[0.3.2]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.1...v0.3.2
[0.3.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.3.0...v0.3.1
[0.3.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.2.1...v0.3.0
[0.2.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.2.0...v0.2.1
[0.2.0]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.1...v0.2.0
[0.1.1]: https://github.com/serversdwn/seismo-fleet-manager/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/serversdwn/seismo-fleet-manager/releases/tag/v0.1.0