Migration cleanup: SLM dashboard restored, db migration

CHANGELOG.md

@@ -1,10 +1,99 @@
 # Changelog
 
-All notable changes to Seismo Fleet Manager will be documented in this file.
+All notable changes to Terra-View will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.0] - 2026-01-09
+
+### Added
+- **Unified Modular Monolith Architecture**: Complete architectural refactoring to modular monolith pattern
+  - **Three Feature Modules**: Seismo (seismograph fleet), SLM (sound level meters), UI (shared templates/static)
+  - **Module Isolation**: Each module has its own database, models, services, and routers
+  - **Shared Infrastructure**: Common utilities and API aggregation layer
+  - **Multi-Container Deployment**: Three Docker containers (terra-view, sfm, slmm) built from single codebase
+- **SLMM Integration**: Sound Level Meter Manager fully integrated as `app/slm/` module
+  - Migrated from separate repository to unified codebase
+  - Complete NL43 device management API (`/api/nl43/*`)
+  - Database models for NL43Config and NL43Status
+  - NL43Client service for device communication
+  - FTP, TCP, and web interface support for NL43 devices
+- **SLM Dashboard API Layer**: New dashboard endpoints bridge UI and device APIs
+  - `GET /api/slm-dashboard/stats` - Aggregate statistics (total units, online/offline, measuring/idle)
+  - `GET /api/slm-dashboard/units` - List all units with latest status
+  - `GET /api/slm-dashboard/live-view/{unit_id}` - Real-time measurement data
+  - `GET /api/slm-dashboard/config/{unit_id}` - Retrieve unit configuration
+  - `POST /api/slm-dashboard/config/{unit_id}` - Update unit configuration
+  - `POST /api/slm-dashboard/control/{unit_id}/{action}` - Send control commands (start, stop, pause, resume, reset, sleep, wake)
+  - `GET /api/slm-dashboard/test-modem/{unit_id}` - Test device connectivity
+- **Repository Rebranding**: Renamed from `seismo-fleet-manager` to `terra-view`
+  - Reflects unified platform nature (seismo + SLM + future modules)
+  - Git remote updated to `terra-view.git`
+  - All references updated throughout codebase
+
+### Changed
+- **Project Structure**: Complete reorganization following modular monolith pattern
+  - `app/seismo/` - Seismograph fleet module (formerly `backend/`)
+  - `app/slm/` - Sound level meter module (integrated from SLMM)
+  - `app/ui/` - Shared templates and static assets
+  - `app/api/` - Cross-module API aggregation layer
+  - Removed `backend/` and `templates/` directories
+- **Import Paths**: All imports updated from `backend.*` to `app.seismo.*` or `app.slm.*`
+- **Database Initialization**: Each module initializes its own database tables
+  - Seismo database: `app/seismo/database.py`
+  - SLM database: `app/slm/database.py`
+- **Docker Architecture**: Three-container deployment from single codebase
+  - `terra-view` (port 8001): Main UI/orchestrator with all modules
+  - `sfm` (port 8002): Seismograph Fleet Module API
+  - `slmm` (port 8100): Sound Level Meter Manager API
+  - All containers built from same unified codebase with different entry points
+
+### Fixed
+- **Template Path Issues**: Fixed seismo dashboard template references
+  - Updated `app/seismo/routers/dashboard.py` to use `app/ui/templates` directory
+  - Resolved 404 errors for `partials/benched_table.html` and `partials/active_table.html`
+- **Module Import Errors**: Corrected SLMM module structure
+  - Fixed `app/slm/main.py` to import from `app.slm.routers` instead of `app.routers`
+  - Updated all SLMM internal imports to use `app.slm.*` namespace
+- **Docker Build Issues**: Resolved file permission problems
+  - Fixed dashboard.py permissions for Docker COPY operations
+  - Ensured all source files readable during container builds
+
+### Technical Details
+- **Modular Monolith Benefits**:
+  - Single repository for easier development and deployment
+  - Module boundaries enforced through folder structure
+  - Shared dependencies managed in single requirements.txt
+  - Independent database schemas per module
+  - Clean separation of concerns with explicit module APIs
+- **Migration Path**: Existing installations automatically migrate
+  - Import path updates applied programmatically
+  - Database schemas remain compatible
+  - No data migration required
+- **Module Structure**: Each module follows consistent pattern
+  - `database.py` - SQLAlchemy models and session management
+  - `models.py` - Pydantic schemas and database models
+  - `routers.py` - FastAPI route definitions
+  - `services.py` - Business logic and external integrations
+- **Container Communication**: Containers use host networking
+  - terra-view proxies to sfm and slmm containers
+  - Environment variables configure API URLs
+  - Health checks ensure container availability
+
+### Migration Notes
+- **Breaking Changes**: Import paths changed for all modules
+  - Old: `from backend.models import RosterUnit`
+  - New: `from app.seismo.models import RosterUnit`
+- **Configuration Updates**: Environment variables for multi-container setup
+  - `SFM_API_URL=http://localhost:8002` - SFM backend endpoint
+  - `SLMM_API_URL=http://localhost:8100` - SLMM backend endpoint
+  - `MODULE_MODE=sfm|slmm` - Future flag for API-only containers
+- **Repository Migration**: Update git remotes for renamed repository
+  ```bash
+  git remote set-url origin ssh://git@10.0.0.2:2222/serversdown/terra-view.git
+  ```
+
 ## [0.4.2] - 2026-01-05
 
 ### Added

README.md

@@ -1,5 +1,31 @@
-# Seismo Fleet Manager v0.4.2
-Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.
+# Terra-View v0.5.0
+Unified platform for managing seismograph fleets and sound level meter deployments. Built as a modular monolith with independent feature modules (Seismo, SLM) sharing a common UI layer. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your entire fleet through a unified database and dashboard.
+
+## Architecture
+
+Terra-View follows a **modular monolith** architecture with independent feature modules in a single codebase:
+
+- **app/seismo/** - Seismograph Fleet Module (SFM)
+  - Device roster and deployment tracking
+  - Series 3/4 telemetry ingestion
+  - Status monitoring (OK/Pending/Missing)
+  - Photo management and location tracking
+- **app/slm/** - Sound Level Meter Manager (SLMM)
+  - NL43 device configuration and control
+  - Real-time measurement monitoring
+  - TCP/FTP/Web interface support
+  - Dashboard statistics and unit management
+- **app/ui/** - Shared UI layer
+  - Templates, static assets, and common components
+  - Progressive Web App (PWA) support
+- **app/api/** - API aggregation layer
+  - Cross-module endpoints
+  - Future unified dashboard APIs
+
+**Multi-Container Deployment**: Three Docker containers built from the same codebase:
+- `terra-view` (port 8001) - Main UI with all modules integrated
+- `sfm` (port 8002) - Seismo API backend
+- `slmm` (port 8100) - SLM API backend
 
 ## Features
 

app/main.py

@@ -39,6 +39,7 @@ from app.seismo import routes as seismo_legacy_routes
 
 # Import feature module routers (SLM)
 from app.slm.routers import router as slm_router
+from app.slm.dashboard import router as slm_dashboard_router
 
 # Import API aggregation layer (placeholder for now)
 from app.api import dashboard as api_dashboard
@@ -48,6 +49,9 @@ from app.api import roster as api_roster
 from app.seismo.database import engine as seismo_engine, Base as SeismoBase
 SeismoBase.metadata.create_all(bind=seismo_engine)
 
+from app.slm.database import engine as slm_engine, Base as SlmBase
+SlmBase.metadata.create_all(bind=slm_engine)
+
 # Initialize FastAPI app
 app = FastAPI(
     title=APP_NAME,
@@ -104,6 +108,7 @@ app.include_router(seismo_legacy_routes.router)
 
 # SLM Feature Module APIs
 app.include_router(slm_router)
+app.include_router(slm_dashboard_router)
 
 # API Aggregation Layer (future cross-feature endpoints)
 # app.include_router(api_dashboard.router)  # TODO: Implement aggregation

app/seismo/routers/dashboard.py

@@ -4,7 +4,7 @@ from fastapi.templating import Jinja2Templates
 from app.seismo.services.snapshot import emit_status_snapshot
 
 router = APIRouter()
-templates = Jinja2Templates(directory="templates")
+templates = Jinja2Templates(directory="app/ui/templates")
 
 
 @router.get("/dashboard/active")

app/slm/dashboard.py

@@ -0,0 +1,241 @@
+"""
+Dashboard API endpoints for SLM/NL43 devices.
+This layer aggregates and transforms data from the device API for UI consumption.
+"""
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy.orm import Session
+from sqlalchemy import func
+from typing import List, Dict, Any
+import logging
+
+from app.slm.database import get_db
+from app.slm.models import NL43Config, NL43Status
+from app.slm.services import NL43Client
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/slm-dashboard", tags=["slm-dashboard"])
+
+
+@router.get("/stats")
+async def get_dashboard_stats(db: Session = Depends(get_db)):
+    """Get aggregate statistics for the SLM dashboard."""
+    total_units = db.query(func.count(NL43Config.unit_id)).scalar() or 0
+
+    # Count units with recent status updates (within last 5 minutes)
+    from datetime import datetime, timedelta
+    five_min_ago = datetime.utcnow() - timedelta(minutes=5)
+    online_units = db.query(func.count(NL43Status.unit_id)).filter(
+        NL43Status.last_seen >= five_min_ago
+    ).scalar() or 0
+
+    # Count units currently measuring
+    measuring_units = db.query(func.count(NL43Status.unit_id)).filter(
+        NL43Status.measurement_state == "Measure"
+    ).scalar() or 0
+
+    return {
+        "total_units": total_units,
+        "online_units": online_units,
+        "offline_units": total_units - online_units,
+        "measuring_units": measuring_units,
+        "idle_units": online_units - measuring_units
+    }
+
+
+@router.get("/units")
+async def get_units_list(db: Session = Depends(get_db)):
+    """Get list of all NL43 units with their latest status."""
+    configs = db.query(NL43Config).all()
+    units = []
+
+    for config in configs:
+        status = db.query(NL43Status).filter_by(unit_id=config.unit_id).first()
+
+        # Determine if unit is online (status updated within last 5 minutes)
+        from datetime import datetime, timedelta
+        is_online = False
+        if status and status.last_seen:
+            five_min_ago = datetime.utcnow() - timedelta(minutes=5)
+            is_online = status.last_seen >= five_min_ago
+
+        unit_data = {
+            "unit_id": config.unit_id,
+            "host": config.host,
+            "tcp_port": config.tcp_port,
+            "tcp_enabled": config.tcp_enabled,
+            "is_online": is_online,
+            "measurement_state": status.measurement_state if status else "unknown",
+            "last_seen": status.last_seen.isoformat() if status and status.last_seen else None,
+            "lp": status.lp if status else None,
+            "leq": status.leq if status else None,
+            "lmax": status.lmax if status else None,
+            "battery_level": status.battery_level if status else None,
+        }
+        units.append(unit_data)
+
+    return {"units": units}
+
+
+@router.get("/live-view/{unit_id}")
+async def get_live_view(unit_id: str, db: Session = Depends(get_db)):
+    """Get live measurement data for a specific unit."""
+    status = db.query(NL43Status).filter_by(unit_id=unit_id).first()
+    if not status:
+        raise HTTPException(status_code=404, detail="Unit not found")
+
+    return {
+        "unit_id": unit_id,
+        "last_seen": status.last_seen.isoformat() if status.last_seen else None,
+        "measurement_state": status.measurement_state,
+        "measurement_start_time": status.measurement_start_time.isoformat() if status.measurement_start_time else None,
+        "counter": status.counter,
+        "lp": status.lp,
+        "leq": status.leq,
+        "lmax": status.lmax,
+        "lmin": status.lmin,
+        "lpeak": status.lpeak,
+        "battery_level": status.battery_level,
+        "power_source": status.power_source,
+        "sd_remaining_mb": status.sd_remaining_mb,
+        "sd_free_ratio": status.sd_free_ratio,
+    }
+
+
+@router.get("/config/{unit_id}")
+async def get_unit_config(unit_id: str, db: Session = Depends(get_db)):
+    """Get configuration for a specific unit."""
+    config = db.query(NL43Config).filter_by(unit_id=unit_id).first()
+    if not config:
+        raise HTTPException(status_code=404, detail="Unit configuration not found")
+
+    return {
+        "unit_id": config.unit_id,
+        "host": config.host,
+        "tcp_port": config.tcp_port,
+        "tcp_enabled": config.tcp_enabled,
+        "ftp_enabled": config.ftp_enabled,
+        "ftp_username": config.ftp_username,
+        "ftp_password": config.ftp_password,
+        "web_enabled": config.web_enabled,
+    }
+
+
+@router.post("/config/{unit_id}")
+async def update_unit_config(unit_id: str, config_data: Dict[str, Any], db: Session = Depends(get_db)):
+    """Update configuration for a specific unit."""
+    config = db.query(NL43Config).filter_by(unit_id=unit_id).first()
+
+    if not config:
+        # Create new config
+        config = NL43Config(unit_id=unit_id)
+        db.add(config)
+
+    # Update fields
+    if "host" in config_data:
+        config.host = config_data["host"]
+    if "tcp_port" in config_data:
+        config.tcp_port = config_data["tcp_port"]
+    if "tcp_enabled" in config_data:
+        config.tcp_enabled = config_data["tcp_enabled"]
+    if "ftp_enabled" in config_data:
+        config.ftp_enabled = config_data["ftp_enabled"]
+    if "ftp_username" in config_data:
+        config.ftp_username = config_data["ftp_username"]
+    if "ftp_password" in config_data:
+        config.ftp_password = config_data["ftp_password"]
+    if "web_enabled" in config_data:
+        config.web_enabled = config_data["web_enabled"]
+
+    db.commit()
+    db.refresh(config)
+
+    return {"success": True, "unit_id": unit_id}
+
+
+@router.post("/control/{unit_id}/{action}")
+async def control_unit(unit_id: str, action: str, db: Session = Depends(get_db)):
+    """Send control command to a unit (start, stop, pause, resume, etc.)."""
+    config = db.query(NL43Config).filter_by(unit_id=unit_id).first()
+    if not config:
+        raise HTTPException(status_code=404, detail="Unit configuration not found")
+
+    if not config.tcp_enabled:
+        raise HTTPException(status_code=400, detail="TCP control not enabled for this unit")
+
+    # Create NL43Client
+    client = NL43Client(
+        host=config.host,
+        port=config.tcp_port,
+        timeout=5.0,
+        ftp_username=config.ftp_username,
+        ftp_password=config.ftp_password
+    )
+
+    # Map action to command
+    action_map = {
+        "start": "start_measurement",
+        "stop": "stop_measurement",
+        "pause": "pause_measurement",
+        "resume": "resume_measurement",
+        "reset": "reset_measurement",
+        "sleep": "sleep_mode",
+        "wake": "wake_from_sleep",
+    }
+
+    if action not in action_map:
+        raise HTTPException(status_code=400, detail=f"Unknown action: {action}")
+
+    method_name = action_map[action]
+    method = getattr(client, method_name, None)
+
+    if not method:
+        raise HTTPException(status_code=500, detail=f"Method {method_name} not implemented")
+
+    try:
+        result = await method()
+        return {"success": True, "action": action, "result": result}
+    except Exception as e:
+        logger.error(f"Error executing {action} on {unit_id}: {e}")
+        raise HTTPException(status_code=500, detail=str(e))
+
+
+@router.get("/test-modem/{unit_id}")
+async def test_modem(unit_id: str, db: Session = Depends(get_db)):
+    """Test connectivity to a unit's modem/device."""
+    config = db.query(NL43Config).filter_by(unit_id=unit_id).first()
+    if not config:
+        raise HTTPException(status_code=404, detail="Unit configuration not found")
+
+    if not config.tcp_enabled:
+        raise HTTPException(status_code=400, detail="TCP control not enabled for this unit")
+
+    client = NL43Client(
+        host=config.host,
+        port=config.tcp_port,
+        timeout=5.0,
+        ftp_username=config.ftp_username,
+        ftp_password=config.ftp_password
+    )
+
+    try:
+        # Try to get measurement state as a connectivity test
+        state = await client.get_measurement_state()
+        return {
+            "success": True,
+            "unit_id": unit_id,
+            "host": config.host,
+            "port": config.tcp_port,
+            "reachable": True,
+            "measurement_state": state
+        }
+    except Exception as e:
+        logger.warning(f"Modem test failed for {unit_id}: {e}")
+        return {
+            "success": False,
+            "unit_id": unit_id,
+            "host": config.host,
+            "port": config.tcp_port,
+            "reachable": False,
+            "error": str(e)
+        }