Compare commits

..

20 Commits

Author SHA1 Message Date
serversdwn
bf5f222511 Add:
- db cache dump on diagnostics request.
- individual device logs, db and files.
-Device logs api endpoints and diagnostics UI.

Fix:
- slmm standalone now uses local TZ (was UTC only before)
- fixed measurement start time logic.
2026-01-29 18:50:47 +00:00
serversdwn
eb39a9d1d0 add: device communication lock, Now to send a tcp command, slmm must establish a connection lock to prevent flooding unit.
fixed: Background poller intervals increased.
2026-01-29 07:54:49 +00:00
serversdwn
67d63b4173 Merge branch 'main' of ssh://10.0.0.2:2222/serversdown/slmm 2026-01-23 08:29:27 +00:00
serversdwn
25cf9528d0 docs: update to 0.2.1 2026-01-23 08:26:23 +00:00
738ad7878e doc update 2026-01-22 15:30:06 -05:00
serversdwn
152377d608 feat: terra-view scheduler implementation added. Start_cylce and stop_cycle functions added. 2026-01-22 20:25:47 +00:00
serversdwn
4868381053 Enhance FTP logging with detailed phases for connection, authentication, and data transfer 2026-01-21 08:05:38 +00:00
serversdwn
b4bbfd2b01 chore:fixed api.md to confirm FTP/TCP interactions are working. 2026-01-17 08:13:19 +00:00
serversdwn
82651f71b5 Add roster management interface and related API endpoints
- Implemented a new `/roster` endpoint to retrieve and manage device configurations.
- Added HTML template for the roster page with a table to display device status and actions.
- Introduced functionality to add, edit, and delete devices via the roster interface.
- Enhanced `ConfigPayload` model to include polling options.
- Updated the main application to serve the new roster page and link to it from the index.
- Added validation for polling interval in the configuration payload.
- Created detailed documentation for the roster management features and API endpoints.
2026-01-17 08:00:05 +00:00
serversdwn
182920809d chore: docs and scripts organized. clutter cleared. 2026-01-16 19:06:38 +00:00
serversdwn
2a3589ca5c Add endpoint to delete device configuration and associated status data 2026-01-16 07:39:26 +00:00
serversdwn
d43ef7427f v0.2.0: async status polling added. 2026-01-16 06:24:13 +00:00
serversdwn
d2b47156d8 Simple diagnostics heartbeat test program added, for debugging. 2026-01-15 20:52:08 +00:00
serversdwn
5b31c2e567 Add endpoint to sync measurement start time from FTP folder timestamp 2026-01-14 21:58:45 +00:00
serversdwn
b74360b6bb Implement automatic sleep mode disable for NL43/NL53 during config updates and measurements 2026-01-14 19:58:22 +00:00
serversdwn
3d445daf1f fixed FTP port support to NL43 configuration and client 2026-01-14 01:44:53 +00:00
serversdwn
2cb96a7a1c Add configurable timezone support with environment variables 2026-01-12 16:31:33 +00:00
serversdwn
6b363b0788 Added: Ability to change store name and overwrite protection 2026-01-08 19:16:59 +00:00
serversdwn
1fb786c262 Fix NL43 DRD field mapping to match official specification
Corrected the parsing of NL43 DRD (Dynamic Range Data) and DOD (Data On Demand)
responses according to the NL43 Communications Guide. The previous implementation
incorrectly mapped d0 (counter field) as a measurement.

Changes:
- Updated DRD/DOD parsing to skip d0 (counter: 1-600)
- Correctly map d1-d5 to lp/leq/lmax/lmin/lpeak measurements
- Added inline documentation referencing DRD format specification
- Included database migration script to revert incorrect field names

DRD format per NL43 spec:
- d0 = counter (1-600) - NOT a measurement
- d1 = Lp (instantaneous sound pressure level)
- d2 = Leq (equivalent continuous sound level)
- d3 = Lmax (maximum level)
- d4 = Lmin (minimum level)
- d5 = Lpeak (peak level)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 03:42:26 +00:00
serversdwn
50c9370b8e Containerized for TV deployment 2026-01-07 01:32:25 +00:00
33 changed files with 5289 additions and 224 deletions

18
.env.example Normal file
View File

@@ -0,0 +1,18 @@
# SLMM Configuration
# Copy this file to .env and customize as needed
# Timezone Configuration
# Set the timezone offset from UTC (in hours)
# Examples:
# -5 = EST (Eastern Standard Time)
# -4 = EDT (Eastern Daylight Time)
# 0 = UTC
# +1 = CET (Central European Time)
# -8 = PST (Pacific Standard Time)
TIMEZONE_OFFSET=-5
# Optional: Timezone name for logging (cosmetic only)
TIMEZONE_NAME=EST
# CORS Configuration (comma-separated list of allowed origins)
CORS_ORIGINS=*

151
CHANGELOG.md Normal file
View File

@@ -0,0 +1,151 @@
# Changelog
All notable changes to SLMM (Sound Level Meter Manager) will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.1] - 2026-01-23
### Added
- **Roster management**: UI and API endpoints for managing device rosters.
- **Delete config endpoint**: Remove device configuration alongside cached status data.
- **Scheduler hooks**: `start_cycle` and `stop_cycle` helpers for Terra-View scheduling integration.
### Changed
- **FTP logging**: Connection, authentication, and transfer phases now log explicitly.
- **Documentation**: Reorganized docs/scripts and updated API notes for FTP/TCP verification.
## [0.2.0] - 2026-01-15
### Added
#### Background Polling System
- **Continuous automatic device polling** - Background service that continuously polls configured devices
- **Per-device configurable intervals** - Each device can have custom polling interval (10-3600 seconds, default 60)
- **Automatic offline detection** - Devices automatically marked unreachable after 3 consecutive failures
- **Reachability tracking** - Database fields track device health with failure counters and error messages
- **Dynamic sleep scheduling** - Polling service adjusts sleep intervals based on device configurations
- **Graceful lifecycle management** - Background poller starts on application startup and stops cleanly on shutdown
#### New API Endpoints
- `GET /api/nl43/{unit_id}/polling/config` - Get device polling configuration
- `PUT /api/nl43/{unit_id}/polling/config` - Update polling interval and enable/disable per-device polling
- `GET /api/nl43/_polling/status` - Get global polling status for all devices with reachability info
#### Database Schema Changes
- **NL43Config table**:
- `poll_interval_seconds` (Integer, default 60) - Polling interval in seconds
- `poll_enabled` (Boolean, default true) - Enable/disable background polling per device
- **NL43Status table**:
- `is_reachable` (Boolean, default true) - Current device reachability status
- `consecutive_failures` (Integer, default 0) - Count of consecutive poll failures
- `last_poll_attempt` (DateTime) - Last time background poller attempted to poll
- `last_success` (DateTime) - Last successful poll timestamp
- `last_error` (Text) - Last error message (truncated to 500 chars)
#### New Files
- `app/background_poller.py` - Background polling service implementation
- `migrate_add_polling_fields.py` - Database migration script for v0.2.0 schema changes
- `test_polling.sh` - Comprehensive test script for polling functionality
- `CHANGELOG.md` - This changelog file
### Changed
- **Enhanced status endpoint** - `GET /api/nl43/{unit_id}/status` now includes polling-related fields (is_reachable, consecutive_failures, last_poll_attempt, last_success, last_error)
- **Application startup** - Added lifespan context manager in `app/main.py` to manage background poller lifecycle
- **Performance improvement** - Terra-View requests now return cached data instantly (<100ms) instead of waiting for device queries (1-2 seconds)
### Technical Details
#### Architecture
- Background poller runs as async task using `asyncio.create_task()`
- Uses existing `NL43Client` and `persist_snapshot()` functions - no code duplication
- Respects existing 1-second rate limiting per device
- Efficient resource usage - skips work when no devices configured
- WebSocket streaming remains unaffected - separate real-time data path
#### Default Behavior
- Existing devices automatically get 60-second polling interval
- Existing status records default to `is_reachable=true`
- Migration is additive-only - no data loss
- Polling can be disabled per-device via `poll_enabled=false`
#### Recommended Intervals
- Critical monitoring: 30 seconds
- Normal monitoring: 60 seconds (default)
- Battery conservation: 300 seconds (5 minutes)
- Development/testing: 10 seconds (minimum allowed)
### Migration Notes
To upgrade from v0.1.x to v0.2.0:
1. **Stop the service** (if running):
```bash
docker compose down slmm
# OR
# Stop your uvicorn process
```
2. **Update code**:
```bash
git pull
# OR copy new files
```
3. **Run migration**:
```bash
cd slmm
python3 migrate_add_polling_fields.py
```
4. **Restart service**:
```bash
docker compose up -d --build slmm
# OR
uvicorn app.main:app --host 0.0.0.0 --port 8100
```
5. **Verify polling is active**:
```bash
curl http://localhost:8100/api/nl43/_polling/status | jq '.'
```
You should see `"poller_running": true` and all configured devices listed.
### Breaking Changes
None. This release is fully backward-compatible with v0.1.x. All existing endpoints and functionality remain unchanged.
---
## [0.1.0] - 2025-12-XX
### Added
- Initial release
- REST API for NL43/NL53 sound level meter control
- TCP command protocol implementation
- FTP file download support
- WebSocket streaming for real-time data (DRD)
- Device configuration management
- Measurement control (start, stop, pause, resume, reset, store)
- Device information endpoints (battery, clock, results)
- Measurement settings management (frequency/time weighting)
- Sleep mode control
- Rate limiting (1-second minimum between commands)
- SQLite database for device configs and status cache
- Health check endpoints
- Comprehensive API documentation
- NL43 protocol documentation
### Database Schema (v0.1.0)
- **NL43Config table** - Device connection configuration
- **NL43Status table** - Measurement snapshot cache
---
## Version History Summary
- **v0.2.1** (2026-01-23) - Roster management, scheduler hooks, FTP logging, doc cleanup
- **v0.2.0** (2026-01-15) - Background Polling System
- **v0.1.0** (2025-12-XX) - Initial Release

21
Dockerfile Normal file
View File

@@ -0,0 +1,21 @@
FROM python:3.11-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && \
apt-get install -y --no-install-recommends curl && \
rm -rf /var/lib/apt/lists/*
# Copy requirements and install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Expose port
EXPOSE 8100
# Run the application
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8100"]

116
README.md
View File

@@ -1,15 +1,19 @@
# SLMM - Sound Level Meter Manager
**Version 0.2.1**
Backend API service for controlling and monitoring Rion NL-43/NL-53 Sound Level Meters via TCP and FTP protocols.
## Overview
SLMM is a standalone backend module that provides REST API routing and command translation for NL43/NL53 sound level meters. This service acts as a bridge between the hardware devices and frontend applications, handling all device communication, data persistence, and protocol management.
**Note:** This is a backend-only service. Actual user interfacing is done via [SFM/Terra-View](https://github.com/your-org/terra-view) frontend applications.
**Note:** This is a backend-only service. Actual user interfacing is done via customized front ends or cli.
## Features
- **Background Polling** ⭐ NEW: Continuous automatic polling of devices with configurable intervals
- **Offline Detection** ⭐ NEW: Automatic device reachability tracking with failure counters
- **Device Management**: Configure and manage multiple NL43/NL53 devices
- **Real-time Monitoring**: Stream live measurement data via WebSocket
- **Measurement Control**: Start, stop, pause, resume, and reset measurements
@@ -22,18 +26,33 @@ SLMM is a standalone backend module that provides REST API routing and command t
## Architecture
```
┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐
Terra-View UI │◄───────►│ SLMM API │◄───────►│ NL43/NL53 │
│ (Frontend) │ HTTP │ (Backend) │ TCP │ Sound Meters │
└─────────────────┘ └──────────────┘ └─────────────────┘
┌──────────────┐
│ SQLite DB │
│ (Cache)
──────────────┘
┌─────────────────┐ ┌──────────────────────────────┐ ┌─────────────────┐
│◄───────►│ SLMM API │◄───────►│ NL43/NL53 │
│ (Frontend) │ HTTP │ • REST Endpoints │ TCP │ Sound Meters │
└─────────────────┘ │ • WebSocket Streaming │ └─────────────────┘
• Background Poller ⭐ NEW │ ▲
└──────────────────────────────┘ │
│ Continuous
▼ Polling
┌──────────────┐
│ SQLite DB │◄─────────────────────┘
│ • Config │
│ • Status │
└──────────────┘
```
### Background Polling (v0.2.0)
SLMM now includes a background polling service that continuously queries devices and updates the status cache:
- **Automatic Updates**: Devices are polled at configurable intervals (10-3600 seconds)
- **Offline Detection**: Devices marked unreachable after 3 consecutive failures
- **Per-Device Configuration**: Each device can have a custom polling interval
- **Resource Efficient**: Dynamic sleep intervals and smart scheduling
- **Graceful Shutdown**: Background task stops cleanly on service shutdown
This makes Terra-View significantly more responsive - status requests return cached data instantly (<100ms) instead of waiting for device queries (1-2 seconds).
## Quick Start
### Prerequisites
@@ -103,10 +122,18 @@ Logs are written to:
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/nl43/{unit_id}/status` | Get cached measurement snapshot |
| GET | `/api/nl43/{unit_id}/live` | Request fresh DOD data from device |
| GET | `/api/nl43/{unit_id}/status` | Get cached measurement snapshot (updated by background poller) |
| GET | `/api/nl43/{unit_id}/live` | Request fresh DOD data from device (bypasses cache) |
| WS | `/api/nl43/{unit_id}/stream` | WebSocket stream for real-time DRD data |
### Background Polling Configuration ⭐ NEW
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/nl43/{unit_id}/polling/config` | Get device polling configuration |
| PUT | `/api/nl43/{unit_id}/polling/config` | Update polling interval and enable/disable polling |
| GET | `/api/nl43/_polling/status` | Get global polling status for all devices |
### Measurement Control
| Method | Endpoint | Description |
@@ -167,6 +194,7 @@ slmm/
│ ├── routers.py # API route definitions
│ ├── models.py # SQLAlchemy database models
│ ├── services.py # NL43Client and business logic
│ ├── background_poller.py # Background polling service ⭐ NEW
│ └── database.py # Database configuration
├── data/
│ ├── slmm.db # SQLite database (auto-created)
@@ -175,9 +203,12 @@ slmm/
├── templates/
│ └── index.html # Simple web interface (optional)
├── manuals/ # Device documentation
├── migrate_add_polling_fields.py # Database migration for v0.2.0 ⭐ NEW
├── test_polling.sh # Polling feature test script ⭐ NEW
├── API.md # Detailed API documentation
├── COMMUNICATION_GUIDE.md # NL43 protocol documentation
├── NL43_COMMANDS.md # Command reference
├── CHANGELOG.md # Version history ⭐ NEW
├── requirements.txt # Python dependencies
└── README.md # This file
```
@@ -194,12 +225,16 @@ Stores device connection configuration:
- `ftp_username`: FTP authentication username
- `ftp_password`: FTP authentication password
- `web_enabled`: Enable/disable web interface access
- `poll_interval_seconds`: Polling interval in seconds (10-3600, default: 60) ⭐ NEW
- `poll_enabled`: Enable/disable background polling for this device ⭐ NEW
### NL43Status Table
Caches latest measurement snapshot:
- `unit_id` (PK): Unique device identifier
- `last_seen`: Timestamp of last update
- `measurement_state`: Current state (Measure/Stop)
- `measurement_start_time`: When measurement started (UTC)
- `counter`: Measurement interval counter (1-600)
- `lp`: Instantaneous sound pressure level
- `leq`: Equivalent continuous sound level
- `lmax`: Maximum sound level
@@ -210,6 +245,11 @@ Caches latest measurement snapshot:
- `sd_remaining_mb`: Free SD card space (MB)
- `sd_free_ratio`: SD card free space ratio
- `raw_payload`: Raw device response data
- `is_reachable`: Device reachability status (Boolean) ⭐ NEW
- `consecutive_failures`: Count of consecutive poll failures ⭐ NEW
- `last_poll_attempt`: Last time background poller attempted to poll ⭐ NEW
- `last_success`: Last successful poll timestamp ⭐ NEW
- `last_error`: Last error message (truncated to 500 chars) ⭐ NEW
## Protocol Details
@@ -224,6 +264,7 @@ Caches latest measurement snapshot:
- Uses active mode FTP (requires device to connect back)
- TCP and FTP are mutually exclusive on the device
- Credentials configurable per device
- **Default NL43 FTP Credentials**: Username: `USER`, Password: `0000`
### Data Formats
@@ -241,8 +282,9 @@ curl -X PUT http://localhost:8100/api/nl43/meter-001/config \
"host": "192.168.1.100",
"tcp_port": 2255,
"tcp_enabled": true,
"ftp_username": "admin",
"ftp_password": "password"
"ftp_enabled": true,
"ftp_username": "USER",
"ftp_password": "0000"
}'
```
@@ -251,11 +293,33 @@ curl -X PUT http://localhost:8100/api/nl43/meter-001/config \
curl -X POST http://localhost:8100/api/nl43/meter-001/start
```
### Get Live Status
### Get Cached Status (Fast - from background poller)
```bash
curl http://localhost:8100/api/nl43/meter-001/status
```
### Get Live Status (Bypasses cache)
```bash
curl http://localhost:8100/api/nl43/meter-001/live
```
### Configure Background Polling ⭐ NEW
```bash
# Set polling interval to 30 seconds
curl -X PUT http://localhost:8100/api/nl43/meter-001/polling/config \
-H "Content-Type: application/json" \
-d '{
"poll_interval_seconds": 30,
"poll_enabled": true
}'
# Get polling configuration
curl http://localhost:8100/api/nl43/meter-001/polling/config
# Check global polling status
curl http://localhost:8100/api/nl43/_polling/status
```
### Verify Device Settings
```bash
curl http://localhost:8100/api/nl43/meter-001/settings
@@ -354,13 +418,31 @@ pytest
### Database Migrations
```bash
# Migrate existing database to add FTP credentials
# Migrate to v0.2.0 (add background polling fields)
python3 migrate_add_polling_fields.py
# Legacy: Migrate to add FTP credentials
python migrate_add_ftp_credentials.py
# Set FTP credentials for a device
python set_ftp_credentials.py <unit_id> <username> <password>
```
### Testing Background Polling
```bash
# Run comprehensive polling tests
./test_polling.sh [unit_id]
# Test settings endpoint
python3 test_settings_endpoint.py <unit_id>
# Test sleep mode auto-disable
python3 test_sleep_mode_auto_disable.py <unit_id>
```
### Legacy Scripts
Old migration scripts and manual polling tools have been moved to `archive/` for reference. See [archive/README.md](archive/README.md) for details.
## Contributing
This is a standalone module kept separate from the SFM/Terra-View codebase. When contributing:

343
app/background_poller.py Normal file
View File

@@ -0,0 +1,343 @@
"""
Background polling service for NL43 devices.
This module provides continuous, automatic polling of configured NL43 devices
at configurable intervals. Status snapshots are persisted to the database
for fast API access without querying devices on every request.
"""
import asyncio
import logging
from datetime import datetime, timedelta
from typing import Optional
from sqlalchemy.orm import Session
from app.database import SessionLocal
from app.models import NL43Config, NL43Status
from app.services import NL43Client, persist_snapshot, sync_measurement_start_time_from_ftp
from app.device_logger import log_device_event, cleanup_old_logs
logger = logging.getLogger(__name__)
class BackgroundPoller:
"""
Background task that continuously polls NL43 devices and updates status cache.
Features:
- Per-device configurable poll intervals (30 seconds to 6 hours)
- Automatic offline detection (marks unreachable after 3 consecutive failures)
- Dynamic sleep intervals based on device configurations
- Graceful shutdown on application stop
- Respects existing rate limiting (1-second minimum between commands)
"""
def __init__(self):
self._task: Optional[asyncio.Task] = None
self._running = False
self._logger = logger
self._last_cleanup = None # Track last log cleanup time
async def start(self):
"""Start the background polling task."""
if self._running:
self._logger.warning("Background poller already running")
return
self._running = True
self._task = asyncio.create_task(self._poll_loop())
self._logger.info("Background poller task created")
async def stop(self):
"""Gracefully stop the background polling task."""
if not self._running:
return
self._logger.info("Stopping background poller...")
self._running = False
if self._task:
try:
await asyncio.wait_for(self._task, timeout=5.0)
except asyncio.TimeoutError:
self._logger.warning("Background poller task did not stop gracefully, cancelling...")
self._task.cancel()
try:
await self._task
except asyncio.CancelledError:
pass
self._logger.info("Background poller stopped")
async def _poll_loop(self):
"""Main polling loop that runs continuously."""
self._logger.info("Background polling loop started")
while self._running:
try:
await self._poll_all_devices()
except Exception as e:
self._logger.error(f"Error in poll loop: {e}", exc_info=True)
# Run log cleanup once per hour
try:
now = datetime.utcnow()
if self._last_cleanup is None or (now - self._last_cleanup).total_seconds() > 3600:
cleanup_old_logs()
self._last_cleanup = now
except Exception as e:
self._logger.warning(f"Log cleanup failed: {e}")
# Calculate dynamic sleep interval
sleep_time = self._calculate_sleep_interval()
self._logger.debug(f"Sleeping for {sleep_time} seconds until next poll cycle")
# Sleep in small intervals to allow graceful shutdown
for _ in range(int(sleep_time)):
if not self._running:
break
await asyncio.sleep(1)
self._logger.info("Background polling loop exited")
async def _poll_all_devices(self):
"""Poll all configured devices that are due for polling."""
db: Session = SessionLocal()
try:
# Get all devices with TCP and polling enabled
configs = db.query(NL43Config).filter_by(
tcp_enabled=True,
poll_enabled=True
).all()
if not configs:
self._logger.debug("No devices configured for polling")
return
self._logger.debug(f"Checking {len(configs)} devices for polling")
now = datetime.utcnow()
polled_count = 0
for cfg in configs:
if not self._running:
break
# Get current status
status = db.query(NL43Status).filter_by(unit_id=cfg.unit_id).first()
# Check if device should be polled
if self._should_poll(cfg, status, now):
await self._poll_device(cfg, db)
polled_count += 1
else:
self._logger.debug(f"Skipping {cfg.unit_id} - interval not elapsed")
if polled_count > 0:
self._logger.info(f"Polled {polled_count}/{len(configs)} devices")
finally:
db.close()
def _should_poll(self, cfg: NL43Config, status: Optional[NL43Status], now: datetime) -> bool:
"""
Determine if a device should be polled based on interval and last poll time.
Args:
cfg: Device configuration
status: Current device status (may be None if never polled)
now: Current UTC timestamp
Returns:
True if device should be polled, False otherwise
"""
# If never polled before, poll now
if not status or not status.last_poll_attempt:
self._logger.debug(f"Device {cfg.unit_id} never polled, polling now")
return True
# Calculate elapsed time since last poll attempt
interval = cfg.poll_interval_seconds or 60
elapsed = (now - status.last_poll_attempt).total_seconds()
should_poll = elapsed >= interval
if should_poll:
self._logger.debug(
f"Device {cfg.unit_id} due for polling: {elapsed:.1f}s elapsed, interval={interval}s"
)
return should_poll
async def _poll_device(self, cfg: NL43Config, db: Session):
"""
Poll a single device and update its status in the database.
Args:
cfg: Device configuration
db: Database session
"""
unit_id = cfg.unit_id
self._logger.info(f"Polling device {unit_id} at {cfg.host}:{cfg.tcp_port}")
# Get or create status record
status = db.query(NL43Status).filter_by(unit_id=unit_id).first()
if not status:
status = NL43Status(unit_id=unit_id)
db.add(status)
# Update last_poll_attempt immediately
status.last_poll_attempt = datetime.utcnow()
db.commit()
# Create client and attempt to poll
client = NL43Client(
cfg.host,
cfg.tcp_port,
timeout=5.0,
ftp_username=cfg.ftp_username,
ftp_password=cfg.ftp_password,
ftp_port=cfg.ftp_port or 21
)
try:
# Send DOD? command to get device status
snap = await client.request_dod()
snap.unit_id = unit_id
# Success - persist snapshot and reset failure counter
persist_snapshot(snap, db)
status.is_reachable = True
status.consecutive_failures = 0
status.last_success = datetime.utcnow()
status.last_error = None
db.commit()
self._logger.info(f"✓ Successfully polled {unit_id}")
# Log to device log
log_device_event(
unit_id, "INFO", "POLL",
f"Poll success: state={snap.measurement_state}, Leq={snap.leq}, Lp={snap.lp}",
db
)
# Check if device is measuring but has no start time recorded
# This happens if measurement was started before SLMM began polling
# or after a service restart
status = db.query(NL43Status).filter_by(unit_id=unit_id).first()
# Reset the sync flag when measurement stops (so next measurement can sync)
if status and status.measurement_state != "Start":
if status.start_time_sync_attempted:
status.start_time_sync_attempted = False
db.commit()
self._logger.debug(f"Reset FTP sync flag for {unit_id} (measurement stopped)")
log_device_event(unit_id, "DEBUG", "STATE", "Measurement stopped, reset FTP sync flag", db)
# Attempt FTP sync if:
# - Device is measuring
# - No start time recorded
# - FTP sync not already attempted for this measurement
# - FTP is configured
if (status and
status.measurement_state == "Start" and
status.measurement_start_time is None and
not status.start_time_sync_attempted and
cfg.ftp_enabled and
cfg.ftp_username and
cfg.ftp_password):
self._logger.info(
f"Device {unit_id} is measuring but has no start time - "
f"attempting FTP sync"
)
log_device_event(unit_id, "INFO", "SYNC", "Attempting FTP sync for measurement start time", db)
# Mark that we attempted sync (prevents repeated attempts on failure)
status.start_time_sync_attempted = True
db.commit()
try:
synced = await sync_measurement_start_time_from_ftp(
unit_id=unit_id,
host=cfg.host,
tcp_port=cfg.tcp_port,
ftp_port=cfg.ftp_port or 21,
ftp_username=cfg.ftp_username,
ftp_password=cfg.ftp_password,
db=db
)
if synced:
self._logger.info(f"✓ FTP sync succeeded for {unit_id}")
log_device_event(unit_id, "INFO", "SYNC", "FTP sync succeeded - measurement start time updated", db)
else:
self._logger.warning(f"FTP sync returned False for {unit_id}")
log_device_event(unit_id, "WARNING", "SYNC", "FTP sync returned False", db)
except Exception as sync_err:
self._logger.warning(
f"FTP sync failed for {unit_id}: {sync_err}"
)
log_device_event(unit_id, "ERROR", "SYNC", f"FTP sync failed: {sync_err}", db)
except Exception as e:
# Failure - increment counter and potentially mark offline
status.consecutive_failures += 1
error_msg = str(e)[:500] # Truncate to prevent bloat
status.last_error = error_msg
# Mark unreachable after 3 consecutive failures
if status.consecutive_failures >= 3:
if status.is_reachable: # Only log transition
self._logger.warning(
f"Device {unit_id} marked unreachable after {status.consecutive_failures} failures: {error_msg}"
)
log_device_event(unit_id, "ERROR", "POLL", f"Device marked UNREACHABLE after {status.consecutive_failures} failures: {error_msg}", db)
status.is_reachable = False
else:
self._logger.warning(
f"Poll failed for {unit_id} (attempt {status.consecutive_failures}/3): {error_msg}"
)
log_device_event(unit_id, "WARNING", "POLL", f"Poll failed (attempt {status.consecutive_failures}/3): {error_msg}", db)
db.commit()
def _calculate_sleep_interval(self) -> int:
"""
Calculate the next sleep interval based on all device poll intervals.
Returns a dynamic sleep time that ensures responsive polling:
- Minimum 30 seconds (prevents tight loops)
- Maximum 300 seconds / 5 minutes (ensures reasonable responsiveness for long intervals)
- Generally half the minimum device interval
Returns:
Sleep interval in seconds
"""
db: Session = SessionLocal()
try:
configs = db.query(NL43Config).filter_by(
tcp_enabled=True,
poll_enabled=True
).all()
if not configs:
return 60 # Default sleep when no devices configured
# Get all intervals
intervals = [cfg.poll_interval_seconds or 60 for cfg in configs]
min_interval = min(intervals)
# Use half the minimum interval, but cap between 30-300 seconds
# This allows longer sleep times when polling intervals are long (e.g., hourly)
sleep_time = max(30, min(300, min_interval // 2))
return sleep_time
finally:
db.close()
# Global singleton instance
poller = BackgroundPoller()

277
app/device_logger.py Normal file
View File

@@ -0,0 +1,277 @@
"""
Per-device logging system.
Provides dual output: database entries for structured queries and file logs for backup.
Each device gets its own log file in data/logs/{unit_id}.log with rotation.
"""
import logging
import os
from datetime import datetime, timedelta
from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import Optional
from sqlalchemy.orm import Session
from app.database import SessionLocal
from app.models import DeviceLog
# Configure base logger
logger = logging.getLogger(__name__)
# Log directory (persisted in Docker volume)
LOG_DIR = Path(os.path.dirname(os.path.dirname(__file__))) / "data" / "logs"
LOG_DIR.mkdir(parents=True, exist_ok=True)
# Per-device file loggers (cached)
_device_file_loggers: dict = {}
# Log retention (days)
LOG_RETENTION_DAYS = int(os.getenv("LOG_RETENTION_DAYS", "7"))
def _get_file_logger(unit_id: str) -> logging.Logger:
"""Get or create a file logger for a specific device."""
if unit_id in _device_file_loggers:
return _device_file_loggers[unit_id]
# Create device-specific logger
device_logger = logging.getLogger(f"device.{unit_id}")
device_logger.setLevel(logging.DEBUG)
# Avoid duplicate handlers
if not device_logger.handlers:
# Create rotating file handler (5 MB max, keep 3 backups)
log_file = LOG_DIR / f"{unit_id}.log"
handler = RotatingFileHandler(
log_file,
maxBytes=5 * 1024 * 1024, # 5 MB
backupCount=3,
encoding="utf-8"
)
handler.setLevel(logging.DEBUG)
# Format: timestamp [LEVEL] [CATEGORY] message
formatter = logging.Formatter(
"%(asctime)s [%(levelname)s] [%(category)s] %(message)s",
datefmt="%Y-%m-%d %H:%M:%S"
)
handler.setFormatter(formatter)
device_logger.addHandler(handler)
# Don't propagate to root logger
device_logger.propagate = False
_device_file_loggers[unit_id] = device_logger
return device_logger
def log_device_event(
unit_id: str,
level: str,
category: str,
message: str,
db: Optional[Session] = None
):
"""
Log an event for a specific device.
Writes to both:
1. Database (DeviceLog table) for structured queries
2. File (data/logs/{unit_id}.log) for backup/debugging
Args:
unit_id: Device identifier
level: Log level (DEBUG, INFO, WARNING, ERROR)
category: Event category (TCP, FTP, POLL, COMMAND, STATE, SYNC)
message: Log message
db: Optional database session (creates one if not provided)
"""
timestamp = datetime.utcnow()
# Write to file log
try:
file_logger = _get_file_logger(unit_id)
log_func = getattr(file_logger, level.lower(), file_logger.info)
# Pass category as extra for formatter
log_func(message, extra={"category": category})
except Exception as e:
logger.warning(f"Failed to write file log for {unit_id}: {e}")
# Write to database
close_db = False
try:
if db is None:
db = SessionLocal()
close_db = True
log_entry = DeviceLog(
unit_id=unit_id,
timestamp=timestamp,
level=level.upper(),
category=category.upper(),
message=message
)
db.add(log_entry)
db.commit()
except Exception as e:
logger.warning(f"Failed to write DB log for {unit_id}: {e}")
if db:
db.rollback()
finally:
if close_db and db:
db.close()
def cleanup_old_logs(retention_days: Optional[int] = None, db: Optional[Session] = None):
"""
Delete log entries older than retention period.
Args:
retention_days: Days to retain (default: LOG_RETENTION_DAYS env var or 7)
db: Optional database session
"""
if retention_days is None:
retention_days = LOG_RETENTION_DAYS
cutoff = datetime.utcnow() - timedelta(days=retention_days)
close_db = False
try:
if db is None:
db = SessionLocal()
close_db = True
deleted = db.query(DeviceLog).filter(DeviceLog.timestamp < cutoff).delete()
db.commit()
if deleted > 0:
logger.info(f"Cleaned up {deleted} log entries older than {retention_days} days")
except Exception as e:
logger.error(f"Failed to cleanup old logs: {e}")
if db:
db.rollback()
finally:
if close_db and db:
db.close()
def get_device_logs(
unit_id: str,
limit: int = 100,
offset: int = 0,
level: Optional[str] = None,
category: Optional[str] = None,
since: Optional[datetime] = None,
db: Optional[Session] = None
) -> list:
"""
Query log entries for a specific device.
Args:
unit_id: Device identifier
limit: Max entries to return (default: 100)
offset: Number of entries to skip (default: 0)
level: Filter by level (DEBUG, INFO, WARNING, ERROR)
category: Filter by category (TCP, FTP, POLL, COMMAND, STATE, SYNC)
since: Filter entries after this timestamp
db: Optional database session
Returns:
List of log entries as dicts
"""
close_db = False
try:
if db is None:
db = SessionLocal()
close_db = True
query = db.query(DeviceLog).filter(DeviceLog.unit_id == unit_id)
if level:
query = query.filter(DeviceLog.level == level.upper())
if category:
query = query.filter(DeviceLog.category == category.upper())
if since:
query = query.filter(DeviceLog.timestamp >= since)
# Order by newest first
query = query.order_by(DeviceLog.timestamp.desc())
# Apply pagination
entries = query.offset(offset).limit(limit).all()
return [
{
"id": e.id,
"timestamp": e.timestamp.isoformat() + "Z",
"level": e.level,
"category": e.category,
"message": e.message
}
for e in entries
]
finally:
if close_db and db:
db.close()
def get_log_stats(unit_id: str, db: Optional[Session] = None) -> dict:
"""
Get log statistics for a device.
Returns:
Dict with counts by level and category
"""
close_db = False
try:
if db is None:
db = SessionLocal()
close_db = True
total = db.query(DeviceLog).filter(DeviceLog.unit_id == unit_id).count()
# Count by level
level_counts = {}
for level in ["DEBUG", "INFO", "WARNING", "ERROR"]:
count = db.query(DeviceLog).filter(
DeviceLog.unit_id == unit_id,
DeviceLog.level == level
).count()
if count > 0:
level_counts[level] = count
# Count by category
category_counts = {}
for category in ["TCP", "FTP", "POLL", "COMMAND", "STATE", "SYNC", "GENERAL"]:
count = db.query(DeviceLog).filter(
DeviceLog.unit_id == unit_id,
DeviceLog.category == category
).count()
if count > 0:
category_counts[category] = count
# Get oldest and newest
oldest = db.query(DeviceLog).filter(
DeviceLog.unit_id == unit_id
).order_by(DeviceLog.timestamp.asc()).first()
newest = db.query(DeviceLog).filter(
DeviceLog.unit_id == unit_id
).order_by(DeviceLog.timestamp.desc()).first()
return {
"total": total,
"by_level": level_counts,
"by_category": category_counts,
"oldest": oldest.timestamp.isoformat() + "Z" if oldest else None,
"newest": newest.timestamp.isoformat() + "Z" if newest else None
}
finally:
if close_db and db:
db.close()

View File

@@ -1,5 +1,6 @@
import os
import logging
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import HTMLResponse
@@ -7,6 +8,7 @@ from fastapi.templating import Jinja2Templates
from app.database import Base, engine
from app import routers
from app.background_poller import poller
# Configure logging
logging.basicConfig(
@@ -23,10 +25,28 @@ logger = logging.getLogger(__name__)
Base.metadata.create_all(bind=engine)
logger.info("Database tables initialized")
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Manage application lifecycle - startup and shutdown events."""
# Startup
logger.info("Starting background poller...")
await poller.start()
logger.info("Background poller started")
yield # Application runs
# Shutdown
logger.info("Stopping background poller...")
await poller.stop()
logger.info("Background poller stopped")
app = FastAPI(
title="SLMM NL43 Addon",
description="Standalone module for NL43 configuration and status APIs",
version="0.1.0",
description="Standalone module for NL43 configuration and status APIs with background polling",
version="0.2.0",
lifespan=lifespan,
)
# CORS configuration - use environment variable for allowed origins
@@ -52,6 +72,11 @@ def index(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
@app.get("/roster", response_class=HTMLResponse)
def roster(request: Request):
return templates.TemplateResponse("roster.html", {"request": request})
@app.get("/health")
async def health():
"""Basic health check endpoint."""

View File

@@ -11,13 +11,18 @@ class NL43Config(Base):
unit_id = Column(String, primary_key=True, index=True)
host = Column(String, default="127.0.0.1")
tcp_port = Column(Integer, default=80) # NL43 TCP control port (via RX55)
tcp_port = Column(Integer, default=2255) # NL43 TCP control port (standard: 2255)
tcp_enabled = Column(Boolean, default=True)
ftp_enabled = Column(Boolean, default=False)
ftp_port = Column(Integer, default=21) # FTP port (standard: 21)
ftp_username = Column(String, nullable=True) # FTP login username
ftp_password = Column(String, nullable=True) # FTP login password
web_enabled = Column(Boolean, default=False)
# Background polling configuration
poll_interval_seconds = Column(Integer, nullable=True, default=60) # Polling interval (10-3600 seconds)
poll_enabled = Column(Boolean, default=True) # Enable/disable background polling for this device
class NL43Status(Base):
"""
@@ -29,13 +34,41 @@ class NL43Status(Base):
unit_id = Column(String, primary_key=True, index=True)
last_seen = Column(DateTime, default=func.now())
measurement_state = Column(String, default="unknown") # Measure/Stop
lp = Column(String, nullable=True)
leq = Column(String, nullable=True)
lmax = Column(String, nullable=True)
lmin = Column(String, nullable=True)
lpeak = Column(String, nullable=True)
measurement_start_time = Column(DateTime, nullable=True) # When measurement started (UTC)
counter = Column(String, nullable=True) # d0: Measurement interval counter (1-600)
lp = Column(String, nullable=True) # Instantaneous sound pressure level
leq = Column(String, nullable=True) # Equivalent continuous sound level
lmax = Column(String, nullable=True) # Maximum level
lmin = Column(String, nullable=True) # Minimum level
lpeak = Column(String, nullable=True) # Peak level
battery_level = Column(String, nullable=True)
power_source = Column(String, nullable=True)
sd_remaining_mb = Column(String, nullable=True)
sd_free_ratio = Column(String, nullable=True)
raw_payload = Column(Text, nullable=True)
# Background polling status
is_reachable = Column(Boolean, default=True) # Device reachability status
consecutive_failures = Column(Integer, default=0) # Count of consecutive poll failures
last_poll_attempt = Column(DateTime, nullable=True) # Last time background poller attempted to poll
last_success = Column(DateTime, nullable=True) # Last successful poll timestamp
last_error = Column(Text, nullable=True) # Last error message (truncated to 500 chars)
# FTP start time sync tracking
start_time_sync_attempted = Column(Boolean, default=False) # True if FTP sync was attempted for current measurement
class DeviceLog(Base):
"""
Per-device log entries for debugging and audit trail.
Stores events like commands, state changes, errors, and FTP operations.
"""
__tablename__ = "device_logs"
id = Column(Integer, primary_key=True, autoincrement=True)
unit_id = Column(String, index=True, nullable=False)
timestamp = Column(DateTime, default=func.now(), index=True)
level = Column(String, default="INFO") # DEBUG, INFO, WARNING, ERROR
category = Column(String, default="GENERAL") # TCP, FTP, POLL, COMMAND, STATE, SYNC
message = Column(Text, nullable=False)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

67
archive/README.md Normal file
View File

@@ -0,0 +1,67 @@
# SLMM Archive
This directory contains legacy scripts that are no longer needed for normal operation but are preserved for reference.
## Legacy Migrations (`legacy_migrations/`)
These migration scripts were used during SLMM development (v0.1.x) to incrementally add database fields. They are **no longer needed** because:
1. **Fresh databases** get the complete schema automatically from `app/models.py`
2. **Existing databases** should already have these fields from previous runs
3. **Current migration** is `migrate_add_polling_fields.py` (v0.2.0) in the parent directory
### Archived Migration Files
- `migrate_add_counter.py` - Added `counter` field to NL43Status
- `migrate_add_measurement_start_time.py` - Added `measurement_start_time` field
- `migrate_add_ftp_port.py` - Added `ftp_port` field to NL43Config
- `migrate_field_names.py` - Renamed fields for consistency (one-time fix)
- `migrate_revert_field_names.py` - Rollback for the rename migration
**Do not delete** - These provide historical context for database schema evolution.
---
## Legacy Tools
### `nl43_dod_poll.py`
Manual polling script that queries a single NL-43 device for DOD (Device On-Demand) data.
**Status**: Replaced by background polling system in v0.2.0
**Why archived**:
- Background poller (`app/background_poller.py`) now handles continuous polling automatically
- No need for manual polling scripts
- Kept for reference in case manual querying is needed for debugging
**How to use** (if needed):
```bash
cd /home/serversdown/tmi/slmm/archive
python3 nl43_dod_poll.py <host> <port> <unit_id>
```
---
## Active Scripts (Still in Parent Directory)
These scripts are **actively used** and documented in the main README:
### Migrations
- `migrate_add_polling_fields.py` - **v0.2.0 migration** - Adds background polling fields
- `migrate_add_ftp_credentials.py` - **Legacy FTP migration** - Adds FTP auth fields
### Testing
- `test_polling.sh` - Comprehensive test suite for background polling features
- `test_settings_endpoint.py` - Tests device settings API
- `test_sleep_mode_auto_disable.py` - Tests automatic sleep mode handling
### Utilities
- `set_ftp_credentials.py` - Command-line tool to set FTP credentials for a device
---
## Version History
- **v0.2.0** (2026-01-15) - Background polling system added, manual polling scripts archived
- **v0.1.0** (2025-12-XX) - Initial release with incremental migrations

View File

@@ -0,0 +1,57 @@
#!/usr/bin/env python3
"""
Database migration: Add counter field to nl43_status table
This adds the d0 (measurement interval counter) field to track the device's
actual measurement progress for accurate timer synchronization.
"""
import sqlite3
import sys
DB_PATH = "data/slmm.db"
def migrate():
print(f"Adding counter field to: {DB_PATH}")
try:
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
# Check if counter column already exists
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cursor.fetchall()]
if 'counter' in columns:
print("✓ Counter column already exists, no migration needed")
conn.close()
return
print("Starting migration...")
# Add counter column
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN counter TEXT
""")
conn.commit()
print("✓ Added counter column")
# Verify
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cursor.fetchall()]
if 'counter' not in columns:
raise Exception("Counter column was not added successfully")
print("✓ Migration completed successfully")
conn.close()
except Exception as e:
print(f"✗ Migration failed: {e}")
sys.exit(1)
if __name__ == "__main__":
migrate()

View File

@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""
Migration script to add ftp_port column to nl43_config table.
Usage:
python migrate_add_ftp_port.py
"""
import sqlite3
import sys
from pathlib import Path
def migrate():
db_path = Path("data/slmm.db")
if not db_path.exists():
print(f"❌ Database not found at {db_path}")
print(" Run this script from the slmm directory")
return False
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check if column already exists
cursor.execute("PRAGMA table_info(nl43_config)")
columns = [row[1] for row in cursor.fetchall()]
if "ftp_port" in columns:
print("✓ ftp_port column already exists")
conn.close()
return True
print("Adding ftp_port column to nl43_config table...")
# Add the ftp_port column with default value of 21
cursor.execute("""
ALTER TABLE nl43_config
ADD COLUMN ftp_port INTEGER DEFAULT 21
""")
conn.commit()
print("✓ Migration completed successfully")
print(" Added ftp_port column (default: 21)")
conn.close()
return True
except Exception as e:
print(f"❌ Migration failed: {e}")
return False
if __name__ == "__main__":
success = migrate()
sys.exit(0 if success else 1)

View File

@@ -0,0 +1,58 @@
#!/usr/bin/env python3
"""
Database migration: Add measurement_start_time field to nl43_status table
This tracks when a measurement session started by detecting the state transition
from "Stop" to "Measure", enabling accurate elapsed time display even for
manually-started measurements.
"""
import sqlite3
import sys
DB_PATH = "data/slmm.db"
def migrate():
print(f"Adding measurement_start_time field to: {DB_PATH}")
try:
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
# Check if measurement_start_time column already exists
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cursor.fetchall()]
if 'measurement_start_time' in columns:
print("✓ measurement_start_time column already exists, no migration needed")
conn.close()
return
print("Starting migration...")
# Add measurement_start_time column
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN measurement_start_time TEXT
""")
conn.commit()
print("✓ Added measurement_start_time column")
# Verify
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cursor.fetchall()]
if 'measurement_start_time' not in columns:
raise Exception("measurement_start_time column was not added successfully")
print("✓ Migration completed successfully")
conn.close()
except Exception as e:
print(f"✗ Migration failed: {e}")
sys.exit(1)
if __name__ == "__main__":
migrate()

View File

@@ -0,0 +1,111 @@
#!/usr/bin/env python3
"""
Migration script to rename NL43 measurement field names to match actual device output.
Changes:
- lp -> laeq (A-weighted equivalent continuous sound level)
- leq -> lae (A-weighted sound exposure level)
- lmax -> lasmax (A-weighted slow maximum)
- lmin -> lasmin (A-weighted slow minimum)
- lpeak -> lapeak (A-weighted peak)
"""
import sqlite3
import sys
from pathlib import Path
def migrate_database(db_path: str):
"""Migrate the database schema to use correct field names."""
print(f"Migrating database: {db_path}")
# Connect to database
conn = sqlite3.connect(db_path)
cur = conn.cursor()
try:
# Check if migration is needed
cur.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cur.fetchall()]
if 'laeq' in columns:
print("✓ Database already migrated")
return
if 'lp' not in columns:
print("✗ Database schema does not match expected format")
sys.exit(1)
print("Starting migration...")
# SQLite doesn't support column renaming directly, so we need to:
# 1. Create new table with correct column names
# 2. Copy data from old table
# 3. Drop old table
# 4. Rename new table
# Create new table with correct column names
cur.execute("""
CREATE TABLE nl43_status_new (
unit_id VARCHAR PRIMARY KEY,
last_seen DATETIME,
measurement_state VARCHAR,
laeq VARCHAR,
lae VARCHAR,
lasmax VARCHAR,
lasmin VARCHAR,
lapeak VARCHAR,
battery_level VARCHAR,
power_source VARCHAR,
sd_remaining_mb VARCHAR,
sd_free_ratio VARCHAR,
raw_payload TEXT
)
""")
print("✓ Created new table with correct column names")
# Copy data from old table to new table
cur.execute("""
INSERT INTO nl43_status_new
(unit_id, last_seen, measurement_state, laeq, lae, lasmax, lasmin, lapeak,
battery_level, power_source, sd_remaining_mb, sd_free_ratio, raw_payload)
SELECT
unit_id, last_seen, measurement_state, lp, leq, lmax, lmin, lpeak,
battery_level, power_source, sd_remaining_mb, sd_free_ratio, raw_payload
FROM nl43_status
""")
rows_copied = cur.rowcount
print(f"✓ Copied {rows_copied} rows from old table")
# Drop old table
cur.execute("DROP TABLE nl43_status")
print("✓ Dropped old table")
# Rename new table
cur.execute("ALTER TABLE nl43_status_new RENAME TO nl43_status")
print("✓ Renamed new table to nl43_status")
# Commit changes
conn.commit()
print("✓ Migration completed successfully")
except Exception as e:
conn.rollback()
print(f"✗ Migration failed: {e}")
sys.exit(1)
finally:
conn.close()
if __name__ == "__main__":
# Default database path
db_path = Path(__file__).parent / "data" / "slmm.db"
# Allow custom path as command line argument
if len(sys.argv) > 1:
db_path = Path(sys.argv[1])
if not db_path.exists():
print(f"✗ Database not found: {db_path}")
sys.exit(1)
migrate_database(str(db_path))

View File

@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""
Migration script to revert NL43 measurement field names back to correct DRD format.
The previous migration was incorrect. According to NL43 DRD documentation:
- d0 = counter (1-600) - NOT a measurement!
- d1 = Lp (instantaneous sound pressure level)
- d2 = Leq (equivalent continuous sound level)
- d3 = Lmax (maximum level)
- d4 = Lmin (minimum level)
- d5 = Lpeak (peak level)
Changes:
- laeq -> lp (was incorrectly mapped to counter field!)
- lae -> leq
- lasmax -> lmax
- lasmin -> lmin
- lapeak -> lpeak
"""
import sqlite3
import sys
from pathlib import Path
def migrate_database(db_path: str):
"""Revert database schema to correct DRD field names."""
print(f"Reverting database migration: {db_path}")
# Connect to database
conn = sqlite3.connect(db_path)
cur = conn.cursor()
try:
# Check if migration is needed
cur.execute("PRAGMA table_info(nl43_status)")
columns = [row[1] for row in cur.fetchall()]
if 'lp' in columns:
print("✓ Database already has correct field names")
return
if 'laeq' not in columns:
print("✗ Database schema does not match expected format")
sys.exit(1)
print("Starting revert migration...")
# Create new table with correct column names
cur.execute("""
CREATE TABLE nl43_status_new (
unit_id VARCHAR PRIMARY KEY,
last_seen DATETIME,
measurement_state VARCHAR,
lp VARCHAR,
leq VARCHAR,
lmax VARCHAR,
lmin VARCHAR,
lpeak VARCHAR,
battery_level VARCHAR,
power_source VARCHAR,
sd_remaining_mb VARCHAR,
sd_free_ratio VARCHAR,
raw_payload TEXT
)
""")
print("✓ Created new table with correct DRD field names")
# Copy data from old table to new table
# Note: laeq was incorrectly mapped to d0 (counter), so we discard it
# The actual measurements start from d1
cur.execute("""
INSERT INTO nl43_status_new
(unit_id, last_seen, measurement_state, lp, leq, lmax, lmin, lpeak,
battery_level, power_source, sd_remaining_mb, sd_free_ratio, raw_payload)
SELECT
unit_id, last_seen, measurement_state, lae, lasmax, lasmin, lapeak, NULL,
battery_level, power_source, sd_remaining_mb, sd_free_ratio, raw_payload
FROM nl43_status
""")
rows_copied = cur.rowcount
print(f"✓ Copied {rows_copied} rows (note: discarded incorrect 'laeq' counter field)")
# Drop old table
cur.execute("DROP TABLE nl43_status")
print("✓ Dropped old table")
# Rename new table
cur.execute("ALTER TABLE nl43_status_new RENAME TO nl43_status")
print("✓ Renamed new table to nl43_status")
# Commit changes
conn.commit()
print("✓ Revert migration completed successfully")
print("\nNote: The 'lp' field will be populated correctly on next device measurement")
except Exception as e:
conn.rollback()
print(f"✗ Migration failed: {e}")
sys.exit(1)
finally:
conn.close()
if __name__ == "__main__":
# Default database path
db_path = Path(__file__).parent / "data" / "slmm.db"
# Allow custom path as command line argument
if len(sys.argv) > 1:
db_path = Path(sys.argv[1])
if not db_path.exists():
print(f"✗ Database not found: {db_path}")
sys.exit(1)
migrate_database(str(db_path))

100
archive/nl43_dod_poll.py Normal file
View File

@@ -0,0 +1,100 @@
#!/usr/bin/env python3
"""
Diagnostic poller for NL-43 TCP connectivity.
Every interval, open a TCP connection, send DOD?, read response, and log results.
"""
from __future__ import annotations
import datetime as dt
import socket
import time
from pathlib import Path
# ---- Configuration (edit as needed) ----
HOST = "192.168.0.10"
PORT = 2255
INTERVAL_SECONDS = 5 * 60
CONNECT_TIMEOUT_SECONDS = 5.0
READ_TIMEOUT_SECONDS = 5.0
LOG_PATH = Path("nl43_dod_poll.log")
# ---------------------------------------
def _timestamp() -> str:
return dt.datetime.utcnow().isoformat(timespec="seconds") + "Z"
def _read_line(sock_file) -> str:
line = sock_file.readline()
if not line:
raise ConnectionError("Socket closed before full response")
return line.decode("ascii", errors="ignore").strip()
def _poll_once() -> tuple[bool, str, str, str, str]:
sock = None
result_code = ""
data_line = ""
try:
sock = socket.create_connection((HOST, PORT), timeout=CONNECT_TIMEOUT_SECONDS)
sock.settimeout(READ_TIMEOUT_SECONDS)
sock.sendall(b"DOD?\r\n")
with sock.makefile("rb") as sock_file:
result_code = _read_line(sock_file)
if result_code.startswith("$"):
result_code = result_code[1:].strip()
if result_code != "R+0000":
return False, "other", f"device_result={result_code}", result_code, data_line
data_line = _read_line(sock_file)
if data_line.startswith("$"):
data_line = data_line[1:].strip()
return True, "none", "ok", result_code, data_line
except socket.timeout:
return False, "timeout", "socket_timeout", result_code, data_line
except ConnectionRefusedError:
return False, "refused", "connection_refused", result_code, data_line
except OSError as exc:
return False, "other", f"os_error={exc.__class__.__name__}", result_code, data_line
except Exception as exc:
return False, "other", f"error={exc.__class__.__name__}", result_code, data_line
finally:
if sock is not None:
try:
sock.shutdown(socket.SHUT_RDWR)
except OSError:
pass
sock.close()
def _log_line(text: str) -> None:
print(text, flush=True)
with LOG_PATH.open("a", encoding="ascii") as handle:
handle.write(text + "\n")
def main() -> None:
while True:
start = time.monotonic()
ok, error_type, detail, result_code, data_line = _poll_once()
status = "success" if ok else "failure"
msg = (
f"ts={_timestamp()} status={status} error_type={error_type} "
f"detail={detail} result_code={result_code} data={data_line}"
)
_log_line(msg)
elapsed = time.monotonic() - start
sleep_for = max(0.0, INTERVAL_SECONDS - elapsed)
time.sleep(sleep_for)
if __name__ == "__main__":
main()

View File

@@ -48,6 +48,8 @@ Update device configuration.
}
```
**Important:** When TCP is enabled and connection details are provided, sleep mode will be automatically disabled on the device. This is necessary because sleep/eco mode turns off TCP communications, which would prevent remote monitoring and control.
## Device Status
### Get Cached Status
@@ -96,6 +98,8 @@ POST /{unit_id}/start
```
Starts measurement on the device.
**Important:** Before starting the measurement, sleep mode is automatically disabled to ensure TCP communications remain active throughout the measurement session.
### Stop Measurement
```
POST /{unit_id}/stop
@@ -445,6 +449,12 @@ Enables Sleep Mode on the device. When enabled, the device will automatically en
**Note:** This is a SETTING, not a command to sleep immediately. Sleep Mode only applies when using Timer Auto measurements.
**Warning:** Sleep/eco mode turns off TCP communications, which will prevent remote monitoring and control. For this reason, SLMM automatically disables sleep mode when:
- Device configuration is created or updated with TCP enabled
- Measurements are started
If you need to enable sleep mode for battery conservation, be aware that TCP connectivity will be lost until the device is physically accessed or wakes for a scheduled measurement.
### Wake Device
```
POST /{unit_id}/wake
@@ -473,7 +483,7 @@ POST /{unit_id}/ftp/enable
```
Enables FTP server on the device.
**Note:** FTP and TCP are mutually exclusive. Enabling FTP will temporarily disable TCP control.
**Note:** ~~FTP and TCP are mutually exclusive. Enabling FTP will temporarily disable TCP control.~~ As of v0.2.0, FTP and TCP are working fine in tandem. Just dont spam them a bunch.
### Disable FTP
```

246
docs/ROSTER.md Normal file
View File

@@ -0,0 +1,246 @@
# SLMM Roster Management
The SLMM standalone application now includes a roster management interface for viewing and configuring all Sound Level Meter devices.
## Features
### Web Interface
Access the roster at: **http://localhost:8100/roster**
The roster page provides:
- **Device List Table**: View all configured SLMs with their connection details
- **Real-time Status**: See device connectivity status (Online/Offline/Stale)
- **Add Device**: Create new device configurations with a user-friendly modal form
- **Edit Device**: Modify existing device configurations
- **Delete Device**: Remove device configurations (does not affect physical devices)
- **Test Connection**: Run diagnostics on individual devices
### Table Columns
| Column | Description |
|--------|-------------|
| Unit ID | Unique identifier for the device |
| Host / IP | Device IP address or hostname |
| TCP Port | TCP control port (default: 2255) |
| FTP Port | FTP file transfer port (default: 21) |
| TCP | Whether TCP control is enabled |
| FTP | Whether FTP file transfer is enabled |
| Polling | Whether background polling is enabled |
| Status | Device connectivity status (Online/Offline/Stale) |
| Actions | Test, Edit, Delete buttons |
### Status Indicators
- **Online** (green): Device responded within the last 5 minutes
- **Stale** (yellow): Device hasn't responded recently but was seen before
- **Offline** (red): Device is unreachable or has consecutive failures
- **Unknown** (gray): No status data available yet
## API Endpoints
### List All Devices
```bash
GET /api/nl43/roster
```
Returns all configured devices with their status information.
**Response:**
```json
{
"status": "ok",
"devices": [
{
"unit_id": "SLM-43-01",
"host": "192.168.1.100",
"tcp_port": 2255,
"ftp_port": 21,
"tcp_enabled": true,
"ftp_enabled": true,
"ftp_username": "USER",
"ftp_password": "0000",
"web_enabled": false,
"poll_enabled": true,
"poll_interval_seconds": 60,
"status": {
"last_seen": "2026-01-16T20:00:00",
"measurement_state": "Start",
"is_reachable": true,
"consecutive_failures": 0,
"last_success": "2026-01-16T20:00:00",
"last_error": null
}
}
],
"total": 1
}
```
### Create New Device
```bash
POST /api/nl43/roster
Content-Type: application/json
{
"unit_id": "SLM-43-01",
"host": "192.168.1.100",
"tcp_port": 2255,
"ftp_port": 21,
"tcp_enabled": true,
"ftp_enabled": false,
"poll_enabled": true,
"poll_interval_seconds": 60
}
```
**Required Fields:**
- `unit_id`: Unique device identifier
- `host`: IP address or hostname
**Optional Fields:**
- `tcp_port`: TCP control port (default: 2255)
- `ftp_port`: FTP port (default: 21)
- `tcp_enabled`: Enable TCP control (default: true)
- `ftp_enabled`: Enable FTP transfers (default: false)
- `ftp_username`: FTP username (only if ftp_enabled)
- `ftp_password`: FTP password (only if ftp_enabled)
- `poll_enabled`: Enable background polling (default: true)
- `poll_interval_seconds`: Polling interval 10-3600 seconds (default: 60)
**Response:**
```json
{
"status": "ok",
"message": "Device SLM-43-01 created successfully",
"data": {
"unit_id": "SLM-43-01",
"host": "192.168.1.100",
"tcp_port": 2255,
"tcp_enabled": true,
"ftp_enabled": false,
"poll_enabled": true,
"poll_interval_seconds": 60
}
}
```
### Update Device
```bash
PUT /api/nl43/{unit_id}/config
Content-Type: application/json
{
"host": "192.168.1.101",
"tcp_port": 2255,
"poll_interval_seconds": 120
}
```
All fields are optional. Only include fields you want to update.
### Delete Device
```bash
DELETE /api/nl43/{unit_id}/config
```
Removes the device configuration and associated status data. Does not affect the physical device.
**Response:**
```json
{
"status": "ok",
"message": "Deleted device SLM-43-01"
}
```
## Usage Examples
### Via Web Interface
1. Navigate to http://localhost:8100/roster
2. Click "Add Device" to create a new configuration
3. Fill in the device details (unit ID, IP address, ports)
4. Configure TCP, FTP, and polling settings
5. Click "Save Device"
6. Use "Test" button to verify connectivity
7. Edit or delete devices as needed
### Via API (curl)
**Add a new device:**
```bash
curl -X POST http://localhost:8100/api/nl43/roster \
-H "Content-Type: application/json" \
-d '{
"unit_id": "slm-site-a",
"host": "192.168.1.100",
"tcp_port": 2255,
"tcp_enabled": true,
"ftp_enabled": true,
"ftp_username": "USER",
"ftp_password": "0000",
"poll_enabled": true,
"poll_interval_seconds": 60
}'
```
**Update device host:**
```bash
curl -X PUT http://localhost:8100/api/nl43/slm-site-a/config \
-H "Content-Type: application/json" \
-d '{"host": "192.168.1.101"}'
```
**Delete device:**
```bash
curl -X DELETE http://localhost:8100/api/nl43/slm-site-a/config
```
**List all devices:**
```bash
curl http://localhost:8100/api/nl43/roster | python3 -m json.tool
```
## Integration with Terra-View
When SLMM is used as a module within Terra-View:
1. Terra-View manages device configurations in its own database
2. Terra-View syncs configurations to SLMM via `PUT /api/nl43/{unit_id}/config`
3. Terra-View can query device status via `GET /api/nl43/{unit_id}/status`
4. SLMM's roster page can be used for standalone testing and diagnostics
## Background Polling
Devices with `poll_enabled: true` are automatically polled at their configured interval:
- Polls device status every `poll_interval_seconds` (10-3600 seconds)
- Updates `NL43Status` table with latest measurements
- Tracks device reachability and failure counts
- Provides real-time status updates in the roster
**Note**: Polling respects the NL43 protocol's 1-second rate limit between commands.
## Validation
The roster system validates:
- **Unit ID**: Must be unique across all devices
- **Host**: Valid IP address or hostname format
- **Ports**: Must be between 1-65535
- **Poll Interval**: Must be between 10-3600 seconds
- **Duplicate Check**: Returns 409 Conflict if unit_id already exists
## Notes
- Deleting a device from the roster does NOT affect the physical device
- Device configurations are stored in the SLMM database (`data/slmm.db`)
- Status information is updated by the background polling system
- The roster page auto-refreshes status indicators
- Test button runs full diagnostics (connectivity, TCP, FTP if enabled)

26
docs/features/README.md Normal file
View File

@@ -0,0 +1,26 @@
# SLMM Feature Documentation
This directory contains detailed documentation for specific SLMM features and enhancements.
## Feature Documents
### FEATURE_SUMMARY.md
Overview of all major features in SLMM.
### SETTINGS_ENDPOINT.md
Documentation of the device settings endpoint and verification system.
### TIMEZONE_CONFIGURATION.md
Timezone handling and configuration for SLMM timestamps.
### SLEEP_MODE_AUTO_DISABLE.md
Automatic sleep mode wake-up system for background polling.
### UI_UPDATE.md
UI/UX improvements and interface updates.
## Related Documentation
- [../README.md](../../README.md) - Main SLMM documentation
- [../CHANGELOG.md](../../CHANGELOG.md) - Version history
- [../API.md](../../API.md) - Complete API reference

View File

@@ -0,0 +1,154 @@
# Sleep Mode Auto-Disable Feature
## Problem Statement
NL-43/NL-53 sound level meters have a sleep/eco mode feature that conserves battery power. However, when these devices enter sleep mode, **they turn off TCP communications**, which completely breaks remote monitoring and control capabilities. This makes it impossible to:
- Query device status remotely
- Start/stop measurements
- Stream real-time data
- Download files via FTP
- Perform any remote management tasks
This is particularly problematic in deployed scenarios where physical access to devices is limited or impossible.
## Solution
SLMM now automatically disables sleep mode in two key scenarios:
### 1. Device Configuration
When a device configuration is created or updated with TCP enabled, SLMM automatically:
- Checks the current sleep mode status on the device
- Disables sleep mode if it's enabled
- Logs the operation for visibility
**Endpoint:** `PUT /api/nl43/{unit_id}/config`
### 2. Measurement Start
Before starting any measurement, SLMM:
- Proactively disables sleep mode
- Ensures TCP remains active throughout the measurement session
- Allows remote monitoring to work reliably
**Endpoint:** `POST /api/nl43/{unit_id}/start`
## Implementation Details
### Helper Function
A new async helper function was added to [app/routers.py](app/routers.py:21-38):
```python
async def ensure_sleep_mode_disabled(client: NL43Client, unit_id: str):
"""
Helper function to ensure sleep mode is disabled on the device.
Sleep/eco mode turns off TCP communications, preventing remote monitoring.
This should be called when configuring a device or starting measurements.
"""
try:
current_status = await client.get_sleep_status()
logger.info(f"Current sleep mode status for {unit_id}: {current_status}")
# If sleep mode is on, disable it
if "On" in current_status or "on" in current_status:
logger.info(f"Sleep mode is enabled on {unit_id}, disabling it to maintain TCP connectivity")
await client.wake()
logger.info(f"Successfully disabled sleep mode on {unit_id}")
else:
logger.info(f"Sleep mode already disabled on {unit_id}")
except Exception as e:
logger.warning(f"Could not verify/disable sleep mode on {unit_id}: {e}")
# Don't raise - we want configuration to succeed even if sleep mode check fails
```
### Non-Blocking Design
The sleep mode check is **non-blocking**:
- If the device is unreachable, the operation logs a warning but continues
- Configuration updates succeed even if sleep mode can't be verified
- Measurement starts proceed even if sleep mode check fails
- This prevents device communication issues from blocking critical operations
### Logging
All sleep mode operations are logged with appropriate levels:
- **INFO**: Successful operations and status checks
- **WARNING**: Failed operations (device unreachable, timeout, etc.)
Example logs:
```
2026-01-14 18:37:12,889 - app.routers - INFO - TCP enabled for test-nl43-001, ensuring sleep mode is disabled
2026-01-14 18:37:12,889 - app.services - INFO - Sending command to 192.168.1.100:2255: Sleep Mode?
2026-01-14 18:37:17,890 - app.routers - WARNING - Could not verify/disable sleep mode on test-nl43-001: Failed to connect to device at 192.168.1.100:2255
```
## Testing
A comprehensive test script is available: [test_sleep_mode_auto_disable.py](test_sleep_mode_auto_disable.py)
Run it with:
```bash
python3 test_sleep_mode_auto_disable.py
```
The test verifies:
1. Config updates trigger sleep mode check
2. Config retrieval works correctly
3. Start measurement triggers sleep mode check
4. Operations succeed even without a physical device (non-blocking)
## API Documentation Updates
The following documentation files were updated to reflect this feature:
### [docs/API.md](docs/API.md)
- Updated config endpoint documentation with sleep mode auto-disable note
- Added warning to start measurement endpoint
- Enhanced power management section with detailed warnings about sleep mode behavior
Key additions:
- Configuration section now explains that sleep mode is automatically disabled when TCP is enabled
- Measurement control section notes that sleep mode is disabled before starting measurements
- Power management section includes comprehensive warnings about sleep mode affecting TCP connectivity
## Usage Notes
### For Operators
- You no longer need to manually disable sleep mode before starting remote monitoring
- Sleep mode will be automatically disabled when you configure a device or start measurements
- Check logs to verify sleep mode operations if experiencing connectivity issues
### For Developers
- The `ensure_sleep_mode_disabled()` helper can be called from any endpoint that requires reliable TCP connectivity
- Always use it before long-running operations that depend on continuous device communication
- The function is designed to fail gracefully - don't worry about exception handling
### Battery Conservation
If battery conservation is a concern:
- Consider using Timer Auto mode with scheduled measurements
- Sleep mode can be manually re-enabled between measurements using `POST /{unit_id}/sleep`
- Be aware that TCP connectivity will be lost until the device wakes or is physically accessed
## Deployment
The feature is automatically included when building the SLMM container:
```bash
cd /home/serversdown/tmi/terra-view
docker compose build slmm
docker compose up -d slmm
```
No configuration changes are required - the feature is active by default.
## Future Enhancements
Potential improvements for future versions:
- Add a user preference to optionally skip sleep mode disable
- Implement smart sleep mode scheduling (enable between measurements, disable during)
- Add sleep mode status to device health checks
- Create alerts when sleep mode is detected as enabled
## References
- NL-43 Command Reference: [docs/nl43_Command_ref.md](docs/nl43_Command_ref.md)
- Communication Guide: [docs/COMMUNICATION_GUIDE.md](docs/COMMUNICATION_GUIDE.md) (page 65, Sleep Mode)
- API Documentation: [docs/API.md](docs/API.md)
- SLMM Services: [app/services.py](app/services.py:395-417) (sleep mode commands)

View File

@@ -0,0 +1,164 @@
# Timezone Configuration for SLMM
## Overview
The SLMM system now supports configurable timezone settings. All timestamps are stored internally in UTC for consistency, but the system can interpret FTP timestamps and display times in your local timezone.
## Configuration
### Environment Variables
Set the following environment variables to configure your timezone:
#### `TIMEZONE_OFFSET` (required)
The number of hours offset from UTC. Use negative numbers for zones west of UTC, positive for east.
**Examples:**
- `-5` = EST (Eastern Standard Time, UTC-5)
- `-4` = EDT (Eastern Daylight Time, UTC-4)
- `0` = UTC (Coordinated Universal Time)
- `+1` = CET (Central European Time, UTC+1)
- `-8` = PST (Pacific Standard Time, UTC-8)
**Default:** `-5` (EST)
#### `TIMEZONE_NAME` (optional)
A friendly name for your timezone, used in log messages.
**Examples:**
- `EST`
- `EDT`
- `UTC`
- `PST`
**Default:** Auto-generated from offset (e.g., "UTC-5")
### Setup Instructions
#### Option 1: Using .env file (Recommended)
1. Copy the example file:
```bash
cd /home/serversdown/slmm
cp .env.example .env
```
2. Edit `.env` and set your timezone:
```bash
TIMEZONE_OFFSET=-5
TIMEZONE_NAME=EST
```
3. Make sure your application loads the .env file (you may need to install `python-dotenv`):
```bash
pip install python-dotenv
```
4. Update `app/main.py` to load the .env file (add at the top):
```python
from dotenv import load_dotenv
load_dotenv()
```
#### Option 2: System Environment Variables
Set the environment variables in your shell or service configuration:
```bash
export TIMEZONE_OFFSET=-5
export TIMEZONE_NAME=EST
```
Or add to your systemd service file if running as a service.
#### Option 3: Docker/Docker Compose
If using Docker, add to your `docker-compose.yml`:
```yaml
services:
slmm:
environment:
- TIMEZONE_OFFSET=-5
- TIMEZONE_NAME=EST
```
Or pass via command line:
```bash
docker run -e TIMEZONE_OFFSET=-5 -e TIMEZONE_NAME=EST ...
```
## How It Works
### Data Flow
1. **FTP Timestamps**: When the system reads file timestamps via FTP from the NL43 device, they are assumed to be in your configured timezone
2. **Conversion**: Timestamps are immediately converted to UTC for internal storage
3. **Database**: All timestamps in the database are stored in UTC
4. **API Responses**: Timestamps are sent to the frontend as UTC ISO strings
5. **Frontend Display**: The browser automatically converts UTC timestamps to the user's local timezone for display
### Example
If you're in EST (UTC-5) and the FTP shows a file timestamp of "Jan 11 21:57":
1. System interprets: `Jan 11 21:57 EST`
2. Converts to UTC: `Jan 12 02:57 UTC` (adds 5 hours)
3. Stores in database: `2026-01-12T02:57:00`
4. Sends to frontend: `2026-01-12T02:57:00` (with 'Z' added = UTC)
5. Browser displays: `Jan 11, 9:57 PM EST` (converts back to user's local time)
### Timer Calculation
The measurement timer calculates elapsed time correctly because:
- `measurement_start_time` is stored in UTC
- FTP folder timestamps are converted to UTC
- Frontend calculates `Date.now() - startTime` using UTC milliseconds
- All timezone offsets cancel out, giving accurate elapsed time
## Troubleshooting
### Timer shows wrong elapsed time
1. **Check your timezone setting**: Make sure `TIMEZONE_OFFSET` matches your actual timezone
```bash
# Check current setting in logs when SLMM starts:
grep "Using timezone" data/slmm.log
```
2. **Verify FTP timestamps**: FTP timestamps from the device should be in your local timezone
- If the device is configured for a different timezone, adjust `TIMEZONE_OFFSET` accordingly
3. **Restart the service**: Changes to environment variables require restarting the SLMM service
### Logs show unexpected timezone
Check the startup logs:
```bash
tail -f data/slmm.log | grep timezone
```
You should see:
```
Using timezone: EST (UTC-5)
```
If not, the environment variable may not be loaded correctly.
## Daylight Saving Time (DST)
**Important:** This configuration uses a fixed offset. If you need to account for Daylight Saving Time:
- **During DST (summer)**: Set `TIMEZONE_OFFSET=-4` (EDT)
- **During standard time (winter)**: Set `TIMEZONE_OFFSET=-5` (EST)
- You'll need to manually update the setting when DST changes (typically March and November)
**Future Enhancement:** Automatic DST handling could be implemented using Python's `zoneinfo` module with named timezones (e.g., "America/New_York").
## Default Behavior
If no environment variables are set:
- **TIMEZONE_OFFSET**: Defaults to `-5` (EST)
- **TIMEZONE_NAME**: Defaults to `UTC-5`
This means the system will work correctly for EST deployments out of the box.

View File

@@ -0,0 +1,73 @@
#!/usr/bin/env python3
"""
Database migration: Add device_logs table.
This table stores per-device log entries for debugging and audit trail.
Run this once to add the new table.
"""
import sqlite3
import os
# Path to the SLMM database
DB_PATH = os.path.join(os.path.dirname(__file__), "data", "slmm.db")
def migrate():
print(f"Adding device_logs table to: {DB_PATH}")
if not os.path.exists(DB_PATH):
print("Database does not exist yet. Table will be created automatically on first run.")
return
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
try:
# Check if table already exists
cursor.execute("""
SELECT name FROM sqlite_master
WHERE type='table' AND name='device_logs'
""")
if cursor.fetchone():
print("✓ device_logs table already exists, no migration needed")
return
# Create the table
print("Creating device_logs table...")
cursor.execute("""
CREATE TABLE device_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
unit_id VARCHAR NOT NULL,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
level VARCHAR DEFAULT 'INFO',
category VARCHAR DEFAULT 'GENERAL',
message TEXT NOT NULL
)
""")
# Create indexes for efficient querying
print("Creating indexes...")
cursor.execute("CREATE INDEX ix_device_logs_unit_id ON device_logs (unit_id)")
cursor.execute("CREATE INDEX ix_device_logs_timestamp ON device_logs (timestamp)")
conn.commit()
print("✓ Created device_logs table with indexes")
# Verify
cursor.execute("""
SELECT name FROM sqlite_master
WHERE type='table' AND name='device_logs'
""")
if not cursor.fetchone():
raise Exception("device_logs table was not created successfully")
print("✓ Migration completed successfully")
finally:
conn.close()
if __name__ == "__main__":
migrate()

View File

@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""
Migration script to add polling-related fields to nl43_config and nl43_status tables.
Adds to nl43_config:
- poll_interval_seconds (INTEGER, default 60)
- poll_enabled (BOOLEAN, default 1/True)
Adds to nl43_status:
- is_reachable (BOOLEAN, default 1/True)
- consecutive_failures (INTEGER, default 0)
- last_poll_attempt (DATETIME, nullable)
- last_success (DATETIME, nullable)
- last_error (TEXT, nullable)
Usage:
python migrate_add_polling_fields.py
"""
import sqlite3
import sys
from pathlib import Path
def migrate():
db_path = Path("data/slmm.db")
if not db_path.exists():
print(f"❌ Database not found at {db_path}")
print(" Run this script from the slmm directory")
return False
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Check nl43_config columns
cursor.execute("PRAGMA table_info(nl43_config)")
config_columns = [row[1] for row in cursor.fetchall()]
# Check nl43_status columns
cursor.execute("PRAGMA table_info(nl43_status)")
status_columns = [row[1] for row in cursor.fetchall()]
changes_made = False
# Add nl43_config columns
if "poll_interval_seconds" not in config_columns:
print("Adding poll_interval_seconds to nl43_config...")
cursor.execute("""
ALTER TABLE nl43_config
ADD COLUMN poll_interval_seconds INTEGER DEFAULT 60
""")
changes_made = True
else:
print("✓ poll_interval_seconds already exists in nl43_config")
if "poll_enabled" not in config_columns:
print("Adding poll_enabled to nl43_config...")
cursor.execute("""
ALTER TABLE nl43_config
ADD COLUMN poll_enabled BOOLEAN DEFAULT 1
""")
changes_made = True
else:
print("✓ poll_enabled already exists in nl43_config")
# Add nl43_status columns
if "is_reachable" not in status_columns:
print("Adding is_reachable to nl43_status...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN is_reachable BOOLEAN DEFAULT 1
""")
changes_made = True
else:
print("✓ is_reachable already exists in nl43_status")
if "consecutive_failures" not in status_columns:
print("Adding consecutive_failures to nl43_status...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN consecutive_failures INTEGER DEFAULT 0
""")
changes_made = True
else:
print("✓ consecutive_failures already exists in nl43_status")
if "last_poll_attempt" not in status_columns:
print("Adding last_poll_attempt to nl43_status...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN last_poll_attempt DATETIME
""")
changes_made = True
else:
print("✓ last_poll_attempt already exists in nl43_status")
if "last_success" not in status_columns:
print("Adding last_success to nl43_status...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN last_success DATETIME
""")
changes_made = True
else:
print("✓ last_success already exists in nl43_status")
if "last_error" not in status_columns:
print("Adding last_error to nl43_status...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN last_error TEXT
""")
changes_made = True
else:
print("✓ last_error already exists in nl43_status")
if changes_made:
conn.commit()
print("\n✓ Migration completed successfully")
print(" Added polling-related fields to nl43_config and nl43_status")
else:
print("\n✓ All polling fields already exist - no changes needed")
conn.close()
return True
except Exception as e:
print(f"❌ Migration failed: {e}")
return False
if __name__ == "__main__":
success = migrate()
sys.exit(0 if success else 1)

View File

@@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""
Database migration: Add start_time_sync_attempted field to nl43_status table.
This field tracks whether FTP sync has been attempted for the current measurement,
preventing repeated sync attempts when FTP fails.
Run this once to add the new column.
"""
import sqlite3
import os
# Path to the SLMM database
DB_PATH = os.path.join(os.path.dirname(__file__), "data", "slmm.db")
def migrate():
print(f"Adding start_time_sync_attempted field to: {DB_PATH}")
if not os.path.exists(DB_PATH):
print("Database does not exist yet. Column will be created automatically.")
return
conn = sqlite3.connect(DB_PATH)
cursor = conn.cursor()
try:
# Check if column already exists
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [col[1] for col in cursor.fetchall()]
if 'start_time_sync_attempted' in columns:
print("✓ start_time_sync_attempted column already exists, no migration needed")
return
# Add the column
print("Adding start_time_sync_attempted column...")
cursor.execute("""
ALTER TABLE nl43_status
ADD COLUMN start_time_sync_attempted BOOLEAN DEFAULT 0
""")
conn.commit()
print("✓ Added start_time_sync_attempted column")
# Verify
cursor.execute("PRAGMA table_info(nl43_status)")
columns = [col[1] for col in cursor.fetchall()]
if 'start_time_sync_attempted' not in columns:
raise Exception("start_time_sync_attempted column was not added successfully")
print("✓ Migration completed successfully")
finally:
conn.close()
if __name__ == "__main__":
migrate()

View File

@@ -3,3 +3,5 @@ uvicorn
sqlalchemy
pydantic
aioftp
jinja2
websockets

View File

@@ -31,6 +31,11 @@
<body>
<h1>SLMM NL43 Standalone</h1>
<p>Configure a unit (host/port), then use controls to Start/Stop and fetch live status.</p>
<p style="margin-bottom: 16px;">
<a href="/roster" style="color: #0969da; text-decoration: none; font-weight: 600;">📊 View Device Roster</a>
<span style="margin: 0 8px; color: #d0d7de;">|</span>
<a href="/docs" style="color: #0969da; text-decoration: none;">API Documentation</a>
</p>
<fieldset>
<legend>🔍 Connection Diagnostics</legend>
@@ -40,13 +45,34 @@
</fieldset>
<fieldset>
<legend>Unit Config</legend>
<label>Unit ID</label>
<input id="unitId" value="nl43-1" />
<label>Host</label>
<input id="host" value="127.0.0.1" />
<label>Port</label>
<input id="port" type="number" value="80" />
<legend>Unit Selection & Config</legend>
<div style="display: flex; gap: 8px; align-items: flex-end; margin-bottom: 12px;">
<div style="flex: 1;">
<label>Select Device</label>
<select id="deviceSelector" onchange="loadSelectedDevice()" style="width: 100%; padding: 8px; margin-bottom: 0;">
<option value="">-- Select a device --</option>
</select>
</div>
<button onclick="refreshDeviceList()" style="padding: 8px 12px;">↻ Refresh</button>
</div>
<div style="padding: 12px; background: #f6f8fa; border: 1px solid #d0d7de; border-radius: 4px; margin-bottom: 12px;">
<div style="display: flex; gap: 16px;">
<div style="flex: 1;">
<label>Unit ID</label>
<input id="unitId" value="nl43-1" />
</div>
<div style="flex: 2;">
<label>Host</label>
<input id="host" value="127.0.0.1" />
</div>
<div style="flex: 1;">
<label>TCP Port</label>
<input id="port" type="number" value="2255" />
</div>
</div>
</div>
<div style="margin: 12px 0;">
<label style="display: inline-flex; align-items: center; margin-right: 16px;">
@@ -66,8 +92,10 @@
<input id="ftpPassword" type="password" value="0000" />
</div>
<button onclick="saveConfig()" style="margin-top: 12px;">Save Config</button>
<button onclick="loadConfig()">Load Config</button>
<div style="margin-top: 12px;">
<button onclick="saveConfig()">Save Config</button>
<button onclick="loadConfig()">Load Config</button>
</div>
</fieldset>
<fieldset>
@@ -148,6 +176,7 @@
let ws = null;
let streamUpdateCount = 0;
let availableDevices = [];
function log(msg) {
logEl.textContent += msg + "\n";
@@ -160,9 +189,97 @@
ftpCredentials.style.display = ftpEnabled ? 'block' : 'none';
}
// Add event listener for FTP checkbox
// Load device list from roster
async function refreshDeviceList() {
try {
const res = await fetch('/api/nl43/roster');
const data = await res.json();
if (!res.ok) {
log('Failed to load device list');
return;
}
availableDevices = data.devices || [];
const selector = document.getElementById('deviceSelector');
// Save current selection
const currentSelection = selector.value;
// Clear and rebuild options
selector.innerHTML = '<option value="">-- Select a device --</option>';
availableDevices.forEach(device => {
const option = document.createElement('option');
option.value = device.unit_id;
// Add status indicator
let statusIcon = '⚪';
if (device.status) {
if (device.status.is_reachable === false) {
statusIcon = '🔴';
} else if (device.status.last_success) {
const lastSeen = new Date(device.status.last_success);
const ageMinutes = Math.floor((Date.now() - lastSeen) / 60000);
statusIcon = ageMinutes < 5 ? '🟢' : '🟡';
}
}
option.textContent = `${statusIcon} ${device.unit_id} (${device.host})`;
selector.appendChild(option);
});
// Restore selection if it still exists
if (currentSelection && availableDevices.find(d => d.unit_id === currentSelection)) {
selector.value = currentSelection;
}
log(`Loaded ${availableDevices.length} device(s) from roster`);
} catch (err) {
log(`Error loading device list: ${err.message}`);
}
}
// Load selected device configuration
function loadSelectedDevice() {
const selector = document.getElementById('deviceSelector');
const unitId = selector.value;
if (!unitId) {
return;
}
const device = availableDevices.find(d => d.unit_id === unitId);
if (!device) {
log(`Device ${unitId} not found in list`);
return;
}
// Populate form fields
document.getElementById('unitId').value = device.unit_id;
document.getElementById('host').value = device.host;
document.getElementById('port').value = device.tcp_port || 2255;
document.getElementById('tcpEnabled').checked = device.tcp_enabled || false;
document.getElementById('ftpEnabled').checked = device.ftp_enabled || false;
if (device.ftp_username) {
document.getElementById('ftpUsername').value = device.ftp_username;
}
if (device.ftp_password) {
document.getElementById('ftpPassword').value = device.ftp_password;
}
toggleFtpCredentials();
log(`Loaded configuration for ${device.unit_id}`);
}
// Add event listeners
document.addEventListener('DOMContentLoaded', function() {
document.getElementById('ftpEnabled').addEventListener('change', toggleFtpCredentials);
// Load device list on page load
refreshDeviceList();
});
async function runDiagnostics() {
@@ -216,6 +333,134 @@
html += `<p style="margin-top: 12px; font-size: 0.9em; color: #666;">Last run: ${new Date(data.timestamp).toLocaleString()}</p>`;
// Add database dump section if available
if (data.database_dump) {
html += `<div style="margin-top: 16px; border-top: 1px solid #d0d7de; padding-top: 12px;">`;
html += `<h4 style="margin: 0 0 12px 0;">📦 Database Dump</h4>`;
// Config section
if (data.database_dump.config) {
const cfg = data.database_dump.config;
html += `<div style="background: #f0f4f8; padding: 12px; border-radius: 4px; margin-bottom: 12px;">`;
html += `<strong>Configuration (nl43_config)</strong>`;
html += `<table style="width: 100%; margin-top: 8px; font-size: 0.9em;">`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Host</td><td>${cfg.host}:${cfg.tcp_port}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">TCP Enabled</td><td>${cfg.tcp_enabled ? '✓' : '✗'}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">FTP Enabled</td><td>${cfg.ftp_enabled ? '✓' : '✗'}${cfg.ftp_enabled ? ` (port ${cfg.ftp_port}, user: ${cfg.ftp_username || 'none'})` : ''}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Background Polling</td><td>${cfg.poll_enabled ? `✓ every ${cfg.poll_interval_seconds}s` : '✗ disabled'}</td></tr>`;
html += `</table></div>`;
}
// Status cache section
if (data.database_dump.status_cache) {
const cache = data.database_dump.status_cache;
html += `<div style="background: #f0f8f4; padding: 12px; border-radius: 4px; margin-bottom: 12px;">`;
html += `<strong>Status Cache (nl43_status)</strong>`;
html += `<table style="width: 100%; margin-top: 8px; font-size: 0.9em;">`;
// Measurement state and timing
html += `<tr><td style="padding: 2px 8px; color: #666;">Measurement State</td><td><strong>${cache.measurement_state || 'unknown'}</strong></td></tr>`;
if (cache.measurement_start_time) {
const startTime = new Date(cache.measurement_start_time);
const elapsed = Math.floor((Date.now() - startTime) / 1000);
const elapsedStr = elapsed > 3600 ? `${Math.floor(elapsed/3600)}h ${Math.floor((elapsed%3600)/60)}m` : elapsed > 60 ? `${Math.floor(elapsed/60)}m ${elapsed%60}s` : `${elapsed}s`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Measurement Started</td><td>${startTime.toLocaleString()} (${elapsedStr} ago)</td></tr>`;
}
html += `<tr><td style="padding: 2px 8px; color: #666;">Counter (d0)</td><td>${cache.counter || 'N/A'}</td></tr>`;
// Sound levels
html += `<tr><td colspan="2" style="padding: 8px 8px 2px 8px; font-weight: 600; border-top: 1px solid #d0d7de;">Sound Levels (dB)</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Lp (Instantaneous)</td><td>${cache.lp || 'N/A'}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Leq (Equivalent)</td><td>${cache.leq || 'N/A'}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Lmax / Lmin</td><td>${cache.lmax || 'N/A'} / ${cache.lmin || 'N/A'}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Lpeak</td><td>${cache.lpeak || 'N/A'}</td></tr>`;
// Device status
html += `<tr><td colspan="2" style="padding: 8px 8px 2px 8px; font-weight: 600; border-top: 1px solid #d0d7de;">Device Status</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Battery</td><td>${cache.battery_level || 'N/A'}${cache.power_source ? ` (${cache.power_source})` : ''}</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">SD Card</td><td>${cache.sd_remaining_mb ? `${cache.sd_remaining_mb} MB` : 'N/A'}${cache.sd_free_ratio ? ` (${cache.sd_free_ratio} free)` : ''}</td></tr>`;
// Polling status
html += `<tr><td colspan="2" style="padding: 8px 8px 2px 8px; font-weight: 600; border-top: 1px solid #d0d7de;">Polling Status</td></tr>`;
html += `<tr><td style="padding: 2px 8px; color: #666;">Reachable</td><td>${cache.is_reachable ? '🟢 Yes' : '🔴 No'}</td></tr>`;
if (cache.last_seen) {
html += `<tr><td style="padding: 2px 8px; color: #666;">Last Seen</td><td>${new Date(cache.last_seen).toLocaleString()}</td></tr>`;
}
if (cache.last_success) {
html += `<tr><td style="padding: 2px 8px; color: #666;">Last Success</td><td>${new Date(cache.last_success).toLocaleString()}</td></tr>`;
}
if (cache.last_poll_attempt) {
html += `<tr><td style="padding: 2px 8px; color: #666;">Last Poll Attempt</td><td>${new Date(cache.last_poll_attempt).toLocaleString()}</td></tr>`;
}
html += `<tr><td style="padding: 2px 8px; color: #666;">Consecutive Failures</td><td>${cache.consecutive_failures || 0}</td></tr>`;
if (cache.last_error) {
html += `<tr><td style="padding: 2px 8px; color: #666;">Last Error</td><td style="color: #d00; font-size: 0.85em;">${cache.last_error}</td></tr>`;
}
html += `</table></div>`;
// Raw payload (collapsible)
if (cache.raw_payload) {
html += `<details style="margin-top: 8px;"><summary style="cursor: pointer; color: #666; font-size: 0.9em;">📄 Raw Payload</summary>`;
html += `<pre style="background: #f6f8fa; padding: 8px; border-radius: 4px; font-size: 0.8em; overflow-x: auto; margin-top: 8px;">${cache.raw_payload}</pre></details>`;
}
} else {
html += `<p style="color: #888; font-style: italic;">No cached status available for this unit.</p>`;
}
html += `</div>`;
}
// Fetch and display device logs
try {
const logsRes = await fetch(`/api/nl43/${unitId}/logs?limit=50`);
if (logsRes.ok) {
const logsData = await logsRes.json();
if (logsData.logs && logsData.logs.length > 0) {
html += `<div style="margin-top: 16px; border-top: 1px solid #d0d7de; padding-top: 12px;">`;
html += `<h4 style="margin: 0 0 12px 0;">📋 Device Logs (${logsData.stats.total} total)</h4>`;
// Stats summary
if (logsData.stats.by_level) {
html += `<div style="margin-bottom: 8px; font-size: 0.85em; color: #666;">`;
const levels = logsData.stats.by_level;
const parts = [];
if (levels.ERROR) parts.push(`<span style="color: #d00;">${levels.ERROR} errors</span>`);
if (levels.WARNING) parts.push(`<span style="color: #fa0;">${levels.WARNING} warnings</span>`);
if (levels.INFO) parts.push(`${levels.INFO} info`);
html += parts.join(' · ');
html += `</div>`;
}
// Log entries (collapsible)
html += `<details open><summary style="cursor: pointer; font-size: 0.9em; margin-bottom: 8px;">Recent entries (${logsData.logs.length})</summary>`;
html += `<div style="max-height: 300px; overflow-y: auto; background: #f6f8fa; border: 1px solid #d0d7de; border-radius: 4px; padding: 8px; font-size: 0.8em; font-family: monospace;">`;
logsData.logs.forEach(entry => {
const levelColor = {
'ERROR': '#d00',
'WARNING': '#b86e00',
'INFO': '#0969da',
'DEBUG': '#888'
}[entry.level] || '#666';
const time = new Date(entry.timestamp).toLocaleString();
html += `<div style="margin-bottom: 4px; border-bottom: 1px solid #eee; padding-bottom: 4px;">`;
html += `<span style="color: #888;">${time}</span> `;
html += `<span style="color: ${levelColor}; font-weight: 600;">[${entry.level}]</span> `;
html += `<span style="color: #666;">[${entry.category}]</span> `;
html += `${entry.message}`;
html += `</div>`;
});
html += `</div></details>`;
html += `</div>`;
}
}
} catch (logErr) {
console.log('Could not fetch device logs:', logErr);
}
resultsEl.innerHTML = html;
log(`Diagnostics complete: ${data.overall_status}`);

624
templates/roster.html Normal file
View File

@@ -0,0 +1,624 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>SLMM Roster - Sound Level Meter Configuration</title>
<style>
* { box-sizing: border-box; }
body {
font-family: system-ui, -apple-system, sans-serif;
margin: 0;
padding: 24px;
background: #f6f8fa;
}
.container { max-width: 1400px; margin: 0 auto; }
.header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 24px;
padding: 16px;
background: white;
border-radius: 6px;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
h1 { margin: 0; font-size: 24px; }
.nav { display: flex; gap: 12px; }
.btn {
padding: 8px 16px;
border: 1px solid #d0d7de;
background: white;
border-radius: 6px;
cursor: pointer;
text-decoration: none;
color: #24292f;
font-size: 14px;
transition: background 0.2s;
}
.btn:hover { background: #f6f8fa; }
.btn-primary {
background: #2da44e;
color: white;
border-color: #2da44e;
}
.btn-primary:hover { background: #2c974b; }
.btn-danger {
background: #cf222e;
color: white;
border-color: #cf222e;
}
.btn-danger:hover { background: #a40e26; }
.btn-small {
padding: 4px 8px;
font-size: 12px;
margin-right: 4px;
}
.table-container {
background: white;
border-radius: 6px;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
overflow-x: auto;
}
table {
width: 100%;
border-collapse: collapse;
}
th {
background: #f6f8fa;
padding: 12px;
text-align: left;
font-weight: 600;
border-bottom: 2px solid #d0d7de;
font-size: 13px;
white-space: nowrap;
}
td {
padding: 12px;
border-bottom: 1px solid #d0d7de;
font-size: 13px;
}
tr:hover { background: #f6f8fa; }
.status-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 12px;
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
}
.status-ok {
background: #dafbe1;
color: #1a7f37;
}
.status-unknown {
background: #eaeef2;
color: #57606a;
}
.status-error {
background: #ffebe9;
color: #cf222e;
}
.checkbox-cell {
text-align: center;
width: 80px;
}
.checkbox-cell input[type="checkbox"] {
cursor: pointer;
width: 16px;
height: 16px;
}
.actions-cell {
white-space: nowrap;
width: 200px;
}
.empty-state {
text-align: center;
padding: 48px;
color: #57606a;
}
.empty-state-icon {
font-size: 48px;
margin-bottom: 16px;
}
.modal {
display: none;
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0,0,0,0.5);
z-index: 1000;
align-items: center;
justify-content: center;
}
.modal.active { display: flex; }
.modal-content {
background: white;
padding: 24px;
border-radius: 6px;
max-width: 600px;
width: 90%;
max-height: 80vh;
overflow-y: auto;
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
}
.modal-header h2 {
margin: 0;
font-size: 20px;
}
.close-btn {
background: none;
border: none;
font-size: 24px;
cursor: pointer;
color: #57606a;
padding: 0;
width: 32px;
height: 32px;
}
.close-btn:hover { color: #24292f; }
.form-group {
margin-bottom: 16px;
}
.form-group label {
display: block;
margin-bottom: 6px;
font-weight: 600;
font-size: 14px;
}
.form-group input[type="text"],
.form-group input[type="number"],
.form-group input[type="password"] {
width: 100%;
padding: 8px 12px;
border: 1px solid #d0d7de;
border-radius: 6px;
font-size: 14px;
}
.form-group input[type="checkbox"] {
width: auto;
margin-right: 8px;
}
.checkbox-label {
display: flex;
align-items: center;
font-weight: normal;
cursor: pointer;
}
.form-actions {
display: flex;
justify-content: flex-end;
gap: 8px;
margin-top: 24px;
}
.toast {
position: fixed;
top: 24px;
right: 24px;
padding: 12px 16px;
background: #24292f;
color: white;
border-radius: 6px;
box-shadow: 0 4px 12px rgba(0,0,0,0.15);
z-index: 2000;
display: none;
min-width: 300px;
}
.toast.active {
display: block;
animation: slideIn 0.3s ease-out;
}
@keyframes slideIn {
from {
transform: translateX(400px);
opacity: 0;
}
to {
transform: translateX(0);
opacity: 1;
}
}
.toast-success { background: #2da44e; }
.toast-error { background: #cf222e; }
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>📊 Sound Level Meter Roster</h1>
<div class="nav">
<a href="/" class="btn">← Back to Control Panel</a>
<button class="btn btn-primary" onclick="openAddModal()">+ Add Device</button>
</div>
</div>
<div class="table-container">
<table id="rosterTable">
<thead>
<tr>
<th>Unit ID</th>
<th>Host / IP</th>
<th>TCP Port</th>
<th>FTP Port</th>
<th class="checkbox-cell">TCP</th>
<th class="checkbox-cell">FTP</th>
<th class="checkbox-cell">Polling</th>
<th>Status</th>
<th class="actions-cell">Actions</th>
</tr>
</thead>
<tbody id="rosterBody">
<tr>
<td colspan="9" style="text-align: center; padding: 24px;">
Loading...
</td>
</tr>
</tbody>
</table>
</div>
</div>
<!-- Add/Edit Modal -->
<div id="deviceModal" class="modal">
<div class="modal-content">
<div class="modal-header">
<h2 id="modalTitle">Add Device</h2>
<button class="close-btn" onclick="closeModal()">&times;</button>
</div>
<form id="deviceForm" onsubmit="saveDevice(event)">
<div class="form-group">
<label for="unitId">Unit ID *</label>
<input type="text" id="unitId" required placeholder="e.g., nl43-1, slm-site-a" />
</div>
<div class="form-group">
<label for="host">Host / IP Address *</label>
<input type="text" id="host" required placeholder="e.g., 192.168.1.100" />
</div>
<div class="form-group">
<label for="tcpPort">TCP Port *</label>
<input type="number" id="tcpPort" required value="2255" min="1" max="65535" />
</div>
<div class="form-group">
<label for="ftpPort">FTP Port</label>
<input type="number" id="ftpPort" value="21" min="1" max="65535" />
</div>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="tcpEnabled" checked />
TCP Enabled (required for remote control)
</label>
</div>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="ftpEnabled" onchange="toggleFtpCredentials()" />
FTP Enabled (for file downloads)
</label>
</div>
<div id="ftpCredentialsSection" style="display: none; padding: 12px; background: #f6f8fa; border-radius: 6px; margin-bottom: 16px;">
<div class="form-group">
<label for="ftpUsername">FTP Username</label>
<input type="text" id="ftpUsername" placeholder="Default: USER" />
</div>
<div class="form-group">
<label for="ftpPassword">FTP Password</label>
<input type="password" id="ftpPassword" placeholder="Default: 0000" />
</div>
</div>
<div class="form-group">
<label class="checkbox-label">
<input type="checkbox" id="pollEnabled" checked />
Enable background polling (status updates)
</label>
</div>
<div class="form-group">
<label for="pollInterval">Polling Interval (seconds)</label>
<input type="number" id="pollInterval" value="60" min="10" max="3600" />
</div>
<div class="form-actions">
<button type="button" class="btn" onclick="closeModal()">Cancel</button>
<button type="submit" class="btn btn-primary">Save Device</button>
</div>
</form>
</div>
</div>
<!-- Toast Notification -->
<div id="toast" class="toast"></div>
<script>
let devices = [];
let editingDeviceId = null;
// Load roster on page load
document.addEventListener('DOMContentLoaded', () => {
loadRoster();
});
async function loadRoster() {
try {
const res = await fetch('/api/nl43/roster');
const data = await res.json();
if (!res.ok) {
showToast('Failed to load roster', 'error');
return;
}
devices = data.devices || [];
renderRoster();
} catch (err) {
showToast('Error loading roster: ' + err.message, 'error');
console.error('Load roster error:', err);
}
}
function renderRoster() {
const tbody = document.getElementById('rosterBody');
if (devices.length === 0) {
tbody.innerHTML = `
<tr>
<td colspan="9" class="empty-state">
<div class="empty-state-icon">📭</div>
<div><strong>No devices configured</strong></div>
<div style="margin-top: 8px; font-size: 14px;">Click "Add Device" to configure your first sound level meter</div>
</td>
</tr>
`;
return;
}
tbody.innerHTML = devices.map(device => `
<tr>
<td><strong>${escapeHtml(device.unit_id)}</strong></td>
<td>${escapeHtml(device.host)}</td>
<td>${device.tcp_port}</td>
<td>${device.ftp_port || 21}</td>
<td class="checkbox-cell">
<input type="checkbox" ${device.tcp_enabled ? 'checked' : ''} disabled />
</td>
<td class="checkbox-cell">
<input type="checkbox" ${device.ftp_enabled ? 'checked' : ''} disabled />
</td>
<td class="checkbox-cell">
<input type="checkbox" ${device.poll_enabled ? 'checked' : ''} disabled />
</td>
<td>
${getStatusBadge(device)}
</td>
<td class="actions-cell">
<button class="btn btn-small" onclick="testDevice('${escapeHtml(device.unit_id)}')">Test</button>
<button class="btn btn-small" onclick="openEditModal('${escapeHtml(device.unit_id)}')">Edit</button>
<button class="btn btn-small btn-danger" onclick="deleteDevice('${escapeHtml(device.unit_id)}')">Delete</button>
</td>
</tr>
`).join('');
}
function getStatusBadge(device) {
if (!device.status) {
return '<span class="status-badge status-unknown">Unknown</span>';
}
if (device.status.is_reachable === false) {
return '<span class="status-badge status-error">Offline</span>';
}
if (device.status.last_success) {
const lastSeen = new Date(device.status.last_success);
const ago = Math.floor((Date.now() - lastSeen) / 1000);
if (ago < 300) { // Less than 5 minutes
return '<span class="status-badge status-ok">Online</span>';
} else {
return `<span class="status-badge status-unknown">Stale (${Math.floor(ago / 60)}m ago)</span>`;
}
}
return '<span class="status-badge status-unknown">Unknown</span>';
}
function escapeHtml(text) {
const map = {
'&': '&amp;',
'<': '&lt;',
'>': '&gt;',
'"': '&quot;',
"'": '&#039;'
};
return String(text).replace(/[&<>"']/g, m => map[m]);
}
function openAddModal() {
editingDeviceId = null;
document.getElementById('modalTitle').textContent = 'Add Device';
document.getElementById('deviceForm').reset();
document.getElementById('unitId').disabled = false;
document.getElementById('tcpEnabled').checked = true;
document.getElementById('ftpEnabled').checked = false;
document.getElementById('pollEnabled').checked = true;
document.getElementById('tcpPort').value = 2255;
document.getElementById('ftpPort').value = 21;
document.getElementById('pollInterval').value = 60;
toggleFtpCredentials();
document.getElementById('deviceModal').classList.add('active');
}
function openEditModal(unitId) {
const device = devices.find(d => d.unit_id === unitId);
if (!device) {
showToast('Device not found', 'error');
return;
}
editingDeviceId = unitId;
document.getElementById('modalTitle').textContent = 'Edit Device';
document.getElementById('unitId').value = device.unit_id;
document.getElementById('unitId').disabled = true;
document.getElementById('host').value = device.host;
document.getElementById('tcpPort').value = device.tcp_port;
document.getElementById('ftpPort').value = device.ftp_port || 21;
document.getElementById('tcpEnabled').checked = device.tcp_enabled;
document.getElementById('ftpEnabled').checked = device.ftp_enabled;
document.getElementById('ftpUsername').value = device.ftp_username || '';
document.getElementById('ftpPassword').value = device.ftp_password || '';
document.getElementById('pollEnabled').checked = device.poll_enabled;
document.getElementById('pollInterval').value = device.poll_interval_seconds || 60;
toggleFtpCredentials();
document.getElementById('deviceModal').classList.add('active');
}
function closeModal() {
document.getElementById('deviceModal').classList.remove('active');
editingDeviceId = null;
}
function toggleFtpCredentials() {
const ftpEnabled = document.getElementById('ftpEnabled').checked;
document.getElementById('ftpCredentialsSection').style.display = ftpEnabled ? 'block' : 'none';
}
async function saveDevice(event) {
event.preventDefault();
const unitId = document.getElementById('unitId').value.trim();
const payload = {
host: document.getElementById('host').value.trim(),
tcp_port: parseInt(document.getElementById('tcpPort').value),
ftp_port: parseInt(document.getElementById('ftpPort').value),
tcp_enabled: document.getElementById('tcpEnabled').checked,
ftp_enabled: document.getElementById('ftpEnabled').checked,
poll_enabled: document.getElementById('pollEnabled').checked,
poll_interval_seconds: parseInt(document.getElementById('pollInterval').value)
};
if (payload.ftp_enabled) {
const username = document.getElementById('ftpUsername').value.trim();
const password = document.getElementById('ftpPassword').value.trim();
if (username) payload.ftp_username = username;
if (password) payload.ftp_password = password;
}
try {
const url = editingDeviceId
? `/api/nl43/${editingDeviceId}/config`
: `/api/nl43/roster`;
const method = editingDeviceId ? 'PUT' : 'POST';
const body = editingDeviceId
? payload
: { unit_id: unitId, ...payload };
const res = await fetch(url, {
method,
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body)
});
const data = await res.json();
if (!res.ok) {
showToast(data.detail || 'Failed to save device', 'error');
return;
}
showToast(editingDeviceId ? 'Device updated successfully' : 'Device added successfully', 'success');
closeModal();
await loadRoster();
} catch (err) {
showToast('Error saving device: ' + err.message, 'error');
console.error('Save device error:', err);
}
}
async function deleteDevice(unitId) {
if (!confirm(`Are you sure you want to delete "${unitId}"?\n\nThis will remove the device configuration but will not affect the physical device.`)) {
return;
}
try {
const res = await fetch(`/api/nl43/${unitId}/config`, {
method: 'DELETE'
});
const data = await res.json();
if (!res.ok) {
showToast(data.detail || 'Failed to delete device', 'error');
return;
}
showToast('Device deleted successfully', 'success');
await loadRoster();
} catch (err) {
showToast('Error deleting device: ' + err.message, 'error');
console.error('Delete device error:', err);
}
}
async function testDevice(unitId) {
showToast('Testing device connection...', 'success');
try {
const res = await fetch(`/api/nl43/${unitId}/diagnostics`);
const data = await res.json();
if (!res.ok) {
showToast('Device test failed', 'error');
return;
}
const statusText = {
'pass': 'All systems operational ✓',
'fail': 'Connection failed ✗',
'degraded': 'Partial connectivity ⚠'
};
showToast(statusText[data.overall_status] || 'Test complete',
data.overall_status === 'pass' ? 'success' : 'error');
// Reload to update status
await loadRoster();
} catch (err) {
showToast('Error testing device: ' + err.message, 'error');
console.error('Test device error:', err);
}
}
function showToast(message, type = 'success') {
const toast = document.getElementById('toast');
toast.textContent = message;
toast.className = `toast toast-${type} active`;
setTimeout(() => {
toast.classList.remove('active');
}, 3000);
}
// Close modal when clicking outside
document.getElementById('deviceModal').addEventListener('click', (e) => {
if (e.target.id === 'deviceModal') {
closeModal();
}
});
</script>
</body>
</html>

167
test_polling.sh Executable file
View File

@@ -0,0 +1,167 @@
#!/bin/bash
# Manual test script for background polling functionality
# Usage: ./test_polling.sh [UNIT_ID]
BASE_URL="http://localhost:8100/api/nl43"
UNIT_ID="${1:-NL43-001}"
echo "=========================================="
echo "Background Polling Test Script"
echo "=========================================="
echo "Testing device: $UNIT_ID"
echo "Base URL: $BASE_URL"
echo ""
# Color codes for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
# Function to print test header
test_header() {
echo ""
echo "=========================================="
echo "$1"
echo "=========================================="
}
# Function to print success
success() {
echo -e "${GREEN}${NC} $1"
}
# Function to print warning
warning() {
echo -e "${YELLOW}${NC} $1"
}
# Function to print error
error() {
echo -e "${RED}${NC} $1"
}
# Test 1: Get current polling configuration
test_header "Test 1: Get Current Polling Configuration"
RESPONSE=$(curl -s "$BASE_URL/$UNIT_ID/polling/config")
echo "$RESPONSE" | jq '.'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
success "Successfully retrieved polling configuration"
CURRENT_INTERVAL=$(echo "$RESPONSE" | jq -r '.data.poll_interval_seconds')
CURRENT_ENABLED=$(echo "$RESPONSE" | jq -r '.data.poll_enabled')
echo " Current interval: ${CURRENT_INTERVAL}s"
echo " Polling enabled: $CURRENT_ENABLED"
else
error "Failed to retrieve polling configuration"
exit 1
fi
# Test 2: Update polling interval to 30 seconds
test_header "Test 2: Update Polling Interval to 30 Seconds"
RESPONSE=$(curl -s -X PUT "$BASE_URL/$UNIT_ID/polling/config" \
-H "Content-Type: application/json" \
-d '{"poll_interval_seconds": 30}')
echo "$RESPONSE" | jq '.'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
success "Successfully updated polling interval to 30s"
else
error "Failed to update polling interval"
fi
# Test 3: Check global polling status
test_header "Test 3: Check Global Polling Status"
RESPONSE=$(curl -s "$BASE_URL/_polling/status")
echo "$RESPONSE" | jq '.'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
success "Successfully retrieved global polling status"
POLLER_RUNNING=$(echo "$RESPONSE" | jq -r '.data.poller_running')
TOTAL_DEVICES=$(echo "$RESPONSE" | jq -r '.data.total_devices')
echo " Poller running: $POLLER_RUNNING"
echo " Total devices: $TOTAL_DEVICES"
else
error "Failed to retrieve global polling status"
fi
# Test 4: Wait for automatic poll to occur
test_header "Test 4: Wait for Automatic Poll (35 seconds)"
warning "Waiting 35 seconds for automatic poll to occur..."
for i in {35..1}; do
echo -ne " ${i}s remaining...\r"
sleep 1
done
echo ""
success "Wait complete"
# Test 5: Check if status was updated by background poller
test_header "Test 5: Verify Background Poll Occurred"
RESPONSE=$(curl -s "$BASE_URL/$UNIT_ID/status")
echo "$RESPONSE" | jq '{last_poll_attempt, last_success, is_reachable, consecutive_failures}'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
LAST_POLL=$(echo "$RESPONSE" | jq -r '.data.last_poll_attempt')
IS_REACHABLE=$(echo "$RESPONSE" | jq -r '.data.is_reachable')
FAILURES=$(echo "$RESPONSE" | jq -r '.data.consecutive_failures')
if [ "$LAST_POLL" != "null" ]; then
success "Device was polled by background poller"
echo " Last poll: $LAST_POLL"
echo " Reachable: $IS_REACHABLE"
echo " Failures: $FAILURES"
else
warning "No automatic poll detected yet"
fi
else
error "Failed to retrieve device status"
fi
# Test 6: Disable polling
test_header "Test 6: Disable Background Polling"
RESPONSE=$(curl -s -X PUT "$BASE_URL/$UNIT_ID/polling/config" \
-H "Content-Type: application/json" \
-d '{"poll_enabled": false}')
echo "$RESPONSE" | jq '.'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
success "Successfully disabled background polling"
else
error "Failed to disable polling"
fi
# Test 7: Verify polling is disabled
test_header "Test 7: Verify Polling Disabled in Global Status"
RESPONSE=$(curl -s "$BASE_URL/_polling/status")
DEVICE_ENABLED=$(echo "$RESPONSE" | jq --arg uid "$UNIT_ID" '.data.devices[] | select(.unit_id == $uid) | .poll_enabled')
if [ "$DEVICE_ENABLED" == "false" ]; then
success "Polling correctly shows as disabled for $UNIT_ID"
else
warning "Device still appears in polling list or shows as enabled"
fi
# Test 8: Re-enable polling with original interval
test_header "Test 8: Re-enable Polling with Original Interval"
RESPONSE=$(curl -s -X PUT "$BASE_URL/$UNIT_ID/polling/config" \
-H "Content-Type: application/json" \
-d "{\"poll_enabled\": true, \"poll_interval_seconds\": $CURRENT_INTERVAL}")
echo "$RESPONSE" | jq '.'
if echo "$RESPONSE" | jq -e '.status == "ok"' > /dev/null; then
success "Successfully re-enabled polling with ${CURRENT_INTERVAL}s interval"
else
error "Failed to re-enable polling"
fi
# Summary
test_header "Test Summary"
echo "All tests completed!"
echo ""
echo "Key endpoints tested:"
echo " GET $BASE_URL/{unit_id}/polling/config"
echo " PUT $BASE_URL/{unit_id}/polling/config"
echo " GET $BASE_URL/_polling/status"
echo " GET $BASE_URL/{unit_id}/status (with polling fields)"
echo ""
success "Background polling feature is working correctly"

View File

@@ -0,0 +1,128 @@
#!/usr/bin/env python3
"""
Test script to verify that sleep mode is automatically disabled when:
1. Device configuration is created/updated with TCP enabled
2. Measurements are started
This script tests the API endpoints, not the actual device communication.
"""
import requests
import json
BASE_URL = "http://localhost:8100/api/nl43"
UNIT_ID = "test-nl43-001"
def test_config_update():
"""Test that config update works (actual sleep mode disable requires real device)"""
print("\n=== Testing Config Update ===")
# Create/update a device config
config_data = {
"host": "192.168.1.100",
"tcp_port": 2255,
"tcp_enabled": True,
"ftp_enabled": False,
"ftp_username": "admin",
"ftp_password": "password"
}
print(f"Updating config for {UNIT_ID}...")
response = requests.put(f"{BASE_URL}/{UNIT_ID}/config", json=config_data)
if response.status_code == 200:
print("✓ Config updated successfully")
print(f"Response: {json.dumps(response.json(), indent=2)}")
print("\nNote: Sleep mode disable was attempted (will succeed if device is reachable)")
return True
else:
print(f"✗ Config update failed: {response.status_code}")
print(f"Error: {response.text}")
return False
def test_get_config():
"""Test retrieving the config"""
print("\n=== Testing Get Config ===")
response = requests.get(f"{BASE_URL}/{UNIT_ID}/config")
if response.status_code == 200:
print("✓ Config retrieved successfully")
print(f"Response: {json.dumps(response.json(), indent=2)}")
return True
elif response.status_code == 404:
print("✗ Config not found (create one first)")
return False
else:
print(f"✗ Request failed: {response.status_code}")
print(f"Error: {response.text}")
return False
def test_start_measurement():
"""Test that start measurement attempts to disable sleep mode"""
print("\n=== Testing Start Measurement ===")
print(f"Attempting to start measurement on {UNIT_ID}...")
response = requests.post(f"{BASE_URL}/{UNIT_ID}/start")
if response.status_code == 200:
print("✓ Start command accepted")
print(f"Response: {json.dumps(response.json(), indent=2)}")
print("\nNote: Sleep mode was disabled before starting measurement")
return True
elif response.status_code == 404:
print("✗ Device config not found (create config first)")
return False
elif response.status_code == 502:
print("✗ Device not reachable (expected if no physical device)")
print(f"Response: {response.text}")
print("\nNote: This is expected behavior when testing without a physical device")
return True # This is actually success - the endpoint tried to communicate
else:
print(f"✗ Request failed: {response.status_code}")
print(f"Error: {response.text}")
return False
def main():
print("=" * 60)
print("Sleep Mode Auto-Disable Test")
print("=" * 60)
print("\nThis test verifies that sleep mode is automatically disabled")
print("when device configs are updated or measurements are started.")
print("\nNote: Without a physical device, some operations will fail at")
print("the device communication level, but the API logic will execute.")
# Run tests
results = []
# Test 1: Update config (should attempt to disable sleep mode)
results.append(("Config Update", test_config_update()))
# Test 2: Get config
results.append(("Get Config", test_get_config()))
# Test 3: Start measurement (should attempt to disable sleep mode)
results.append(("Start Measurement", test_start_measurement()))
# Summary
print("\n" + "=" * 60)
print("Test Summary")
print("=" * 60)
for test_name, result in results:
status = "✓ PASS" if result else "✗ FAIL"
print(f"{status}: {test_name}")
print("\n" + "=" * 60)
print("Implementation Details:")
print("=" * 60)
print("1. Config endpoint is now async and calls ensure_sleep_mode_disabled()")
print(" when TCP is enabled")
print("2. Start measurement endpoint calls ensure_sleep_mode_disabled()")
print(" before starting the measurement")
print("3. Sleep mode check is non-blocking - config/start will succeed")
print(" even if the device is unreachable")
print("=" * 60)
if __name__ == "__main__":
main()