Two related operator-facing improvements after the nav reorg.
1) Events as a top-level sidebar entry.
The /sfm page (fleet-wide event database) was demoted to Settings →
Developer in the previous reorg. Bringing it back to main nav as
"Events" — operators do reach for the cross-project, sortable
event list, so it earns a top-level slot.
Sidebar now (7 items):
Dashboard · Devices · Projects · Events · Tools · Job Planner · Settings
Settings → Developer card pointing at /sfm is removed. /sfm page
title/subtitle updated from "SFM Event Data" to just "Events". URL
unchanged.
2) "Peak PVS" KPI tile becomes "Overall Peak" and excludes false
triggers from the calculation.
When operators ask "what's the biggest event at this location/unit/
project?" they mean the biggest REAL event, not the biggest sensor
glitch. A single mis-flagged false trigger could otherwise dominate
the tile (the 14.13 in/s spike at Loc 1 was a prime example).
backend/services/sfm_events.py:
- _compute_stats() skips false_trigger=True events when computing
peak_pvs / peak_pvs_at / peak_pvs_serial. Continues counting them
in false_trigger_count so the separate "False Triggers" tile still
reflects what got filtered out. last_event unchanged (recency, not
magnitude).
- Same change automatically propagates to events_for_unit() and
vibration_summary_for_project() — both call _compute_stats().
Templates: "Peak PVS" → "Overall Peak" in 3 KPI tile locations
(vibration_location_detail.html, partials/projects/vibration_summary
.html, unit_detail.html). The physical-quantity name "Peak Vector
Sum" in the event-detail modal stays — that's the actual physics
term, not a summary stat.
Verified end-to-end: Overall Peak renders on real data; peak event
false_trigger flag confirmed False.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sidebar evolved from "Fleet defaults to seismograph dashboard" to
"Devices defaults to unified roster" + a new "Tools" entry housing the
active operator workflows.
Sidebar (6 items):
Dashboard · Devices · Projects · Tools · Job Planner · Settings
Changes:
- templates/base.html: renamed Fleet → Devices. Default route changed
from /seismographs to /roster — clicking Devices now lands on the
unified all-devices view, then operators drill into type-specific
layouts via the tab strip. Tools entry added between Projects and
Job Planner; highlights when on /tools or any of its linked workflow
pages.
- templates/partials/fleet_tab_strip.html: reordered tabs so "All
Devices" comes first (matches the new default landing).
Seismographs → SLMs → Modems follow.
- templates/tools.html (new) + /tools route in main.py: card grid hub
for active workflows.
• Pair Devices — links to /pair-devices
• Project Tidy — links to /settings/developer/project-tidy
• Backfill from event metadata — /settings/developer/metadata-backfill
• Reports — info card pointing to project detail pages where
Excel report generation actually lives (per-project context)
• Swap Detection — greyed-out placeholder for Phase 5c
- templates/settings.html: removed Project Tidy + Metadata Backfill
cards from Settings → Developer. They now live in Tools. Settings
→ Developer retains the truly admin/dev surfaces (Watcher Manager,
SFM Admin).
The workflow page URLs (/settings/developer/project-tidy,
/settings/developer/metadata-backfill) stay where they are — only the
nav entry point changes. Bookmarks still work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The sidebar had 10 entries with 5 of them (Devices, Seismographs, Sound
Level Meters, Modems, Pair Devices) all about the physical fleet plus
SFM Events as a debug surface. Operators kept asking "where do I find
BE11529?" without knowing whether it was a seismograph / SLM / modem.
This collapses those 5+1 into a single "Fleet" sidebar entry that opens
into a unified tab strip across the top of the four device pages. Each
page keeps its existing custom layout (seismograph-specific
calibration/deployment columns, SLM live-status panel, modem pairing
view, all-devices roster). The strip just provides the navigation +
the "Pair Devices" button as an action.
Sidebar before (10 items):
Dashboard · Devices · Seismographs · SFM Events · Sound Level Meters
Modems · Pair Devices · Projects · Job Planner · Settings
Sidebar after (5 items):
Dashboard · Fleet · Projects · Job Planner · Settings
Changes:
- templates/partials/fleet_tab_strip.html (new): the shared tab strip.
Auto-detects the active tab from request.url.path. 4 tabs
(Seismographs / Sound Level Meters / Modems / All Devices) plus a
"Pair Devices" button on the right.
- templates/{seismographs,sound_level_meters,modems,roster}.html: added
{% include 'partials/fleet_tab_strip.html' %} as the first thing
inside the content block. No other changes to those templates'
existing layouts.
- templates/base.html: replaced the 6 device-related sidebar links with
one "Fleet" link to /seismographs. The Fleet entry is highlighted
when the current URL is any of /seismographs, /sound-level-meters,
/modems, /roster, /pair-devices, /unit/*, or /slm/*.
- templates/settings.html: SFM Events moved out of the main nav into a
new "SFM Admin" card under Settings → Developer. Daily event
browsing already lives on project / location / unit pages (Phases
1+2+3); the standalone /sfm page is now admin / cross-project debug
surface only.
URLs unchanged — all bookmarks / deep links still work. /sfm still
serves the standalone page, it's just no longer in the main nav.
Mobile bottom-nav unaffected.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 5b first slice. Surfaces near-duplicate projects (typo variants,
abbreviation differences, spacing variations like "SR81" vs "SR 81")
as side-by-side pairs the operator can merge with one click.
Backend (backend/services/project_tidy.py):
- find_duplicate_pairs(db, threshold=0.85) walks all active projects and
computes rapidfuzz.WRatio similarity for every pair. Pre-filters
too-short normalised names (< 4 chars) to avoid noise. Skips
soft-deleted projects. Returns pairs sorted by score desc, then by
total content (more assignments → review first).
- Each pair carries a suggested merge target with a human-readable
reason. Priorities (in order): manual source over parser source,
populated project_number, more locations, more assignments, shorter
name. Operator can override the suggestion by clicking the OTHER
direction button.
- O(N^2) over project count. Fine up to ~500 projects. Token-prefix
blocking is the obvious next optimisation if it becomes slow.
Backend (backend/routers/projects.py):
- GET /api/projects/admin/duplicate_pairs?threshold=&max_pairs= returns
pairs as JSON for the Tidy page.
Frontend (templates/admin/project_tidy.html):
- New admin page at /settings/developer/project-tidy. Threshold selector
(95% / 90% / 85% / 80%) at the top; rescan button next to it; auto-
scans on load.
- Each pair card shows side-by-side project summaries (name, project_
number, client, source-badge, location/assignment counts) with the
suggested target visually highlighted (orange border). Three buttons:
"Merge A → B", "Merge B → A", "Not a dup" (hide locally).
- Click-to-merge opens a native confirm with the preview totals
(assignments/sessions/data files moving, consolidations) — same data
the project_header.html merge modal shows. On confirm, hits the
existing /merge_into endpoint and re-scans automatically.
- Source badges distinguish parser-created (`metadata_backfill`) from
manual projects — at a glance the operator can see "this duplicate is
parser-generated; safe to merge into the manual one".
Frontend (templates/admin/metadata_backfill.html):
- Apply-result handling now surfaces failed[] cluster reasons in a
dedicated failure panel (bottom-left, dismissable). Previously a 200
OK with all-failures showed a misleading "1 cluster applied" success
toast because the count and the failure list weren't being reconciled.
This bit us during the DB-revert recovery earlier — the
project_modules table was missing, every apply silently rolled back,
user saw success toasts. Fixed.
Smoke-verified against current state (10K events, 9 projects, post-
merge): tool correctly finds 0 pairs at threshold 0.85 (data is clean),
1 false-positive at 0.70 (two unrelated projects sharing the token "81"
— example of why the 0.85 default is correct).
Settings link added under Developer → Project Tidy.
Phase 5c (swap-detection daily background job + notification inbox)
remains deferred to the next session.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operator clicks one button. Parser reads SFM's events table (operator-typed
project / client / sensor_location strings), clusters by serial + time +
metadata, fuzzy-matches against existing projects, and proposes
Project / MonitoringLocation / UnitAssignment chains to create.
Auto-applies high-confidence non-conflicting clusters in bulk; queues
medium/low confidence for individual review.
Verified against real data: 10,052 events → 59 clusters → 37 high-
confidence + 14 medium + 8 low. Test-applied one cluster end-to-end;
Project + Module + Location + Assignment + UnitHistory + Decision rows
all created correctly, and Phase 2's attribution walk picked up the
events automatically on the new location's detail page.
Pipeline (backend/services/metadata_backfill.py, ~700 lines):
1. Pull all SFM events via /db/events per serial.
2. Pre-filter: drop events already covered by an existing UnitAssignment
window (Phase 2 handles those automatically).
3. Time-cluster what's left: serial + 7-day gap is the cluster identity.
4. Metadata-split each time-cluster on persistent metadata transitions
(≥ 2 consecutive events) so a single typo doesn't fork the cluster.
5. Match against existing graph (rapidfuzz.WRatio multi-signal scoring,
normalisation that handles abbreviations / reorders / separator
variations). Thresholds: 0.95 exact, 0.80 fuzzy, min-shorter-input
5 chars to guardrail false positives on single common words.
6. Score confidence (high/medium/low) using event count, span,
blank-meta, conflict, ambiguity rules.
7. Detect conflicts: overlap with existing UnitAssignment at a different
location for the same serial → blocking. Operator must reconcile.
8. Apply: ensure auto_imported ProjectType exists, ensure
vibration_monitoring ProjectModule on the project, write
Project / MonitoringLocation / UnitAssignment / UnitHistory all in
one transaction.
Migration (backend/migrate_add_metadata_backfill.py): adds
unit_assignments.source column (default 'manual') and
metadata_backfill_decisions table. Idempotent, non-destructive.
API (backend/routers/metadata_backfill.py):
GET /api/admin/metadata_backfill/scan — clusters + suggestions
POST /api/admin/metadata_backfill/apply — bulk apply by cluster_ids
w/ optional per-cluster
project/location overrides
POST /api/admin/metadata_backfill/skip — mark skipped (persistent)
UI (templates/admin/metadata_backfill.html, accessible at
/settings/developer/metadata-backfill via the Developer tab of Settings):
- One-button "Run scan" entry.
- Summary KPI tiles (scanned / already attributed / pending / conflicts).
- "Apply all high-confidence" bulk button at the top — primary path.
- Per-cluster cards below with Apply / Skip / Preview event actions.
- Blank-meta clusters get inline input fields for operator-typed project +
location names before applying.
- Blocking-conflict clusters render with the conflicting assignment
information and a disabled Apply button.
- Live progress toast during apply.
- Reuses the Phase 1+2+4 event-detail modal for "Preview event" — operator
can sanity-check the BW report data against the cluster's sample event.
Dependencies: rapidfuzz==3.10.1 added to requirements.txt. Pre-built C
wheels for all platforms, ~5s docker build hit.
Phase 5b (deferred to next session): swap-detection daily background job,
notification inbox for auto-applied swaps, recently-applied audit view,
"Tidy" page for renaming/merging auto-created projects.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Introduced a new section for displaying soft-deleted projects.
- Implemented loading of deleted projects via an API call.
- Added restore and permanently delete options for each deleted project.
- Integrated loading of deleted projects when the data tab is shown.
- Updated roster.html to include a new option for Sound Level Meter in the device type selection.
- Added specific fields for Sound Level Meter information, including model, host/IP address, TCP and FTP ports, serial number, frequency weighting, and time weighting.
- Enhanced JavaScript to handle the visibility and state of Sound Level Meter fields based on the selected device type.
- Modified the unit editing functionality to populate Sound Level Meter fields with existing data when editing a unit.
- Updated settings.html to change the deployment status display from badges to radio buttons for better user interaction.
- Adjusted the toggleDeployed function to accept the new state directly instead of the current state.
- Changed the edit button in unit_detail.html to redirect to the roster edit page with the appropriate unit ID.
serversdown
claude