diff --git a/docs/superpowers/specs/2026-09-02-project-cadence-design.md b/docs/superpowers/specs/2026-09-02-project-cadence-design.md new file mode 100644 index 0000000..d946e67 --- /dev/null +++ b/docs/superpowers/specs/2026-09-02-project-cadence-design.md @@ -0,0 +1,460 @@ +# Project cadence — Lyra as BIT's observer and picker + +- **Date:** 2026-09-02 +- **Status:** Spec — approved to build +- **Branch:** `feat/project-cadence` +- **Second repo:** `~/break-it-down` (BIT) — needs its own branch work, see Phase 0 +- **Related:** `lyra/thoughts.py` (surfacing machinery), `docs/ROADMAP.md` + +--- + +## The problem + +Brian's long-horizon projects — the garage, the music backup, the side repos — +have no due date and no end state. The failure isn't laziness or tracking; it's +**initiation**, driven by a specific loop: + +> "If I can't do it perfectly, why do it at all." → "I'll start tomorrow." → nothing. + +Three things feed that loop, and all three are fixable. + +### 1. The record is false, in the direction of despair + +BIT records every project as last-updated **2026-03-21**. Meanwhile `project-lyra` +alone has **177 commits across 30 distinct working days** since that date, most +recently two days ago. + +BIT isn't recording a dead project. It's blind to a very alive one. + +This matters more than any feature. A manual tracker doesn't decay to *neutral* +when you stop feeding it — it decays to **actively demoralizing**, because absence +of records is indistinguishable from absence of work. Open BIT today and it shows +a graveyard with March on the headstone. That is "why bother" fuel, manufactured +by the tool built to fight it. + +### 2. Nothing initiates + +BIT is a passive store. Using it requires remembering to go to it — which is the +same broken step as remembering to start a pomodoro timer. (Note: a pomodoro +timer *was* built, in February, and has sat on an undeployed branch since. The +timer was never the missing piece.) + +### 3. `/actionable` enumerates but does not pick + +BIT already has a "⚡ Now — what can I do right now?" view. Today it returns +**68 tasks**. That is another wall — the exact wall it was built to knock down. + +Picking requires context BIT structurally cannot have: what time it is, how long +Brian has, what he was warm on yesterday, what's gone stale, whether he's at the +desk or on his phone. + +--- + +## The frame + +Mirrors the frame already committed to for the pokerlog: + +> **BIT is the system of record. Lyra is a client of it, not its container.** + +| | Owns | Rationale | +|---|---|---| +| **BIT** | Facts: projects, tasks, blockers, estimates, **time logs** | Standalone value. Human-editable UI. Works with Lyra dead. | +| **Lyra** | The relationship: cadence config, nudge state, dormancy, **the pick** | None of this is a fact about the garage. It's her state *about* the garage. | + +Same distinction as poker's *ledger* (facts) vs *relationship* (her memory of the +sessions). + +**One sentence:** BIT knows what's possible. Lyra picks one, and knows why. + +--- + +## Design principles + +1. **Sessions, not completion.** The unit of progress is a logged session, never + a completion percentage. A % bar on an endless project reads 12% forever and + is a "why bother" generator. BIT's own v0.2.0 roadmap lists "Progress tracking + (% complete)" — **keep it off long-horizon projects.** +2. **Cadence health, not deadlines.** A project is `on_cadence`, `drifting`, or + `dormant`. **Dormant is a legal, non-shameful state**, not a failure. +3. **Observe, don't ask.** The record must become true without Brian maintaining + it. This is the highest-value thing in the whole design. +4. **No model call in the read/write path.** Reading projects, logging a session, + marking dormant — pure HTTP and SQL, working with the MI50 down and every API + key expired. The LLM appears **only** in the nudging and picking paths, which + are allowed to be flaky because a missed nudge costs nothing. +5. **Push has a budget; ignoring changes state.** Habituation is automatic and + cannot be willpowered. The anti-stacking mechanism is scarcity, not + notification priority flags. Two ignored nudges puts a project to sleep and + she stops asking. +6. **Additive-only to BIT's schema.** (Moot with a fresh DB, but keep the + discipline: SQLAlchemy `create_all()` adds missing tables safely; altering + existing ones is where data dies.) + +--- + +## What already exists — do not rebuild + +Substantially more than expected. Verified against the running instance at +`10.0.0.40:3002` and the cloned repo at `~/break-it-down`. + +### On BIT `origin/dev` (commit `2ee75f7`, Feb 2026) — built, never deployed + +```python +class TimeLog(Base): + __tablename__ = "time_logs" + task_id = Column(Integer, ForeignKey("tasks.id"), nullable=False) + minutes = Column(Integer, nullable=False) + note = Column(Text, nullable=True) + session_type = Column(String(50), default="manual") # 'pomodoro' | 'manual' + logged_at = Column(DateTime, default=datetime.utcnow) +``` + +- `Project.weekly_hours_goal` and `Project.total_hours_goal` — cadence targets, + already modeled. **Trap: both are named `*_hours_goal` but the source comment + says they store MINUTES.** Do not multiply by 60. +- `POST /api/tasks/{id}/time-logs`, `GET /api/tasks/{id}/time-logs` +- `GET /api/projects/{id}/time-summary` — the accumulation-evidence endpoint +- `migrate_add_time_logs.py`, `migrate_add_project_goals.py` — idempotent + `CREATE TABLE IF NOT EXISTS` migrations, hand-written +- `PomodoroWidget.jsx`, `PomodoroContext.jsx` — a working timer UI + +`session_type` is a free-text `String(50)`, so adding `'git'` is **not** a schema +change. + +### On BIT `origin/blockers` (commit `5da6e07`, Mar 2026) — this is what's deployed + +- `task_blockers` many-to-many association table; `Task.blockers` / `Task.blocking` +- `GET|POST|DELETE /api/tasks/{id}/blockers[/{id}]` +- `GET /api/actionable` — unblocked, not-done leaf tasks across all projects +- `is_archived` on Project, Active/Archived/All tabs + +### Branch divergence + +Three commits total: **2 on dev** (pomodoro + a chore), **1 on blockers**. Neither +branch has both halves. `main` (Feb 17) has neither. + +### Already in Lyra + +- `lyra/thoughts.py` — a threaded, decaying, self-surfacing, backs-off-when-ignored + nag engine with quiet hours and a push channel. `new_thread:210`, `decay:265`, + `record_response:292`, `maybe_surface:326`, `maybe_ping:377`, + `maybe_daily_digest:429`. +- `lyra/notify.py` — ntfy push with tap-through +- `lyra/clock.py:56` — `humanize_gap` ("3 days") +- `lyra/tools.py` — tool spec + dispatch pattern +- `lyra/mind.py:168` — `build_messages`, where context notes get injected +- `lyra/web/` on `:7078` — RTO black/orange theme, matching BIT's existing look + +### API facts verified against the running instance + +- API is served at **`:3002/api/*` through nginx**. Port 8002 is not exposed. +- `GET /api/projects/{id}/tree` **404s** on the deployed branch. Use + `GET /api/projects/{id}/tasks`. +- The README is stale in both directions: it claims v0.1.6 while the deploy + reports v0.1.5, it documents `/tree` which doesn't exist, and it omits + blockers, `/actionable`, and `is_archived` — listing blockers as "v0.2.0 + planned" when they're live. **Treat the code as truth, never the README.** + +--- + +## Architecture + +``` + ┌────────────────────────┐ + │ git repos on disk │ + │ (7 of 8 BIT projects) │ + └───────────┬────────────┘ + │ walk history, group by local day + ▼ + ┌────────────────────────┐ ┌──────────────────────────────┐ + │ lyra/gitobserver.py │ │ chat: "spent an hour in the │ + │ deterministic, no LLM │ │ garage" → log_work tool │ + └───────────┬────────────┘ └──────────────┬───────────────┘ + │ │ + └──────────────┬─────────────────────┘ + ▼ POST /api/tasks/{id}/time-logs + ┌───────────────────────────┐ + │ BIT (facts) │ + │ projects · tasks · blockers│ + │ time_logs · goals │ + └───────────┬───────────────┘ + │ GET /actionable, /time-summary, /tasks + ▼ + ┌───────────────────────────┐ + │ lyra/projects.py │ + │ BIT client + cadence math │ + │ (pure, no LLM) │ + └───────────┬───────────────┘ + │ + ┌──────────────┼──────────────────┐ + ▼ ▼ ▼ + lyra/tools.py cadence state thoughts.py surfacing + (she can talk (Lyra's DB: (salience → surface → + about it) goals, nudge ping → response, + state, dormancy) budget + backoff) +``` + +**Data flow, one line:** git and chat write facts into BIT; Lyra reads BIT, computes +cadence, and decides whether and how to open her mouth. + +--- + +## Phase 0 — BIT consolidation + +There is no production data. The 8 projects / 77 tasks on `10.0.0.40` were test +builds; a snapshot lives at `~/bit-seed-snapshot/` if any of it is worth reseeding +through the existing JSON import. + +1. Merge `origin/dev` and `origin/blockers` into a single branch. Three commits; + the backend touch points barely overlap (`models.py` gains `TimeLog` from one + and `task_blockers` from the other). +2. Deploy BIT **on this machine** via its existing `docker-compose.yml`. Pick + ports that don't collide with `lyra-web` (`:7078`) or `lyra-ntfy`. +3. Fresh `bit.db` — `create_all()` builds the full schema. The migration scripts + become unnecessary but stay in the repo for the old instance. +4. Verify post-merge: `/api/actionable`, `/api/tasks/{id}/time-logs`, + `/api/projects/{id}/time-summary`, and the Pomodoro widget all respond. +5. Create the real projects. `Clean up march 26` becomes actual physical projects + (garage, music backup, spare closet). + +**Exit criteria:** one BIT instance on this box serving blockers *and* time logs, +reachable from Lyra. + +--- + +## Phase 1 — the git observer + +`lyra/gitobserver.py`. **Deterministic. No LLM anywhere in this module** (principle 4), +which keeps the backfill reproducible and unit-testable. + +### Config + +A repo↔project mapping in Lyra's config: local repo path → BIT project id. +Explicit, not auto-discovered — auto-discovery would guess wrong and pollute the +ledger. + +### Session derivation + +For each repo, group commits **by local calendar day** (Brian's configured +timezone via `clock.py`, not UTC — a 11pm commit belongs to that evening). + +For each day with commits: + +- `minutes = (last_commit_ts - first_commit_ts) + LEAD_PADDING` +- `LEAD_PADDING = 10` — work precedes the first commit +- `MIN_MINUTES = 15` — a single-commit day would otherwise be 0 +- `MAX_MINUTES = 240` — a 9am and an 11pm commit is not a 14-hour session + +These are heuristics and must be **config, not constants**, and documented as +estimates in the spec and in the code. + +- `session_type = "git"` — never `'manual'` or `'pomodoro'`. Estimated time must + never masquerade as measured time. +- `note` = commit count and short SHA range, for traceability. + +### Attribution + +`time_logs.task_id` is `NOT NULL`, but a commit maps to a *repo* (project), not a +task. Two deterministic tiers: + +1. **Conventional-commit scope match.** `feat(persona): …` → match `persona` + against task titles and tags in that project, case-insensitive substring. Plain + string matching, no model. +2. **Fallback:** a per-project `Unfiled work` task, created on demand. + +Attribution is a nice-to-have; **the totals are the point**. Attribution must never +block or fail a backfill. LLM-assisted attribution is explicitly deferred (see +Non-goals). + +### Idempotency — the critical correctness property + +The observer runs repeatedly and must never double-log. + +Lyra owns a `git_sessions` table: `(repo_path, local_date)` unique → `bit_time_log_id`, +`last_sha`. A day is logged only if that key is absent. This is idempotent by +construction and survives BIT being reset — on a fresh BIT, clear the table and +re-backfill cleanly. + +A re-run over a day that gained new commits **updates** the existing time log +rather than adding a second one. + +### Backfill + +One-shot run over history since 2026-03-21 (or repo start). `project-lyra` alone +should produce ~30 sessions. **This is the emotional payload of the entire +project** — the first time BIT is opened after this, it says "30 days worked since +March" instead of showing a tombstone. + +### Non-code projects in v1 + +The git observer does nothing for the garage or the music collection. Those are +covered in v1 by **conversational logging**: "I spent an hour in the garage" → +`log_work` tool → a real `time_logs` row. Zero new infrastructure. Automated +observation of physical work is deferred past v1 (see Non-goals). + +**Exit criteria:** BIT shows true history for every configured repo; re-running the +observer changes nothing. + +--- + +## Phase 2 — cadence + +`lyra/projects.py` — BIT HTTP client plus **pure cadence functions** (easy to test, +no I/O). + +### Health states + +Computed from `time_logs` + `Project.weekly_hours_goal`: + +- `on_cadence` — trailing-7-day minutes ≥ `weekly_hours_goal` +- `drifting` — some activity in the window, below goal +- `dormant` — no session in `DORMANT_AFTER_DAYS` (default 30), or set explicitly +- `untracked` — no goal set; fall back to days-since-last-session alone + +`weekly_hours_goal` is nullable, so `untracked` is the common initial state and +must render sensibly. + +### Lyra-side state + +A small table keyed by BIT project id: cadence target, last surfaced, ignore +count, dormancy, salience. **Not in BIT** — it's her state about the project, not +a fact about it. + +--- + +## Phase 3 — Lyra tools + +Following the `lyra/tools.py` spec + dispatch pattern: + +- `projects_overview()` — every project with cadence health and accumulated time +- `project_status(name)` — detail, recent sessions, what's actionable +- `log_work(project, task, minutes, note)` — conversational session logging +- `pick_next(minutes_available)` — see Phase 4 +- `break_down(task_id)` — generate a subtask tree, push it through BIT's existing + JSON import +- `set_goal(project, weekly_hours)` +- `set_dormant(project)` / `wake(project)` + +--- + +## Phase 4 — surfacing and the pick + +### The pick + +The one place an LLM is genuinely required. Input: `/api/actionable`, minutes +available, recent sessions, cadence health, time of day. Output: **one task and one +sentence of why.** Not a list. A list is what BIT already does badly. + +### The decomposition trigger + +Live data shows tasks like "Multi-user authentication — est 480", "Historical data +tracking — 360". **You cannot start an eight-hour task**, and these sit in +`/actionable` forever radiating "why bother." + +Rule: `estimated_minutes > DECOMPOSE_THRESHOLD` (default 60) means decomposition +hasn't happened yet. When the best candidates are all oversized, Lyra offers to +break one down instead of proposing it — closing the anti-perfectionism loop +through `break_down`, using the JSON import that already exists for exactly this. + +### Surfacing + +Reuse `thoughts.py` machinery; **do not put projects in the `threads` table.** A +thought is her interiority; a project is a fact about Brian's life. Conflating them +pollutes both — her dream cycle would generate the garage as a thought, and the +garage would appear in her self-narrative. Separate storage, shared +salience → surface → ping → response loop. + +**Channel priority:** + +1. **Conversation (default).** Next time Brian's talking to her anyway, she raises + it: *"the music backup's been sitting three weeks — dead, or just asleep?"* Not + a notification. A person asking. This is the unfair advantage no to-do app has. +2. **Push (rare, budgeted).** Two legal uses only: + - the **receipt** — "you've been in there 20 minutes, want me to count it?" + (makes no demand, so there's nothing to blow off) + - the **backoff** — "I've raised the garage twice and you didn't bite, so I'm + putting it to sleep. Say the word and it wakes up." + +No scheduled daily ping. A clock tick is the most habituation-prone trigger there +is; she speaks when something is *true*. + +**Ignoring changes state:** two ignored surfaces → dormant → she stops. Dormancy +needs no BIT schema change — BIT already has per-project custom statuses. + +--- + +## Non-goals — explicitly parked + +Named to hold the line, because "actually get this functional and helpful for my +every day life" is the stated goal and feature creep is the named enemy. + +- **The ambient card / desktop widget** — the right long-term interface, but a + surface on top of a thing that has to work first. +- **Generated wallpaper** — same. +- **Windows active-window reporter** — the "backwards timer" for physical and + non-git work. Deferred past v1. Note it needs no Pieces: ~30 lines reporting active + window title covers "which project am I in." +- **PiecesOS / MCP integration** — richer, but a cross-machine dependency + (PiecesOS is local to the Windows box, Lyra is on the homelab) that must be + spiked before anything rests on it. Never load-bearing on day one. +- **Phone widget / home-screen surface** — depends on iOS vs Android, unresolved. +- **Physical display on the rack** — unblocked (a spare monitor exists; drive it + with a Pi or old laptop running a kiosk browser, **not** by installing Xorg on + the Proxmox host). It's a browser pointed at a URL this design already produces, + so it changes nothing and can be added any time. Put it on the desk, not the rack. +- **Automatic blocker detection** — Brian's dependency-graph idea (paint → move + furniture → closet access → clothes + desk). Manual blockers are already built + and deployed; *auto-identifying* them is the parked part, and it's a natural + Lyra job later. **This is the strongest answer to "where do I start" for physical + projects** — worth un-parking once v1 is honest. +- **Cross-device syncing sticky notes** — a separate product. The sticky's *form* + (a small always-visible card) is worth having and needs no sync, because its + content is generated server-side. The sync engine is the whole product, and + Trilium already does it. +- **`% complete` on long-horizon projects** — actively harmful. See principle 1. + +--- + +## Error handling + +- **BIT unreachable:** every Lyra tool degrades to a clear "BIT is down" message. + Never a stack trace into her context, never a fabricated answer about project + state. Follows the `notify.py` precedent: a down dependency must not break the + loop. +- **Git observer failure on one repo:** log and continue. One bad repo must not + abort a backfill. +- **Duplicate protection:** the `git_sessions` unique key is the guard. A + crashed mid-backfill run resumes cleanly. +- **Clock skew / commits with future dates:** clamp to today; never create a + session in the future. +- **Attribution failure:** always falls back to `Unfiled work`. Never raises. + +--- + +## Testing + +Follows existing conventions (`tests/`, pytest, replay evals for LLM paths). + +- **Git observer:** fixture repo built in a tmpdir with controlled commit + timestamps. Assert day grouping across timezone boundaries, MIN/MAX/padding + clamps, scope-match attribution, `Unfiled work` fallback, and — most + importantly — **that a second run produces zero new time logs**. +- **Cadence functions:** pure, table-driven. Boundary cases: no goal set, goal met + exactly, dormancy threshold ±1 day, zero sessions ever. +- **BIT client:** against a stub server. Include the `/tree` 404 case so a README + regression can't silently break it. +- **The pick and the surfacing copy:** replay eval, parameterized by + `EVAL_BACKEND`/`EVAL_MODEL`, matching the persona eval pattern. Assert it + returns *one* task, and that it offers decomposition when candidates are + oversized. + +--- + +## Open questions + +1. **Ports for BIT on this box.** `:3002/:8002` are free here; confirm at deploy. +2. **Which of the 77 snapshotted tasks are worth reseeding**, versus starting the + real projects clean. Brian's call during Phase 0. +3. **Whether `weekly_hours_goal` gets set per project at all in v1**, or whether + `untracked` + days-since-last-session is enough to start. Leaning: ship without + goals, add them once real session data shows what a realistic cadence is.