Files
project-lyra/docs/superpowers/specs/2026-09-02-project-cadence-design.md
T
serversdown 5560c7c3b0 docs(spec): pin why the git observer runs on the host, not in BIT's container
Batch job not request handler; volume mounts would rot as projects are added;
HTTP keeps the observer relocatable to another machine. BIT stays in Docker.
2026-09-02 04:29:17 +00:00

24 KiB

Project cadence — BIT observes, Lyra picks

  • Date: 2026-09-02
  • Status: Spec — approved to build
  • Branch: feat/project-cadence
  • Two repos, both worked on: ~/break-it-down (BIT) — Part A; project-lyra — Part B
  • Related: lyra/thoughts.py (surfacing machinery), docs/ROADMAP.md

The problem

Brian's long-horizon projects — the garage, the music backup, the side repos — have no due date and no end state. The failure isn't laziness or tracking; it's initiation, driven by a specific loop:

"If I can't do it perfectly, why do it at all." → "I'll start tomorrow." → nothing.

Three things feed that loop, and all three are fixable.

1. The record is false, in the direction of despair

BIT records every project as last-updated 2026-03-21. Meanwhile project-lyra alone has 177 commits across 30 distinct working days since that date, most recently two days ago.

BIT isn't recording a dead project. It's blind to a very alive one.

This matters more than any feature. A manual tracker doesn't decay to neutral when you stop feeding it — it decays to actively demoralizing, because absence of records is indistinguishable from absence of work. Open BIT today and it shows a graveyard with March on the headstone. That is "why bother" fuel, manufactured by the tool built to fight it.

2. Nothing initiates

BIT is a passive store. Using it requires remembering to go to it — which is the same broken step as remembering to start a pomodoro timer. (Note: a pomodoro timer was built, in February, and has sat on an undeployed branch since. The timer was never the missing piece.)

3. /actionable enumerates but does not pick

BIT already has a "⚡ Now — what can I do right now?" view. Today it returns 68 tasks. That is another wall — the exact wall it was built to knock down.

Picking requires context BIT structurally cannot have: what time it is, how long Brian has, what he was warm on yesterday, what's gone stale, whether he's at the desk or on his phone.


The frame

Mirrors the frame already committed to for the pokerlog:

BIT is the system of record. Lyra is a client of it, not its container.

The test that decides where any piece of work goes:

Does it produce or derive facts? → BIT. Does it require knowing Brian? → Lyra.

Owns Rationale
BIT Facts and everything derived from them: projects, tasks, blockers, estimates, time logs, the git observer, cadence health Standalone value. Human-editable UI. Works with Lyra dead — and the observer is what keeps that true.
Lyra The relationship: the pick, nudge state, backoff, conversational logging, decomposition offers Every one of these needs to know it's 9pm, that he has 20 minutes, that he was warm on seismo-relay yesterday.

An earlier draft of this spec put the git observer and cadence math in Lyra. That was wrong and violated the frame above: the observer produces facts and knows nothing about Brian. Worse, it would have made BIT's data true only while Lyra was running — reintroducing exactly the coupling the separable frame exists to prevent. Auto-logging and cadence health also make BIT meaningfully better as a standalone product, which is the justification for keeping it separable at all.

Same distinction as poker's ledger (facts) vs relationship (her memory of the sessions).

One sentence: BIT knows what's possible. Lyra picks one, and knows why.


Design principles

  1. Sessions, not completion. The unit of progress is a logged session, never a completion percentage. A % bar on an endless project reads 12% forever and is a "why bother" generator. BIT's own v0.2.0 roadmap lists "Progress tracking (% complete)" — keep it off long-horizon projects.
  2. Cadence health, not deadlines. A project is on_cadence, drifting, or dormant. Dormant is a legal, non-shameful state, not a failure.
  3. Observe, don't ask. The record must become true without Brian maintaining it. This is the highest-value thing in the whole design.
  4. No model call in the read/write path. Reading projects, logging a session, marking dormant — pure HTTP and SQL, working with the MI50 down and every API key expired. The LLM appears only in the nudging and picking paths, which are allowed to be flaky because a missed nudge costs nothing.
  5. Push has a budget; ignoring changes state. Habituation is automatic and cannot be willpowered. The anti-stacking mechanism is scarcity, not notification priority flags. Two ignored nudges puts a project to sleep and she stops asking.
  6. Additive-only to BIT's schema. (Moot with a fresh DB, but keep the discipline: SQLAlchemy create_all() adds missing tables safely; altering existing ones is where data dies.)

What already exists — do not rebuild

Substantially more than expected. Verified against the running instance at 10.0.0.40:3002 and the cloned repo at ~/break-it-down.

On BIT origin/dev (commit 2ee75f7, Feb 2026) — built, never deployed

class TimeLog(Base):
    __tablename__ = "time_logs"
    task_id      = Column(Integer, ForeignKey("tasks.id"), nullable=False)
    minutes      = Column(Integer, nullable=False)
    note         = Column(Text, nullable=True)
    session_type = Column(String(50), default="manual")  # 'pomodoro' | 'manual'
    logged_at    = Column(DateTime, default=datetime.utcnow)
  • Project.weekly_hours_goal and Project.total_hours_goal — cadence targets, already modeled. Trap: both are named *_hours_goal but the source comment says they store MINUTES. Do not multiply by 60.
  • POST /api/tasks/{id}/time-logs, GET /api/tasks/{id}/time-logs
  • GET /api/projects/{id}/time-summary — the accumulation-evidence endpoint
  • migrate_add_time_logs.py, migrate_add_project_goals.py — idempotent CREATE TABLE IF NOT EXISTS migrations, hand-written
  • PomodoroWidget.jsx, PomodoroContext.jsx — a working timer UI

session_type is a free-text String(50), so adding 'git' is not a schema change.

On BIT origin/blockers (commit 5da6e07, Mar 2026) — this is what's deployed

  • task_blockers many-to-many association table; Task.blockers / Task.blocking
  • GET|POST|DELETE /api/tasks/{id}/blockers[/{id}]
  • GET /api/actionable — unblocked, not-done leaf tasks across all projects
  • is_archived on Project, Active/Archived/All tabs

Branch divergence

Three commits total: 2 on dev (pomodoro + a chore), 1 on blockers. Neither branch has both halves. main (Feb 17) has neither.

Already in Lyra

  • lyra/thoughts.py — a threaded, decaying, self-surfacing, backs-off-when-ignored nag engine with quiet hours and a push channel. new_thread:210, decay:265, record_response:292, maybe_surface:326, maybe_ping:377, maybe_daily_digest:429.
  • lyra/notify.py — ntfy push with tap-through
  • lyra/clock.py:56 — humanize_gap ("3 days")
  • lyra/tools.py — tool spec + dispatch pattern
  • lyra/mind.py:168 — build_messages, where context notes get injected
  • lyra/web/ on :7078 — RTO black/orange theme, matching BIT's existing look

API facts verified against the running instance

  • API is served at :3002/api/* through nginx. Port 8002 is not exposed.
  • GET /api/projects/{id}/tree 404s on the deployed branch. Use GET /api/projects/{id}/tasks.
  • The README is stale in both directions: it claims v0.1.6 while the deploy reports v0.1.5, it documents /tree which doesn't exist, and it omits blockers, /actionable, and is_archived — listing blockers as "v0.2.0 planned" when they're live. Treat the code as truth, never the README.

Architecture

  ┌────────────────────────┐
  │ git repos on host      │
  │ (7 of 8 BIT projects)  │
  └───────────┬────────────┘
              │  git_observer.py — BIT's code, runs on the HOST
              │  walk history, group by local day, no LLM
              ▼  POST /api/tasks/{id}/time-logs
  ┌──────────────────────────────────────────────────┐
  │ BIT — facts, and everything derived from them    │
  │   projects · tasks · blockers · estimates        │
  │   time_logs · goals · git_sessions               │
  │   GET /api/actionable                            │
  │   GET /api/projects/health   (cadence, new)      │
  │   cadence health rendered on the project cards   │
  └────────────────────┬─────────────────────────────┘
                       │ HTTP
                       ▼
  ┌──────────────────────────────────────────────────┐
  │ lyra/bit.py — thin client. No cadence math here.  │
  └────────────────────┬─────────────────────────────┘
           ┌───────────┴───────────┐
           ▼                       ▼
   lyra/tools.py            lyra/projects.py
   log_work, pick_next,     nudge state, ignore counts,
   break_down, status       backoff, quiet hours
           ▲                       │
           │                       ▼
   chat: "spent an hour     thoughts.py surfacing
   in the garage"           (salience → surface →
                             ping → response)

Data flow, one line: git and chat write facts into BIT; BIT derives cadence; Lyra reads BIT and decides whether, when, and how to open her mouth.


Part A — work in ~/break-it-down (BIT)

Everything here is deterministic. No LLM appears anywhere in Part A. All of it makes BIT better as a standalone product, independently of Lyra.

A0 — consolidation and deploy

There is no production data. The 8 projects / 77 tasks on 10.0.0.40 were test builds; a snapshot lives at ~/bit-seed-snapshot/ if any of it is worth reseeding through the existing JSON import.

  1. Merge origin/dev and origin/blockers into a single branch. Three commits; the backend touch points barely overlap (models.py gains TimeLog from one and task_blockers from the other).
  2. Deploy BIT on this machine via its existing docker-compose.yml. Pick ports that don't collide with lyra-web (:7078) or lyra-ntfy.
  3. Fresh bit.db — create_all() builds the full schema. The migration scripts become unnecessary but stay in the repo for the old instance.
  4. Verify post-merge: /api/actionable, /api/tasks/{id}/time-logs, /api/projects/{id}/time-summary, and the Pomodoro widget all respond.
  5. Create the real projects. Clean up march 26 becomes actual physical projects (garage, music backup, spare closet).

Exit criteria: one BIT instance on this box serving blockers and time logs.

A1 — the git observer

Ships with BIT, in BIT's repo. Deterministic Python, no model, no knowledge of Brian — it produces facts, which is why it lives here.

Deployment note: BIT stays in Docker. The observer ships in BIT's repo but runs on the host (systemd timer preferred over cron — better logging and systemctl status), POSTing to BIT's API like any other client.

This is not a workaround for Docker; it is correct regardless:

  1. It's a batch job, not a request handler. It runs on a schedule and exits. That doesn't belong in a web container's lifecycle either way.
  2. Volume-mounting repos would rot. Every new project would require editing docker-compose.yml — exactly the kind of manual upkeep step that goes stale, which is the failure mode this whole project exists to fix. A path in a config table costs nothing.
  3. HTTP decouples it usefully. The observer can later run on a different machine entirely (the Windows box, for repos that live there) with no code changes. Putting it inside the container throws that away.

Requires git and Python on the host, both already present.

Config

A repo↔project mapping: local repo path → BIT project id. Stored in BIT (a small observed_repos table with a settings UI later, or config file for v1). Explicit, never auto-discovered — auto-discovery would guess wrong and pollute the ledger.

Session derivation

For each repo, group commits by local calendar day in Brian's timezone, not UTC — an 11pm commit belongs to that evening.

For each day with commits:

  • minutes = (last_commit_ts - first_commit_ts) + LEAD_PADDING
  • LEAD_PADDING = 10 — work precedes the first commit
  • MIN_MINUTES = 15 — a single-commit day would otherwise be 0
  • MAX_MINUTES = 240 — a 9am and an 11pm commit is not a 14-hour session

These are heuristics. They must be config, not constants, and documented as estimates in both the spec and the code.

  • session_type = "git" — never manual or pomodoro. Estimated time must never masquerade as measured time. The field is a free-text String(50), so this is not a schema change.
  • note = commit count and short SHA range, for traceability.

Attribution

time_logs.task_id is NOT NULL, but a commit maps to a repo (project), not a task. Two deterministic tiers:

  1. Conventional-commit scope match. feat(persona): … → match persona against task titles and tags in that project, case-insensitive substring. Plain string matching, no model.
  2. Fallback: a per-project Unfiled work task, created on demand.

Attribution is a nice-to-have; the totals are the point. It must never block or fail a backfill. LLM-assisted attribution is explicitly deferred (see Non-goals).

Idempotency — the critical correctness property

The observer runs repeatedly and must never double-log.

A git_sessions table in BIT: (repo_path, local_date) unique → time_log_id, last_sha. A day is logged only if that key is absent. Idempotent by construction. A re-run over a day that gained new commits updates the existing time log rather than adding a second one.

Backfill

One-shot run over history since 2026-03-21 (or repo start). project-lyra alone should produce ~30 sessions. This is the emotional payload of the entire project — the first time BIT is opened afterwards it says "30 days worked since March" instead of showing a tombstone.

Exit criteria: BIT shows true history for every configured repo; re-running the observer changes nothing.

A2 — cadence health

Derived from time_logs + Project.weekly_hours_goal, both of which are BIT's. No knowledge of Brian required, so it belongs here rather than in Lyra.

States

  • on_cadence — trailing-7-day minutes ≥ weekly_hours_goal
  • drifting — some activity in the window, below goal
  • dormant — no session in DORMANT_AFTER_DAYS (default 30), or set explicitly
  • untracked — no goal set; fall back to days-since-last-session alone

weekly_hours_goal is nullable, so untracked is the common initial state and must render sensibly.

Trap: weekly_hours_goal and total_hours_goal are named hours but the source comment says they store minutes. Do not multiply by 60.

Surface it

  • GET /api/projects/health — cadence state and accumulated time per project
  • Render it on the project cards. They currently show Created 3/20/2026 and nothing about activity, which is precisely how the graveyard effect happens. Last-touched plus cadence state plus total logged time turns that wall of dead cards into a readable status board — and this is the payoff of A1 becoming visible to a human without Lyra involved at all.

Exit criteria: opening BIT shows, at a glance, which projects are alive.


Part B — work in project-lyra

Everything here needs to know something about Brian. This is where the LLM lives, and it is allowed to be flaky, because a missed nudge costs nothing.

B1 — client and tools

  • lyra/bit.py — thin HTTP client over BIT's API. No cadence math; BIT already computed it. Degrades to a clear "BIT is down" message, never a stack trace into her context and never a fabricated answer about project state.
  • lyra/tools.py additions, following the existing spec + dispatch pattern:
    • projects_overview() — every project with cadence health and logged time
    • project_status(name) — detail, recent sessions, what's actionable
    • log_work(project, task, minutes, note) — conversational session logging
    • pick_next(minutes_available) — see B2
    • break_down(task_id) — generate a subtask tree, push it through BIT's existing JSON import
    • set_goal(project, weekly_hours), set_dormant(project), wake(project)

Conversational logging is how non-code projects get tracked in v1. The git observer does nothing for the garage or the music collection; "I spent an hour in the garage" → log_work → a real time_logs row. Zero new infrastructure. Automated observation of physical work is deferred past v1 (see Non-goals).

B2 — the pick

The one place an LLM is genuinely required. Input: /api/actionable, minutes available, recent sessions, cadence health, time of day. Output: one task and one sentence of why. Not a list — a list is what BIT already does badly, at 68 items.

The decomposition trigger

Live data shows tasks like "Multi-user authentication — est 480" and "Historical data tracking — 360". You cannot start an eight-hour task, and these sit in /actionable forever radiating "why bother."

Rule: estimated_minutes > DECOMPOSE_THRESHOLD (default 60) means decomposition hasn't happened yet. When the best candidates are all oversized, Lyra offers to break one down instead of proposing it — closing the anti-perfectionism loop through break_down, using the JSON import that already exists for exactly this.

B3 — surfacing

Reuse thoughts.py machinery; do not put projects in the threads table. A thought is her interiority; a project is a fact about Brian's life. Conflating them pollutes both — her dream cycle would generate the garage as a thought, and the garage would appear in her self-narrative. Separate storage, shared salience → surface → ping → response loop.

Lyra owns a small table keyed by BIT project id holding nudge state only: last surfaced, ignore count, snooze, salience. Cadence itself is read from BIT, never recomputed here.

Channel priority:

  1. Conversation (default). Next time Brian's talking to her anyway, she raises it: "the music backup's been sitting three weeks — dead, or just asleep?" Not a notification. A person asking. This is the unfair advantage no to-do app has.
  2. Push (rare, budgeted). Two legal uses only:
    • the receipt — "you've been in there 20 minutes, want me to count it?" (makes no demand, so there's nothing to blow off)
    • the backoff — "I've raised the garage twice and you didn't bite, so I'm putting it to sleep. Say the word and it wakes up."

No scheduled daily ping. A clock tick is the most habituation-prone trigger there is; she speaks when something is true.

Ignoring changes state: two ignored surfaces → dormant → she stops. Dormancy needs no BIT schema change — BIT already has per-project custom statuses.


Non-goals — explicitly parked

Named to hold the line, because "actually get this functional and helpful for my every day life" is the stated goal and feature creep is the named enemy.

  • The ambient card / desktop widget — the right long-term interface, but a surface on top of a thing that has to work first.
  • Generated wallpaper — same.
  • Windows active-window reporter — the "backwards timer" for physical and non-git work. Deferred past v1. Note it needs no Pieces: ~30 lines reporting active window title covers "which project am I in."
  • PiecesOS / MCP integration — richer, but a cross-machine dependency (PiecesOS is local to the Windows box, Lyra is on the homelab) that must be spiked before anything rests on it. Never load-bearing on day one.
  • Phone widget / home-screen surface — depends on iOS vs Android, unresolved.
  • Physical display on the rack — unblocked (a spare monitor exists; drive it with a Pi or old laptop running a kiosk browser, not by installing Xorg on the Proxmox host). It's a browser pointed at a URL this design already produces, so it changes nothing and can be added any time. Put it on the desk, not the rack.
  • Automatic blocker detection — Brian's dependency-graph idea (paint → move furniture → closet access → clothes + desk). Manual blockers are already built and deployed; auto-identifying them is the parked part, and it's a natural Lyra job later. This is the strongest answer to "where do I start" for physical projects — worth un-parking once v1 is honest.
  • Cross-device syncing sticky notes — a separate product. The sticky's form (a small always-visible card) is worth having and needs no sync, because its content is generated server-side. The sync engine is the whole product, and Trilium already does it.
  • % complete on long-horizon projects — actively harmful. See principle 1.

Error handling

  • BIT unreachable: every Lyra tool degrades to a clear "BIT is down" message. Never a stack trace into her context, never a fabricated answer about project state. Follows the notify.py precedent: a down dependency must not break the loop.
  • Git observer failure on one repo: log and continue. One bad repo must not abort a backfill.
  • Duplicate protection: the git_sessions unique key is the guard. A crashed mid-backfill run resumes cleanly.
  • Clock skew / commits with future dates: clamp to today; never create a session in the future.
  • Attribution failure: always falls back to Unfiled work. Never raises.

Testing

Split by repo, matching each one's existing conventions.

In BIT (pytest, backend)

  • Git observer: fixture repo built in a tmpdir with controlled commit timestamps. Assert day grouping across timezone boundaries, MIN/MAX/padding clamps, conventional-commit scope attribution, Unfiled work fallback, and — most importantly — that a second run produces zero new time logs.
  • Cadence functions: pure, table-driven. Boundary cases: no goal set, goal met exactly, dormancy threshold ±1 day, zero sessions ever, and the hours-named-but-minutes-valued goal fields.
  • /api/projects/health: shape and states against a seeded DB.

In project-lyra (pytest, existing tests/)

  • lyra/bit.py: against a stub server. Include a 404 case so a BIT API change can't silently break it, and assert graceful degradation when BIT is down.
  • Nudge state: ignore counts, backoff to dormant, quiet hours, push budget. Pure and deterministic — no model.
  • The pick and the surfacing copy: replay eval, parameterized by EVAL_BACKEND/EVAL_MODEL, matching the persona eval pattern. Assert it returns one task, and that it offers decomposition when candidates are oversized.

Open questions

  1. Ports for BIT on this box. :3002/:8002 are free here; confirm at deploy.
  2. Which of the 77 snapshotted tasks are worth reseeding, versus starting the real projects clean. Brian's call during A0.
  3. Whether weekly_hours_goal gets set per project at all in v1, or whether untracked + days-since-last-session is enough to start. Leaning: ship without goals, add them once real session data shows what a realistic cadence is.