Nail down which backend each LLM path uses, and make the dream cycle resilient to
a backend being down (the MI50 outage left profile/era/narrative — pinned to mi50
with no fallback — aborting every dream cycle).
- Consolidation (summaries + profile/era/narrative) -> cloud via SUMMARY_BACKEND=
cloud (.env, not committed). Matches the documented lesson that the MI50 is too
slow/hot for bulk consolidation; nothing background touches the card now.
- llm.complete_with_fallback(): try the primary backend, fall back to cloud on
error (re-raise if already cloud / no key). Wired into reflect + think so the
introspection voice (3090/dolphin) survives the gaming PC being powered off.
- dream coherence stage is now fault-isolated: a rebuild failure logs + continues
instead of sinking the whole pass (reflection still runs).
- .env: removed stale INTROSPECTION_BACKEND=mi50 (live routing is the web-switchable
introspection_mode DB setting = dolphin/3090; the var only fed a dead fallback).
Verified: forced cycle runs consolidation on cloud, introspection on the 3090,
completes with zero MI50 calls. 206 pass, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015yrEb5qpPGv2FjyxrB7LLk
Belt-and-suspenders so no dream pass can run unchecked for hours:
- llm.complete() now always bounds the OpenAI/mi50 request: default 300s +
max_retries=0 instead of the SDK's 600s x2 (~30 min). One change bounds every
consolidation/introspection call (profile/era/narrative/reflect/think), not
just summaries. Live chat (chat_call*) is a separate path, unaffected.
- dream_cycle() enforces a 20-min wall-clock budget, checked between stages;
once past it, remaining stages are skipped, it logs 'stopped early (over
budget)', and notify.push() pings Brian. Paired with the host watchdog (A) as
an independent fallback.
Tests: default timeout/max_retries threaded into complete(); an over-budget pass
skips later stages + pings. 178 pass, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015yrEb5qpPGv2FjyxrB7LLk
The dream cycle's summarize_all ran uncapped against the MI50: no max_tokens
and no timeout, so the OpenAI SDK's 600s x2-retry default meant ~30 min per
call. Combined with summary.py's own retry loop, one unsummarizable session
pegged the GPU for hours (observed 2026-07-04: stuck since 23:02, nothing saved
since 00:56, 7-8k-token runaway generations, all 4 llama.cpp slots busy). Not
context overflow (0 shifts/truncations) - purely unbounded length on a slow
backend timing out and retrying.
- llm.complete(): add optional max_tokens (caps generation; num_predict for
Ollama) and timeout (bounds the request and sets max_retries=0 so the caller
owns retry policy). Both default None -> unchanged for every existing caller.
- summary.py: cap gists at 768 tokens, 150s/call fast-fail, 2 MI50 attempts
then one cloud fallback (when primary isn't already cloud and a key exists).
Known limitation (scoped out per decision): the fallback triggers on
timeouts/exceptions, not on a degraded backend returning garbage as a 200.
Tests: fallback fires after 2 MI50 failures; no fallback when primary is cloud
or no key; cap+timeout threaded into every complete() call; llm bounds tests.
172 pass, ruff clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015yrEb5qpPGv2FjyxrB7LLk
Background MI50 work was invisible — the dream loop logs one line per cycle, so a
multi-minute consolidation or a chat-on-mi50 showed nothing while the GPU pegged.
Now complete/chat_call/chat_call_stream each emit 'llm call' (kind, backend, model,
~tokens) and 'llm done' (ms, output size, tools). Watch what's hitting any backend
live via journalctl --user -fu lyra-dream -u lyra-web. No signature change, so
test stubs that replace complete() are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Embeddings shared LOCAL_BASE_URL with the local chat backend (the 3090's Ollama),
so the 3090 being powered off killed all chat (every turn embeds to recall + to
store). Add a separate EMBED_BASE_URL (defaults to LOCAL_BASE_URL, so existing
setups are unchanged) and use it in llm.embed.
Deployed: a user-level Ollama (CPU) now runs nomic-embed-text on lyra-cortex
itself; EMBED_BASE_URL points at 127.0.0.1:11434 while LOCAL_BASE_URL still points
the local chat backend at the 3090. Local embeddings verified identical to the
3090's (cosine 0.999994, 768-dim) so existing vectors stay valid — no re-embed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- llm.chat_call_stream: streaming generator for all 3 backends (Ollama NDJSON,
OpenAI/MI50 SSE), accumulating tool-call fragments by index.
- chat.respond_stream: mirrors respond()'s tool loop and persistence/compaction,
yielding ("delta", text) / ("tool", name) / ("done", reply).
- POST /v1/chat/stream: SSE endpoint; blocking generator bridged to async via a
worker thread + asyncio.Queue. Old completions endpoint kept as fallback.
- Client streams into a live bubble with a blinking caret; rAF-throttled render
(no full re-parse per token) and instant scroll during stream — fixes iOS
Safari ghosting from per-token smooth-scroll. Falls back to the blocking
endpoint only if nothing streamed (no double-persist).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
She can now *do* things mid-conversation, not just reply. Adds a tool-calling
loop to the chat path and her first two tools; the same mechanism will carry the
poker tools (start_session, log_result, get_stats, solver) next.
- tools.py: registry of OpenAI-style tool specs + handlers + safe dispatch;
journal_write (knowing journaling) and note (tagged notepad, e.g. poker reads)
- llm.chat_call(): OpenAI-style call that returns tool_calls (cloud/mi50);
local has no tool support and returns plain content
- chat.respond(): tool loop — offer tools, run any calls, feed results back,
repeat until a text reply (capped at MAX_TOOL_ROUNDS); persists final reply
- tests: dispatch + full chat loop (tool call -> result -> reply)
Verified live: she invoked `note`, tagged it 'poker', stored a villain read.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mid-size models (gpt-4o-mini, qwen2.5-14b) resist persona instructions —
help-desk closers and feelings-disclaimers leak through regardless. Route live
chat to a stronger model while keeping bulk consolidation cheap:
- config: CHAT_MODEL (default gpt-4o), distinct from CLOUD_MODEL (gpt-4o-mini)
- llm.complete gains a `model` override; chat.respond uses chat_model on cloud,
consolidation paths keep cloud_model
- persona: reword the "no sign-off" rule so genuine questions are welcome and
only reflexive customer-service closers are discouraged
Verified: on gpt-4o she owns her mood without disclaimers and drops most
help-desk tails — clearly more in-character than mini/qwen.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MI50 box (CT202) runs an OpenAI-compatible llama.cpp server on
10.0.0.44:8080. Wire it in as a third backend:
- llm.complete gains backend="mi50" (OpenAI client pointed at MI50_BASE_URL)
- config: MI50_BASE_URL (default http://10.0.0.44:8080/v1) + MI50_MODEL
- chat.respond labels the model per backend; web _backend_for maps "mi50"
- UI backend selector adds "MI50 — local GPU"
Verified end-to-end: llm.complete(backend="mi50") returns from the live server.
See homelab-inference memory for the box topology.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 — persona + persistent memory chat loop:
- lyra/persona.py + personas/lyra.md: editable identity/voice (friend-first,
honest, never invents poker math)
- lyra/chat.py: turn loop assembling persona + cross-session recall + recent
context, persisting both sides to SQLite
- lyra/session.py, lyra/__main__.py: session lifecycle + `lyra` REPL
Phase 1.25 — reuse the old web UI:
- vendored the prior single-page UI into lyra/web/static, repointed to
same-origin
- lyra/web/server.py (FastAPI): serves the UI and backs its endpoint contract
(/v1/chat/completions, session CRUD, health, inert thinking-stream) with the
new chat loop + memory; SQLite stays the single source of truth
- `lyra-web` console script
Local backends — test for free, no OpenAI key:
- llm.embed routes via EMBED_BACKEND (cloud=OpenAI, local=Ollama /api/embed)
- simplified UI backend selector to Local (Ollama) / Cloud (OpenAI), default local
- memory connection opened check_same_thread=False for the threaded server
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- lyra.config.load() reads env into a frozen Config dataclass
- lyra.llm.complete(messages, backend) routes to Ollama /api/chat or
OpenAI chat completions
- lyra.llm.embed(texts) calls OpenAI embeddings
- .env.example switched from Anthropic to OpenAI to match available key
serversdown
Claude