OpenAI's export changed: conversations.json is now sharded into
conversations-000.json..NNN.json, each a JSON array of conversations with the
mapping tree and per-message create_time.
ingest now reads that format directly (supersedes the old convert/trim/split
scripts): walks each conversation's mapping ordered by create_time, keeps text
and multimodal_text (drops thoughts/reasoning_recap), captures real per-message
timestamps, and imports idempotently by conversation_id. `lyra-import <dir>`
auto-detects raw-export vs legacy {title,messages} dirs; optional limit arg.
Verified on 15 conversations: real dates, correct ordering, recall returns
dated poker history.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Import the parser's {title, messages} JSON into Lyra's memory so past
conversations seed recall (and, later, the era-rollup tier).
- lyra/ingest.py: one conversation -> one session, text messages -> exchanges;
skips non-text (image asset) messages and non user/assistant roles; embeddings
batched; idempotent by filename-derived session id; `lyra-import <dir>` CLI
- memory.add_exchanges_bulk: batched insert of pre-embedded rows
Format has no timestamps yet, so imports are stamped at import time; a future
dated export will let era memory group by real calendar time.
Verified on the 68-file lyra dev set: 7519 exchanges, idempotent re-run, recall
returns relevant history.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "context built" event now carries the fully-rendered prompt (persona, gists,
recalled details, recent turns, the new message) plus a total char count. The
log panel renders it as a collapsed "view full prompt" block — clean by default,
one click to see exactly what hit the model.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Older sessions fade to a general idea; details stay retrievable.
- memory: summaries table (one compacted gist per session, embedded), plus
store_summary/get_summary/recall_summaries and unsummarized_count (tracks
exchanges newer than the current summary)
- lyra/summary.py: summarize_session compacts a session's raw turns into a
third-person gist (default SUMMARY_BACKEND=local, so compaction is free);
maybe_summarize re-summarizes once SUMMARIZE_AFTER new turns accumulate
- chat.build_messages now layers context in tiers: persona -> gists of other
sessions -> a few sharp raw cross-session details -> current session raw
turns -> new message; respond() compacts the session after each turn
- web: POST /sessions/{id}/summarize to compact on demand
- summarization activity surfaces in the live log
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the inert "Show Work" thinking panel into a real live activity log:
- lyra/logbus.py: thread-safe in-memory ring buffer other modules publish to
- chat.respond logs backend/model/embed per turn, recall counts, reply size;
web layer logs chat errors
- server: replace the keep-alive /stream/thinking stub with /stream/logs, an
SSE endpoint that replays the recent buffer then streams new events
- UI: repurpose the panel as a global "Live Log" — connects on load, renders
level/time/msg/fields, drops the old per-session localStorage + dead popup
Every turn now shows its backend + model in-app, so local-vs-cloud (free vs
paid) is visible at a glance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 — persona + persistent memory chat loop:
- lyra/persona.py + personas/lyra.md: editable identity/voice (friend-first,
honest, never invents poker math)
- lyra/chat.py: turn loop assembling persona + cross-session recall + recent
context, persisting both sides to SQLite
- lyra/session.py, lyra/__main__.py: session lifecycle + `lyra` REPL
Phase 1.25 — reuse the old web UI:
- vendored the prior single-page UI into lyra/web/static, repointed to
same-origin
- lyra/web/server.py (FastAPI): serves the UI and backs its endpoint contract
(/v1/chat/completions, session CRUD, health, inert thinking-stream) with the
new chat loop + memory; SQLite stays the single source of truth
- `lyra-web` console script
Local backends — test for free, no OpenAI key:
- llm.embed routes via EMBED_BACKEND (cloud=OpenAI, local=Ollama /api/embed)
- simplified UI backend selector to Local (Ollama) / Cloud (OpenAI), default local
- memory connection opened check_same_thread=False for the threaded server
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- lyra.memory.remember(session_id, role, content) embeds and stores
- lyra.memory.recent(session_id, n) returns the last N from a session
- lyra.memory.recall(query, k, session_id=None) returns top-k by cosine
similarity across the chosen scope (all sessions by default)
- Embeddings live in the exchanges.embedding BLOB column as float32 bytes
- Connection reopens automatically if LYRA_DB_PATH changes (test-friendly)
- lyra.config.load() reads env into a frozen Config dataclass
- lyra.llm.complete(messages, backend) routes to Ollama /api/chat or
OpenAI chat completions
- lyra.llm.embed(texts) calls OpenAI embeddings
- .env.example switched from Anthropic to OpenAI to match available key
- Added `trillium.py` for searching and creating notes with Trillium's ETAPI.
- Implemented `search_notes` and `create_note` functions with appropriate error handling and validation.
feat: Add web search functionality using DuckDuckGo
- Introduced `web_search.py` for performing web searches without API keys.
- Implemented `search_web` function with result handling and validation.
feat: Create provider-agnostic function caller for iterative tool calling
- Developed `function_caller.py` to manage LLM interactions with tools.
- Implemented iterative calling logic with error handling and tool execution.
feat: Establish a tool registry for managing available tools
- Created `registry.py` to define and manage tool availability and execution.
- Integrated feature flags for enabling/disabling tools based on environment variables.
feat: Implement event streaming for tool calling processes
- Added `stream_events.py` to manage Server-Sent Events (SSE) for tool calling.
- Enabled real-time updates during tool execution for enhanced user experience.
test: Add tests for tool calling system components
- Created `test_tools.py` to validate functionality of code execution, web search, and tool registry.
- Implemented asynchronous tests to ensure proper execution and result handling.
chore: Add Dockerfile for sandbox environment setup
- Created `Dockerfile` to set up a Python environment with necessary dependencies for code execution.
chore: Add debug regex script for testing XML parsing
- Introduced `debug_regex.py` to validate regex patterns against XML tool calls.
chore: Add HTML template for displaying thinking stream events
- Created `test_thinking_stream.html` for visualizing tool calling events in a user-friendly format.
test: Add tests for OllamaAdapter XML parsing
- Developed `test_ollama_parser.py` to validate XML parsing with various test cases, including malformed XML.
serversdown
Claude
claude