Initial clean commit - unified Lyra stack

core/PROJECT_SUMMARY.md

@@ -0,0 +1,71 @@
+# Lyra Core — Project Summary
+
+## v0.4 (2025-10-03)
+
+### 🧠 High-Level Architecture
+- **Lyra Core (v0.3.1)** — Orchestration layer.  
+  - Accepts chat requests (`/v1/chat/completions`).  
+  - Routes through Cortex for subconscious annotation.  
+  - Stores everything in Mem0 (no discard).  
+  - Fetches persona + relevant memories.  
+  - Injects context back into LLM.  
+
+- **Cortex (v0.3.0)** — Subconscious annotator.  
+  - Runs locally via `llama.cpp` (Phi-3.5-mini Q4_K_M).  
+  - Strict JSON schema:  
+    ```json
+    {
+      "sentiment": "positive" | "neutral" | "negative",
+      "novelty": 0.0–1.0,
+      "tags": ["keyword", "keyword"],
+      "notes": "short string"
+    }
+    ```  
+  - Normalizes keys (lowercase).  
+  - Strips Markdown fences before parsing.  
+  - Configurable via `.env` (`CORTEX_ENABLED=true|false`).  
+  - Currently generates annotations, but not yet persisted into Mem0 payloads (stored as empty `{cortex:{}}`).  
+
+- **Mem0 (v0.4.0)** — Persistent memory layer.  
+  - Handles embeddings, graph storage, and retrieval.  
+  - Dual embedder support:  
+    - **OpenAI Cloud** (`text-embedding-3-small`, 1536-dim).  
+    - **HuggingFace TEI** (gte-Qwen2-1.5B-instruct, 1536-dim, hosted on 3090).  
+  - Environment toggle for provider (`.env.openai` vs `.env.3090`).  
+  - Memory persistence in Postgres (`payload` JSON).  
+  - CSV export pipeline confirmed (id, user_id, data, created_at).  
+
+- **Persona Sidecar**  
+  - Provides personality, style, and protocol instructions.  
+  - Injected at runtime into Core prompt building.  
+
+---
+
+### 🚀 Recent Changes
+- **Mem0**  
+  - Added HuggingFace TEI integration (local 3090 embedder).  
+  - Enabled dual-mode environment switch (OpenAI cloud ↔ local TEI).  
+  - Fixed `.env` line ending mismatch (CRLF vs LF).  
+  - Added memory dump/export commands for Postgres.  
+
+- **Core/Relay**  
+  - No major changes since v0.3.1 (still routing input → Cortex → Mem0).  
+
+- **Cortex**  
+  - Still outputs annotations, but not yet persisted into Mem0 payloads.  
+
+---
+
+### 📈 Versioning
+- **Lyra Core** → v0.3.1  
+- **Cortex** → v0.3.0  
+- **Mem0** → v0.4.0  
+
+---
+
+### 📋 Next Steps
+- [ ] Wire Cortex annotations into Mem0 payloads (`cortex` object).  
+- [ ] Add “export all memories” script to standard workflow.  
+- [ ] Consider async embedding for faster `mem.add`.  
+- [ ] Build visual diagram of data flow (Core ↔ Cortex ↔ Mem0 ↔ Persona).  
+- [ ] Explore larger LLMs for Cortex (Qwen2-7B, etc.) for richer subconscious annotation.