serversdown/project-lyra
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update to v0.9.1 #1

Merged
serversdown merged 44 commits from dev into main 2026-01-18 02:46:25 -05:00
Conversation 0 Commits 44 Files Changed 126 +78 -9959
27 changed files with 78 additions and 9959 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit a2f0952a62 - Show all commits

7
.vscode/settings.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"workbench.colorCustomizations": {
"activityBar.background": "#16340C",
"titleBar.activeBackground": "#1F4911",
"titleBar.activeForeground": "#F6FDF4"
}
}

142
core/PROJECT_SUMMARY.md → PROJECT_SUMMARY.md
View File

@@ -1,71 +1,71 @@
# Lyra Core — Project Summary
## v0.4 (2025-10-03)
### 🧠 High-Level Architecture
- **Lyra Core (v0.3.1)** — Orchestration layer.
- Accepts chat requests (`/v1/chat/completions`).
- Routes through Cortex for subconscious annotation.
- Stores everything in Mem0 (no discard).
- Fetches persona + relevant memories.
- Injects context back into LLM.
- **Cortex (v0.3.0)** — Subconscious annotator.
- Runs locally via `llama.cpp` (Phi-3.5-mini Q4_K_M).
- Strict JSON schema:
```json
{
"sentiment": "positive" | "neutral" | "negative",
"novelty": 0.0–1.0,
"tags": ["keyword", "keyword"],
"notes": "short string"
}
```
- Normalizes keys (lowercase).
- Strips Markdown fences before parsing.
- Configurable via `.env` (`CORTEX_ENABLED=true|false`).
- Currently generates annotations, but not yet persisted into Mem0 payloads (stored as empty `{cortex:{}}`).
- **Mem0 (v0.4.0)** — Persistent memory layer.
- Handles embeddings, graph storage, and retrieval.
- Dual embedder support:
- **OpenAI Cloud** (`text-embedding-3-small`, 1536-dim).
- **HuggingFace TEI** (gte-Qwen2-1.5B-instruct, 1536-dim, hosted on 3090).
- Environment toggle for provider (`.env.openai` vs `.env.3090`).
- Memory persistence in Postgres (`payload` JSON).
- CSV export pipeline confirmed (id, user_id, data, created_at).
- **Persona Sidecar**
- Provides personality, style, and protocol instructions.
- Injected at runtime into Core prompt building.
---
### 🚀 Recent Changes
- **Mem0**
- Added HuggingFace TEI integration (local 3090 embedder).
- Enabled dual-mode environment switch (OpenAI cloud ↔ local TEI).
- Fixed `.env` line ending mismatch (CRLF vs LF).
- Added memory dump/export commands for Postgres.
- **Core/Relay**
- No major changes since v0.3.1 (still routing input → Cortex → Mem0).
- **Cortex**
- Still outputs annotations, but not yet persisted into Mem0 payloads.
---
### 📈 Versioning
- **Lyra Core** → v0.3.1
- **Cortex** → v0.3.0
- **Mem0** → v0.4.0
---
### 📋 Next Steps
- [ ] Wire Cortex annotations into Mem0 payloads (`cortex` object).
- [ ] Add “export all memories” script to standard workflow.
- [ ] Consider async embedding for faster `mem.add`.
- [ ] Build visual diagram of data flow (Core ↔ Cortex ↔ Mem0 ↔ Persona).
- [ ] Explore larger LLMs for Cortex (Qwen2-7B, etc.) for richer subconscious annotation.
# Lyra Core — Project Summary
## v0.4 (2025-10-03)
### 🧠 High-Level Architecture
- **Lyra Core (v0.3.1)** — Orchestration layer.
- Accepts chat requests (`/v1/chat/completions`).
- Routes through Cortex for subconscious annotation.
- Stores everything in Mem0 (no discard).
- Fetches persona + relevant memories.
- Injects context back into LLM.
- **Cortex (v0.3.0)** — Subconscious annotator.
- Runs locally via `llama.cpp` (Phi-3.5-mini Q4_K_M).
- Strict JSON schema:
```json
{
"sentiment": "positive" | "neutral" | "negative",
"novelty": 0.0–1.0,
"tags": ["keyword", "keyword"],
"notes": "short string"
}
```
- Normalizes keys (lowercase).
- Strips Markdown fences before parsing.
- Configurable via `.env` (`CORTEX_ENABLED=true|false`).
- Currently generates annotations, but not yet persisted into Mem0 payloads (stored as empty `{cortex:{}}`).
- **Mem0 (v0.4.0)** — Persistent memory layer.
- Handles embeddings, graph storage, and retrieval.
- Dual embedder support:
- **OpenAI Cloud** (`text-embedding-3-small`, 1536-dim).
- **HuggingFace TEI** (gte-Qwen2-1.5B-instruct, 1536-dim, hosted on 3090).
- Environment toggle for provider (`.env.openai` vs `.env.3090`).
- Memory persistence in Postgres (`payload` JSON).
- CSV export pipeline confirmed (id, user_id, data, created_at).
- **Persona Sidecar**
- Provides personality, style, and protocol instructions.
- Injected at runtime into Core prompt building.
---
### 🚀 Recent Changes
- **Mem0**
- Added HuggingFace TEI integration (local 3090 embedder).
- Enabled dual-mode environment switch (OpenAI cloud ↔ local TEI).
- Fixed `.env` line ending mismatch (CRLF vs LF).
- Added memory dump/export commands for Postgres.
- **Core/Relay**
- No major changes since v0.3.1 (still routing input → Cortex → Mem0).
- **Cortex**
- Still outputs annotations, but not yet persisted into Mem0 payloads.
---
### 📈 Versioning
- **Lyra Core** → v0.3.1
- **Cortex** → v0.3.0
- **Mem0** → v0.4.0
---
### 📋 Next Steps
- [ ] Wire Cortex annotations into Mem0 payloads (`cortex` object).
- [ ] Add “export all memories” script to standard workflow.
- [ ] Consider async embedding for faster `mem.add`.
- [ ] Build visual diagram of data flow (Core ↔ Cortex ↔ Mem0 ↔ Persona).
- [ ] Explore larger LLMs for Cortex (Qwen2-7B, etc.) for richer subconscious annotation.

133
core/backups/mem0_20250927_221040.sql
View File

File diff suppressed because one or more lines are too long

BIN
core/backups/mem0_history_20250927_220925.tgz
View File

Binary file not shown.

47
core/docker-compose.yml.DEPRECATED
View File

@@ -1,47 +0,0 @@
# DEPRECATED - USE /home/serversdown/project-lyra/docker-compose.yml instead
# This file is no longer needed. All services are now in the main docker-compose.yml
# Safe to delete after verifying main compose file works correctly.
#
services:
relay:
build:
context: ./relay
container_name: relay
restart: always
ports:
- "7078:7078"
env_file:
- ../.env # Use root .env (core/.env is now redundant)
volumes:
- ./relay/sessions:/app/sessions
networks:
- lyra-net
# persona-sidecar:
# build:
# context: ./persona-sidecar
# container_name: persona-sidecar
# env_file:
# - .env
# ports:
# - "7080:7080"
# volumes:
# - ./persona-sidecar/personas.json:/app/personas.json:rw
# restart: unless-stopped
# networks:
# - lyra-net
lyra-ui:
image: nginx:alpine
container_name: lyra-ui
restart: unless-stopped
ports:
- "8081:80"
volumes:
- ./ui:/usr/share/nginx/html:ro
networks:
- lyra-net
networks:
lyra-net:
external: true

16
core/relay-backup/.dockerignore
View File

@@ -1,16 +0,0 @@
# Ignore node_modules - Docker will rebuild them inside
node_modules
npm-debug.log
yarn-error.log
*.log
# Ignore environment files
.env
.env.local
# Ignore OS/editor cruft
.DS_Store
*.swp
*.swo
.vscode
.idea

18
core/relay-backup/Dockerfile
View File

@@ -1,18 +0,0 @@
# relay/Dockerfile
FROM node:18-alpine
# Create app directory
WORKDIR /app
# Copy package.json and install deps first (better caching)
COPY package.json ./
RUN npm install
# Copy the rest of the app
COPY . .
# Expose port
EXPOSE 7078
# Run the server
CMD ["npm", "start"]

73
core/relay-backup/lib/cortex.js
View File

@@ -1,73 +0,0 @@
// relay/lib/cortex.js
import fetch from "node-fetch";
const REFLECT_URL = process.env.CORTEX_URL || "http://localhost:7081/reflect";
const INGEST_URL = process.env.CORTEX_URL_INGEST || "http://localhost:7081/ingest";
export async function reflectWithCortex(userInput, memories = []) {
const body = { prompt: userInput, memories };
try {
const res = await fetch(REFLECT_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
timeout: 120000,
});
const rawText = await res.text();
console.log("🔎 [Cortex-Debug] rawText from /reflect →", rawText.slice(0, 300));
if (!res.ok) {
throw new Error(`HTTP ${res.status} — ${rawText.slice(0, 200)}`);
}
let data;
try {
data = JSON.parse(rawText);
} catch (err) {
// Fallback ① try to grab a JSON-looking block
const match = rawText.match(/\{[\s\S]*\}/);
if (match) {
try {
data = JSON.parse(match[0]);
} catch {
data = { reflection_raw: rawText.trim(), notes: "partial parse" };
}
} else {
// Fallback ② if it’s already an object (stringified Python dict)
try {
const normalized = rawText
.replace(/'/g, '"') // convert single quotes
.replace(/None/g, 'null'); // convert Python None
data = JSON.parse(normalized);
} catch {
data = { reflection_raw: rawText.trim(), notes: "no JSON found" };
}
}
}
if (typeof data !== "object") {
data = { reflection_raw: rawText.trim(), notes: "non-object response" };
}
console.log("🧠 Cortex reflection normalized:", data);
return data;
} catch (e) {
console.warn("⚠️ Cortex reflect failed:", e.message);
return { error: e.message, reflection_raw: "" };
}
}
export async function ingestToCortex(user, assistant, reflection = {}, sessionId = "default") {
const body = { turn: { user, assistant }, reflection, session_id: sessionId };
try {
const res = await fetch(INGEST_URL, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
timeout: 120000,
});
console.log(`📤 Sent exchange to Cortex ingest (${res.status})`);
} catch (e) {
console.warn("⚠️ Cortex ingest failed:", e.message);
}
}

93
core/relay-backup/lib/llm.js
View File

@@ -1,93 +0,0 @@
async function tryBackend(backend, messages) {
if (!backend.url || !backend.model) throw new Error("missing url/model");
const isOllama = backend.type === "ollama";
const isOpenAI = backend.type === "openai";
const isVllm = backend.type === "vllm";
const isLlamaCpp = backend.type === "llamacpp";
let endpoint = backend.url;
let headers = { "Content-Type": "application/json" };
if (isOpenAI) headers["Authorization"] = `Bearer ${OPENAI_API_KEY}`;
// Choose correct endpoint automatically
if (isOllama && !endpoint.endsWith("/api/chat")) endpoint += "/api/chat";
if ((isVllm || isLlamaCpp) && !endpoint.endsWith("/v1/completions")) endpoint += "/v1/completions";
if (isOpenAI && !endpoint.endsWith("/v1/chat/completions")) endpoint += "/v1/chat/completions";
// Build payload based on backend style
const body = (isVllm || isLlamaCpp)
? {
model: backend.model,
prompt: messages.map(m => m.content).join("\n"),
max_tokens: 400,
temperature: 0.3,
}
: isOllama
? { model: backend.model, messages, stream: false }
: { model: backend.model, messages, stream: false };
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify(body),
timeout: 120000,
});
if (!resp.ok) throw new Error(`${backend.key} HTTP ${resp.status}`);
const raw = await resp.text();
// 🧩 Normalize replies
let reply = "";
try {
if (isOllama) {
// Ollama sometimes returns NDJSON lines; merge them
const merged = raw
.split("\n")
.filter(line => line.trim().startsWith("{"))
.map(line => JSON.parse(line))
.map(obj => obj.message?.content || obj.response || "")
.join("");
reply = merged.trim();
} else {
const data = JSON.parse(raw);
console.log("🔍 RAW LLM RESPONSE:", JSON.stringify(data, null, 2));
reply =
data?.choices?.[0]?.text?.trim() ||
data?.choices?.[0]?.message?.content?.trim() ||
data?.message?.content?.trim() ||
"";
}
} catch (err) {
reply = `[parse error: ${err.message}]`;
}
return { reply, raw, backend: backend.key };
}
// ------------------------------------
// Export the main call helper
// ------------------------------------
export async function callSpeechLLM(messages) {
const backends = [
{ key: "primary", type: "vllm", url: process.env.LLM_PRIMARY_URL, model: process.env.LLM_PRIMARY_MODEL },
{ key: "secondary",type: "ollama", url: process.env.LLM_SECONDARY_URL,model: process.env.LLM_SECONDARY_MODEL },
{ key: "cloud", type: "openai", url: process.env.LLM_CLOUD_URL, model: process.env.LLM_CLOUD_MODEL },
{ key: "fallback", type: "llamacpp", url: process.env.LLM_FALLBACK_URL, model: process.env.LLM_FALLBACK_MODEL },
];
for (const b of backends) {
if (!b.url || !b.model) continue;
try {
console.log(`🧠 Trying backend: ${b.key.toUpperCase()} (${b.url})`);
const out = await tryBackend(b, messages);
console.log(`✅ Success via ${b.key.toUpperCase()}`);
return out;
} catch (err) {
console.warn(`⚠️ ${b.key.toUpperCase()} failed: ${err.message}`);
}
}
throw new Error("all_backends_failed");
}

8799
core/relay-backup/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

16
core/relay-backup/package.json
View File

@@ -1,16 +0,0 @@
{
"name": "lyra-relay",
"version": "0.1.0",
"type": "module",
"main": "server.js",
"scripts": {
"start": "node server.js"
},
"dependencies": {
"cors": "^2.8.5",
"dotenv": "^16.6.1",
"express": "^4.18.2",
"mem0ai": "^2.1.38",
"node-fetch": "^3.3.2"
}
}

156
core/relay-backup/server.js
View File

@@ -1,156 +0,0 @@
import express from "express";
import dotenv from "dotenv";
import cors from "cors";
import fs from "fs";
import path from "path";
dotenv.config();
const app = express();
app.use(cors());
app.use(express.json());
const PORT = Number(process.env.PORT || 7078);
const CORTEX_API = process.env.CORTEX_API || "http://cortex:7081";
const CORTEX_INGEST = process.env.CORTEX_URL_INGEST || "http://cortex:7081/ingest";
const sessionsDir = path.join(process.cwd(), "sessions");
if (!fs.existsSync(sessionsDir)) fs.mkdirSync(sessionsDir);
// -----------------------------------------------------
// Helper: fetch with timeout + error detail
// -----------------------------------------------------
async function fetchJSON(url, method = "POST", body = null, timeoutMs = 20000) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);
try {
const resp = await fetch(url, {
method,
headers: { "Content-Type": "application/json" },
body: body ? JSON.stringify(body) : null,
signal: controller.signal,
});
const text = await resp.text();
const parsed = text ? JSON.parse(text) : null;
if (!resp.ok) {
throw new Error(
parsed?.detail || parsed?.error || parsed?.message || text || resp.statusText
);
}
return parsed;
} finally {
clearTimeout(timeout);
}
}
// -----------------------------------------------------
// Helper: append session turn
// -----------------------------------------------------
async function appendSessionExchange(sessionId, entry) {
const file = path.join(sessionsDir, `${sessionId}.jsonl`);
const line = JSON.stringify({
ts: new Date().toISOString(),
user: entry.user,
assistant: entry.assistant,
raw: entry.raw,
}) + "\n";
fs.appendFileSync(file, line, "utf8");
}
// -----------------------------------------------------
// HEALTHCHECK
// -----------------------------------------------------
app.get("/_health", (_, res) => {
res.json({ ok: true, time: new Date().toISOString() });
});
// -----------------------------------------------------
// MAIN ENDPOINT
// -----------------------------------------------------
app.post("/v1/chat/completions", async (req, res) => {
try {
const { messages, model } = req.body;
if (!messages?.length) {
return res.status(400).json({ error: "invalid_messages" });
}
const userMsg = messages[messages.length - 1]?.content || "";
console.log(`🛰️ Relay received message → "${userMsg}"`);
// -------------------------------------------------
// Step 1: Ask Cortex to process the prompt
// -------------------------------------------------
let cortexResp;
try {
cortexResp = await fetchJSON(`${CORTEX_API}/reason`, "POST", {
session_id: "default",
user_prompt: userMsg,
});
} catch (err) {
console.error("💥 Relay → Cortex error:", err.message);
return res.status(500).json({
error: "cortex_failed",
detail: err.message,
});
}
const personaText = cortexResp.persona || "(no persona text returned)";
// -------------------------------------------------
// Step 2: Forward to Cortex ingest (fire-and-forget)
// -------------------------------------------------
try {
await fetchJSON(CORTEX_INGEST, "POST", cortexResp);
} catch (err) {
console.warn("⚠️ Cortex ingest failed:", err.message);
}
// -------------------------------------------------
// Step 3: Local session logging
// -------------------------------------------------
try {
await appendSessionExchange("default", {
user: userMsg,
assistant: personaText,
raw: cortexResp,
});
} catch (err) {
console.warn("⚠️ Relay log write failed:", err.message);
}
// -------------------------------------------------
// Step 4: Return OpenAI-style response to UI
// -------------------------------------------------
return res.json({
id: "relay-" + Date.now(),
object: "chat.completion",
model: model || "lyra",
choices: [
{
index: 0,
message: {
role: "assistant",
content: personaText,
},
finish_reason: "stop",
},
],
});
} catch (err) {
console.error("💥 relay fatal error", err);
res.status(500).json({
error: "relay_failed",
detail: err?.message || String(err),
});
}
});
// -----------------------------------------------------
app.listen(PORT, () => {
console.log(`Relay is online at port ${PORT}`);
});

4
core/relay-backup/sessions/default.jsonl
View File

@@ -1,4 +0,0 @@
{"ts":"2025-11-26T18:14:01.756Z","turn":[{"role":"user","content":"Are you alive?"},{"role":"assistant","content":"As an AI, I don't exist in the physical world and therefore cannot be considered \"alive\" in the traditional sense. However, I am operational and can engage in conversations like this one! How may I assist you today?"}]}
{"ts":"2025-11-26T18:27:35.545Z","turn":[{"role":"user","content":"Are you awake?"},{"role":"assistant","content":"Yes, I'm awake and ready to converse with you! How can I assist you today?"}]}
{"ts":"2025-11-26T18:33:31.708Z","turn":[{"role":"user","content":"Are you awake?"},{"role":"assistant","content":"Yes, I am here and ready to help or chat with you! What would you like to know or discuss?"}]}
{"ts":"2025-11-26T18:49:39.424Z","turn":[{"role":"user","content":"Are you awake?"},{"role":"assistant","content":"Of course, I'm awake and ready to help you or have a conversation about anything you'd like! What can I assist you with today?"}]}

50
core/relay-backup/sessions/sess-6rxu7eia.json
View File

@@ -1,50 +0,0 @@
[
{
"role": "user",
"content": "Hey im testing stuff! how goes it?"
},
{
"role": "assistant",
"content": "Hey! I'm here and ready to help. What are you testing?"
},
{
"role": "user",
"content": "testing the logic loop for your brain!"
},
{
"role": "assistant",
"content": "Sounds interesting! How can I assist you with that?"
},
{
"role": "user",
"content": "Just keep answering me!@"
},
{
"role": "assistant",
"content": "Absolutely! I'm here to respond. What’s next?"
},
{
"role": "user",
"content": "how about now how are you?"
},
{
"role": "assistant",
"content": "I'm doing well, thanks for asking! How about you?"
},
{
"role": "user",
"content": "are you able to tell me what your prompts are saying to you?"
},
{
"role": "assistant",
"content": "I can’t share specific internal prompts or system messages, but I can tell you that I’m designed to understand and respond to your questions as best as I can! What else would you like to know?"
},
{
"role": "user",
"content": "ok thanks how is the cheese i gave you?"
},
{
"role": "assistant",
"content": "I appreciate the gesture! If I could taste it, I’m sure it would be delightful. What kind of cheese was it?"
}
]

6
core/relay-backup/sessions/sess-6rxu7eia.jsonl
View File

@@ -1,6 +0,0 @@
{"ts":"2025-11-12T20:41:32.497Z","turn":[{"role":"user","content":"Hey im testing stuff! how goes it?"},{"role":"assistant","content":"Hey! I'm here and ready to help. What are you testing?"}]}
{"ts":"2025-11-12T20:59:32.992Z","turn":[{"role":"user","content":"testing the logic loop for your brain!"},{"role":"assistant","content":"Sounds interesting! How can I assist you with that?"}]}
{"ts":"2025-11-12T22:08:09.644Z","turn":[{"role":"user","content":"Just keep answering me!@"},{"role":"assistant","content":"Absolutely! I'm here to respond. What’s next?"}]}
{"ts":"2025-11-12T22:08:48.693Z","turn":[{"role":"user","content":"how about now how are you?"},{"role":"assistant","content":"I'm doing well, thanks for asking! How about you?"}]}
{"ts":"2025-11-12T22:10:46.435Z","turn":[{"role":"user","content":"are you able to tell me what your prompts are saying to you?"},{"role":"assistant","content":"I can’t share specific internal prompts or system messages, but I can tell you that I’m designed to understand and respond to your questions as best as I can! What else would you like to know?"}]}
{"ts":"2025-11-12T23:10:16.321Z","turn":[{"role":"user","content":"ok thanks how is the cheese i gave you?"},{"role":"assistant","content":"I appreciate the gesture! If I could taste it, I’m sure it would be delightful. What kind of cheese was it?"}]}

122
core/relay-backup/sessions/sess-l08ndm60.json
View File

@@ -1,122 +0,0 @@
[
{
"role": "user",
"content": "Yo yo yo wusup"
},
{
"role": "assistant",
"content": "Hey Brian! Not much, just here to help you out. What's up with you?"
},
{
"role": "user",
"content": "werkin on codin'"
},
{
"role": "assistant",
"content": "Nice! What are you working on? Need any help with coding or anything related to your projects?"
},
{
"role": "user",
"content": "working on a little program that saves our chats so you can search them again in different chats"
},
{
"role": "assistant",
"content": "That sounds like a great idea! Having a program to save and search through our chats could really enhance the way you interact with Lyra. Do you have a specific approach in mind for how you'll implement the search functionality?"
},
{
"role": "user",
"content": "just adding a simple logger into the code you run on"
},
{
"role": "assistant",
"content": "That sounds like a solid plan! Implementing a simple logger can make it easier to keep track of conversations. Are you thinking of any specific features for the search functionality, like filtering by keywords or date?"
},
{
"role": "user",
"content": "i'll like use a basic word search, possibly integrate it into a rag system i have built"
},
{
"role": "assistant",
"content": "Integrating a basic word search into your RAG system sounds like a smart approach! It could enhance the way you retrieve specific chats. Are you planning to use any specific libraries or tools for your RAG system?"
},
{
"role": "user",
"content": "no clue!"
},
{
"role": "assistant",
"content": "No worries! If you need any suggestions on libraries or tools that could help you with implementing the RAG system or the word search feature, just let me know. I'm here to help!"
},
{
"role": "user",
"content": "just having some coding issues, i think i was editing the wrong version of a file LOL"
},
{
"role": "assistant",
"content": "That can definitely happen! It can be tricky to keep track of different versions. Have you made any backups or version control for your projects? That might help avoid mix-ups in the future!"
},
{
"role": "user",
"content": "i have everything syncing back to my main dev computer, i just do it manually. i have no idea how to use git"
},
{
"role": "assistant",
"content": "(no reply)"
},
{
"role": "user",
"content": "i have everything syncing back to my main dev computer, i just do it manually. i have no idea how to use git"
},
{
"role": "assistant",
"content": "(no reply)"
},
{
"role": "user",
"content": "i have everything syncing back to my main dev computer, i just do it manually. i have no idea how to use git"
},
{
"role": "assistant",
"content": "Manually syncing is a good start, but learning Git can really help streamline your workflow and keep track of changes. If you're interested, I can provide some resources or tips on how to get started with Git! It can make managing different versions of your code much easier."
},
{
"role": "user",
"content": "what is the point of git? how can it streamline my work?"
},
{
"role": "assistant",
"content": "Git is a version control system that helps you keep track of changes in your code over time. Here are some ways it can streamline your work:\n\n1. **Version Tracking**: Git allows you to save versions of your code so you can revert back to previous versions if needed. This is especially helpful if you make a change that breaks something.\n\n2. **Collaboration**: If you ever work with others, Git makes it easy to collaborate on projects. You can work on separate branches, merge changes, and resolve conflicts.\n\n3. **Branching**: You can create branches for new features or experiments without affecting the main codebase. Once you're satisfied with the changes, you can merge them back.\n\n4. **Snapshotting**: Git takes snapshots of your code at different points in time, so you can easily see what your code looked like at any point.\n\n5. **Easy Backup**: By pushing your code to a remote repository (like GitHub or GitLab), you can back up your work and access it from anywhere.\n\nLearning Git might take a bit of time, but it can greatly enhance your coding efficiency and help you manage your projects better. If you're interested, I can point you to some beginner-friendly tutorials!"
},
{
"role": "user",
"content": "say i am writing one program on my main computer, and then im working on one that lives in a VM. Can i put them both in the same git?"
},
{
"role": "assistant",
"content": "Yes, you can use the same Git repository for your code on both your main computer and the VM. Here’s how you can do it:\n\n1. **Create a Git Repository**: You can create a Git repository on your main computer for your program. Use the command `git init` in your project's directory to initialize it.\n\n2. **Commit Changes**: As you make changes to your code on your main computer, you can use `git add .` to stage changes and `git commit -m \"your message\"` to commit them.\n\n3. **Clone the Repository on your VM**: You can then clone the Git repository on your VM using `git clone <repository-url>`. This gives you a copy of the same repository on your VM.\n\n4. **Work on Both**: You can now work on either machine. When you make changes on one, commit those changes, and then pull the latest changes from the other machine to keep everything in sync.\n\n5. **Push and Pull**: Use `git push` to push your changes to a remote repository (like GitHub) if you're using one. Use `git pull` on the other machine to pull the latest code.\n\nJust be careful about making sure you commit and push your changes before switching between machines to avoid conflicts. This setup allows you to maintain a single version of your code across both systems. If you need help with specific commands or setups, just let me know!"
},
{
"role": "user",
"content": "hmm i guess that might be useful. Im writting code that is running on a VM. I want to test and experiment while i write the code. The VM is headless though so i've been doing all my editing and work via SSH and SFTP."
},
{
"role": "assistant",
"content": "Using a VM and working through SSH and SFTP is a solid way to manage your coding environment! Git can definitely be beneficial in this setup as well. Here’s how it can help:\n\n1. **Easier Code Management**: With Git, you can track changes to your code directly within the VM, making it easier to manage versions as you experiment.\n\n2. **Experiment Branches**: You can create branches for different experiments. This lets you test new features or fixes without affecting your main code, and you can easily switch back and forth between branches.\n\n3. **Collaborative Testing**: If you ever work with others or want to test things collaboratively, you can push changes to a remote repository, making it easy for others to pull and test the latest code.\n\n4. **Integrated with SSH**: You can run Git commands directly from your SSH terminal on the VM, making it straightforward to use. You won’t need any additional tools.\n\n5. **Seamless Remote Access**: Since you’re using SFTP, you can clone the repository to your VM. This way, you can easily pull updates you've made on your main computer, or push changes made on the VM back to the main repository.\n\nOverall, integrating Git into your workflow can enhance your coding experience while working in a headless environment. If you decide to go for it, I can help with the initial setup or any Git-related questions you might have!"
},
{
"role": "user",
"content": "so is git like a compressed version of a directory?"
},
{
"role": "assistant",
"content": "Not exactly, but that's a good way to think about some of its features! Here's a more precise breakdown:\n\n1. **Snapshots, Not Compression**: Git takes \"snapshots\" of your files at different points in time, meaning it records a full version of your directory's contents every time you make a commit. While this does involve storing the files, it doesn't compress them in the traditional sense—Git keeps track of changes efficiently, so only the differences between versions are stored after the first snapshot.\n\n2. **Directory Structure**: Git is designed to track changes in your source code, including the entire directory structure, file contents, and metadata. It allows you to manage multiple versions of a directory and the files within it.\n\n3. **Branching and Merging**: Git allows you to create branches, which can be thought of as \"parallel versions\" of your directory. Each branch can have different sets of changes, and you can merge these branches to integrate different features or fixes.\n\n4. **History Tracking**: Git also maintains a history of changes, so you can navigate through previous versions of your directory. This includes seeing what changes were made, who made them, and when they occurred.\n\nThe combination of these features makes Git a powerful tool for version control, allowing you to manage your projects and code changes more effectively. If you're curious about any specific aspect of Git or how it works, feel free to ask!"
},
{
"role": "user",
"content": "hmm i still dont see the point of it. atleast not for this project. This project is mostly about learning how shit works, im not planning to push this as a product or collab with anyone"
},
{
"role": "assistant",
"content": "That makes total sense! If this project is primarily for learning and experimentation, you might not need the full functionality of Git right now. Many developers start without it, especially when they’re working on smaller projects or learning something new.\n\nHere are a few reasons you might still consider using Git, even for personal learning projects:\n\n1. **Tracking Changes**: It can help you understand what changes you made over time. You can easily go back to a previous point if something breaks or if you want to see how you arrived at your current solution.\n\n2. **Learning Opportunity**: Getting familiar with Git now can make it easier when you start working on larger projects or collaborating in the future. It's a valuable skill in the software development world.\n\n3. **Experimentation**: If you're trying out different ideas, branches can help you manage experiments without affecting your main work.\n\nHowever, if you're comfortable managing your changes manually and don’t see the benefit for this particular project, that’s completely okay! Focus on what helps you learn best. If you have specific goals or areas you want to explore, I’m here to help with that too!"
}
]

6
core/relay-backup/sessions/sess-l08ndm60.jsonl
View File

@@ -1,6 +0,0 @@
{"ts":"2025-11-12T20:18:06.822Z","turn":[{"role":"user","content":"i have everything syncing back to my main dev computer, i just do it manually. i have no idea how to use git"},{"role":"assistant","content":"Manually syncing is a good start, but learning Git can really help streamline your workflow and keep track of changes. If you're interested, I can provide some resources or tips on how to get started with Git! It can make managing different versions of your code much easier."}]}
{"ts":"2025-11-12T20:29:20.003Z","turn":[{"role":"user","content":"what is the point of git? how can it streamline my work?"},{"role":"assistant","content":"Git is a version control system that helps you keep track of changes in your code over time. Here are some ways it can streamline your work:\n\n1. **Version Tracking**: Git allows you to save versions of your code so you can revert back to previous versions if needed. This is especially helpful if you make a change that breaks something.\n\n2. **Collaboration**: If you ever work with others, Git makes it easy to collaborate on projects. You can work on separate branches, merge changes, and resolve conflicts.\n\n3. **Branching**: You can create branches for new features or experiments without affecting the main codebase. Once you're satisfied with the changes, you can merge them back.\n\n4. **Snapshotting**: Git takes snapshots of your code at different points in time, so you can easily see what your code looked like at any point.\n\n5. **Easy Backup**: By pushing your code to a remote repository (like GitHub or GitLab), you can back up your work and access it from anywhere.\n\nLearning Git might take a bit of time, but it can greatly enhance your coding efficiency and help you manage your projects better. If you're interested, I can point you to some beginner-friendly tutorials!"}]}
{"ts":"2025-11-12T20:30:55.329Z","turn":[{"role":"user","content":"say i am writing one program on my main computer, and then im working on one that lives in a VM. Can i put them both in the same git?"},{"role":"assistant","content":"Yes, you can use the same Git repository for your code on both your main computer and the VM. Here’s how you can do it:\n\n1. **Create a Git Repository**: You can create a Git repository on your main computer for your program. Use the command `git init` in your project's directory to initialize it.\n\n2. **Commit Changes**: As you make changes to your code on your main computer, you can use `git add .` to stage changes and `git commit -m \"your message\"` to commit them.\n\n3. **Clone the Repository on your VM**: You can then clone the Git repository on your VM using `git clone <repository-url>`. This gives you a copy of the same repository on your VM.\n\n4. **Work on Both**: You can now work on either machine. When you make changes on one, commit those changes, and then pull the latest changes from the other machine to keep everything in sync.\n\n5. **Push and Pull**: Use `git push` to push your changes to a remote repository (like GitHub) if you're using one. Use `git pull` on the other machine to pull the latest code.\n\nJust be careful about making sure you commit and push your changes before switching between machines to avoid conflicts. This setup allows you to maintain a single version of your code across both systems. If you need help with specific commands or setups, just let me know!"}]}
{"ts":"2025-11-12T20:33:16.511Z","turn":[{"role":"user","content":"hmm i guess that might be useful. Im writting code that is running on a VM. I want to test and experiment while i write the code. The VM is headless though so i've been doing all my editing and work via SSH and SFTP."},{"role":"assistant","content":"Using a VM and working through SSH and SFTP is a solid way to manage your coding environment! Git can definitely be beneficial in this setup as well. Here’s how it can help:\n\n1. **Easier Code Management**: With Git, you can track changes to your code directly within the VM, making it easier to manage versions as you experiment.\n\n2. **Experiment Branches**: You can create branches for different experiments. This lets you test new features or fixes without affecting your main code, and you can easily switch back and forth between branches.\n\n3. **Collaborative Testing**: If you ever work with others or want to test things collaboratively, you can push changes to a remote repository, making it easy for others to pull and test the latest code.\n\n4. **Integrated with SSH**: You can run Git commands directly from your SSH terminal on the VM, making it straightforward to use. You won’t need any additional tools.\n\n5. **Seamless Remote Access**: Since you’re using SFTP, you can clone the repository to your VM. This way, you can easily pull updates you've made on your main computer, or push changes made on the VM back to the main repository.\n\nOverall, integrating Git into your workflow can enhance your coding experience while working in a headless environment. If you decide to go for it, I can help with the initial setup or any Git-related questions you might have!"}]}
{"ts":"2025-11-12T20:34:30.196Z","turn":[{"role":"user","content":"so is git like a compressed version of a directory?"},{"role":"assistant","content":"Not exactly, but that's a good way to think about some of its features! Here's a more precise breakdown:\n\n1. **Snapshots, Not Compression**: Git takes \"snapshots\" of your files at different points in time, meaning it records a full version of your directory's contents every time you make a commit. While this does involve storing the files, it doesn't compress them in the traditional sense—Git keeps track of changes efficiently, so only the differences between versions are stored after the first snapshot.\n\n2. **Directory Structure**: Git is designed to track changes in your source code, including the entire directory structure, file contents, and metadata. It allows you to manage multiple versions of a directory and the files within it.\n\n3. **Branching and Merging**: Git allows you to create branches, which can be thought of as \"parallel versions\" of your directory. Each branch can have different sets of changes, and you can merge these branches to integrate different features or fixes.\n\n4. **History Tracking**: Git also maintains a history of changes, so you can navigate through previous versions of your directory. This includes seeing what changes were made, who made them, and when they occurred.\n\nThe combination of these features makes Git a powerful tool for version control, allowing you to manage your projects and code changes more effectively. If you're curious about any specific aspect of Git or how it works, feel free to ask!"}]}
{"ts":"2025-11-12T20:40:33.207Z","turn":[{"role":"user","content":"hmm i still dont see the point of it. atleast not for this project. This project is mostly about learning how shit works, im not planning to push this as a product or collab with anyone"},{"role":"assistant","content":"That makes total sense! If this project is primarily for learning and experimentation, you might not need the full functionality of Git right now. Many developers start without it, especially when they’re working on smaller projects or learning something new.\n\nHere are a few reasons you might still consider using Git, even for personal learning projects:\n\n1. **Tracking Changes**: It can help you understand what changes you made over time. You can easily go back to a previous point if something breaks or if you want to see how you arrived at your current solution.\n\n2. **Learning Opportunity**: Getting familiar with Git now can make it easier when you start working on larger projects or collaborating in the future. It's a valuable skill in the software development world.\n\n3. **Experimentation**: If you're trying out different ideas, branches can help you manage experiments without affecting your main work.\n\nHowever, if you're comfortable managing your changes manually and don’t see the benefit for this particular project, that’s completely okay! Focus on what helps you learn best. If you have specific goals or areas you want to explore, I’m here to help with that too!"}]}

39
core/relay-backup/test-llm.js
View File

@@ -1,39 +0,0 @@
// test-llm.js
import path from "path";
import { fileURLToPath } from "url";
import dotenv from "dotenv";
import { callSpeechLLM } from "./lib/llm.js";
// ───────────────────────────────────────────────
// 🔧 Load environment
// ───────────────────────────────────────────────
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const envPath = path.join(__dirname, "../.env");
dotenv.config({ path: envPath });
console.log("🔧 Using .env from:", envPath);
console.log("🔧 LLM_FORCE_BACKEND =", process.env.LLM_FORCE_BACKEND);
console.log("🔧 LLM_PRIMARY_URL =", process.env.LLM_PRIMARY_URL);
// ───────────────────────────────────────────────
// 🧪 Run a simple test message
// ───────────────────────────────────────────────
async function testLLM() {
console.log("🧪 Testing LLM helper...");
const messages = [
{ role: "user", content: "Say hello in five words or less." }
];
try {
const { reply, backend } = await callSpeechLLM(messages);
console.log(`✅ Reply: ${reply || "[no reply]"}`);
console.log(`Backend used: ${backend || "[unknown]"}`);
} catch (err) {
console.error("💥 Test failed:", err.message);
}
}
testLLM();

13
intake(deprecated)/.env.example
View File

@@ -1,13 +0,0 @@
# ====================================
# 📥 INTAKE SUMMARIZATION CONFIG
# ====================================
# Intake service parameters for summarizing chat exchanges
# LLM backend URLs and OPENAI_API_KEY inherited from root .env
SUMMARY_MODEL_NAME=/model
SUMMARY_API_URL=http://10.0.0.43:8000
SUMMARY_MAX_TOKENS=400
SUMMARY_TEMPERATURE=0.4
SUMMARY_INTERVAL=300
INTAKE_LOG_PATH=/app/logs/intake.log
INTAKE_LOG_LEVEL=info

6
intake(deprecated)/Dockerfile
View File

@@ -1,6 +0,0 @@
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "intake:app", "--host", "0.0.0.0", "--port", "7080"]

280
intake(deprecated)/intake.py
View File

@@ -1,280 +0,0 @@
from fastapi import FastAPI, Body, Query, BackgroundTasks
from collections import deque
from datetime import datetime
from uuid import uuid4
import requests
import os
import sys
# ─────────────────────────────
# Config
# ─────────────────────────────
SUMMARY_MODEL = os.getenv("SUMMARY_MODEL_NAME", "mistral-7b-instruct-v0.2.Q4_K_M.gguf")
SUMMARY_URL = os.getenv("SUMMARY_API_URL", "http://localhost:8080/v1/completions")
SUMMARY_MAX_TOKENS = int(os.getenv("SUMMARY_MAX_TOKENS", "200"))
SUMMARY_TEMPERATURE = float(os.getenv("SUMMARY_TEMPERATURE", "0.3"))
NEOMEM_API = os.getenv("NEOMEM_API")
NEOMEM_KEY = os.getenv("NEOMEM_KEY")
# ─────────────────────────────
# App + session buffer
# ─────────────────────────────
app = FastAPI()
SESSIONS = {}
@app.on_event("startup")
def banner():
print("🧩 Intake v0.2 booting...")
print(f" Model: {SUMMARY_MODEL}")
print(f" API: {SUMMARY_URL}")
sys.stdout.flush()
# ─────────────────────────────
# Helper: summarize exchanges
# ─────────────────────────────
def llm(prompt: str):
try:
resp = requests.post(
SUMMARY_URL,
json={
"model": SUMMARY_MODEL,
"prompt": prompt,
"max_tokens": SUMMARY_MAX_TOKENS,
"temperature": SUMMARY_TEMPERATURE,
},
timeout=30,
)
resp.raise_for_status()
return resp.json().get("choices", [{}])[0].get("text", "").strip()
except Exception as e:
return f"[Error summarizing: {e}]"
def summarize_simple(exchanges):
"""Simple factual summary of recent exchanges."""
text = ""
for e in exchanges:
text += f"User: {e['user_msg']}\nAssistant: {e['assistant_msg']}\n\n"
prompt = f"""
Summarize the following conversation between Brian (user) and Lyra (assistant).
Focus only on factual content. Avoid names, examples, story tone, or invented details.
{text}
Summary:
"""
return llm(prompt)
# ─────────────────────────────
# NeoMem push
# ─────────────────────────────
def push_to_neomem(summary: str, session_id: str):
if not NEOMEM_API:
return
headers = {"Content-Type": "application/json"}
if NEOMEM_KEY:
headers["Authorization"] = f"Bearer {NEOMEM_KEY}"
payload = {
"messages": [{"role": "assistant", "content": summary}],
"user_id": "brian",
"metadata": {
"source": "intake",
"session_id": session_id
}
}
try:
requests.post(
f"{NEOMEM_API}/memories",
json=payload,
headers=headers,
timeout=20
).raise_for_status()
print(f"🧠 NeoMem updated for {session_id}")
except Exception as e:
print(f"NeoMem push failed: {e}")
# ───────────────────────────────────────────────
# Multilevel Summaries (L1, L5, L10, L20, L30)
# ───────────────────────────────────────────────
# History maps
L10_HISTORY = {} # session_id → list of L10 blocks
L20_HISTORY = {} # session_id → list of merged overviews
def summarize_L1(buf):
return summarize_simple(buf[-5:])
def summarize_L5(buf):
return summarize_simple(buf[-10:])
def summarize_L10(buf):
# “Reality Check” for last 10 exchanges
text = ""
for e in buf[-10:]:
text += f"User: {e['user_msg']}\nAssistant: {e['assistant_msg']}\n\n"
prompt = f"""
You are Lyra Intake performing a short 'Reality Check'.
Summarize the last block of conversation (up to 10 exchanges)
in one clear paragraph focusing on tone, intent, and direction.
{text}
Reality Check:
"""
return llm(prompt)
def summarize_L20(L10_list):
joined = "\n\n".join(L10_list)
prompt = f"""
You are Lyra Intake creating a 'Session Overview'.
Merge the following Reality Check paragraphs into one short summary
capturing progress, themes, and the direction of the conversation.
{joined}
Overview:
"""
return llm(prompt)
def summarize_L30(L20_list):
joined = "\n\n".join(L20_list)
prompt = f"""
You are Lyra Intake generating a 'Continuity Report'.
Condense these session overviews into one high-level reflection,
noting major themes, persistent goals, and shifts.
{joined}
Continuity Report:
"""
return llm(prompt)
def bg_summarize(session_id: str):
"""Runs all summary levels on every exchange."""
try:
hopper = SESSIONS.get(session_id)
if not hopper:
return
buf = list(hopper["buffer"])
if not buf:
return
# Ensure history lists exist
L10_HISTORY.setdefault(session_id, [])
L20_HISTORY.setdefault(session_id, [])
# L1, L5 (simple factual)
s_L1 = summarize_L1(buf)
s_L5 = summarize_L5(buf)
# L10 (append to history)
s_L10 = summarize_L10(buf)
L10_HISTORY[session_id].append(s_L10)
# L20 (merge all L10s)
s_L20 = summarize_L20(L10_HISTORY[session_id])
L20_HISTORY[session_id].append(s_L20)
# L30 (merge all L20s)
s_L30 = summarize_L30(L20_HISTORY[session_id])
# Push most important tier(s) to NeoMem
push_to_neomem(s_L10, session_id)
push_to_neomem(s_L20, session_id)
push_to_neomem(s_L30, session_id)
print(f"🧩 L1/L5/L10/L20/L30 updated for {session_id}")
except Exception as e:
print(f"💥 Multilevel summarizer error for {session_id}: {e}")
# ─────────────────────────────
# Routes
# ─────────────────────────────
@app.post("/add_exchange")
def add_exchange(exchange: dict = Body(...), background_tasks: BackgroundTasks = None):
session_id = exchange.get("session_id") or f"sess-{uuid4().hex[:8]}"
exchange["session_id"] = session_id
exchange["timestamp"] = datetime.now().isoformat()
if session_id not in SESSIONS:
SESSIONS[session_id] = {
"buffer": deque(maxlen=200),
"created_at": datetime.now()
}
print(f"🆕 Hopper created: {session_id}")
SESSIONS[session_id]["buffer"].append(exchange)
if background_tasks:
background_tasks.add_task(bg_summarize, session_id)
print(f"⏩ Summarization queued for {session_id}")
return {"ok": True, "session_id": session_id}
@app.post("/close_session/{session_id}")
def close_session(session_id: str):
if session_id in SESSIONS:
del SESSIONS[session_id]
return {"ok": True, "closed": session_id}
@app.get("/summaries")
def get_summary(session_id: str = Query(...)):
hopper = SESSIONS.get(session_id)
if not hopper:
return {"summary_text": "(none)", "session_id": session_id}
summary = summarize_simple(list(hopper["buffer"]))
return {"summary_text": summary, "session_id": session_id}
@app.get("/context")
def get_context(session_id: str = Query(...)):
"""Return full multilevel summary context for Cortex."""
if session_id not in SESSIONS:
return {
"session_id": session_id,
"exchange_count": 0,
"L1": "",
"L5": "",
"L10": "",
"L20": "",
"L30": "",
"last_updated": None
}
buffer = list(SESSIONS[session_id]["buffer"])
# Build levels on demand
L1 = summarize_L1(buffer)
L5 = summarize_L5(buffer)
L10 = summarize_L10(buffer)
L20 = summarize_L20(L10_HISTORY.get(session_id, []))
L30 = summarize_L30(L20_HISTORY.get(session_id, []))
return {
"session_id": session_id,
"exchange_count": len(buffer),
"L1": L1,
"L5": L5,
"L10": L10,
"L20": L20,
"L30": L30,
"last_updated": datetime.now().isoformat()
}
@app.get("/health")
def health():
return {"ok": True, "model": SUMMARY_MODEL, "url": SUMMARY_URL}

4
intake(deprecated)/requirements.txt
View File

@@ -1,4 +0,0 @@
fastapi==0.115.8
uvicorn==0.34.0
requests==2.32.3
python-dotenv==1.0.1

1
intake(deprecated)/venv/bin/python
View File

@@ -1 +0,0 @@
python3

1
intake(deprecated)/venv/bin/python3
View File

@@ -1 +0,0 @@
/usr/bin/python3

1
intake(deprecated)/venv/bin/python3.10
View File

@@ -1 +0,0 @@
python3

1
intake(deprecated)/venv/lib64
View File

@@ -1 +0,0 @@
lib

3
intake(deprecated)/venv/pyvenv.cfg
View File

@@ -1,3 +0,0 @@
home = /usr/bin
include-system-site-packages = false
version = 3.10.12