serversdown
30185f3fd8
The MI50 box (CT202) runs an OpenAI-compatible llama.cpp server on 10.0.0.44:8080. Wire it in as a third backend: - llm.complete gains backend="mi50" (OpenAI client pointed at MI50_BASE_URL) - config: MI50_BASE_URL (default http://10.0.0.44:8080/v1) + MI50_MODEL - chat.respond labels the model per backend; web _backend_for maps "mi50" - UI backend selector adds "MI50 — local GPU" Verified end-to-end: llm.complete(backend="mi50") returns from the live server. See homelab-inference memory for the box topology. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
24 lines
815 B
Bash
24 lines
815 B
Bash
# Local backend (Ollama) — free, private. Point this at your home-lab Ollama.
|
|
LOCAL_BASE_URL=http://localhost:11434
|
|
LOCAL_MODEL=qwen2.5:7b-instruct
|
|
|
|
# MI50 backend — OpenAI-compatible llama.cpp server on the home-lab GPU box.
|
|
MI50_BASE_URL=http://10.0.0.44:8080/v1
|
|
MI50_MODEL=local-gpu
|
|
|
|
# Cloud backend (OpenAI) — higher quality, costs money.
|
|
OPENAI_API_KEY=
|
|
CLOUD_MODEL=gpt-4o-mini
|
|
|
|
# Embeddings: "cloud" (OpenAI) or "local" (Ollama). A database is tied to whichever
|
|
# backend created it — don't switch this against an existing DB (vector spaces differ).
|
|
EMBED_BACKEND=cloud
|
|
EMBED_MODEL=text-embedding-3-small
|
|
LOCAL_EMBED_MODEL=nomic-embed-text
|
|
|
|
# Backend used to compact old sessions into summaries ("local" keeps it free).
|
|
SUMMARY_BACKEND=local
|
|
|
|
# Where Lyra stores her memory.
|
|
LYRA_DB_PATH=data/lyra.db
|