feat: MI50 as a Lyra backend (OpenAI-compatible local GPU)
The MI50 box (CT202) runs an OpenAI-compatible llama.cpp server on 10.0.0.44:8080. Wire it in as a third backend: - llm.complete gains backend="mi50" (OpenAI client pointed at MI50_BASE_URL) - config: MI50_BASE_URL (default http://10.0.0.44:8080/v1) + MI50_MODEL - chat.respond labels the model per backend; web _backend_for maps "mi50" - UI backend selector adds "MI50 — local GPU" Verified end-to-end: llm.complete(backend="mi50") returns from the live server. See homelab-inference memory for the box topology. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+3
-1
@@ -81,7 +81,9 @@ def build_messages(session_id: str, user_msg: str) -> list[Message]:
|
||||
def respond(session_id: str, user_msg: str, backend: Backend = "cloud") -> str:
|
||||
"""Produce Lyra's reply to a single user message and persist the exchange."""
|
||||
cfg = config.load()
|
||||
model = cfg.local_model if backend == "local" else cfg.cloud_model
|
||||
model = {"local": cfg.local_model, "cloud": cfg.cloud_model, "mi50": cfg.mi50_model}.get(
|
||||
backend, backend
|
||||
)
|
||||
logbus.log(
|
||||
"info", "chat request", session=session_id, backend=backend,
|
||||
model=model, embed=cfg.embed_backend,
|
||||
|
||||
Reference in New Issue
Block a user