feat: cloud-first consolidation routing + graceful backend fallback
Nail down which backend each LLM path uses, and make the dream cycle resilient to a backend being down (the MI50 outage left profile/era/narrative — pinned to mi50 with no fallback — aborting every dream cycle). - Consolidation (summaries + profile/era/narrative) -> cloud via SUMMARY_BACKEND= cloud (.env, not committed). Matches the documented lesson that the MI50 is too slow/hot for bulk consolidation; nothing background touches the card now. - llm.complete_with_fallback(): try the primary backend, fall back to cloud on error (re-raise if already cloud / no key). Wired into reflect + think so the introspection voice (3090/dolphin) survives the gaming PC being powered off. - dream coherence stage is now fault-isolated: a rebuild failure logs + continues instead of sinking the whole pass (reflection still runs). - .env: removed stale INTROSPECTION_BACKEND=mi50 (live routing is the web-switchable introspection_mode DB setting = dolphin/3090; the var only fed a dead fallback). Verified: forced cycle runs consolidation on cloud, introspection on the 3090, completes with zero MI50 calls. 206 pass, ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015yrEb5qpPGv2FjyxrB7LLk
This commit is contained in:
+20
@@ -92,6 +92,26 @@ def complete(messages: list[Message], backend: Backend = "local", model: str | N
|
||||
return out
|
||||
|
||||
|
||||
def complete_with_fallback(messages: list[Message], backend: Backend, model: str | None = None,
|
||||
*, fallback: Backend = "cloud",
|
||||
max_tokens: int | None = None, timeout: float | None = None) -> str:
|
||||
"""`complete()` but if the primary backend errors (e.g. a local GPU that's
|
||||
powered off or down), retry once on `fallback` (cloud) instead of failing.
|
||||
Lets local/GPU-routed work (introspection, consolidation) degrade gracefully.
|
||||
Re-raises if the primary is already the fallback or no cloud key is configured."""
|
||||
try:
|
||||
return complete(messages, backend=backend, model=model,
|
||||
max_tokens=max_tokens, timeout=timeout)
|
||||
except Exception as exc:
|
||||
can_fallback = backend != fallback and (fallback != "cloud" or load().openai_api_key)
|
||||
if not can_fallback:
|
||||
raise
|
||||
logbus.log("info", "llm fell back", primary=backend, to=fallback, error=str(exc)[:80])
|
||||
# Drop the primary's model on fallback — let the fallback pick its own default.
|
||||
return complete(messages, backend=fallback, model=None,
|
||||
max_tokens=max_tokens, timeout=timeout)
|
||||
|
||||
|
||||
def chat_call(
|
||||
messages: list, backend: Backend = "cloud", model: str | None = None,
|
||||
tools: list | None = None,
|
||||
|
||||
Reference in New Issue
Block a user