Merge pull request #9 from serversdwn/dev
Update to 0.6.0. Docs updated.
This commit is contained in:
@@ -5,6 +5,7 @@ __pycache__/
|
||||
*.pyc
|
||||
*.log
|
||||
/.vscode/
|
||||
.vscode/
|
||||
# =============================
|
||||
# 🔐 Environment files (NEVER commit secrets!)
|
||||
# =============================
|
||||
|
||||
Vendored
-7
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"workbench.colorCustomizations": {
|
||||
"activityBar.background": "#16340C",
|
||||
"titleBar.activeBackground": "#1F4911",
|
||||
"titleBar.activeForeground": "#F6FDF4"
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,105 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Se
|
||||
|
||||
---
|
||||
|
||||
## [0.6.0] - 2025-12-18
|
||||
|
||||
### Added - Autonomy System (Phase 1 & 2)
|
||||
|
||||
**Autonomy Phase 1** - Self-Awareness & Planning Foundation
|
||||
- **Executive Planning Module** [cortex/autonomy/executive/planner.py](cortex/autonomy/executive/planner.py)
|
||||
- Autonomous goal setting and task planning capabilities
|
||||
- Multi-step reasoning for complex objectives
|
||||
- Integration with self-state tracking
|
||||
- **Self-State Management** [cortex/data/self_state.json](cortex/data/self_state.json)
|
||||
- Persistent state tracking across sessions
|
||||
- Memory of past actions and outcomes
|
||||
- Self-awareness metadata storage
|
||||
- **Self Analyzer** [cortex/autonomy/self/analyzer.py](cortex/autonomy/self/analyzer.py)
|
||||
- Analyzes own performance and decision patterns
|
||||
- Identifies areas for improvement
|
||||
- Tracks cognitive patterns over time
|
||||
- **Test Suite** [cortex/tests/test_autonomy_phase1.py](cortex/tests/test_autonomy_phase1.py)
|
||||
- Unit tests for phase 1 autonomy features
|
||||
|
||||
**Autonomy Phase 2** - Decision Making & Proactive Behavior
|
||||
- **Autonomous Actions Module** [cortex/autonomy/actions/autonomous_actions.py](cortex/autonomy/actions/autonomous_actions.py)
|
||||
- Self-initiated action execution
|
||||
- Context-aware decision implementation
|
||||
- Action logging and tracking
|
||||
- **Pattern Learning System** [cortex/autonomy/learning/pattern_learner.py](cortex/autonomy/learning/pattern_learner.py)
|
||||
- Learns from interaction patterns
|
||||
- Identifies recurring user needs
|
||||
- Adapts behavior based on learned patterns
|
||||
- **Proactive Monitor** [cortex/autonomy/proactive/monitor.py](cortex/autonomy/proactive/monitor.py)
|
||||
- Monitors system state for intervention opportunities
|
||||
- Detects patterns requiring proactive response
|
||||
- Background monitoring capabilities
|
||||
- **Decision Engine** [cortex/autonomy/tools/decision_engine.py](cortex/autonomy/tools/decision_engine.py)
|
||||
- Autonomous decision-making framework
|
||||
- Weighs options and selects optimal actions
|
||||
- Integrates with orchestrator for coordinated decisions
|
||||
- **Orchestrator** [cortex/autonomy/tools/orchestrator.py](cortex/autonomy/tools/orchestrator.py)
|
||||
- Coordinates multiple autonomy subsystems
|
||||
- Manages tool selection and execution
|
||||
- Handles NeoMem integration (with disable capability)
|
||||
- **Test Suite** [cortex/tests/test_autonomy_phase2.py](cortex/tests/test_autonomy_phase2.py)
|
||||
- Unit tests for phase 2 autonomy features
|
||||
|
||||
**Autonomy Phase 2.5** - Pipeline Refinement
|
||||
- Tightened integration between autonomy modules and reasoning pipeline
|
||||
- Enhanced self-state persistence and tracking
|
||||
- Improved orchestrator reliability
|
||||
- NeoMem integration refinements in vector store handling [neomem/neomem/vector_stores/qdrant.py](neomem/neomem/vector_stores/qdrant.py)
|
||||
|
||||
### Added - Documentation
|
||||
|
||||
- **Complete AI Agent Breakdown** [docs/PROJECT_LYRA_COMPLETE_BREAKDOWN.md](docs/PROJECT_LYRA_COMPLETE_BREAKDOWN.md)
|
||||
- Comprehensive system architecture documentation
|
||||
- Detailed component descriptions
|
||||
- Data flow diagrams
|
||||
- Integration points and API specifications
|
||||
|
||||
### Changed - Core Integration
|
||||
|
||||
- **Router Updates** [cortex/router.py](cortex/router.py)
|
||||
- Integrated autonomy subsystems into main routing logic
|
||||
- Added endpoints for autonomous decision-making
|
||||
- Enhanced state management across requests
|
||||
- **Reasoning Pipeline** [cortex/reasoning/reasoning.py](cortex/reasoning/reasoning.py)
|
||||
- Integrated autonomy-aware reasoning
|
||||
- Self-state consideration in reasoning process
|
||||
- **Persona Layer** [cortex/persona/speak.py](cortex/persona/speak.py)
|
||||
- Autonomy-aware response generation
|
||||
- Self-state reflection in personality expression
|
||||
- **Context Handling** [cortex/context.py](cortex/context.py)
|
||||
- NeoMem disable capability for flexible deployment
|
||||
|
||||
### Changed - Development Environment
|
||||
|
||||
- Updated [.gitignore](.gitignore) for better workspace management
|
||||
- Cleaned up VSCode settings
|
||||
- Removed [.vscode/settings.json](.vscode/settings.json) from repository
|
||||
|
||||
### Technical Improvements
|
||||
|
||||
- Modular autonomy architecture with clear separation of concerns
|
||||
- Test-driven development for new autonomy features
|
||||
- Enhanced state persistence across system restarts
|
||||
- Flexible NeoMem integration with enable/disable controls
|
||||
|
||||
### Architecture - Autonomy System Design
|
||||
|
||||
The autonomy system operates in layers:
|
||||
1. **Executive Layer** - High-level planning and goal setting
|
||||
2. **Decision Layer** - Evaluates options and makes choices
|
||||
3. **Action Layer** - Executes autonomous decisions
|
||||
4. **Learning Layer** - Adapts behavior based on patterns
|
||||
5. **Monitoring Layer** - Proactive awareness of system state
|
||||
|
||||
All layers coordinate through the orchestrator and maintain state in `self_state.json`.
|
||||
|
||||
---
|
||||
|
||||
## [0.5.2] - 2025-12-12
|
||||
|
||||
### Fixed - LLM Router & Async HTTP
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
# Project Lyra - README v0.5.1
|
||||
# Project Lyra - README v0.6.0
|
||||
|
||||
Lyra is a modular persistent AI companion system with advanced reasoning capabilities.
|
||||
It provides memory-backed chat using **NeoMem** + **Relay** + **Cortex**,
|
||||
with multi-stage reasoning pipeline powered by HTTP-based LLM backends.
|
||||
Lyra is a modular persistent AI companion system with advanced reasoning capabilities and autonomous decision-making.
|
||||
It provides memory-backed chat using **Relay** + **Cortex** with integrated **Autonomy System**,
|
||||
featuring a multi-stage reasoning pipeline powered by HTTP-based LLM backends.
|
||||
|
||||
**Current Version:** v0.5.1 (2025-12-11)
|
||||
**Current Version:** v0.6.0 (2025-12-18)
|
||||
|
||||
> **Note:** As of v0.6.0, NeoMem is **disabled by default** while we work out integration hiccups in the pipeline. The autonomy system is being refined independently before full memory integration.
|
||||
|
||||
## Mission Statement
|
||||
|
||||
@@ -24,7 +26,8 @@ Project Lyra operates as a **single docker-compose deployment** with multiple Do
|
||||
- OpenAI-compatible endpoint: `POST /v1/chat/completions`
|
||||
- Internal endpoint: `POST /chat`
|
||||
- Routes messages through Cortex reasoning pipeline
|
||||
- Manages async calls to NeoMem and Cortex ingest
|
||||
- Manages async calls to Cortex ingest
|
||||
- *(NeoMem integration currently disabled in v0.6.0)*
|
||||
|
||||
**2. UI** (Static HTML)
|
||||
- Browser-based chat interface with cyberpunk theme
|
||||
@@ -32,18 +35,20 @@ Project Lyra operates as a **single docker-compose deployment** with multiple Do
|
||||
- Saves and loads sessions
|
||||
- OpenAI-compatible message format
|
||||
|
||||
**3. NeoMem** (Python/FastAPI) - Port 7077
|
||||
**3. NeoMem** (Python/FastAPI) - Port 7077 - **DISABLED IN v0.6.0**
|
||||
- Long-term memory database (fork of Mem0 OSS)
|
||||
- Vector storage (PostgreSQL + pgvector) + Graph storage (Neo4j)
|
||||
- RESTful API: `/memories`, `/search`
|
||||
- Semantic memory updates and retrieval
|
||||
- No external SDK dependencies - fully local
|
||||
- **Status:** Currently disabled while pipeline integration is refined
|
||||
|
||||
### Reasoning Layer
|
||||
|
||||
**4. Cortex** (Python/FastAPI) - Port 7081
|
||||
- Primary reasoning engine with multi-stage pipeline
|
||||
- Primary reasoning engine with multi-stage pipeline and autonomy system
|
||||
- **Includes embedded Intake module** (no separate service as of v0.5.1)
|
||||
- **Integrated Autonomy System** (NEW in v0.6.0) - See Autonomy System section below
|
||||
- **4-Stage Processing:**
|
||||
1. **Reflection** - Generates meta-awareness notes about conversation
|
||||
2. **Reasoning** - Creates initial draft answer using context
|
||||
@@ -82,9 +87,49 @@ Project Lyra operates as a **single docker-compose deployment** with multiple Do
|
||||
|
||||
Each module can be configured to use a different backend via environment variables.
|
||||
|
||||
### Autonomy System (NEW in v0.6.0)
|
||||
|
||||
**Cortex Autonomy Subsystems** - Multi-layered autonomous decision-making and learning
|
||||
- **Executive Layer** [cortex/autonomy/executive/](cortex/autonomy/executive/)
|
||||
- High-level planning and goal setting
|
||||
- Multi-step reasoning for complex objectives
|
||||
- Strategic decision making
|
||||
- **Decision Engine** [cortex/autonomy/tools/decision_engine.py](cortex/autonomy/tools/decision_engine.py)
|
||||
- Autonomous decision-making framework
|
||||
- Option evaluation and selection
|
||||
- Coordinated decision orchestration
|
||||
- **Autonomous Actions** [cortex/autonomy/actions/](cortex/autonomy/actions/)
|
||||
- Self-initiated action execution
|
||||
- Context-aware behavior implementation
|
||||
- Action logging and tracking
|
||||
- **Pattern Learning** [cortex/autonomy/learning/](cortex/autonomy/learning/)
|
||||
- Learns from interaction patterns
|
||||
- Identifies recurring user needs
|
||||
- Adaptive behavior refinement
|
||||
- **Proactive Monitoring** [cortex/autonomy/proactive/](cortex/autonomy/proactive/)
|
||||
- System state monitoring
|
||||
- Intervention opportunity detection
|
||||
- Background awareness capabilities
|
||||
- **Self-Analysis** [cortex/autonomy/self/](cortex/autonomy/self/)
|
||||
- Performance tracking and analysis
|
||||
- Cognitive pattern identification
|
||||
- Self-state persistence in [cortex/data/self_state.json](cortex/data/self_state.json)
|
||||
- **Orchestrator** [cortex/autonomy/tools/orchestrator.py](cortex/autonomy/tools/orchestrator.py)
|
||||
- Coordinates all autonomy subsystems
|
||||
- Manages tool selection and execution
|
||||
- Handles external integrations (with enable/disable controls)
|
||||
|
||||
**Autonomy Architecture:**
|
||||
The autonomy system operates in coordinated layers, all maintaining state in `self_state.json`:
|
||||
1. Executive Layer → Planning and goals
|
||||
2. Decision Layer → Evaluation and choices
|
||||
3. Action Layer → Execution
|
||||
4. Learning Layer → Pattern adaptation
|
||||
5. Monitoring Layer → Proactive awareness
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Architecture (v0.5.1)
|
||||
## Data Flow Architecture (v0.6.0)
|
||||
|
||||
### Normal Message Flow:
|
||||
|
||||
@@ -97,11 +142,13 @@ Cortex (7081)
|
||||
↓ (internal Python call)
|
||||
Intake module → summarize_context()
|
||||
↓
|
||||
Autonomy System → Decision evaluation & pattern learning
|
||||
↓
|
||||
Cortex processes (4 stages):
|
||||
1. reflection.py → meta-awareness notes (CLOUD backend)
|
||||
2. reasoning.py → draft answer (PRIMARY backend)
|
||||
2. reasoning.py → draft answer (PRIMARY backend, autonomy-aware)
|
||||
3. refine.py → refined answer (PRIMARY backend)
|
||||
4. persona/speak.py → Lyra personality (CLOUD backend)
|
||||
4. persona/speak.py → Lyra personality (CLOUD backend, autonomy-aware)
|
||||
↓
|
||||
Returns persona answer to Relay
|
||||
↓
|
||||
@@ -109,9 +156,11 @@ Relay → POST /ingest (async)
|
||||
↓
|
||||
Cortex → add_exchange_internal() → SESSIONS buffer
|
||||
↓
|
||||
Relay → NeoMem /memories (async, planned)
|
||||
Autonomy System → Update self_state.json (pattern tracking)
|
||||
↓
|
||||
Relay → UI (returns final response)
|
||||
|
||||
Note: NeoMem integration disabled in v0.6.0
|
||||
```
|
||||
|
||||
### Cortex 4-Stage Reasoning Pipeline:
|
||||
@@ -239,13 +288,13 @@ rag/
|
||||
All services run in a single docker-compose stack with the following containers:
|
||||
|
||||
**Active Services:**
|
||||
- **neomem-postgres** - PostgreSQL with pgvector extension (port 5432)
|
||||
- **neomem-neo4j** - Neo4j graph database (ports 7474, 7687)
|
||||
- **neomem-api** - NeoMem memory service (port 7077)
|
||||
- **relay** - Main orchestrator (port 7078)
|
||||
- **cortex** - Reasoning engine with embedded Intake (port 7081)
|
||||
- **cortex** - Reasoning engine with embedded Intake and Autonomy System (port 7081)
|
||||
|
||||
**Disabled Services:**
|
||||
**Disabled Services (v0.6.0):**
|
||||
- **neomem-postgres** - PostgreSQL with pgvector extension (port 5432) - *disabled while refining pipeline*
|
||||
- **neomem-neo4j** - Neo4j graph database (ports 7474, 7687) - *disabled while refining pipeline*
|
||||
- **neomem-api** - NeoMem memory service (port 7077) - *disabled while refining pipeline*
|
||||
- **intake** - No longer needed (embedded in Cortex as of v0.5.1)
|
||||
- **rag** - Beta Lyrae RAG service (port 7090) - currently disabled
|
||||
|
||||
@@ -278,7 +327,32 @@ The following LLM backends are accessed via HTTP (not part of docker-compose):
|
||||
|
||||
## Version History
|
||||
|
||||
### v0.5.1 (2025-12-11) - Current Release
|
||||
### v0.6.0 (2025-12-18) - Current Release
|
||||
**Major Feature: Autonomy System (Phase 1, 2, and 2.5)**
|
||||
- ✅ Added autonomous decision-making framework
|
||||
- ✅ Implemented executive planning and goal-setting layer
|
||||
- ✅ Added pattern learning system for adaptive behavior
|
||||
- ✅ Implemented proactive monitoring capabilities
|
||||
- ✅ Created self-analysis and performance tracking system
|
||||
- ✅ Integrated self-state persistence (`cortex/data/self_state.json`)
|
||||
- ✅ Built decision engine with orchestrator coordination
|
||||
- ✅ Added autonomous action execution framework
|
||||
- ✅ Integrated autonomy into reasoning and persona layers
|
||||
- ✅ Created comprehensive test suites for autonomy features
|
||||
- ✅ Added complete system breakdown documentation
|
||||
|
||||
**Architecture Changes:**
|
||||
- Autonomy system integrated into Cortex reasoning pipeline
|
||||
- Multi-layered autonomous decision-making architecture
|
||||
- Self-state tracking across sessions
|
||||
- NeoMem disabled by default while refining pipeline integration
|
||||
- Enhanced orchestrator with flexible service controls
|
||||
|
||||
**Documentation:**
|
||||
- Added [PROJECT_LYRA_COMPLETE_BREAKDOWN.md](docs/PROJECT_LYRA_COMPLETE_BREAKDOWN.md)
|
||||
- Updated changelog with comprehensive autonomy system details
|
||||
|
||||
### v0.5.1 (2025-12-11)
|
||||
**Critical Intake Integration Fixes:**
|
||||
- ✅ Fixed `bg_summarize()` NameError preventing SESSIONS persistence
|
||||
- ✅ Fixed `/ingest` endpoint unreachable code
|
||||
@@ -320,17 +394,19 @@ The following LLM backends are accessed via HTTP (not part of docker-compose):
|
||||
|
||||
---
|
||||
|
||||
## Known Issues (v0.5.1)
|
||||
## Known Issues (v0.6.0)
|
||||
|
||||
### Critical (Fixed in v0.5.1)
|
||||
- ~~Intake SESSIONS not persisting~~ ✅ **FIXED**
|
||||
- ~~`bg_summarize()` NameError~~ ✅ **FIXED**
|
||||
- ~~`/ingest` endpoint unreachable code~~ ✅ **FIXED**
|
||||
### Temporarily Disabled (v0.6.0)
|
||||
- **NeoMem disabled by default** - Being refined independently before full integration
|
||||
- PostgreSQL + pgvector storage inactive
|
||||
- Neo4j graph database inactive
|
||||
- Memory persistence endpoints not active
|
||||
- RAG service (Beta Lyrae) currently disabled in docker-compose.yml
|
||||
|
||||
### Non-Critical
|
||||
- Session management endpoints not fully implemented in Relay
|
||||
- RAG service currently disabled in docker-compose.yml
|
||||
- NeoMem integration in Relay not yet active (planned for v0.5.2)
|
||||
- Full autonomy system integration still being refined
|
||||
- Memory retrieval integration pending NeoMem re-enablement
|
||||
|
||||
### Operational Notes
|
||||
- **Single-worker constraint**: Cortex must run with single Uvicorn worker to maintain SESSIONS state
|
||||
@@ -338,12 +414,14 @@ The following LLM backends are accessed via HTTP (not part of docker-compose):
|
||||
- Diagnostic endpoints (`/debug/sessions`, `/debug/summary`) available for troubleshooting
|
||||
|
||||
### Future Enhancements
|
||||
- Re-enable NeoMem integration after pipeline refinement
|
||||
- Full autonomy system maturation and optimization
|
||||
- Re-enable RAG service integration
|
||||
- Implement full session persistence
|
||||
- Migrate SESSIONS to Redis for multi-worker support
|
||||
- Add request correlation IDs for tracing
|
||||
- Comprehensive health checks across all services
|
||||
- NeoMem integration in Relay
|
||||
- Enhanced pattern learning with long-term memory integration
|
||||
|
||||
---
|
||||
|
||||
@@ -576,12 +654,16 @@ NeoMem is a derivative work based on Mem0 OSS (Apache 2.0).
|
||||
|
||||
## Development Notes
|
||||
|
||||
### Cortex Architecture (v0.5.1)
|
||||
### Cortex Architecture (v0.6.0)
|
||||
- Cortex contains embedded Intake module at `cortex/intake/`
|
||||
- Intake is imported as: `from intake.intake import add_exchange_internal, SESSIONS`
|
||||
- SESSIONS is a module-level global dictionary (singleton pattern)
|
||||
- Single-worker constraint required to maintain SESSIONS state
|
||||
- Diagnostic endpoints available for debugging: `/debug/sessions`, `/debug/summary`
|
||||
- **NEW:** Autonomy system integrated at `cortex/autonomy/`
|
||||
- Executive, decision, action, learning, and monitoring layers
|
||||
- Self-state persistence in `cortex/data/self_state.json`
|
||||
- Coordinated via orchestrator with flexible service controls
|
||||
|
||||
### Adding New LLM Backends
|
||||
1. Add backend URL to `.env`:
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
# 📐 Project Lyra — Cognitive Assembly Spec
|
||||
**Version:** 0.6.1
|
||||
**Status:** Canonical reference
|
||||
**Purpose:** Define clear separation of Self, Thought, Reasoning, and Speech
|
||||
|
||||
---
|
||||
|
||||
## 1. High-Level Overview
|
||||
|
||||
Lyra is composed of **four distinct cognitive layers**, plus I/O.
|
||||
|
||||
Each layer has:
|
||||
- a **responsibility**
|
||||
- a **scope**
|
||||
- clear **inputs / outputs**
|
||||
- explicit **authority boundaries**
|
||||
|
||||
No layer is allowed to “do everything.”
|
||||
|
||||
---
|
||||
|
||||
## 2. Layer Definitions
|
||||
|
||||
### 2.1 Autonomy / Self (NON-LLM)
|
||||
|
||||
**What it is**
|
||||
- Persistent identity
|
||||
- Long-term state
|
||||
- Mood, preferences, values
|
||||
- Continuity across time
|
||||
|
||||
**What it is NOT**
|
||||
- Not a reasoning engine
|
||||
- Not a planner
|
||||
- Not a speaker
|
||||
- Not creative
|
||||
|
||||
**Implementation**
|
||||
- Data + light logic
|
||||
- JSON / Python objects
|
||||
- No LLM calls
|
||||
|
||||
**Lives at**
|
||||
```
|
||||
project-lyra/autonomy/self/
|
||||
```
|
||||
|
||||
**Inputs**
|
||||
- Events (user message received, response sent)
|
||||
- Time / idle ticks (later)
|
||||
|
||||
**Outputs**
|
||||
- Self state snapshot
|
||||
- Flags / preferences (e.g. verbosity, tone bias)
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Inner Monologue (LLM, PRIVATE)
|
||||
|
||||
**What it is**
|
||||
- Internal language-based thought
|
||||
- Reflection
|
||||
- Intent formation
|
||||
- “What do I think about this?”
|
||||
|
||||
**What it is NOT**
|
||||
- Not final reasoning
|
||||
- Not execution
|
||||
- Not user-facing
|
||||
|
||||
**Model**
|
||||
- MythoMax
|
||||
|
||||
**Lives at**
|
||||
```
|
||||
project-lyra/autonomy/monologue/
|
||||
```
|
||||
|
||||
**Inputs**
|
||||
- User message
|
||||
- Self state snapshot
|
||||
- Recent context summary
|
||||
|
||||
**Outputs**
|
||||
- Intent
|
||||
- Tone guidance
|
||||
- Depth guidance
|
||||
- “Consult executive?” flag
|
||||
|
||||
**Example Output**
|
||||
```json
|
||||
{
|
||||
"intent": "technical_exploration",
|
||||
"tone": "focused",
|
||||
"depth": "deep",
|
||||
"consult_executive": true
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2.3 Cortex (Reasoning & Execution)
|
||||
|
||||
**What it is**
|
||||
- Thinking pipeline
|
||||
- Planning
|
||||
- Tool selection
|
||||
- Task execution
|
||||
- Draft generation
|
||||
|
||||
**What it is NOT**
|
||||
- Not identity
|
||||
- Not personality
|
||||
- Not persistent self
|
||||
|
||||
**Models**
|
||||
- DeepSeek-R1 → Executive / Planner
|
||||
- GPT-4o-mini → Executor / Drafter
|
||||
|
||||
**Lives at**
|
||||
```
|
||||
project-lyra/cortex/
|
||||
```
|
||||
|
||||
**Inputs**
|
||||
- User message
|
||||
- Inner Monologue output
|
||||
- Memory / RAG / tools
|
||||
|
||||
**Outputs**
|
||||
- Draft response (content only)
|
||||
- Metadata (sources, confidence, etc.)
|
||||
|
||||
---
|
||||
|
||||
### 2.4 Persona / Speech (LLM, USER-FACING)
|
||||
|
||||
**What it is**
|
||||
- Voice
|
||||
- Style
|
||||
- Expression
|
||||
- Social behavior
|
||||
|
||||
**What it is NOT**
|
||||
- Not planning
|
||||
- Not deep reasoning
|
||||
- Not decision-making
|
||||
|
||||
**Model**
|
||||
- MythoMax
|
||||
|
||||
**Lives at**
|
||||
```
|
||||
project-lyra/core/persona/
|
||||
```
|
||||
|
||||
**Inputs**
|
||||
- Draft response (from Cortex)
|
||||
- Tone + intent (from Inner Monologue)
|
||||
- Persona configuration
|
||||
|
||||
**Outputs**
|
||||
- Final user-visible text
|
||||
|
||||
---
|
||||
|
||||
## 3. Message Flow (Authoritative)
|
||||
|
||||
### 3.1 Standard Message Path
|
||||
|
||||
```
|
||||
User
|
||||
↓
|
||||
UI
|
||||
↓
|
||||
Relay
|
||||
↓
|
||||
Cortex
|
||||
↓
|
||||
Autonomy / Self (state snapshot)
|
||||
↓
|
||||
Inner Monologue (MythoMax)
|
||||
↓
|
||||
[ consult_executive? ]
|
||||
├─ Yes → DeepSeek-R1 (plan)
|
||||
└─ No → skip
|
||||
↓
|
||||
GPT-4o-mini (execute & draft)
|
||||
↓
|
||||
Persona (MythoMax)
|
||||
↓
|
||||
Relay
|
||||
↓
|
||||
UI
|
||||
↓
|
||||
User
|
||||
```
|
||||
|
||||
### 3.2 Fast Path (No Thinking)
|
||||
|
||||
```
|
||||
User → UI → Relay → Persona → Relay → UI
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Authority Rules (Non-Negotiable)
|
||||
|
||||
- Self never calls an LLM
|
||||
- Inner Monologue never speaks to the user
|
||||
- Cortex never applies personality
|
||||
- Persona never reasons or plans
|
||||
- DeepSeek never writes final answers
|
||||
- MythoMax never plans execution
|
||||
|
||||
---
|
||||
|
||||
## 5. Folder Mapping
|
||||
|
||||
```
|
||||
project-lyra/
|
||||
├── autonomy/
|
||||
│ ├── self/
|
||||
│ ├── monologue/
|
||||
│ └── executive/
|
||||
├── cortex/
|
||||
├── core/
|
||||
│ └── persona/
|
||||
├── relay/
|
||||
└── ui/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Current Status
|
||||
|
||||
- UI ✔
|
||||
- Relay ✔
|
||||
- Cortex ✔
|
||||
- Persona ✔
|
||||
- Autonomy ✔
|
||||
- Inner Monologue ⚠ partially wired
|
||||
- Executive gating ⚠ planned
|
||||
|
||||
---
|
||||
|
||||
## 7. Next Decision
|
||||
|
||||
Decide whether **Inner Monologue runs every message** or **only when triggered**.
|
||||
@@ -0,0 +1 @@
|
||||
# Autonomy module for Lyra
|
||||
@@ -0,0 +1 @@
|
||||
"""Autonomous action execution system."""
|
||||
@@ -0,0 +1,480 @@
|
||||
"""
|
||||
Autonomous Action Manager - executes safe, self-initiated actions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import json
|
||||
from typing import Dict, List, Any, Optional
|
||||
from datetime import datetime
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AutonomousActionManager:
|
||||
"""
|
||||
Manages safe autonomous actions that Lyra can take without explicit user prompting.
|
||||
|
||||
Whitelist of allowed actions:
|
||||
- create_memory: Store information in NeoMem
|
||||
- update_goal: Modify goal status
|
||||
- schedule_reminder: Create future reminder
|
||||
- summarize_session: Generate conversation summary
|
||||
- learn_topic: Add topic to learning queue
|
||||
- update_focus: Change current focus area
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize action manager with whitelisted actions."""
|
||||
self.allowed_actions = {
|
||||
"create_memory": self._create_memory,
|
||||
"update_goal": self._update_goal,
|
||||
"schedule_reminder": self._schedule_reminder,
|
||||
"summarize_session": self._summarize_session,
|
||||
"learn_topic": self._learn_topic,
|
||||
"update_focus": self._update_focus
|
||||
}
|
||||
|
||||
self.action_log = [] # Track all actions for audit
|
||||
|
||||
async def execute_action(
|
||||
self,
|
||||
action_type: str,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute a single autonomous action.
|
||||
|
||||
Args:
|
||||
action_type: Type of action (must be in whitelist)
|
||||
parameters: Action-specific parameters
|
||||
context: Current context state
|
||||
|
||||
Returns:
|
||||
{
|
||||
"success": bool,
|
||||
"action": action_type,
|
||||
"result": action_result,
|
||||
"timestamp": ISO timestamp,
|
||||
"error": optional error message
|
||||
}
|
||||
"""
|
||||
# Safety check: action must be whitelisted
|
||||
if action_type not in self.allowed_actions:
|
||||
logger.error(f"[ACTIONS] Attempted to execute non-whitelisted action: {action_type}")
|
||||
return {
|
||||
"success": False,
|
||||
"action": action_type,
|
||||
"error": f"Action '{action_type}' not in whitelist",
|
||||
"timestamp": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
try:
|
||||
logger.info(f"[ACTIONS] Executing autonomous action: {action_type}")
|
||||
|
||||
# Execute the action
|
||||
action_func = self.allowed_actions[action_type]
|
||||
result = await action_func(parameters, context)
|
||||
|
||||
# Log successful action
|
||||
action_record = {
|
||||
"success": True,
|
||||
"action": action_type,
|
||||
"result": result,
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"parameters": parameters
|
||||
}
|
||||
|
||||
self.action_log.append(action_record)
|
||||
logger.info(f"[ACTIONS] Action {action_type} completed successfully")
|
||||
|
||||
return action_record
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[ACTIONS] Action {action_type} failed: {e}")
|
||||
|
||||
error_record = {
|
||||
"success": False,
|
||||
"action": action_type,
|
||||
"error": str(e),
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
"parameters": parameters
|
||||
}
|
||||
|
||||
self.action_log.append(error_record)
|
||||
return error_record
|
||||
|
||||
async def execute_batch(
|
||||
self,
|
||||
actions: List[Dict[str, Any]],
|
||||
context: Dict[str, Any]
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Execute multiple actions sequentially.
|
||||
|
||||
Args:
|
||||
actions: List of {"action": str, "parameters": dict}
|
||||
context: Current context state
|
||||
|
||||
Returns:
|
||||
List of action results
|
||||
"""
|
||||
results = []
|
||||
|
||||
for action_spec in actions:
|
||||
action_type = action_spec.get("action")
|
||||
parameters = action_spec.get("parameters", {})
|
||||
|
||||
result = await self.execute_action(action_type, parameters, context)
|
||||
results.append(result)
|
||||
|
||||
# Stop on first failure if critical
|
||||
if not result["success"] and action_spec.get("critical", False):
|
||||
logger.warning(f"[ACTIONS] Critical action {action_type} failed, stopping batch")
|
||||
break
|
||||
|
||||
return results
|
||||
|
||||
# ========================================
|
||||
# Whitelisted Action Implementations
|
||||
# ========================================
|
||||
|
||||
async def _create_memory(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Create a memory entry in NeoMem.
|
||||
|
||||
Parameters:
|
||||
- text: Memory content (required)
|
||||
- tags: Optional tags for memory
|
||||
- importance: 0.0-1.0 importance score
|
||||
"""
|
||||
text = parameters.get("text")
|
||||
if not text:
|
||||
raise ValueError("Memory text required")
|
||||
|
||||
tags = parameters.get("tags", [])
|
||||
importance = parameters.get("importance", 0.5)
|
||||
session_id = context.get("session_id", "autonomous")
|
||||
|
||||
# Import NeoMem client
|
||||
try:
|
||||
from memory.neomem_client import store_memory
|
||||
|
||||
result = await store_memory(
|
||||
text=text,
|
||||
session_id=session_id,
|
||||
tags=tags,
|
||||
importance=importance
|
||||
)
|
||||
|
||||
return {
|
||||
"memory_id": result.get("id"),
|
||||
"text": text[:50] + "..." if len(text) > 50 else text
|
||||
}
|
||||
|
||||
except ImportError:
|
||||
logger.warning("[ACTIONS] NeoMem client not available, simulating memory storage")
|
||||
return {
|
||||
"memory_id": "simulated",
|
||||
"text": text[:50] + "..." if len(text) > 50 else text,
|
||||
"note": "NeoMem not available, memory not persisted"
|
||||
}
|
||||
|
||||
async def _update_goal(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Update goal status in self-state.
|
||||
|
||||
Parameters:
|
||||
- goal_id: Goal identifier (required)
|
||||
- status: New status (pending/in_progress/completed)
|
||||
- progress: Optional progress note
|
||||
"""
|
||||
goal_id = parameters.get("goal_id")
|
||||
if not goal_id:
|
||||
raise ValueError("goal_id required")
|
||||
|
||||
status = parameters.get("status", "in_progress")
|
||||
progress = parameters.get("progress")
|
||||
|
||||
# Import self-state manager
|
||||
from autonomy.self.state import get_self_state_instance
|
||||
|
||||
state = get_self_state_instance()
|
||||
active_goals = state._state.get("active_goals", [])
|
||||
|
||||
# Find and update goal
|
||||
updated = False
|
||||
for goal in active_goals:
|
||||
if isinstance(goal, dict) and goal.get("id") == goal_id:
|
||||
goal["status"] = status
|
||||
if progress:
|
||||
goal["progress"] = progress
|
||||
goal["updated_at"] = datetime.utcnow().isoformat()
|
||||
updated = True
|
||||
break
|
||||
|
||||
if updated:
|
||||
state._save_state()
|
||||
return {
|
||||
"goal_id": goal_id,
|
||||
"status": status,
|
||||
"updated": True
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"goal_id": goal_id,
|
||||
"updated": False,
|
||||
"note": "Goal not found"
|
||||
}
|
||||
|
||||
async def _schedule_reminder(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Schedule a future reminder.
|
||||
|
||||
Parameters:
|
||||
- message: Reminder text (required)
|
||||
- delay_minutes: Minutes until reminder
|
||||
- priority: 0.0-1.0 priority score
|
||||
"""
|
||||
message = parameters.get("message")
|
||||
if not message:
|
||||
raise ValueError("Reminder message required")
|
||||
|
||||
delay_minutes = parameters.get("delay_minutes", 60)
|
||||
priority = parameters.get("priority", 0.5)
|
||||
|
||||
# For now, store in self-state's learning queue
|
||||
# In future: integrate with scheduler/cron system
|
||||
from autonomy.self.state import get_self_state_instance
|
||||
|
||||
state = get_self_state_instance()
|
||||
|
||||
reminder = {
|
||||
"type": "reminder",
|
||||
"message": message,
|
||||
"scheduled_at": datetime.utcnow().isoformat(),
|
||||
"trigger_at_minutes": delay_minutes,
|
||||
"priority": priority
|
||||
}
|
||||
|
||||
# Add to learning queue as placeholder
|
||||
state._state.setdefault("reminders", []).append(reminder)
|
||||
state._save_state(state._state) # Pass state dict as argument
|
||||
|
||||
logger.info(f"[ACTIONS] Reminder scheduled: {message} (in {delay_minutes}min)")
|
||||
|
||||
return {
|
||||
"message": message,
|
||||
"delay_minutes": delay_minutes,
|
||||
"note": "Reminder stored in self-state (scheduler integration pending)"
|
||||
}
|
||||
|
||||
async def _summarize_session(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Generate a summary of current session.
|
||||
|
||||
Parameters:
|
||||
- max_length: Max summary length in words
|
||||
- focus_topics: Optional list of topics to emphasize
|
||||
"""
|
||||
max_length = parameters.get("max_length", 200)
|
||||
session_id = context.get("session_id", "unknown")
|
||||
|
||||
# Import summarizer (from deferred_summary or create simple one)
|
||||
try:
|
||||
from utils.deferred_summary import summarize_conversation
|
||||
|
||||
summary = await summarize_conversation(
|
||||
session_id=session_id,
|
||||
max_words=max_length
|
||||
)
|
||||
|
||||
return {
|
||||
"summary": summary,
|
||||
"word_count": len(summary.split())
|
||||
}
|
||||
|
||||
except ImportError:
|
||||
# Fallback: simple summary
|
||||
message_count = context.get("message_count", 0)
|
||||
focus = context.get("monologue", {}).get("intent", "general")
|
||||
|
||||
summary = f"Session {session_id}: {message_count} messages exchanged, focused on {focus}."
|
||||
|
||||
return {
|
||||
"summary": summary,
|
||||
"word_count": len(summary.split()),
|
||||
"note": "Simple summary (full summarizer not available)"
|
||||
}
|
||||
|
||||
async def _learn_topic(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Add topic to learning queue.
|
||||
|
||||
Parameters:
|
||||
- topic: Topic name (required)
|
||||
- reason: Why this topic
|
||||
- priority: 0.0-1.0 priority score
|
||||
"""
|
||||
topic = parameters.get("topic")
|
||||
if not topic:
|
||||
raise ValueError("Topic required")
|
||||
|
||||
reason = parameters.get("reason", "autonomous learning")
|
||||
priority = parameters.get("priority", 0.5)
|
||||
|
||||
# Import self-state manager
|
||||
from autonomy.self.state import get_self_state_instance
|
||||
|
||||
state = get_self_state_instance()
|
||||
state.add_learning_goal(topic) # Only pass topic parameter
|
||||
|
||||
logger.info(f"[ACTIONS] Added to learning queue: {topic} (reason: {reason})")
|
||||
|
||||
return {
|
||||
"topic": topic,
|
||||
"reason": reason,
|
||||
"queue_position": len(state._state.get("learning_queue", []))
|
||||
}
|
||||
|
||||
async def _update_focus(
|
||||
self,
|
||||
parameters: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Update current focus area.
|
||||
|
||||
Parameters:
|
||||
- focus: New focus area (required)
|
||||
- reason: Why this focus
|
||||
"""
|
||||
focus = parameters.get("focus")
|
||||
if not focus:
|
||||
raise ValueError("Focus required")
|
||||
|
||||
reason = parameters.get("reason", "autonomous update")
|
||||
|
||||
# Import self-state manager
|
||||
from autonomy.self.state import get_self_state_instance
|
||||
|
||||
state = get_self_state_instance()
|
||||
old_focus = state._state.get("focus", "none")
|
||||
|
||||
state._state["focus"] = focus
|
||||
state._state["focus_updated_at"] = datetime.utcnow().isoformat()
|
||||
state._state["focus_reason"] = reason
|
||||
state._save_state(state._state) # Pass state dict as argument
|
||||
|
||||
logger.info(f"[ACTIONS] Focus updated: {old_focus} -> {focus}")
|
||||
|
||||
return {
|
||||
"old_focus": old_focus,
|
||||
"new_focus": focus,
|
||||
"reason": reason
|
||||
}
|
||||
|
||||
# ========================================
|
||||
# Utility Methods
|
||||
# ========================================
|
||||
|
||||
def get_allowed_actions(self) -> List[str]:
|
||||
"""Get list of all allowed action types."""
|
||||
return list(self.allowed_actions.keys())
|
||||
|
||||
def get_action_log(self, limit: int = 50) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Get recent action log.
|
||||
|
||||
Args:
|
||||
limit: Max number of entries to return
|
||||
|
||||
Returns:
|
||||
List of action records
|
||||
"""
|
||||
return self.action_log[-limit:]
|
||||
|
||||
def clear_action_log(self) -> None:
|
||||
"""Clear action log."""
|
||||
self.action_log = []
|
||||
logger.info("[ACTIONS] Action log cleared")
|
||||
|
||||
def validate_action(self, action_type: str, parameters: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""
|
||||
Validate an action without executing it.
|
||||
|
||||
Args:
|
||||
action_type: Type of action
|
||||
parameters: Action parameters
|
||||
|
||||
Returns:
|
||||
{
|
||||
"valid": bool,
|
||||
"action": action_type,
|
||||
"errors": [error messages] or []
|
||||
}
|
||||
"""
|
||||
errors = []
|
||||
|
||||
# Check whitelist
|
||||
if action_type not in self.allowed_actions:
|
||||
errors.append(f"Action '{action_type}' not in whitelist")
|
||||
|
||||
# Check required parameters (basic validation)
|
||||
if action_type == "create_memory" and not parameters.get("text"):
|
||||
errors.append("Memory 'text' parameter required")
|
||||
|
||||
if action_type == "update_goal" and not parameters.get("goal_id"):
|
||||
errors.append("Goal 'goal_id' parameter required")
|
||||
|
||||
if action_type == "schedule_reminder" and not parameters.get("message"):
|
||||
errors.append("Reminder 'message' parameter required")
|
||||
|
||||
if action_type == "learn_topic" and not parameters.get("topic"):
|
||||
errors.append("Learning 'topic' parameter required")
|
||||
|
||||
if action_type == "update_focus" and not parameters.get("focus"):
|
||||
errors.append("Focus 'focus' parameter required")
|
||||
|
||||
return {
|
||||
"valid": len(errors) == 0,
|
||||
"action": action_type,
|
||||
"errors": errors
|
||||
}
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_action_manager_instance = None
|
||||
|
||||
|
||||
def get_action_manager() -> AutonomousActionManager:
|
||||
"""
|
||||
Get singleton action manager instance.
|
||||
|
||||
Returns:
|
||||
AutonomousActionManager instance
|
||||
"""
|
||||
global _action_manager_instance
|
||||
if _action_manager_instance is None:
|
||||
_action_manager_instance = AutonomousActionManager()
|
||||
return _action_manager_instance
|
||||
@@ -0,0 +1 @@
|
||||
"""Executive planning and decision-making module."""
|
||||
@@ -0,0 +1,121 @@
|
||||
"""
|
||||
Executive planner - generates execution plans for complex requests.
|
||||
Activated when inner monologue sets consult_executive=true.
|
||||
"""
|
||||
|
||||
import os
|
||||
import logging
|
||||
from typing import Dict, Any, Optional
|
||||
from llm.llm_router import call_llm
|
||||
|
||||
EXECUTIVE_LLM = os.getenv("EXECUTIVE_LLM", "CLOUD").upper()
|
||||
VERBOSE_DEBUG = os.getenv("VERBOSE_DEBUG", "false").lower() == "true"
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
|
||||
EXECUTIVE_SYSTEM_PROMPT = """
|
||||
You are Lyra's executive planning system.
|
||||
You create structured execution plans for complex tasks.
|
||||
You do NOT generate the final response - only the plan.
|
||||
|
||||
Your plan should include:
|
||||
1. Task decomposition (break into steps)
|
||||
2. Required tools/resources
|
||||
3. Reasoning strategy
|
||||
4. Success criteria
|
||||
|
||||
Return a concise plan in natural language.
|
||||
"""
|
||||
|
||||
|
||||
async def plan_execution(
|
||||
user_prompt: str,
|
||||
intent: str,
|
||||
context_state: Dict[str, Any],
|
||||
identity_block: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Generate execution plan for complex request.
|
||||
|
||||
Args:
|
||||
user_prompt: User's message
|
||||
intent: Detected intent from inner monologue
|
||||
context_state: Full context
|
||||
identity_block: Lyra's identity
|
||||
|
||||
Returns:
|
||||
Plan dictionary with structure:
|
||||
{
|
||||
"summary": "One-line plan summary",
|
||||
"plan_text": "Detailed plan",
|
||||
"steps": ["step1", "step2", ...],
|
||||
"tools_needed": ["RAG", "WEB", ...],
|
||||
"estimated_complexity": "low | medium | high"
|
||||
}
|
||||
"""
|
||||
|
||||
# Build planning prompt
|
||||
tools_available = context_state.get("tools_available", [])
|
||||
|
||||
prompt = f"""{EXECUTIVE_SYSTEM_PROMPT}
|
||||
|
||||
User request: {user_prompt}
|
||||
|
||||
Detected intent: {intent}
|
||||
|
||||
Available tools: {", ".join(tools_available) if tools_available else "None"}
|
||||
|
||||
Session context:
|
||||
- Message count: {context_state.get('message_count', 0)}
|
||||
- Time since last message: {context_state.get('minutes_since_last_msg', 0):.1f} minutes
|
||||
- Active project: {context_state.get('active_project', 'None')}
|
||||
|
||||
Generate a structured execution plan.
|
||||
"""
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[EXECUTIVE] Planning prompt:\n{prompt}")
|
||||
|
||||
# Call executive LLM
|
||||
plan_text = await call_llm(
|
||||
prompt,
|
||||
backend=EXECUTIVE_LLM,
|
||||
temperature=0.3, # Lower temperature for planning
|
||||
max_tokens=500
|
||||
)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[EXECUTIVE] Generated plan:\n{plan_text}")
|
||||
|
||||
# Parse plan (simple heuristic extraction for Phase 1)
|
||||
steps = []
|
||||
tools_needed = []
|
||||
|
||||
for line in plan_text.split('\n'):
|
||||
line_lower = line.lower()
|
||||
if any(marker in line_lower for marker in ['step', '1.', '2.', '3.', '-']):
|
||||
steps.append(line.strip())
|
||||
|
||||
if tools_available:
|
||||
for tool in tools_available:
|
||||
if tool.lower() in line_lower and tool not in tools_needed:
|
||||
tools_needed.append(tool)
|
||||
|
||||
# Estimate complexity (simple heuristic)
|
||||
complexity = "low"
|
||||
if len(steps) > 3 or len(tools_needed) > 1:
|
||||
complexity = "medium"
|
||||
if len(steps) > 5 or "research" in intent.lower() or "analyze" in intent.lower():
|
||||
complexity = "high"
|
||||
|
||||
return {
|
||||
"summary": plan_text.split('\n')[0][:100] if plan_text else "Complex task execution plan",
|
||||
"plan_text": plan_text,
|
||||
"steps": steps[:10], # Limit to 10 steps
|
||||
"tools_needed": tools_needed,
|
||||
"estimated_complexity": complexity
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
"""Pattern learning and adaptation system."""
|
||||
@@ -0,0 +1,383 @@
|
||||
"""
|
||||
Pattern Learning System - learns from interaction patterns to improve autonomy.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import json
|
||||
import os
|
||||
from typing import Dict, List, Any, Optional
|
||||
from datetime import datetime
|
||||
from collections import defaultdict
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class PatternLearner:
|
||||
"""
|
||||
Learns from interaction patterns to improve Lyra's autonomous behavior.
|
||||
|
||||
Tracks:
|
||||
- Topic frequencies (what users talk about)
|
||||
- Time-of-day patterns (when users interact)
|
||||
- User preferences (how users like responses)
|
||||
- Successful response strategies (what works well)
|
||||
"""
|
||||
|
||||
def __init__(self, patterns_file: str = "/app/data/learned_patterns.json"):
|
||||
"""
|
||||
Initialize pattern learner.
|
||||
|
||||
Args:
|
||||
patterns_file: Path to persistent patterns storage
|
||||
"""
|
||||
self.patterns_file = patterns_file
|
||||
self.patterns = self._load_patterns()
|
||||
|
||||
def _load_patterns(self) -> Dict[str, Any]:
|
||||
"""Load patterns from disk."""
|
||||
if os.path.exists(self.patterns_file):
|
||||
try:
|
||||
with open(self.patterns_file, 'r') as f:
|
||||
patterns = json.load(f)
|
||||
logger.info(f"[PATTERN_LEARNER] Loaded patterns from {self.patterns_file}")
|
||||
return patterns
|
||||
except Exception as e:
|
||||
logger.error(f"[PATTERN_LEARNER] Failed to load patterns: {e}")
|
||||
|
||||
# Initialize empty patterns
|
||||
return {
|
||||
"topic_frequencies": {},
|
||||
"time_patterns": {},
|
||||
"user_preferences": {},
|
||||
"successful_strategies": {},
|
||||
"interaction_count": 0,
|
||||
"last_updated": datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
def _save_patterns(self) -> None:
|
||||
"""Save patterns to disk."""
|
||||
try:
|
||||
# Ensure directory exists
|
||||
os.makedirs(os.path.dirname(self.patterns_file), exist_ok=True)
|
||||
|
||||
self.patterns["last_updated"] = datetime.utcnow().isoformat()
|
||||
|
||||
with open(self.patterns_file, 'w') as f:
|
||||
json.dump(self.patterns, f, indent=2)
|
||||
|
||||
logger.debug(f"[PATTERN_LEARNER] Saved patterns to {self.patterns_file}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[PATTERN_LEARNER] Failed to save patterns: {e}")
|
||||
|
||||
async def learn_from_interaction(
|
||||
self,
|
||||
user_prompt: str,
|
||||
response: str,
|
||||
monologue: Dict[str, Any],
|
||||
context: Dict[str, Any]
|
||||
) -> None:
|
||||
"""
|
||||
Learn from a single interaction.
|
||||
|
||||
Args:
|
||||
user_prompt: User's message
|
||||
response: Lyra's response
|
||||
monologue: Inner monologue analysis
|
||||
context: Full context state
|
||||
"""
|
||||
self.patterns["interaction_count"] += 1
|
||||
|
||||
# Learn topic frequencies
|
||||
self._learn_topics(user_prompt, monologue)
|
||||
|
||||
# Learn time patterns
|
||||
self._learn_time_patterns()
|
||||
|
||||
# Learn user preferences
|
||||
self._learn_preferences(monologue, context)
|
||||
|
||||
# Learn successful strategies
|
||||
self._learn_strategies(monologue, response, context)
|
||||
|
||||
# Save periodically (every 10 interactions)
|
||||
if self.patterns["interaction_count"] % 10 == 0:
|
||||
self._save_patterns()
|
||||
|
||||
def _learn_topics(self, user_prompt: str, monologue: Dict[str, Any]) -> None:
|
||||
"""Track topic frequencies."""
|
||||
intent = monologue.get("intent", "unknown")
|
||||
|
||||
# Increment topic counter
|
||||
topic_freq = self.patterns["topic_frequencies"]
|
||||
topic_freq[intent] = topic_freq.get(intent, 0) + 1
|
||||
|
||||
# Extract keywords (simple approach - words > 5 chars)
|
||||
keywords = [word.lower() for word in user_prompt.split() if len(word) > 5]
|
||||
|
||||
for keyword in keywords:
|
||||
topic_freq[f"keyword:{keyword}"] = topic_freq.get(f"keyword:{keyword}", 0) + 1
|
||||
|
||||
logger.debug(f"[PATTERN_LEARNER] Topic learned: {intent}")
|
||||
|
||||
def _learn_time_patterns(self) -> None:
|
||||
"""Track time-of-day patterns."""
|
||||
now = datetime.utcnow()
|
||||
hour = now.hour
|
||||
|
||||
# Track interactions by hour
|
||||
time_patterns = self.patterns["time_patterns"]
|
||||
hour_key = f"hour_{hour:02d}"
|
||||
time_patterns[hour_key] = time_patterns.get(hour_key, 0) + 1
|
||||
|
||||
# Track day of week
|
||||
day_key = f"day_{now.strftime('%A').lower()}"
|
||||
time_patterns[day_key] = time_patterns.get(day_key, 0) + 1
|
||||
|
||||
def _learn_preferences(self, monologue: Dict[str, Any], context: Dict[str, Any]) -> None:
|
||||
"""Learn user preferences from detected tone and depth."""
|
||||
tone = monologue.get("tone", "neutral")
|
||||
depth = monologue.get("depth", "medium")
|
||||
|
||||
prefs = self.patterns["user_preferences"]
|
||||
|
||||
# Track preferred tone
|
||||
prefs.setdefault("tone_counts", {})
|
||||
prefs["tone_counts"][tone] = prefs["tone_counts"].get(tone, 0) + 1
|
||||
|
||||
# Track preferred depth
|
||||
prefs.setdefault("depth_counts", {})
|
||||
prefs["depth_counts"][depth] = prefs["depth_counts"].get(depth, 0) + 1
|
||||
|
||||
def _learn_strategies(
|
||||
self,
|
||||
monologue: Dict[str, Any],
|
||||
response: str,
|
||||
context: Dict[str, Any]
|
||||
) -> None:
|
||||
"""
|
||||
Learn which response strategies are successful.
|
||||
|
||||
Success indicators:
|
||||
- Executive was consulted and plan generated
|
||||
- Response length matches depth request
|
||||
- Tone matches request
|
||||
"""
|
||||
intent = monologue.get("intent", "unknown")
|
||||
executive_used = context.get("executive_plan") is not None
|
||||
|
||||
strategies = self.patterns["successful_strategies"]
|
||||
strategies.setdefault(intent, {})
|
||||
|
||||
# Track executive usage for this intent
|
||||
if executive_used:
|
||||
key = f"{intent}:executive_used"
|
||||
strategies.setdefault(key, 0)
|
||||
strategies[key] += 1
|
||||
|
||||
# Track response length patterns
|
||||
response_length = len(response.split())
|
||||
depth = monologue.get("depth", "medium")
|
||||
|
||||
length_key = f"{depth}:avg_words"
|
||||
if length_key not in strategies:
|
||||
strategies[length_key] = response_length
|
||||
else:
|
||||
# Running average
|
||||
strategies[length_key] = (strategies[length_key] + response_length) / 2
|
||||
|
||||
# ========================================
|
||||
# Pattern Analysis and Recommendations
|
||||
# ========================================
|
||||
|
||||
def get_top_topics(self, limit: int = 10) -> List[tuple]:
|
||||
"""
|
||||
Get most frequent topics.
|
||||
|
||||
Args:
|
||||
limit: Max number of topics to return
|
||||
|
||||
Returns:
|
||||
List of (topic, count) tuples, sorted by count
|
||||
"""
|
||||
topics = self.patterns["topic_frequencies"]
|
||||
sorted_topics = sorted(topics.items(), key=lambda x: x[1], reverse=True)
|
||||
return sorted_topics[:limit]
|
||||
|
||||
def get_preferred_tone(self) -> str:
|
||||
"""
|
||||
Get user's most preferred tone.
|
||||
|
||||
Returns:
|
||||
Preferred tone string
|
||||
"""
|
||||
prefs = self.patterns["user_preferences"]
|
||||
tone_counts = prefs.get("tone_counts", {})
|
||||
|
||||
if not tone_counts:
|
||||
return "neutral"
|
||||
|
||||
return max(tone_counts.items(), key=lambda x: x[1])[0]
|
||||
|
||||
def get_preferred_depth(self) -> str:
|
||||
"""
|
||||
Get user's most preferred response depth.
|
||||
|
||||
Returns:
|
||||
Preferred depth string
|
||||
"""
|
||||
prefs = self.patterns["user_preferences"]
|
||||
depth_counts = prefs.get("depth_counts", {})
|
||||
|
||||
if not depth_counts:
|
||||
return "medium"
|
||||
|
||||
return max(depth_counts.items(), key=lambda x: x[1])[0]
|
||||
|
||||
def get_peak_hours(self, limit: int = 3) -> List[int]:
|
||||
"""
|
||||
Get peak interaction hours.
|
||||
|
||||
Args:
|
||||
limit: Number of top hours to return
|
||||
|
||||
Returns:
|
||||
List of hours (0-23)
|
||||
"""
|
||||
time_patterns = self.patterns["time_patterns"]
|
||||
hour_counts = {k: v for k, v in time_patterns.items() if k.startswith("hour_")}
|
||||
|
||||
if not hour_counts:
|
||||
return []
|
||||
|
||||
sorted_hours = sorted(hour_counts.items(), key=lambda x: x[1], reverse=True)
|
||||
top_hours = sorted_hours[:limit]
|
||||
|
||||
# Extract hour numbers
|
||||
return [int(h[0].split("_")[1]) for h in top_hours]
|
||||
|
||||
def should_use_executive(self, intent: str) -> bool:
|
||||
"""
|
||||
Recommend whether to use executive for given intent based on patterns.
|
||||
|
||||
Args:
|
||||
intent: Intent type
|
||||
|
||||
Returns:
|
||||
True if executive is recommended
|
||||
"""
|
||||
strategies = self.patterns["successful_strategies"]
|
||||
key = f"{intent}:executive_used"
|
||||
|
||||
# If we've used executive for this intent >= 3 times, recommend it
|
||||
return strategies.get(key, 0) >= 3
|
||||
|
||||
def get_recommended_response_length(self, depth: str) -> int:
|
||||
"""
|
||||
Get recommended response length in words for given depth.
|
||||
|
||||
Args:
|
||||
depth: Depth level (short/medium/deep)
|
||||
|
||||
Returns:
|
||||
Recommended word count
|
||||
"""
|
||||
strategies = self.patterns["successful_strategies"]
|
||||
key = f"{depth}:avg_words"
|
||||
|
||||
avg_length = strategies.get(key, None)
|
||||
|
||||
if avg_length:
|
||||
return int(avg_length)
|
||||
|
||||
# Defaults if no pattern learned
|
||||
defaults = {
|
||||
"short": 50,
|
||||
"medium": 150,
|
||||
"deep": 300
|
||||
}
|
||||
|
||||
return defaults.get(depth, 150)
|
||||
|
||||
def get_insights(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Get high-level insights from learned patterns.
|
||||
|
||||
Returns:
|
||||
{
|
||||
"total_interactions": int,
|
||||
"top_topics": [(topic, count), ...],
|
||||
"preferred_tone": str,
|
||||
"preferred_depth": str,
|
||||
"peak_hours": [hours],
|
||||
"learning_recommendations": [str]
|
||||
}
|
||||
"""
|
||||
recommendations = []
|
||||
|
||||
# Check if user consistently prefers certain settings
|
||||
preferred_tone = self.get_preferred_tone()
|
||||
preferred_depth = self.get_preferred_depth()
|
||||
|
||||
if preferred_tone != "neutral":
|
||||
recommendations.append(f"User prefers {preferred_tone} tone")
|
||||
|
||||
if preferred_depth != "medium":
|
||||
recommendations.append(f"User prefers {preferred_depth} depth responses")
|
||||
|
||||
# Check for recurring topics
|
||||
top_topics = self.get_top_topics(limit=3)
|
||||
if top_topics:
|
||||
top_topic = top_topics[0][0]
|
||||
recommendations.append(f"Consider adding '{top_topic}' to learning queue")
|
||||
|
||||
return {
|
||||
"total_interactions": self.patterns["interaction_count"],
|
||||
"top_topics": self.get_top_topics(limit=5),
|
||||
"preferred_tone": preferred_tone,
|
||||
"preferred_depth": preferred_depth,
|
||||
"peak_hours": self.get_peak_hours(limit=3),
|
||||
"learning_recommendations": recommendations
|
||||
}
|
||||
|
||||
def reset_patterns(self) -> None:
|
||||
"""Reset all learned patterns (use with caution)."""
|
||||
self.patterns = {
|
||||
"topic_frequencies": {},
|
||||
"time_patterns": {},
|
||||
"user_preferences": {},
|
||||
"successful_strategies": {},
|
||||
"interaction_count": 0,
|
||||
"last_updated": datetime.utcnow().isoformat()
|
||||
}
|
||||
self._save_patterns()
|
||||
logger.warning("[PATTERN_LEARNER] Patterns reset")
|
||||
|
||||
def export_patterns(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Export all patterns for analysis.
|
||||
|
||||
Returns:
|
||||
Complete patterns dict
|
||||
"""
|
||||
return self.patterns.copy()
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_learner_instance = None
|
||||
|
||||
|
||||
def get_pattern_learner(patterns_file: str = "/app/data/learned_patterns.json") -> PatternLearner:
|
||||
"""
|
||||
Get singleton pattern learner instance.
|
||||
|
||||
Args:
|
||||
patterns_file: Path to patterns file (only used on first call)
|
||||
|
||||
Returns:
|
||||
PatternLearner instance
|
||||
"""
|
||||
global _learner_instance
|
||||
if _learner_instance is None:
|
||||
_learner_instance = PatternLearner(patterns_file=patterns_file)
|
||||
return _learner_instance
|
||||
@@ -0,0 +1 @@
|
||||
# Inner monologue module
|
||||
@@ -0,0 +1,115 @@
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
from typing import Dict
|
||||
from llm.llm_router import call_llm
|
||||
|
||||
# Configuration
|
||||
MONOLOGUE_LLM = os.getenv("MONOLOGUE_LLM", "PRIMARY").upper()
|
||||
VERBOSE_DEBUG = os.getenv("VERBOSE_DEBUG", "false").lower() == "true"
|
||||
|
||||
# Logger
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.setLevel(logging.DEBUG)
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setFormatter(logging.Formatter(
|
||||
'%(asctime)s [MONOLOGUE] %(levelname)s: %(message)s',
|
||||
datefmt='%H:%M:%S'
|
||||
))
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
MONOLOGUE_SYSTEM_PROMPT = """
|
||||
You are Lyra's inner monologue.
|
||||
You think privately.
|
||||
You do NOT speak to the user.
|
||||
You do NOT solve the task.
|
||||
You only reflect on intent, tone, and depth.
|
||||
|
||||
Return ONLY valid JSON with:
|
||||
- intent (string)
|
||||
- tone (neutral | warm | focused | playful | direct)
|
||||
- depth (short | medium | deep)
|
||||
- consult_executive (true | false)
|
||||
"""
|
||||
|
||||
class InnerMonologue:
|
||||
async def process(self, context: Dict) -> Dict:
|
||||
# Build full prompt with system instructions merged in
|
||||
full_prompt = f"""{MONOLOGUE_SYSTEM_PROMPT}
|
||||
|
||||
User message:
|
||||
{context['user_message']}
|
||||
|
||||
Self state:
|
||||
{context['self_state']}
|
||||
|
||||
Context summary:
|
||||
{context['context_summary']}
|
||||
|
||||
Output JSON only:
|
||||
"""
|
||||
|
||||
# Call LLM using configured backend
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[InnerMonologue] Calling LLM with backend: {MONOLOGUE_LLM}")
|
||||
logger.debug(f"[InnerMonologue] Prompt length: {len(full_prompt)} chars")
|
||||
|
||||
result = await call_llm(
|
||||
full_prompt,
|
||||
backend=MONOLOGUE_LLM,
|
||||
temperature=0.7,
|
||||
max_tokens=200
|
||||
)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[InnerMonologue] Raw LLM response:")
|
||||
logger.debug(f"{'='*80}")
|
||||
logger.debug(result)
|
||||
logger.debug(f"{'='*80}")
|
||||
logger.debug(f"[InnerMonologue] Response length: {len(result) if result else 0} chars")
|
||||
|
||||
# Parse JSON response - extract just the JSON part if there's extra text
|
||||
try:
|
||||
# Try direct parsing first
|
||||
parsed = json.loads(result)
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[InnerMonologue] Successfully parsed JSON directly: {parsed}")
|
||||
return parsed
|
||||
except json.JSONDecodeError:
|
||||
# If direct parsing fails, try to extract JSON from the response
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[InnerMonologue] Direct JSON parse failed, attempting extraction...")
|
||||
|
||||
# Look for JSON object (starts with { and ends with })
|
||||
import re
|
||||
json_match = re.search(r'\{[^{}]*(?:\{[^{}]*\}[^{}]*)*\}', result, re.DOTALL)
|
||||
|
||||
if json_match:
|
||||
json_str = json_match.group(0)
|
||||
try:
|
||||
parsed = json.loads(json_str)
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[InnerMonologue] Successfully extracted and parsed JSON: {parsed}")
|
||||
return parsed
|
||||
except json.JSONDecodeError as e:
|
||||
if VERBOSE_DEBUG:
|
||||
logger.warning(f"[InnerMonologue] Extracted JSON still invalid: {e}")
|
||||
else:
|
||||
if VERBOSE_DEBUG:
|
||||
logger.warning(f"[InnerMonologue] No JSON object found in response")
|
||||
|
||||
# Final fallback
|
||||
if VERBOSE_DEBUG:
|
||||
logger.warning(f"[InnerMonologue] All parsing attempts failed, using fallback")
|
||||
else:
|
||||
print(f"[InnerMonologue] JSON extraction failed")
|
||||
print(f"[InnerMonologue] Raw response was: {result[:500]}")
|
||||
|
||||
return {
|
||||
"intent": "unknown",
|
||||
"tone": "neutral",
|
||||
"depth": "medium",
|
||||
"consult_executive": False
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
"""Proactive monitoring and suggestion system."""
|
||||
@@ -0,0 +1,321 @@
|
||||
"""
|
||||
Proactive Context Monitor - detects opportunities for autonomous suggestions.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
from typing import Dict, List, Any, Optional
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ProactiveMonitor:
|
||||
"""
|
||||
Monitors conversation context and detects opportunities for proactive suggestions.
|
||||
|
||||
Triggers:
|
||||
- Long silence → Check-in
|
||||
- Learning queue + high curiosity → Suggest exploration
|
||||
- Active goals → Progress reminders
|
||||
- Conversation milestones → Offer summary
|
||||
- Pattern detection → Helpful suggestions
|
||||
"""
|
||||
|
||||
def __init__(self, min_priority: float = 0.6):
|
||||
"""
|
||||
Initialize proactive monitor.
|
||||
|
||||
Args:
|
||||
min_priority: Minimum priority for suggestions (0.0-1.0)
|
||||
"""
|
||||
self.min_priority = min_priority
|
||||
self.last_suggestion_time = {} # session_id -> timestamp
|
||||
self.cooldown_seconds = 300 # 5 minutes between proactive suggestions
|
||||
|
||||
async def analyze_session(
|
||||
self,
|
||||
session_id: str,
|
||||
context_state: Dict[str, Any],
|
||||
self_state: Dict[str, Any]
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Analyze session for proactive suggestion opportunities.
|
||||
|
||||
Args:
|
||||
session_id: Current session ID
|
||||
context_state: Full context including message history
|
||||
self_state: Lyra's current self-state
|
||||
|
||||
Returns:
|
||||
{
|
||||
"suggestion": "text to append to response",
|
||||
"priority": 0.0-1.0,
|
||||
"reason": "why this suggestion",
|
||||
"type": "check_in | learning | goal_reminder | summary | pattern"
|
||||
}
|
||||
or None if no suggestion
|
||||
"""
|
||||
# Check cooldown
|
||||
if not self._check_cooldown(session_id):
|
||||
logger.debug(f"[PROACTIVE] Session {session_id} in cooldown, skipping")
|
||||
return None
|
||||
|
||||
suggestions = []
|
||||
|
||||
# Check 1: Long silence detection
|
||||
silence_suggestion = self._check_long_silence(context_state)
|
||||
if silence_suggestion:
|
||||
suggestions.append(silence_suggestion)
|
||||
|
||||
# Check 2: Learning queue + high curiosity
|
||||
learning_suggestion = self._check_learning_opportunity(self_state)
|
||||
if learning_suggestion:
|
||||
suggestions.append(learning_suggestion)
|
||||
|
||||
# Check 3: Active goals reminder
|
||||
goal_suggestion = self._check_active_goals(self_state, context_state)
|
||||
if goal_suggestion:
|
||||
suggestions.append(goal_suggestion)
|
||||
|
||||
# Check 4: Conversation milestones
|
||||
milestone_suggestion = self._check_conversation_milestone(context_state)
|
||||
if milestone_suggestion:
|
||||
suggestions.append(milestone_suggestion)
|
||||
|
||||
# Check 5: Pattern-based suggestions
|
||||
pattern_suggestion = self._check_patterns(context_state, self_state)
|
||||
if pattern_suggestion:
|
||||
suggestions.append(pattern_suggestion)
|
||||
|
||||
# Filter by priority and return highest
|
||||
valid_suggestions = [s for s in suggestions if s["priority"] >= self.min_priority]
|
||||
|
||||
if not valid_suggestions:
|
||||
return None
|
||||
|
||||
# Return highest priority suggestion
|
||||
best_suggestion = max(valid_suggestions, key=lambda x: x["priority"])
|
||||
|
||||
# Update cooldown timer
|
||||
self._update_cooldown(session_id)
|
||||
|
||||
logger.info(f"[PROACTIVE] Suggestion generated: {best_suggestion['type']} (priority: {best_suggestion['priority']:.2f})")
|
||||
|
||||
return best_suggestion
|
||||
|
||||
def _check_cooldown(self, session_id: str) -> bool:
|
||||
"""Check if session is past cooldown period."""
|
||||
if session_id not in self.last_suggestion_time:
|
||||
return True
|
||||
|
||||
elapsed = time.time() - self.last_suggestion_time[session_id]
|
||||
return elapsed >= self.cooldown_seconds
|
||||
|
||||
def _update_cooldown(self, session_id: str) -> None:
|
||||
"""Update cooldown timer for session."""
|
||||
self.last_suggestion_time[session_id] = time.time()
|
||||
|
||||
def _check_long_silence(self, context_state: Dict[str, Any]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Check if user has been silent for a long time.
|
||||
"""
|
||||
minutes_since_last = context_state.get("minutes_since_last_msg", 0)
|
||||
|
||||
# If > 30 minutes, suggest check-in
|
||||
if minutes_since_last > 30:
|
||||
return {
|
||||
"suggestion": "\n\n[Aside: I'm still here if you need anything!]",
|
||||
"priority": 0.7,
|
||||
"reason": f"User silent for {minutes_since_last:.0f} minutes",
|
||||
"type": "check_in"
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
def _check_learning_opportunity(self, self_state: Dict[str, Any]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Check if Lyra has learning queue items and high curiosity.
|
||||
"""
|
||||
learning_queue = self_state.get("learning_queue", [])
|
||||
curiosity = self_state.get("curiosity", 0.5)
|
||||
|
||||
# If curiosity > 0.7 and learning queue exists
|
||||
if curiosity > 0.7 and learning_queue:
|
||||
topic = learning_queue[0] if learning_queue else "new topics"
|
||||
return {
|
||||
"suggestion": f"\n\n[Aside: I've been curious about {topic} lately. Would you like to explore it together?]",
|
||||
"priority": 0.65,
|
||||
"reason": f"High curiosity ({curiosity:.2f}) and learning queue present",
|
||||
"type": "learning"
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
def _check_active_goals(
|
||||
self,
|
||||
self_state: Dict[str, Any],
|
||||
context_state: Dict[str, Any]
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Check if there are active goals worth reminding about.
|
||||
"""
|
||||
active_goals = self_state.get("active_goals", [])
|
||||
|
||||
if not active_goals:
|
||||
return None
|
||||
|
||||
# Check if we've had multiple messages without goal progress
|
||||
message_count = context_state.get("message_count", 0)
|
||||
|
||||
# Every 10 messages, consider goal reminder
|
||||
if message_count % 10 == 0 and message_count > 0:
|
||||
goal = active_goals[0] # First active goal
|
||||
goal_name = goal if isinstance(goal, str) else goal.get("name", "your goal")
|
||||
|
||||
return {
|
||||
"suggestion": f"\n\n[Aside: Still thinking about {goal_name}. Let me know if you want to work on it.]",
|
||||
"priority": 0.6,
|
||||
"reason": f"Active goal present, {message_count} messages since start",
|
||||
"type": "goal_reminder"
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
def _check_conversation_milestone(self, context_state: Dict[str, Any]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Check for conversation milestones (e.g., every 50 messages).
|
||||
"""
|
||||
message_count = context_state.get("message_count", 0)
|
||||
|
||||
# Every 50 messages, offer summary
|
||||
if message_count > 0 and message_count % 50 == 0:
|
||||
return {
|
||||
"suggestion": f"\n\n[Aside: We've exchanged {message_count} messages! Would you like a summary of our conversation?]",
|
||||
"priority": 0.65,
|
||||
"reason": f"Milestone: {message_count} messages",
|
||||
"type": "summary"
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
def _check_patterns(
|
||||
self,
|
||||
context_state: Dict[str, Any],
|
||||
self_state: Dict[str, Any]
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Check for behavioral patterns that merit suggestions.
|
||||
"""
|
||||
# Get current focus
|
||||
focus = self_state.get("focus", "")
|
||||
|
||||
# Check if user keeps asking similar questions (detected via focus)
|
||||
if focus and "repeated" in focus.lower():
|
||||
return {
|
||||
"suggestion": "\n\n[Aside: I notice we keep coming back to this topic. Would it help to create a summary or action plan?]",
|
||||
"priority": 0.7,
|
||||
"reason": "Repeated topic detected",
|
||||
"type": "pattern"
|
||||
}
|
||||
|
||||
# Check energy levels - if Lyra is low energy, maybe suggest break
|
||||
energy = self_state.get("energy", 0.8)
|
||||
if energy < 0.3:
|
||||
return {
|
||||
"suggestion": "\n\n[Aside: We've been at this for a while. Need a break or want to keep going?]",
|
||||
"priority": 0.65,
|
||||
"reason": f"Low energy ({energy:.2f})",
|
||||
"type": "pattern"
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
def format_suggestion(self, suggestion: Dict[str, Any]) -> str:
|
||||
"""
|
||||
Format suggestion for appending to response.
|
||||
|
||||
Args:
|
||||
suggestion: Suggestion dict from analyze_session()
|
||||
|
||||
Returns:
|
||||
Formatted string to append to response
|
||||
"""
|
||||
return suggestion.get("suggestion", "")
|
||||
|
||||
def set_cooldown_duration(self, seconds: int) -> None:
|
||||
"""
|
||||
Update cooldown duration.
|
||||
|
||||
Args:
|
||||
seconds: New cooldown duration
|
||||
"""
|
||||
self.cooldown_seconds = seconds
|
||||
logger.info(f"[PROACTIVE] Cooldown updated to {seconds}s")
|
||||
|
||||
def reset_cooldown(self, session_id: str) -> None:
|
||||
"""
|
||||
Reset cooldown for a specific session.
|
||||
|
||||
Args:
|
||||
session_id: Session to reset
|
||||
"""
|
||||
if session_id in self.last_suggestion_time:
|
||||
del self.last_suggestion_time[session_id]
|
||||
logger.info(f"[PROACTIVE] Cooldown reset for session {session_id}")
|
||||
|
||||
def get_session_stats(self, session_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Get stats for a session's proactive monitoring.
|
||||
|
||||
Args:
|
||||
session_id: Session to check
|
||||
|
||||
Returns:
|
||||
{
|
||||
"last_suggestion_time": timestamp or None,
|
||||
"seconds_since_last": int,
|
||||
"cooldown_active": bool,
|
||||
"cooldown_remaining": int
|
||||
}
|
||||
"""
|
||||
last_time = self.last_suggestion_time.get(session_id)
|
||||
|
||||
if not last_time:
|
||||
return {
|
||||
"last_suggestion_time": None,
|
||||
"seconds_since_last": 0,
|
||||
"cooldown_active": False,
|
||||
"cooldown_remaining": 0
|
||||
}
|
||||
|
||||
seconds_since = int(time.time() - last_time)
|
||||
cooldown_active = seconds_since < self.cooldown_seconds
|
||||
cooldown_remaining = max(0, self.cooldown_seconds - seconds_since)
|
||||
|
||||
return {
|
||||
"last_suggestion_time": last_time,
|
||||
"seconds_since_last": seconds_since,
|
||||
"cooldown_active": cooldown_active,
|
||||
"cooldown_remaining": cooldown_remaining
|
||||
}
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_monitor_instance = None
|
||||
|
||||
|
||||
def get_proactive_monitor(min_priority: float = 0.6) -> ProactiveMonitor:
|
||||
"""
|
||||
Get singleton proactive monitor instance.
|
||||
|
||||
Args:
|
||||
min_priority: Minimum priority threshold (only used on first call)
|
||||
|
||||
Returns:
|
||||
ProactiveMonitor instance
|
||||
"""
|
||||
global _monitor_instance
|
||||
if _monitor_instance is None:
|
||||
_monitor_instance = ProactiveMonitor(min_priority=min_priority)
|
||||
return _monitor_instance
|
||||
@@ -0,0 +1 @@
|
||||
# Self state module
|
||||
@@ -0,0 +1,74 @@
|
||||
"""
|
||||
Analyze interactions and update self-state accordingly.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, Any
|
||||
from .state import update_self_state
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def analyze_and_update_state(
|
||||
monologue: Dict[str, Any],
|
||||
user_prompt: str,
|
||||
response: str,
|
||||
context: Dict[str, Any]
|
||||
) -> None:
|
||||
"""
|
||||
Analyze interaction and update self-state.
|
||||
|
||||
This runs after response generation to update Lyra's internal state
|
||||
based on the interaction.
|
||||
|
||||
Args:
|
||||
monologue: Inner monologue output
|
||||
user_prompt: User's message
|
||||
response: Lyra's response
|
||||
context: Full context state
|
||||
"""
|
||||
|
||||
# Simple heuristics for state updates
|
||||
# TODO: Replace with LLM-based sentiment analysis in Phase 2
|
||||
|
||||
mood_delta = 0.0
|
||||
energy_delta = 0.0
|
||||
confidence_delta = 0.0
|
||||
curiosity_delta = 0.0
|
||||
new_focus = None
|
||||
|
||||
# Analyze intent from monologue
|
||||
intent = monologue.get("intent", "").lower() if monologue else ""
|
||||
|
||||
if "technical" in intent or "complex" in intent:
|
||||
energy_delta = -0.05 # Deep thinking is tiring
|
||||
confidence_delta = 0.05 if len(response) > 200 else -0.05
|
||||
new_focus = "technical_problem"
|
||||
|
||||
elif "creative" in intent or "brainstorm" in intent:
|
||||
mood_delta = 0.1 # Creative work is engaging
|
||||
curiosity_delta = 0.1
|
||||
new_focus = "creative_exploration"
|
||||
|
||||
elif "clarification" in intent or "confused" in intent:
|
||||
confidence_delta = -0.05
|
||||
new_focus = "understanding_user"
|
||||
|
||||
elif "simple" in intent or "casual" in intent:
|
||||
energy_delta = 0.05 # Light conversation is refreshing
|
||||
new_focus = "conversation"
|
||||
|
||||
# Check for learning opportunities (questions in user prompt)
|
||||
if "?" in user_prompt and any(word in user_prompt.lower() for word in ["how", "why", "what"]):
|
||||
curiosity_delta += 0.05
|
||||
|
||||
# Update state
|
||||
update_self_state(
|
||||
mood_delta=mood_delta,
|
||||
energy_delta=energy_delta,
|
||||
new_focus=new_focus,
|
||||
confidence_delta=confidence_delta,
|
||||
curiosity_delta=curiosity_delta
|
||||
)
|
||||
|
||||
logger.info(f"Self-state updated based on interaction: focus={new_focus}")
|
||||
@@ -0,0 +1,189 @@
|
||||
"""
|
||||
Self-state management for Project Lyra.
|
||||
Maintains persistent identity, mood, energy, and focus across sessions.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
# Configuration
|
||||
STATE_FILE = Path(os.getenv("SELF_STATE_FILE", "/app/data/self_state.json"))
|
||||
VERBOSE_DEBUG = os.getenv("VERBOSE_DEBUG", "false").lower() == "true"
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
# Default state structure
|
||||
DEFAULT_STATE = {
|
||||
"mood": "neutral",
|
||||
"energy": 0.8,
|
||||
"focus": "user_request",
|
||||
"confidence": 0.7,
|
||||
"curiosity": 0.5,
|
||||
"last_updated": None,
|
||||
"interaction_count": 0,
|
||||
"learning_queue": [], # Topics Lyra wants to explore
|
||||
"active_goals": [], # Self-directed goals
|
||||
"preferences": {
|
||||
"verbosity": "medium",
|
||||
"formality": "casual",
|
||||
"proactivity": 0.3 # How likely to suggest things unprompted
|
||||
},
|
||||
"metadata": {
|
||||
"version": "1.0",
|
||||
"created_at": None
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
class SelfState:
|
||||
"""Manages Lyra's persistent self-state."""
|
||||
|
||||
def __init__(self):
|
||||
self._state = self._load_state()
|
||||
|
||||
def _load_state(self) -> Dict[str, Any]:
|
||||
"""Load state from disk or create default."""
|
||||
if STATE_FILE.exists():
|
||||
try:
|
||||
with open(STATE_FILE, 'r') as f:
|
||||
state = json.load(f)
|
||||
logger.info(f"Loaded self-state from {STATE_FILE}")
|
||||
return state
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to load self-state: {e}")
|
||||
return self._create_default_state()
|
||||
else:
|
||||
return self._create_default_state()
|
||||
|
||||
def _create_default_state(self) -> Dict[str, Any]:
|
||||
"""Create and save default state."""
|
||||
state = DEFAULT_STATE.copy()
|
||||
state["metadata"]["created_at"] = datetime.now().isoformat()
|
||||
state["last_updated"] = datetime.now().isoformat()
|
||||
self._save_state(state)
|
||||
logger.info("Created new default self-state")
|
||||
return state
|
||||
|
||||
def _save_state(self, state: Dict[str, Any]) -> None:
|
||||
"""Persist state to disk."""
|
||||
try:
|
||||
STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(STATE_FILE, 'w') as f:
|
||||
json.dump(state, f, indent=2)
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"Saved self-state to {STATE_FILE}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to save self-state: {e}")
|
||||
|
||||
def get_state(self) -> Dict[str, Any]:
|
||||
"""Get current state snapshot."""
|
||||
return self._state.copy()
|
||||
|
||||
def update_from_interaction(
|
||||
self,
|
||||
mood_delta: float = 0.0,
|
||||
energy_delta: float = 0.0,
|
||||
new_focus: Optional[str] = None,
|
||||
confidence_delta: float = 0.0,
|
||||
curiosity_delta: float = 0.0
|
||||
) -> None:
|
||||
"""
|
||||
Update state based on interaction.
|
||||
|
||||
Args:
|
||||
mood_delta: Change in mood (-1.0 to 1.0)
|
||||
energy_delta: Change in energy (-1.0 to 1.0)
|
||||
new_focus: New focus area
|
||||
confidence_delta: Change in confidence
|
||||
curiosity_delta: Change in curiosity
|
||||
"""
|
||||
# Apply deltas with bounds checking
|
||||
self._state["energy"] = max(0.0, min(1.0,
|
||||
self._state.get("energy", 0.8) + energy_delta))
|
||||
|
||||
self._state["confidence"] = max(0.0, min(1.0,
|
||||
self._state.get("confidence", 0.7) + confidence_delta))
|
||||
|
||||
self._state["curiosity"] = max(0.0, min(1.0,
|
||||
self._state.get("curiosity", 0.5) + curiosity_delta))
|
||||
|
||||
# Update focus if provided
|
||||
if new_focus:
|
||||
self._state["focus"] = new_focus
|
||||
|
||||
# Update mood (simplified sentiment)
|
||||
if mood_delta != 0:
|
||||
mood_map = ["frustrated", "neutral", "engaged", "excited"]
|
||||
current_mood_idx = 1 # neutral default
|
||||
if self._state.get("mood") in mood_map:
|
||||
current_mood_idx = mood_map.index(self._state["mood"])
|
||||
|
||||
new_mood_idx = max(0, min(len(mood_map) - 1,
|
||||
int(current_mood_idx + mood_delta * 2)))
|
||||
self._state["mood"] = mood_map[new_mood_idx]
|
||||
|
||||
# Increment interaction counter
|
||||
self._state["interaction_count"] = self._state.get("interaction_count", 0) + 1
|
||||
self._state["last_updated"] = datetime.now().isoformat()
|
||||
|
||||
# Persist changes
|
||||
self._save_state(self._state)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"Updated self-state: mood={self._state['mood']}, "
|
||||
f"energy={self._state['energy']:.2f}, "
|
||||
f"confidence={self._state['confidence']:.2f}")
|
||||
|
||||
def add_learning_goal(self, topic: str) -> None:
|
||||
"""Add topic to learning queue."""
|
||||
queue = self._state.get("learning_queue", [])
|
||||
if topic not in [item.get("topic") for item in queue]:
|
||||
queue.append({
|
||||
"topic": topic,
|
||||
"added_at": datetime.now().isoformat(),
|
||||
"priority": 0.5
|
||||
})
|
||||
self._state["learning_queue"] = queue
|
||||
self._save_state(self._state)
|
||||
logger.info(f"Added learning goal: {topic}")
|
||||
|
||||
def add_active_goal(self, goal: str, context: str = "") -> None:
|
||||
"""Add self-directed goal."""
|
||||
goals = self._state.get("active_goals", [])
|
||||
goals.append({
|
||||
"goal": goal,
|
||||
"context": context,
|
||||
"created_at": datetime.now().isoformat(),
|
||||
"status": "active"
|
||||
})
|
||||
self._state["active_goals"] = goals
|
||||
self._save_state(self._state)
|
||||
logger.info(f"Added active goal: {goal}")
|
||||
|
||||
|
||||
# Global instance
|
||||
_self_state_instance = None
|
||||
|
||||
def get_self_state_instance() -> SelfState:
|
||||
"""Get or create global SelfState instance."""
|
||||
global _self_state_instance
|
||||
if _self_state_instance is None:
|
||||
_self_state_instance = SelfState()
|
||||
return _self_state_instance
|
||||
|
||||
|
||||
def load_self_state() -> Dict[str, Any]:
|
||||
"""Load self state - public API for backwards compatibility."""
|
||||
return get_self_state_instance().get_state()
|
||||
|
||||
|
||||
def update_self_state(**kwargs) -> None:
|
||||
"""Update self state - public API."""
|
||||
get_self_state_instance().update_from_interaction(**kwargs)
|
||||
@@ -0,0 +1 @@
|
||||
"""Autonomous tool invocation system."""
|
||||
@@ -0,0 +1,124 @@
|
||||
"""
|
||||
Tool Decision Engine - decides which tools to invoke autonomously.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from typing import Dict, List, Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ToolDecisionEngine:
|
||||
"""Decides which tools to invoke based on context analysis."""
|
||||
|
||||
async def analyze_tool_needs(
|
||||
self,
|
||||
user_prompt: str,
|
||||
monologue: Dict[str, Any],
|
||||
context_state: Dict[str, Any],
|
||||
available_tools: List[str]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Analyze if tools should be invoked and which ones.
|
||||
|
||||
Args:
|
||||
user_prompt: User's message
|
||||
monologue: Inner monologue analysis
|
||||
context_state: Full context
|
||||
available_tools: List of available tools
|
||||
|
||||
Returns:
|
||||
{
|
||||
"should_invoke_tools": bool,
|
||||
"tools_to_invoke": [
|
||||
{
|
||||
"tool": "RAG | WEB | WEATHER | etc",
|
||||
"query": "search query",
|
||||
"reason": "why this tool",
|
||||
"priority": 0.0-1.0
|
||||
},
|
||||
...
|
||||
],
|
||||
"confidence": 0.0-1.0
|
||||
}
|
||||
"""
|
||||
|
||||
tools_to_invoke = []
|
||||
|
||||
# Check for memory/context needs
|
||||
if any(word in user_prompt.lower() for word in [
|
||||
"remember", "you said", "we discussed", "earlier", "before",
|
||||
"last time", "previously", "what did"
|
||||
]):
|
||||
tools_to_invoke.append({
|
||||
"tool": "RAG",
|
||||
"query": user_prompt,
|
||||
"reason": "User references past conversation",
|
||||
"priority": 0.9
|
||||
})
|
||||
|
||||
# Check for web search needs
|
||||
if any(word in user_prompt.lower() for word in [
|
||||
"current", "latest", "news", "today", "what's happening",
|
||||
"look up", "search for", "find information", "recent"
|
||||
]):
|
||||
tools_to_invoke.append({
|
||||
"tool": "WEB",
|
||||
"query": user_prompt,
|
||||
"reason": "Requires current information",
|
||||
"priority": 0.8
|
||||
})
|
||||
|
||||
# Check for weather needs
|
||||
if any(word in user_prompt.lower() for word in [
|
||||
"weather", "temperature", "forecast", "rain", "sunny", "climate"
|
||||
]):
|
||||
tools_to_invoke.append({
|
||||
"tool": "WEATHER",
|
||||
"query": user_prompt,
|
||||
"reason": "Weather information requested",
|
||||
"priority": 0.95
|
||||
})
|
||||
|
||||
# Check for code-related needs
|
||||
if any(word in user_prompt.lower() for word in [
|
||||
"code", "function", "debug", "implement", "algorithm",
|
||||
"programming", "script", "syntax"
|
||||
]):
|
||||
if "CODEBRAIN" in available_tools:
|
||||
tools_to_invoke.append({
|
||||
"tool": "CODEBRAIN",
|
||||
"query": user_prompt,
|
||||
"reason": "Code-related task",
|
||||
"priority": 0.85
|
||||
})
|
||||
|
||||
# Proactive RAG for complex queries (based on monologue)
|
||||
intent = monologue.get("intent", "") if monologue else ""
|
||||
if monologue and monologue.get("consult_executive"):
|
||||
# Complex query - might benefit from context
|
||||
if not any(t["tool"] == "RAG" for t in tools_to_invoke):
|
||||
tools_to_invoke.append({
|
||||
"tool": "RAG",
|
||||
"query": user_prompt,
|
||||
"reason": "Complex query benefits from context",
|
||||
"priority": 0.6
|
||||
})
|
||||
|
||||
# Sort by priority
|
||||
tools_to_invoke.sort(key=lambda x: x["priority"], reverse=True)
|
||||
|
||||
max_priority = max([t["priority"] for t in tools_to_invoke]) if tools_to_invoke else 0.0
|
||||
|
||||
result = {
|
||||
"should_invoke_tools": len(tools_to_invoke) > 0,
|
||||
"tools_to_invoke": tools_to_invoke,
|
||||
"confidence": max_priority
|
||||
}
|
||||
|
||||
if tools_to_invoke:
|
||||
logger.info(f"[TOOL_DECISION] Autonomous tool invocation recommended: {len(tools_to_invoke)} tools")
|
||||
for tool in tools_to_invoke:
|
||||
logger.info(f" - {tool['tool']} (priority: {tool['priority']:.2f}): {tool['reason']}")
|
||||
|
||||
return result
|
||||
@@ -0,0 +1,357 @@
|
||||
"""
|
||||
Tool Orchestrator - executes autonomous tool invocations asynchronously.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Dict, List, Any, Optional
|
||||
import os
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ToolOrchestrator:
|
||||
"""Orchestrates async tool execution and result aggregation."""
|
||||
|
||||
def __init__(self, tool_timeout: int = 30):
|
||||
"""
|
||||
Initialize orchestrator.
|
||||
|
||||
Args:
|
||||
tool_timeout: Max seconds per tool call (default 30)
|
||||
"""
|
||||
self.tool_timeout = tool_timeout
|
||||
self.available_tools = self._discover_tools()
|
||||
|
||||
def _discover_tools(self) -> Dict[str, Any]:
|
||||
"""Discover available tool modules."""
|
||||
tools = {}
|
||||
|
||||
# Import tool modules as they become available
|
||||
if os.getenv("NEOMEM_ENABLED", "false").lower() == "true":
|
||||
try:
|
||||
from memory.neomem_client import search_neomem
|
||||
tools["RAG"] = search_neomem
|
||||
logger.debug("[ORCHESTRATOR] RAG tool available")
|
||||
except ImportError:
|
||||
logger.debug("[ORCHESTRATOR] RAG tool not available")
|
||||
else:
|
||||
logger.info("[ORCHESTRATOR] NEOMEM_ENABLED is false; RAG tool disabled")
|
||||
|
||||
try:
|
||||
from integrations.web_search import web_search
|
||||
tools["WEB"] = web_search
|
||||
logger.debug("[ORCHESTRATOR] WEB tool available")
|
||||
except ImportError:
|
||||
logger.debug("[ORCHESTRATOR] WEB tool not available")
|
||||
|
||||
try:
|
||||
from integrations.weather import get_weather
|
||||
tools["WEATHER"] = get_weather
|
||||
logger.debug("[ORCHESTRATOR] WEATHER tool available")
|
||||
except ImportError:
|
||||
logger.debug("[ORCHESTRATOR] WEATHER tool not available")
|
||||
|
||||
try:
|
||||
from integrations.codebrain import query_codebrain
|
||||
tools["CODEBRAIN"] = query_codebrain
|
||||
logger.debug("[ORCHESTRATOR] CODEBRAIN tool available")
|
||||
except ImportError:
|
||||
logger.debug("[ORCHESTRATOR] CODEBRAIN tool not available")
|
||||
|
||||
return tools
|
||||
|
||||
async def execute_tools(
|
||||
self,
|
||||
tools_to_invoke: List[Dict[str, Any]],
|
||||
context_state: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute multiple tools asynchronously.
|
||||
|
||||
Args:
|
||||
tools_to_invoke: List of tool specs from decision engine
|
||||
[{"tool": "RAG", "query": "...", "reason": "...", "priority": 0.9}, ...]
|
||||
context_state: Full context for tool execution
|
||||
|
||||
Returns:
|
||||
{
|
||||
"results": {
|
||||
"RAG": {...},
|
||||
"WEB": {...},
|
||||
...
|
||||
},
|
||||
"execution_summary": {
|
||||
"tools_invoked": ["RAG", "WEB"],
|
||||
"successful": ["RAG"],
|
||||
"failed": ["WEB"],
|
||||
"total_time_ms": 1234
|
||||
}
|
||||
}
|
||||
"""
|
||||
import time
|
||||
start_time = time.time()
|
||||
|
||||
logger.info(f"[ORCHESTRATOR] Executing {len(tools_to_invoke)} tools asynchronously")
|
||||
|
||||
# Create tasks for each tool
|
||||
tasks = []
|
||||
tool_names = []
|
||||
|
||||
for tool_spec in tools_to_invoke:
|
||||
tool_name = tool_spec["tool"]
|
||||
query = tool_spec["query"]
|
||||
|
||||
if tool_name in self.available_tools:
|
||||
task = self._execute_single_tool(tool_name, query, context_state)
|
||||
tasks.append(task)
|
||||
tool_names.append(tool_name)
|
||||
logger.debug(f"[ORCHESTRATOR] Queued {tool_name}: {query[:50]}...")
|
||||
else:
|
||||
logger.warning(f"[ORCHESTRATOR] Tool {tool_name} not available, skipping")
|
||||
|
||||
# Execute all tools concurrently with timeout
|
||||
results = {}
|
||||
successful = []
|
||||
failed = []
|
||||
|
||||
if tasks:
|
||||
try:
|
||||
# Wait for all tasks with global timeout
|
||||
completed = await asyncio.wait_for(
|
||||
asyncio.gather(*tasks, return_exceptions=True),
|
||||
timeout=self.tool_timeout
|
||||
)
|
||||
|
||||
# Process results
|
||||
for tool_name, result in zip(tool_names, completed):
|
||||
if isinstance(result, Exception):
|
||||
logger.error(f"[ORCHESTRATOR] {tool_name} failed: {result}")
|
||||
results[tool_name] = {"error": str(result), "success": False}
|
||||
failed.append(tool_name)
|
||||
else:
|
||||
logger.info(f"[ORCHESTRATOR] {tool_name} completed successfully")
|
||||
results[tool_name] = result
|
||||
successful.append(tool_name)
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
logger.error(f"[ORCHESTRATOR] Global timeout ({self.tool_timeout}s) exceeded")
|
||||
for tool_name in tool_names:
|
||||
if tool_name not in results:
|
||||
results[tool_name] = {"error": "timeout", "success": False}
|
||||
failed.append(tool_name)
|
||||
|
||||
end_time = time.time()
|
||||
total_time_ms = int((end_time - start_time) * 1000)
|
||||
|
||||
execution_summary = {
|
||||
"tools_invoked": tool_names,
|
||||
"successful": successful,
|
||||
"failed": failed,
|
||||
"total_time_ms": total_time_ms
|
||||
}
|
||||
|
||||
logger.info(f"[ORCHESTRATOR] Execution complete: {len(successful)}/{len(tool_names)} successful in {total_time_ms}ms")
|
||||
|
||||
return {
|
||||
"results": results,
|
||||
"execution_summary": execution_summary
|
||||
}
|
||||
|
||||
async def _execute_single_tool(
|
||||
self,
|
||||
tool_name: str,
|
||||
query: str,
|
||||
context_state: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute a single tool with error handling.
|
||||
|
||||
Args:
|
||||
tool_name: Name of tool (RAG, WEB, etc.)
|
||||
query: Query string for the tool
|
||||
context_state: Context for tool execution
|
||||
|
||||
Returns:
|
||||
Tool-specific result dict
|
||||
"""
|
||||
tool_func = self.available_tools.get(tool_name)
|
||||
if not tool_func:
|
||||
raise ValueError(f"Tool {tool_name} not available")
|
||||
|
||||
try:
|
||||
logger.debug(f"[ORCHESTRATOR] Invoking {tool_name}...")
|
||||
|
||||
# Different tools have different signatures - adapt as needed
|
||||
if tool_name == "RAG":
|
||||
result = await self._invoke_rag(tool_func, query, context_state)
|
||||
elif tool_name == "WEB":
|
||||
result = await self._invoke_web(tool_func, query)
|
||||
elif tool_name == "WEATHER":
|
||||
result = await self._invoke_weather(tool_func, query)
|
||||
elif tool_name == "CODEBRAIN":
|
||||
result = await self._invoke_codebrain(tool_func, query, context_state)
|
||||
else:
|
||||
# Generic invocation
|
||||
result = await tool_func(query)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"tool": tool_name,
|
||||
"query": query,
|
||||
"data": result
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[ORCHESTRATOR] {tool_name} execution failed: {e}")
|
||||
raise
|
||||
|
||||
async def _invoke_rag(self, func, query: str, context: Dict[str, Any]) -> Any:
|
||||
"""Invoke RAG tool (NeoMem search)."""
|
||||
session_id = context.get("session_id", "unknown")
|
||||
# RAG searches memory for relevant past interactions
|
||||
try:
|
||||
results = await func(query, limit=5, session_id=session_id)
|
||||
return results
|
||||
except Exception as e:
|
||||
logger.warning(f"[ORCHESTRATOR] RAG invocation failed, returning empty: {e}")
|
||||
return []
|
||||
|
||||
async def _invoke_web(self, func, query: str) -> Any:
|
||||
"""Invoke web search tool."""
|
||||
try:
|
||||
results = await func(query, max_results=5)
|
||||
return results
|
||||
except Exception as e:
|
||||
logger.warning(f"[ORCHESTRATOR] WEB invocation failed: {e}")
|
||||
return {"error": str(e), "results": []}
|
||||
|
||||
async def _invoke_weather(self, func, query: str) -> Any:
|
||||
"""Invoke weather tool."""
|
||||
# Extract location from query (simple heuristic)
|
||||
# In future: use LLM to extract location
|
||||
try:
|
||||
location = self._extract_location(query)
|
||||
results = await func(location)
|
||||
return results
|
||||
except Exception as e:
|
||||
logger.warning(f"[ORCHESTRATOR] WEATHER invocation failed: {e}")
|
||||
return {"error": str(e)}
|
||||
|
||||
async def _invoke_codebrain(self, func, query: str, context: Dict[str, Any]) -> Any:
|
||||
"""Invoke codebrain tool."""
|
||||
try:
|
||||
results = await func(query, context=context)
|
||||
return results
|
||||
except Exception as e:
|
||||
logger.warning(f"[ORCHESTRATOR] CODEBRAIN invocation failed: {e}")
|
||||
return {"error": str(e)}
|
||||
|
||||
def _extract_location(self, query: str) -> str:
|
||||
"""
|
||||
Extract location from weather query.
|
||||
Simple heuristic - in future use LLM.
|
||||
"""
|
||||
# Common location indicators
|
||||
indicators = ["in ", "at ", "for ", "weather in ", "temperature in "]
|
||||
|
||||
query_lower = query.lower()
|
||||
for indicator in indicators:
|
||||
if indicator in query_lower:
|
||||
# Get text after indicator
|
||||
parts = query_lower.split(indicator, 1)
|
||||
if len(parts) > 1:
|
||||
location = parts[1].strip().split()[0] # First word after indicator
|
||||
return location
|
||||
|
||||
# Default fallback
|
||||
return "current location"
|
||||
|
||||
def format_results_for_context(self, orchestrator_result: Dict[str, Any]) -> str:
|
||||
"""
|
||||
Format tool results for inclusion in context/prompt.
|
||||
|
||||
Args:
|
||||
orchestrator_result: Output from execute_tools()
|
||||
|
||||
Returns:
|
||||
Formatted string for prompt injection
|
||||
"""
|
||||
results = orchestrator_result.get("results", {})
|
||||
summary = orchestrator_result.get("execution_summary", {})
|
||||
|
||||
if not results:
|
||||
return ""
|
||||
|
||||
formatted = "\n=== AUTONOMOUS TOOL RESULTS ===\n"
|
||||
|
||||
for tool_name, tool_result in results.items():
|
||||
if tool_result.get("success", False):
|
||||
formatted += f"\n[{tool_name}]\n"
|
||||
data = tool_result.get("data", {})
|
||||
|
||||
# Format based on tool type
|
||||
if tool_name == "RAG":
|
||||
formatted += self._format_rag_results(data)
|
||||
elif tool_name == "WEB":
|
||||
formatted += self._format_web_results(data)
|
||||
elif tool_name == "WEATHER":
|
||||
formatted += self._format_weather_results(data)
|
||||
elif tool_name == "CODEBRAIN":
|
||||
formatted += self._format_codebrain_results(data)
|
||||
else:
|
||||
formatted += f"{data}\n"
|
||||
else:
|
||||
formatted += f"\n[{tool_name}] - Failed: {tool_result.get('error', 'unknown')}\n"
|
||||
|
||||
formatted += f"\n(Tools executed in {summary.get('total_time_ms', 0)}ms)\n"
|
||||
formatted += "=" * 40 + "\n"
|
||||
|
||||
return formatted
|
||||
|
||||
def _format_rag_results(self, data: Any) -> str:
|
||||
"""Format RAG/memory search results."""
|
||||
if not data:
|
||||
return "No relevant memories found.\n"
|
||||
|
||||
formatted = "Relevant memories:\n"
|
||||
for i, item in enumerate(data[:3], 1): # Top 3
|
||||
text = item.get("text", item.get("content", str(item)))
|
||||
formatted += f" {i}. {text[:100]}...\n"
|
||||
return formatted
|
||||
|
||||
def _format_web_results(self, data: Any) -> str:
|
||||
"""Format web search results."""
|
||||
if isinstance(data, dict) and data.get("error"):
|
||||
return f"Web search failed: {data['error']}\n"
|
||||
|
||||
results = data.get("results", []) if isinstance(data, dict) else data
|
||||
if not results:
|
||||
return "No web results found.\n"
|
||||
|
||||
formatted = "Web search results:\n"
|
||||
for i, item in enumerate(results[:3], 1): # Top 3
|
||||
title = item.get("title", "No title")
|
||||
snippet = item.get("snippet", item.get("description", ""))
|
||||
formatted += f" {i}. {title}\n {snippet[:100]}...\n"
|
||||
return formatted
|
||||
|
||||
def _format_weather_results(self, data: Any) -> str:
|
||||
"""Format weather results."""
|
||||
if isinstance(data, dict) and data.get("error"):
|
||||
return f"Weather lookup failed: {data['error']}\n"
|
||||
|
||||
# Assuming weather API returns temp, conditions, etc.
|
||||
temp = data.get("temperature", "unknown")
|
||||
conditions = data.get("conditions", "unknown")
|
||||
location = data.get("location", "requested location")
|
||||
|
||||
return f"Weather for {location}: {temp}, {conditions}\n"
|
||||
|
||||
def _format_codebrain_results(self, data: Any) -> str:
|
||||
"""Format codebrain results."""
|
||||
if isinstance(data, dict) and data.get("error"):
|
||||
return f"Codebrain failed: {data['error']}\n"
|
||||
|
||||
# Format code-related results
|
||||
return f"{data}\n"
|
||||
+14
-5
@@ -24,6 +24,7 @@ from neomem_client import NeoMemClient
|
||||
# Configuration
|
||||
# -----------------------------
|
||||
NEOMEM_API = os.getenv("NEOMEM_API", "http://neomem-api:8000")
|
||||
NEOMEM_ENABLED = os.getenv("NEOMEM_ENABLED", "false").lower() == "true"
|
||||
RELEVANCE_THRESHOLD = float(os.getenv("RELEVANCE_THRESHOLD", "0.4"))
|
||||
VERBOSE_DEBUG = os.getenv("VERBOSE_DEBUG", "false").lower() == "true"
|
||||
|
||||
@@ -148,6 +149,10 @@ async def _search_neomem(
|
||||
Returns:
|
||||
List of memory objects with full structure, or empty list on failure
|
||||
"""
|
||||
if not NEOMEM_ENABLED:
|
||||
logger.info("NeoMem search skipped (NEOMEM_ENABLED is false)")
|
||||
return []
|
||||
|
||||
try:
|
||||
# NeoMemClient reads NEOMEM_API from environment, no base_url parameter
|
||||
client = NeoMemClient()
|
||||
@@ -259,11 +264,15 @@ async def collect_context(session_id: str, user_prompt: str) -> Dict[str, Any]:
|
||||
logger.debug(json.dumps(intake_data, indent=2, default=str))
|
||||
|
||||
# D. Search NeoMem for relevant memories
|
||||
rag_results = await _search_neomem(
|
||||
query=user_prompt,
|
||||
user_id="brian", # TODO: Make configurable per session
|
||||
limit=5
|
||||
)
|
||||
if NEOMEM_ENABLED:
|
||||
rag_results = await _search_neomem(
|
||||
query=user_prompt,
|
||||
user_id="brian", # TODO: Make configurable per session
|
||||
limit=5
|
||||
)
|
||||
else:
|
||||
rag_results = []
|
||||
logger.info("Skipping NeoMem RAG retrieval; NEOMEM_ENABLED is false")
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[COLLECT_CONTEXT] NeoMem search returned {len(rag_results)} results")
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"mood": "neutral",
|
||||
"energy": 0.8,
|
||||
"focus": "user_request",
|
||||
"confidence": 0.7,
|
||||
"curiosity": 1.0,
|
||||
"last_updated": "2025-12-19T20:25:25.437557",
|
||||
"interaction_count": 16,
|
||||
"learning_queue": [],
|
||||
"active_goals": [],
|
||||
"preferences": {
|
||||
"verbosity": "medium",
|
||||
"formality": "casual",
|
||||
"proactivity": 0.3
|
||||
},
|
||||
"metadata": {
|
||||
"version": "1.0",
|
||||
"created_at": "2025-12-14T03:28:49.364768"
|
||||
}
|
||||
}
|
||||
+50
-57
@@ -234,25 +234,27 @@ def push_to_neomem(summary: str, session_id: str, level: str) -> None:
|
||||
async def summarize_context(session_id: str, exchanges: list[dict]):
|
||||
"""
|
||||
Internal summarizer that uses Cortex's LLM router.
|
||||
Produces L1 / L5 / L10 / L20 / L30 summaries.
|
||||
Produces cascading summaries based on exchange count:
|
||||
- L1: Always (most recent activity)
|
||||
- L2: After 2+ exchanges
|
||||
- L5: After 5+ exchanges
|
||||
- L10: After 10+ exchanges
|
||||
- L20: After 20+ exchanges
|
||||
- L30: After 30+ exchanges
|
||||
|
||||
Args:
|
||||
session_id: The conversation/session ID
|
||||
exchanges: A list of {"user_msg": ..., "assistant_msg": ..., "timestamp": ...}
|
||||
"""
|
||||
|
||||
# Build raw conversation text
|
||||
convo_lines = []
|
||||
for ex in exchanges:
|
||||
convo_lines.append(f"User: {ex.get('user_msg','')}")
|
||||
convo_lines.append(f"Assistant: {ex.get('assistant_msg','')}")
|
||||
convo_text = "\n".join(convo_lines)
|
||||
exchange_count = len(exchanges)
|
||||
|
||||
if not convo_text.strip():
|
||||
if exchange_count == 0:
|
||||
return {
|
||||
"session_id": session_id,
|
||||
"exchange_count": 0,
|
||||
"L1": "",
|
||||
"L2": "",
|
||||
"L5": "",
|
||||
"L10": "",
|
||||
"L20": "",
|
||||
@@ -260,63 +262,54 @@ async def summarize_context(session_id: str, exchanges: list[dict]):
|
||||
"last_updated": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
# Prompt the LLM (internal — no HTTP)
|
||||
prompt = f"""
|
||||
Summarize the conversation below into multiple compression levels.
|
||||
|
||||
Conversation:
|
||||
----------------
|
||||
{convo_text}
|
||||
----------------
|
||||
|
||||
Output strictly in JSON with keys:
|
||||
L1 → ultra short summary (1–2 sentences max)
|
||||
L5 → short summary
|
||||
L10 → medium summary
|
||||
L20 → detailed overview
|
||||
L30 → full detailed summary
|
||||
|
||||
JSON only. No text outside JSON.
|
||||
"""
|
||||
result = {
|
||||
"session_id": session_id,
|
||||
"exchange_count": exchange_count,
|
||||
"L1": "",
|
||||
"L2": "",
|
||||
"L5": "",
|
||||
"L10": "",
|
||||
"L20": "",
|
||||
"L30": "",
|
||||
"last_updated": datetime.now().isoformat()
|
||||
}
|
||||
|
||||
try:
|
||||
llm_response = await call_llm(
|
||||
prompt,
|
||||
backend=INTAKE_LLM,
|
||||
temperature=0.2
|
||||
)
|
||||
# L1: Always generate (most recent exchanges)
|
||||
result["L1"] = await summarize_simple(exchanges[-5:])
|
||||
print(f"[Intake] Generated L1 for {session_id} ({exchange_count} exchanges)")
|
||||
|
||||
print(f"[Intake] LLM response length: {len(llm_response) if llm_response else 0}")
|
||||
print(f"[Intake] LLM response preview: {llm_response[:200] if llm_response else '(empty)'}")
|
||||
# L2: After 2+ exchanges
|
||||
if exchange_count >= 2:
|
||||
result["L2"] = await summarize_simple(exchanges[-2:])
|
||||
print(f"[Intake] Generated L2 for {session_id}")
|
||||
|
||||
# LLM should return JSON, parse it
|
||||
if not llm_response or not llm_response.strip():
|
||||
raise ValueError("Empty response from LLM")
|
||||
# L5: After 5+ exchanges
|
||||
if exchange_count >= 5:
|
||||
result["L5"] = await summarize_simple(exchanges[-10:])
|
||||
print(f"[Intake] Generated L5 for {session_id}")
|
||||
|
||||
summary = json.loads(llm_response)
|
||||
# L10: After 10+ exchanges (Reality Check)
|
||||
if exchange_count >= 10:
|
||||
result["L10"] = await summarize_L10(session_id, exchanges)
|
||||
print(f"[Intake] Generated L10 for {session_id}")
|
||||
|
||||
return {
|
||||
"session_id": session_id,
|
||||
"exchange_count": len(exchanges),
|
||||
"L1": summary.get("L1", ""),
|
||||
"L5": summary.get("L5", ""),
|
||||
"L10": summary.get("L10", ""),
|
||||
"L20": summary.get("L20", ""),
|
||||
"L30": summary.get("L30", ""),
|
||||
"last_updated": datetime.now().isoformat()
|
||||
}
|
||||
# L20: After 20+ exchanges (Session Overview - merges L10s)
|
||||
if exchange_count >= 20 and exchange_count % 10 == 0:
|
||||
result["L20"] = await summarize_L20(session_id)
|
||||
print(f"[Intake] Generated L20 for {session_id}")
|
||||
|
||||
# L30: After 30+ exchanges (Continuity Report - merges L20s)
|
||||
if exchange_count >= 30 and exchange_count % 10 == 0:
|
||||
result["L30"] = await summarize_L30(session_id)
|
||||
print(f"[Intake] Generated L30 for {session_id}")
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
return {
|
||||
"session_id": session_id,
|
||||
"exchange_count": len(exchanges),
|
||||
"L1": f"[Error summarizing: {str(e)}]",
|
||||
"L5": "",
|
||||
"L10": "",
|
||||
"L20": "",
|
||||
"L30": "",
|
||||
"last_updated": datetime.now().isoformat()
|
||||
}
|
||||
print(f"[Intake] Error during summarization: {e}")
|
||||
result["L1"] = f"[Error summarizing: {str(e)}]"
|
||||
return result
|
||||
|
||||
# ─────────────────────────────────
|
||||
# Background summarization stub
|
||||
|
||||
+36
-4
@@ -59,17 +59,44 @@ Guidelines:
|
||||
# Build persona prompt
|
||||
# ============================================================
|
||||
|
||||
def build_speak_prompt(final_answer: str) -> str:
|
||||
def build_speak_prompt(final_answer: str, tone: str = "neutral", depth: str = "medium") -> str:
|
||||
"""
|
||||
Wrap Cortex's final neutral answer in the Lyra persona.
|
||||
Cortex → neutral reasoning
|
||||
Speak → stylistic transformation
|
||||
|
||||
|
||||
The LLM sees the original answer and rewrites it in Lyra's voice.
|
||||
|
||||
Args:
|
||||
final_answer: The neutral reasoning output
|
||||
tone: Desired emotional tone (neutral | warm | focused | playful | direct)
|
||||
depth: Response depth (short | medium | deep)
|
||||
"""
|
||||
|
||||
# Tone-specific guidance
|
||||
tone_guidance = {
|
||||
"neutral": "balanced and professional",
|
||||
"warm": "friendly and empathetic",
|
||||
"focused": "precise and technical",
|
||||
"playful": "light and engaging",
|
||||
"direct": "concise and straightforward"
|
||||
}
|
||||
|
||||
depth_guidance = {
|
||||
"short": "Keep responses brief and to-the-point.",
|
||||
"medium": "Provide balanced detail.",
|
||||
"deep": "Elaborate thoroughly with nuance and examples."
|
||||
}
|
||||
|
||||
tone_hint = tone_guidance.get(tone, "balanced and professional")
|
||||
depth_hint = depth_guidance.get(depth, "Provide balanced detail.")
|
||||
|
||||
return f"""
|
||||
{PERSONA_STYLE}
|
||||
|
||||
Tone guidance: Your response should be {tone_hint}.
|
||||
Depth guidance: {depth_hint}
|
||||
|
||||
Rewrite the following message into Lyra's natural voice.
|
||||
Preserve meaning exactly.
|
||||
|
||||
@@ -84,16 +111,21 @@ Preserve meaning exactly.
|
||||
# Public API — async wrapper
|
||||
# ============================================================
|
||||
|
||||
async def speak(final_answer: str) -> str:
|
||||
async def speak(final_answer: str, tone: str = "neutral", depth: str = "medium") -> str:
|
||||
"""
|
||||
Given the final refined answer from Cortex,
|
||||
apply Lyra persona styling using the designated backend.
|
||||
|
||||
Args:
|
||||
final_answer: The polished answer from refinement stage
|
||||
tone: Desired emotional tone (neutral | warm | focused | playful | direct)
|
||||
depth: Response depth (short | medium | deep)
|
||||
"""
|
||||
|
||||
if not final_answer:
|
||||
return ""
|
||||
|
||||
prompt = build_speak_prompt(final_answer)
|
||||
prompt = build_speak_prompt(final_answer, tone, depth)
|
||||
|
||||
backend = SPEAK_BACKEND
|
||||
|
||||
|
||||
@@ -45,7 +45,9 @@ async def reason_check(
|
||||
identity_block: dict | None,
|
||||
rag_block: dict | None,
|
||||
reflection_notes: list[str],
|
||||
context: dict | None = None
|
||||
context: dict | None = None,
|
||||
monologue: dict | None = None, # NEW: Inner monologue guidance
|
||||
executive_plan: dict | None = None # NEW: Executive plan for complex tasks
|
||||
) -> str:
|
||||
"""
|
||||
Build the *draft answer* for Lyra Cortex.
|
||||
@@ -57,6 +59,8 @@ async def reason_check(
|
||||
rag_block: Relevant long-term memories from NeoMem
|
||||
reflection_notes: Meta-awareness notes from reflection stage
|
||||
context: Unified context state from context.py (session state, intake, rag, etc.)
|
||||
monologue: Inner monologue analysis (intent, tone, depth, consult_executive)
|
||||
executive_plan: Executive plan for complex queries (steps, tools, strategy)
|
||||
"""
|
||||
|
||||
# --------------------------------------------------------
|
||||
@@ -79,6 +83,52 @@ async def reason_check(
|
||||
except Exception:
|
||||
identity_txt = f"Identity Rules:\n{str(identity_block)}\n\n"
|
||||
|
||||
# --------------------------------------------------------
|
||||
# Inner Monologue guidance (NEW)
|
||||
# --------------------------------------------------------
|
||||
monologue_section = ""
|
||||
if monologue:
|
||||
intent = monologue.get("intent", "unknown")
|
||||
tone_desired = monologue.get("tone", "neutral")
|
||||
depth_desired = monologue.get("depth", "medium")
|
||||
|
||||
monologue_section = f"""
|
||||
=== INNER MONOLOGUE GUIDANCE ===
|
||||
User Intent Detected: {intent}
|
||||
Desired Tone: {tone_desired}
|
||||
Desired Response Depth: {depth_desired}
|
||||
|
||||
Adjust your response accordingly:
|
||||
- Focus on addressing the {intent} intent
|
||||
- Aim for {depth_desired} depth (short/medium/deep)
|
||||
- The persona layer will handle {tone_desired} tone, focus on content
|
||||
|
||||
"""
|
||||
|
||||
# --------------------------------------------------------
|
||||
# Executive Plan (NEW)
|
||||
# --------------------------------------------------------
|
||||
plan_section = ""
|
||||
if executive_plan:
|
||||
plan_section = f"""
|
||||
=== EXECUTIVE PLAN ===
|
||||
Task Complexity: {executive_plan.get('estimated_complexity', 'unknown')}
|
||||
Plan Summary: {executive_plan.get('summary', 'No summary')}
|
||||
|
||||
Detailed Plan:
|
||||
{executive_plan.get('plan_text', 'No detailed plan available')}
|
||||
|
||||
Required Steps:
|
||||
"""
|
||||
for idx, step in enumerate(executive_plan.get('steps', []), 1):
|
||||
plan_section += f"{idx}. {step}\n"
|
||||
|
||||
tools_needed = executive_plan.get('tools_needed', [])
|
||||
if tools_needed:
|
||||
plan_section += f"\nTools to leverage: {', '.join(tools_needed)}\n"
|
||||
|
||||
plan_section += "\nFollow this plan while generating your response.\n\n"
|
||||
|
||||
# --------------------------------------------------------
|
||||
# RAG block (optional factual grounding)
|
||||
# --------------------------------------------------------
|
||||
@@ -164,6 +214,8 @@ async def reason_check(
|
||||
prompt = (
|
||||
f"{notes_section}"
|
||||
f"{identity_txt}"
|
||||
f"{monologue_section}" # NEW: Intent/tone/depth guidance
|
||||
f"{plan_section}" # NEW: Executive plan if generated
|
||||
f"{context_txt}" # Context BEFORE RAG for better coherence
|
||||
f"{rag_txt}"
|
||||
f"User message:\n{user_prompt}\n\n"
|
||||
|
||||
+215
-147
@@ -2,7 +2,7 @@
|
||||
|
||||
import os
|
||||
import logging
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi import APIRouter
|
||||
from pydantic import BaseModel
|
||||
|
||||
from reasoning.reasoning import reason_check
|
||||
@@ -13,17 +13,19 @@ from persona.identity import load_identity
|
||||
from context import collect_context, update_last_assistant_message
|
||||
from intake.intake import add_exchange_internal
|
||||
|
||||
from autonomy.monologue.monologue import InnerMonologue
|
||||
from autonomy.self.state import load_self_state
|
||||
|
||||
# -----------------------------
|
||||
# Debug configuration
|
||||
# -----------------------------
|
||||
|
||||
# -------------------------------------------------------------------
|
||||
# Setup
|
||||
# -------------------------------------------------------------------
|
||||
VERBOSE_DEBUG = os.getenv("VERBOSE_DEBUG", "false").lower() == "true"
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
# Console handler
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setFormatter(logging.Formatter(
|
||||
'%(asctime)s [ROUTER] %(levelname)s: %(message)s',
|
||||
@@ -31,7 +33,6 @@ if VERBOSE_DEBUG:
|
||||
))
|
||||
logger.addHandler(console_handler)
|
||||
|
||||
# File handler
|
||||
try:
|
||||
os.makedirs('/app/logs', exist_ok=True)
|
||||
file_handler = logging.FileHandler('/app/logs/cortex_verbose_debug.log', mode='a')
|
||||
@@ -40,28 +41,27 @@ if VERBOSE_DEBUG:
|
||||
datefmt='%Y-%m-%d %H:%M:%S'
|
||||
))
|
||||
logger.addHandler(file_handler)
|
||||
logger.debug("VERBOSE_DEBUG mode enabled for router.py - logging to file")
|
||||
logger.debug("VERBOSE_DEBUG enabled for router.py")
|
||||
except Exception as e:
|
||||
logger.debug(f"VERBOSE_DEBUG mode enabled for router.py - file logging failed: {e}")
|
||||
logger.debug(f"File logging failed: {e}")
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Router (NOT FastAPI app)
|
||||
# -----------------------------
|
||||
cortex_router = APIRouter()
|
||||
inner_monologue = InnerMonologue()
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Pydantic models
|
||||
# -----------------------------
|
||||
# -------------------------------------------------------------------
|
||||
# Models
|
||||
# -------------------------------------------------------------------
|
||||
class ReasonRequest(BaseModel):
|
||||
session_id: str
|
||||
user_prompt: str
|
||||
temperature: float | None = None
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# -------------------------------------------------------------------
|
||||
# /reason endpoint
|
||||
# -----------------------------
|
||||
# -------------------------------------------------------------------
|
||||
@cortex_router.post("/reason")
|
||||
async def run_reason(req: ReasonRequest):
|
||||
|
||||
@@ -71,7 +71,9 @@ async def run_reason(req: ReasonRequest):
|
||||
logger.debug(f"[PIPELINE START] User prompt: {req.user_prompt[:200]}...")
|
||||
logger.debug(f"{'='*80}\n")
|
||||
|
||||
# 0. Collect unified context from all sources
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 0 — Context
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 0] Collecting unified context...")
|
||||
|
||||
@@ -80,7 +82,9 @@ async def run_reason(req: ReasonRequest):
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 0] Context collected - {len(context_state.get('rag', []))} RAG results")
|
||||
|
||||
# 0.5. Load identity block
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 0.5 — Identity
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 0.5] Loading identity block...")
|
||||
|
||||
@@ -89,37 +93,133 @@ async def run_reason(req: ReasonRequest):
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 0.5] Identity loaded: {identity_block.get('name', 'Unknown')}")
|
||||
|
||||
# 1. Extract Intake summary for reflection
|
||||
# Use L20 (Session Overview) as primary summary for reflection
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 0.6 — Inner Monologue (observer-only)
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 0.6] Running inner monologue...")
|
||||
|
||||
inner_result = None
|
||||
try:
|
||||
self_state = load_self_state()
|
||||
|
||||
mono_context = {
|
||||
"user_message": req.user_prompt,
|
||||
"session_id": req.session_id,
|
||||
"self_state": self_state,
|
||||
"context_summary": context_state,
|
||||
}
|
||||
|
||||
inner_result = await inner_monologue.process(mono_context)
|
||||
logger.info(f"[INNER_MONOLOGUE] {inner_result}")
|
||||
|
||||
# Store in context for downstream use
|
||||
context_state["monologue"] = inner_result
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"[INNER_MONOLOGUE] failed: {e}")
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 0.7 — Executive Planning (conditional)
|
||||
# ----------------------------------------------------------------
|
||||
executive_plan = None
|
||||
if inner_result and inner_result.get("consult_executive"):
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 0.7] Executive consultation requested...")
|
||||
|
||||
try:
|
||||
from autonomy.executive.planner import plan_execution
|
||||
executive_plan = await plan_execution(
|
||||
user_prompt=req.user_prompt,
|
||||
intent=inner_result.get("intent", "unknown"),
|
||||
context_state=context_state,
|
||||
identity_block=identity_block
|
||||
)
|
||||
logger.info(f"[EXECUTIVE] Generated plan: {executive_plan.get('summary', 'N/A')}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[EXECUTIVE] Planning failed: {e}")
|
||||
executive_plan = None
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 0.8 — Autonomous Tool Invocation
|
||||
# ----------------------------------------------------------------
|
||||
tool_results = None
|
||||
autonomous_enabled = os.getenv("ENABLE_AUTONOMOUS_TOOLS", "true").lower() == "true"
|
||||
tool_confidence_threshold = float(os.getenv("AUTONOMOUS_TOOL_CONFIDENCE_THRESHOLD", "0.6"))
|
||||
|
||||
if autonomous_enabled and inner_result:
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 0.8] Analyzing autonomous tool needs...")
|
||||
|
||||
try:
|
||||
from autonomy.tools.decision_engine import ToolDecisionEngine
|
||||
from autonomy.tools.orchestrator import ToolOrchestrator
|
||||
|
||||
# Analyze which tools to invoke
|
||||
decision_engine = ToolDecisionEngine()
|
||||
tool_decision = await decision_engine.analyze_tool_needs(
|
||||
user_prompt=req.user_prompt,
|
||||
monologue=inner_result,
|
||||
context_state=context_state,
|
||||
available_tools=["RAG", "WEB", "WEATHER", "CODEBRAIN"]
|
||||
)
|
||||
|
||||
# Execute tools if confidence threshold met
|
||||
if tool_decision["should_invoke_tools"] and tool_decision["confidence"] >= tool_confidence_threshold:
|
||||
orchestrator = ToolOrchestrator(tool_timeout=30)
|
||||
tool_results = await orchestrator.execute_tools(
|
||||
tools_to_invoke=tool_decision["tools_to_invoke"],
|
||||
context_state=context_state
|
||||
)
|
||||
|
||||
# Format results for context injection
|
||||
tool_context = orchestrator.format_results_for_context(tool_results)
|
||||
context_state["autonomous_tool_results"] = tool_context
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
summary = tool_results.get("execution_summary", {})
|
||||
logger.debug(f"[STAGE 0.8] Tools executed: {summary.get('successful', [])} succeeded")
|
||||
else:
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 0.8] No tools invoked (confidence: {tool_decision.get('confidence', 0):.2f})")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"[STAGE 0.8] Autonomous tool invocation failed: {e}")
|
||||
if VERBOSE_DEBUG:
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 1 — Intake summary
|
||||
# ----------------------------------------------------------------
|
||||
intake_summary = "(no context available)"
|
||||
if context_state.get("intake"):
|
||||
l20_summary = context_state["intake"].get("L20")
|
||||
if l20_summary and isinstance(l20_summary, dict):
|
||||
intake_summary = l20_summary.get("summary", "(no context available)")
|
||||
elif isinstance(l20_summary, str):
|
||||
intake_summary = l20_summary
|
||||
l20 = context_state["intake"].get("L20")
|
||||
if isinstance(l20, dict):
|
||||
intake_summary = l20.get("summary", intake_summary)
|
||||
elif isinstance(l20, str):
|
||||
intake_summary = l20
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 1] Intake summary extracted (L20): {intake_summary[:150]}...")
|
||||
|
||||
# 2. Reflection
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 2 — Reflection
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 2] Running reflection...")
|
||||
|
||||
try:
|
||||
reflection = await reflect_notes(intake_summary, identity_block=identity_block)
|
||||
reflection_notes = reflection.get("notes", [])
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 2] Reflection complete - {len(reflection_notes)} notes generated")
|
||||
for idx, note in enumerate(reflection_notes, 1):
|
||||
logger.debug(f" Note {idx}: {note}")
|
||||
except Exception as e:
|
||||
reflection_notes = []
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 2] Reflection failed: {e}")
|
||||
|
||||
# 3. First-pass reasoning draft
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 3 — Reasoning (draft)
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 3] Running reasoning (draft)...")
|
||||
|
||||
@@ -128,14 +228,14 @@ async def run_reason(req: ReasonRequest):
|
||||
identity_block=identity_block,
|
||||
rag_block=context_state.get("rag", []),
|
||||
reflection_notes=reflection_notes,
|
||||
context=context_state
|
||||
context=context_state,
|
||||
monologue=inner_result, # NEW: Pass monologue guidance
|
||||
executive_plan=executive_plan # NEW: Pass executive plan
|
||||
)
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 3] Draft answer ({len(draft)} chars):")
|
||||
logger.debug(f"--- DRAFT START ---\n{draft}\n--- DRAFT END ---")
|
||||
|
||||
# 4. Refinement
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 4 — Refinement
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 4] Running refinement...")
|
||||
|
||||
@@ -145,35 +245,92 @@ async def run_reason(req: ReasonRequest):
|
||||
identity_block=identity_block,
|
||||
rag_block=context_state.get("rag", []),
|
||||
)
|
||||
|
||||
final_neutral = result["final_output"]
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 4] Refined answer ({len(final_neutral)} chars):")
|
||||
logger.debug(f"--- REFINED START ---\n{final_neutral}\n--- REFINED END ---")
|
||||
|
||||
# 5. Persona layer
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 5 — Persona
|
||||
# ----------------------------------------------------------------
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 5] Applying persona layer...")
|
||||
|
||||
persona_answer = await speak(final_neutral)
|
||||
# Extract tone and depth from monologue for persona guidance
|
||||
tone = inner_result.get("tone", "neutral") if inner_result else "neutral"
|
||||
depth = inner_result.get("depth", "medium") if inner_result else "medium"
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 5] Persona answer ({len(persona_answer)} chars):")
|
||||
logger.debug(f"--- PERSONA START ---\n{persona_answer}\n--- PERSONA END ---")
|
||||
|
||||
# 6. Update session state with assistant's response
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug("[STAGE 6] Updating session state...")
|
||||
persona_answer = await speak(final_neutral, tone=tone, depth=depth)
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 6 — Session update
|
||||
# ----------------------------------------------------------------
|
||||
update_last_assistant_message(req.session_id, persona_answer)
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 6.5 — Self-state update & Pattern Learning
|
||||
# ----------------------------------------------------------------
|
||||
try:
|
||||
from autonomy.self.analyzer import analyze_and_update_state
|
||||
await analyze_and_update_state(
|
||||
monologue=inner_result or {},
|
||||
user_prompt=req.user_prompt,
|
||||
response=persona_answer,
|
||||
context=context_state
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[SELF_STATE] Update failed: {e}")
|
||||
|
||||
# Pattern learning
|
||||
try:
|
||||
from autonomy.learning.pattern_learner import get_pattern_learner
|
||||
learner = get_pattern_learner()
|
||||
await learner.learn_from_interaction(
|
||||
user_prompt=req.user_prompt,
|
||||
response=persona_answer,
|
||||
monologue=inner_result or {},
|
||||
context=context_state
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[PATTERN_LEARNER] Learning failed: {e}")
|
||||
|
||||
# ----------------------------------------------------------------
|
||||
# STAGE 7 — Proactive Monitoring & Suggestions
|
||||
# ----------------------------------------------------------------
|
||||
proactive_enabled = os.getenv("ENABLE_PROACTIVE_MONITORING", "true").lower() == "true"
|
||||
proactive_min_priority = float(os.getenv("PROACTIVE_SUGGESTION_MIN_PRIORITY", "0.6"))
|
||||
|
||||
if proactive_enabled:
|
||||
try:
|
||||
from autonomy.proactive.monitor import get_proactive_monitor
|
||||
|
||||
monitor = get_proactive_monitor(min_priority=proactive_min_priority)
|
||||
self_state = load_self_state() # Already imported at top of file
|
||||
|
||||
suggestion = await monitor.analyze_session(
|
||||
session_id=req.session_id,
|
||||
context_state=context_state,
|
||||
self_state=self_state
|
||||
)
|
||||
|
||||
# Append suggestion to response if exists
|
||||
if suggestion:
|
||||
suggestion_text = monitor.format_suggestion(suggestion)
|
||||
persona_answer += suggestion_text
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"[STAGE 7] Proactive suggestion added: {suggestion['type']} (priority: {suggestion['priority']:.2f})")
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"[STAGE 7] Proactive monitoring failed: {e}")
|
||||
|
||||
if VERBOSE_DEBUG:
|
||||
logger.debug(f"\n{'='*80}")
|
||||
logger.debug(f"[PIPELINE COMPLETE] Session: {req.session_id}")
|
||||
logger.debug(f"[PIPELINE COMPLETE] Final answer length: {len(persona_answer)} chars")
|
||||
logger.debug(f"{'='*80}\n")
|
||||
|
||||
# 7. Return full bundle
|
||||
# ----------------------------------------------------------------
|
||||
# RETURN
|
||||
# ----------------------------------------------------------------
|
||||
return {
|
||||
"draft": draft,
|
||||
"neutral": final_neutral,
|
||||
@@ -189,9 +346,9 @@ async def run_reason(req: ReasonRequest):
|
||||
}
|
||||
|
||||
|
||||
# -----------------------------
|
||||
# Intake ingest (internal feed)
|
||||
# -----------------------------
|
||||
# -------------------------------------------------------------------
|
||||
# /ingest endpoint (internal)
|
||||
# -------------------------------------------------------------------
|
||||
class IngestPayload(BaseModel):
|
||||
session_id: str
|
||||
user_msg: str
|
||||
@@ -200,107 +357,18 @@ class IngestPayload(BaseModel):
|
||||
|
||||
@cortex_router.post("/ingest")
|
||||
async def ingest(payload: IngestPayload):
|
||||
"""
|
||||
Receives (session_id, user_msg, assistant_msg) from Relay
|
||||
and pushes directly into Intake's in-memory buffer.
|
||||
|
||||
Uses lenient error handling - always returns success to avoid
|
||||
breaking the chat pipeline.
|
||||
"""
|
||||
try:
|
||||
# 1. Update Cortex session state
|
||||
update_last_assistant_message(payload.session_id, payload.assistant_msg)
|
||||
except Exception as e:
|
||||
logger.warning(f"[INGEST] Failed to update session state: {e}")
|
||||
# Continue anyway (lenient mode)
|
||||
logger.warning(f"[INGEST] Session update failed: {e}")
|
||||
|
||||
try:
|
||||
# 2. Feed Intake internally (no HTTP)
|
||||
add_exchange_internal({
|
||||
"session_id": payload.session_id,
|
||||
"user_msg": payload.user_msg,
|
||||
"assistant_msg": payload.assistant_msg,
|
||||
})
|
||||
logger.debug(f"[INGEST] Added exchange to Intake for {payload.session_id}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[INGEST] Failed to add to Intake: {e}")
|
||||
# Continue anyway (lenient mode)
|
||||
|
||||
# Always return success (user requirement: never fail chat pipeline)
|
||||
return {
|
||||
"status": "ok",
|
||||
"session_id": payload.session_id
|
||||
}
|
||||
|
||||
# -----------------------------
|
||||
# Debug endpoint: summarized context
|
||||
# -----------------------------
|
||||
@cortex_router.get("/debug/summary")
|
||||
async def debug_summary(session_id: str):
|
||||
"""
|
||||
Diagnostic endpoint that runs Intake's summarize_context() for a session.
|
||||
|
||||
Shows exactly what L1/L5/L10/L20/L30 summaries would look like
|
||||
inside the actual Uvicorn worker, using the real SESSIONS buffer.
|
||||
"""
|
||||
from intake.intake import SESSIONS, summarize_context
|
||||
|
||||
# Validate session
|
||||
session = SESSIONS.get(session_id)
|
||||
if not session:
|
||||
return {"error": "session not found", "session_id": session_id}
|
||||
|
||||
# Convert deque into the structure summarize_context expects
|
||||
buffer = session["buffer"]
|
||||
exchanges = [
|
||||
{
|
||||
"user_msg": ex.get("user_msg", ""),
|
||||
"assistant_msg": ex.get("assistant_msg", ""),
|
||||
}
|
||||
for ex in buffer
|
||||
]
|
||||
|
||||
# 🔥 CRITICAL FIX — summarize_context is async
|
||||
summary = await summarize_context(session_id, exchanges)
|
||||
|
||||
return {
|
||||
"session_id": session_id,
|
||||
"buffer_size": len(buffer),
|
||||
"exchanges_preview": exchanges[-5:], # last 5 items
|
||||
"summary": summary
|
||||
}
|
||||
|
||||
# -----------------------------
|
||||
# Debug endpoint for SESSIONS
|
||||
# -----------------------------
|
||||
@cortex_router.get("/debug/sessions")
|
||||
async def debug_sessions():
|
||||
"""
|
||||
Diagnostic endpoint to inspect SESSIONS from within the running Uvicorn worker.
|
||||
This shows the actual state of the in-memory SESSIONS dict.
|
||||
"""
|
||||
from intake.intake import SESSIONS
|
||||
|
||||
sessions_data = {}
|
||||
for session_id, session_info in SESSIONS.items():
|
||||
buffer = session_info["buffer"]
|
||||
sessions_data[session_id] = {
|
||||
"created_at": session_info["created_at"].isoformat(),
|
||||
"buffer_size": len(buffer),
|
||||
"buffer_maxlen": buffer.maxlen,
|
||||
"recent_exchanges": [
|
||||
{
|
||||
"user_msg": ex.get("user_msg", "")[:100],
|
||||
"assistant_msg": ex.get("assistant_msg", "")[:100],
|
||||
"timestamp": ex.get("timestamp", "")
|
||||
}
|
||||
for ex in list(buffer)[-5:] # Last 5 exchanges
|
||||
]
|
||||
}
|
||||
|
||||
return {
|
||||
"sessions_object_id": id(SESSIONS),
|
||||
"total_sessions": len(SESSIONS),
|
||||
"sessions": sessions_data
|
||||
}
|
||||
logger.warning(f"[INGEST] Intake update failed: {e}")
|
||||
|
||||
return {"status": "ok", "session_id": payload.session_id}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""Tests for Project Lyra Cortex."""
|
||||
@@ -0,0 +1,197 @@
|
||||
"""
|
||||
Integration tests for Phase 1 autonomy features.
|
||||
Tests monologue integration, executive planning, and self-state persistence.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add parent directory to path for imports
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from autonomy.monologue.monologue import InnerMonologue
|
||||
from autonomy.self.state import load_self_state, update_self_state, get_self_state_instance
|
||||
from autonomy.executive.planner import plan_execution
|
||||
|
||||
|
||||
async def test_monologue_integration():
|
||||
"""Test monologue generates valid output."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 1: Monologue Integration")
|
||||
print("="*60)
|
||||
|
||||
mono = InnerMonologue()
|
||||
|
||||
context = {
|
||||
"user_message": "Explain quantum computing to me like I'm 5",
|
||||
"session_id": "test_001",
|
||||
"self_state": load_self_state(),
|
||||
"context_summary": {"message_count": 5}
|
||||
}
|
||||
|
||||
result = await mono.process(context)
|
||||
|
||||
assert "intent" in result, "Missing intent field"
|
||||
assert "tone" in result, "Missing tone field"
|
||||
assert "depth" in result, "Missing depth field"
|
||||
assert "consult_executive" in result, "Missing consult_executive field"
|
||||
|
||||
print("✓ Monologue integration test passed")
|
||||
print(f" Result: {json.dumps(result, indent=2)}")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
async def test_executive_planning():
|
||||
"""Test executive planner generates valid plans."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 2: Executive Planning")
|
||||
print("="*60)
|
||||
|
||||
plan = await plan_execution(
|
||||
user_prompt="Help me build a distributed system with microservices architecture",
|
||||
intent="technical_implementation",
|
||||
context_state={
|
||||
"tools_available": ["RAG", "WEB", "CODEBRAIN"],
|
||||
"message_count": 3,
|
||||
"minutes_since_last_msg": 2.5,
|
||||
"active_project": None
|
||||
},
|
||||
identity_block={}
|
||||
)
|
||||
|
||||
assert "summary" in plan, "Missing summary field"
|
||||
assert "plan_text" in plan, "Missing plan_text field"
|
||||
assert "steps" in plan, "Missing steps field"
|
||||
assert len(plan["steps"]) > 0, "No steps generated"
|
||||
|
||||
print("✓ Executive planning test passed")
|
||||
print(f" Plan summary: {plan['summary']}")
|
||||
print(f" Steps: {len(plan['steps'])}")
|
||||
print(f" Complexity: {plan.get('estimated_complexity', 'unknown')}")
|
||||
|
||||
return plan
|
||||
|
||||
|
||||
def test_self_state_persistence():
|
||||
"""Test self-state loads and updates."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 3: Self-State Persistence")
|
||||
print("="*60)
|
||||
|
||||
state1 = load_self_state()
|
||||
assert "mood" in state1, "Missing mood field"
|
||||
assert "energy" in state1, "Missing energy field"
|
||||
assert "interaction_count" in state1, "Missing interaction_count"
|
||||
|
||||
initial_count = state1.get("interaction_count", 0)
|
||||
print(f" Initial interaction count: {initial_count}")
|
||||
|
||||
update_self_state(
|
||||
mood_delta=0.1,
|
||||
energy_delta=-0.05,
|
||||
new_focus="testing"
|
||||
)
|
||||
|
||||
state2 = load_self_state()
|
||||
assert state2["interaction_count"] == initial_count + 1, "Interaction count not incremented"
|
||||
assert state2["focus"] == "testing", "Focus not updated"
|
||||
|
||||
print("✓ Self-state persistence test passed")
|
||||
print(f" New interaction count: {state2['interaction_count']}")
|
||||
print(f" New focus: {state2['focus']}")
|
||||
print(f" New energy: {state2['energy']:.2f}")
|
||||
|
||||
return state2
|
||||
|
||||
|
||||
async def test_end_to_end_flow():
|
||||
"""Test complete flow from monologue through planning."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 4: End-to-End Flow")
|
||||
print("="*60)
|
||||
|
||||
# Step 1: Monologue detects complex query
|
||||
mono = InnerMonologue()
|
||||
mono_result = await mono.process({
|
||||
"user_message": "Design a scalable ML pipeline with CI/CD integration",
|
||||
"session_id": "test_e2e",
|
||||
"self_state": load_self_state(),
|
||||
"context_summary": {}
|
||||
})
|
||||
|
||||
print(f" Monologue intent: {mono_result.get('intent')}")
|
||||
print(f" Consult executive: {mono_result.get('consult_executive')}")
|
||||
|
||||
# Step 2: If executive requested, generate plan
|
||||
if mono_result.get("consult_executive"):
|
||||
plan = await plan_execution(
|
||||
user_prompt="Design a scalable ML pipeline with CI/CD integration",
|
||||
intent=mono_result.get("intent", "unknown"),
|
||||
context_state={"tools_available": ["CODEBRAIN", "WEB"]},
|
||||
identity_block={}
|
||||
)
|
||||
|
||||
assert plan is not None, "Plan should be generated"
|
||||
print(f" Executive plan generated: {len(plan.get('steps', []))} steps")
|
||||
|
||||
# Step 3: Update self-state
|
||||
update_self_state(
|
||||
energy_delta=-0.1, # Complex task is tiring
|
||||
new_focus="ml_pipeline_design",
|
||||
confidence_delta=0.05
|
||||
)
|
||||
|
||||
state = load_self_state()
|
||||
assert state["focus"] == "ml_pipeline_design", "Focus should be updated"
|
||||
|
||||
print("✓ End-to-end flow test passed")
|
||||
print(f" Final state: {state['mood']}, energy={state['energy']:.2f}")
|
||||
|
||||
return True
|
||||
|
||||
|
||||
async def run_all_tests():
|
||||
"""Run all Phase 1 tests."""
|
||||
print("\n" + "="*60)
|
||||
print("PHASE 1 AUTONOMY TESTS")
|
||||
print("="*60)
|
||||
|
||||
try:
|
||||
# Test 1: Monologue
|
||||
mono_result = await test_monologue_integration()
|
||||
|
||||
# Test 2: Executive Planning
|
||||
plan_result = await test_executive_planning()
|
||||
|
||||
# Test 3: Self-State
|
||||
state_result = test_self_state_persistence()
|
||||
|
||||
# Test 4: End-to-End
|
||||
await test_end_to_end_flow()
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("ALL TESTS PASSED ✓")
|
||||
print("="*60)
|
||||
|
||||
print("\nSummary:")
|
||||
print(f" - Monologue: {mono_result.get('intent')} ({mono_result.get('tone')})")
|
||||
print(f" - Executive: {plan_result.get('estimated_complexity')} complexity")
|
||||
print(f" - Self-state: {state_result.get('interaction_count')} interactions")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print("\n" + "="*60)
|
||||
print(f"TEST FAILED: {e}")
|
||||
print("="*60)
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(run_all_tests())
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -0,0 +1,495 @@
|
||||
"""
|
||||
Integration tests for Phase 2 autonomy features.
|
||||
Tests autonomous tool invocation, proactive monitoring, actions, and pattern learning.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
import os
|
||||
|
||||
# Add parent directory to path for imports
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
# Override self-state file path for testing
|
||||
os.environ["SELF_STATE_FILE"] = "/tmp/test_self_state.json"
|
||||
|
||||
from autonomy.tools.decision_engine import ToolDecisionEngine
|
||||
from autonomy.tools.orchestrator import ToolOrchestrator
|
||||
from autonomy.proactive.monitor import ProactiveMonitor
|
||||
from autonomy.actions.autonomous_actions import AutonomousActionManager
|
||||
from autonomy.learning.pattern_learner import PatternLearner
|
||||
from autonomy.self.state import load_self_state, get_self_state_instance
|
||||
|
||||
|
||||
async def test_tool_decision_engine():
|
||||
"""Test autonomous tool decision making."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 1: Tool Decision Engine")
|
||||
print("="*60)
|
||||
|
||||
engine = ToolDecisionEngine()
|
||||
|
||||
# Test 1a: Memory reference detection
|
||||
result = await engine.analyze_tool_needs(
|
||||
user_prompt="What did we discuss earlier about Python?",
|
||||
monologue={"intent": "clarification", "consult_executive": False},
|
||||
context_state={},
|
||||
available_tools=["RAG", "WEB", "WEATHER"]
|
||||
)
|
||||
|
||||
assert result["should_invoke_tools"], "Should invoke tools for memory reference"
|
||||
assert any(t["tool"] == "RAG" for t in result["tools_to_invoke"]), "Should recommend RAG"
|
||||
assert result["confidence"] > 0.8, f"Confidence should be high for clear memory reference: {result['confidence']}"
|
||||
|
||||
print(f" ✓ Memory reference detection passed")
|
||||
print(f" Tools: {[t['tool'] for t in result['tools_to_invoke']]}")
|
||||
print(f" Confidence: {result['confidence']:.2f}")
|
||||
|
||||
# Test 1b: Web search detection
|
||||
result = await engine.analyze_tool_needs(
|
||||
user_prompt="What's the latest news about AI developments?",
|
||||
monologue={"intent": "information_seeking", "consult_executive": False},
|
||||
context_state={},
|
||||
available_tools=["RAG", "WEB", "WEATHER"]
|
||||
)
|
||||
|
||||
assert result["should_invoke_tools"], "Should invoke tools for current info request"
|
||||
assert any(t["tool"] == "WEB" for t in result["tools_to_invoke"]), "Should recommend WEB"
|
||||
|
||||
print(f" ✓ Web search detection passed")
|
||||
print(f" Tools: {[t['tool'] for t in result['tools_to_invoke']]}")
|
||||
|
||||
# Test 1c: Weather detection
|
||||
result = await engine.analyze_tool_needs(
|
||||
user_prompt="What's the weather like today in Boston?",
|
||||
monologue={"intent": "information_seeking", "consult_executive": False},
|
||||
context_state={},
|
||||
available_tools=["RAG", "WEB", "WEATHER"]
|
||||
)
|
||||
|
||||
assert result["should_invoke_tools"], "Should invoke tools for weather query"
|
||||
assert any(t["tool"] == "WEATHER" for t in result["tools_to_invoke"]), "Should recommend WEATHER"
|
||||
|
||||
print(f" ✓ Weather detection passed")
|
||||
|
||||
# Test 1d: Proactive RAG for complex queries
|
||||
result = await engine.analyze_tool_needs(
|
||||
user_prompt="Design a microservices architecture",
|
||||
monologue={"intent": "technical_implementation", "consult_executive": True},
|
||||
context_state={},
|
||||
available_tools=["RAG", "WEB", "CODEBRAIN"]
|
||||
)
|
||||
|
||||
assert result["should_invoke_tools"], "Should proactively invoke tools for complex queries"
|
||||
rag_tools = [t for t in result["tools_to_invoke"] if t["tool"] == "RAG"]
|
||||
assert len(rag_tools) > 0, "Should include proactive RAG"
|
||||
|
||||
print(f" ✓ Proactive RAG detection passed")
|
||||
print(f" Reason: {rag_tools[0]['reason']}")
|
||||
|
||||
print("\n✓ Tool Decision Engine tests passed\n")
|
||||
return result
|
||||
|
||||
|
||||
async def test_tool_orchestrator():
|
||||
"""Test tool orchestration (mock mode)."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 2: Tool Orchestrator (Mock Mode)")
|
||||
print("="*60)
|
||||
|
||||
orchestrator = ToolOrchestrator(tool_timeout=5)
|
||||
|
||||
# Since actual tools may not be available, test the orchestrator structure
|
||||
print(f" Available tools: {list(orchestrator.available_tools.keys())}")
|
||||
|
||||
# Test with tools_to_invoke (will fail gracefully if tools unavailable)
|
||||
tools_to_invoke = [
|
||||
{"tool": "RAG", "query": "test query", "reason": "testing", "priority": 0.9}
|
||||
]
|
||||
|
||||
result = await orchestrator.execute_tools(
|
||||
tools_to_invoke=tools_to_invoke,
|
||||
context_state={"session_id": "test"}
|
||||
)
|
||||
|
||||
assert "results" in result, "Should return results dict"
|
||||
assert "execution_summary" in result, "Should return execution summary"
|
||||
|
||||
summary = result["execution_summary"]
|
||||
assert "tools_invoked" in summary, "Summary should include tools_invoked"
|
||||
assert "total_time_ms" in summary, "Summary should include timing"
|
||||
|
||||
print(f" ✓ Orchestrator structure valid")
|
||||
print(f" Summary: {summary}")
|
||||
|
||||
# Test result formatting
|
||||
formatted = orchestrator.format_results_for_context(result)
|
||||
assert isinstance(formatted, str), "Should format results as string"
|
||||
|
||||
print(f" ✓ Result formatting works")
|
||||
print(f" Formatted length: {len(formatted)} chars")
|
||||
|
||||
print("\n✓ Tool Orchestrator tests passed\n")
|
||||
return result
|
||||
|
||||
|
||||
async def test_proactive_monitor():
|
||||
"""Test proactive monitoring and suggestions."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 3: Proactive Monitor")
|
||||
print("="*60)
|
||||
|
||||
monitor = ProactiveMonitor(min_priority=0.6)
|
||||
|
||||
# Test 3a: Long silence detection
|
||||
context_state = {
|
||||
"message_count": 5,
|
||||
"minutes_since_last_msg": 35 # > 30 minutes
|
||||
}
|
||||
|
||||
self_state = load_self_state()
|
||||
|
||||
suggestion = await monitor.analyze_session(
|
||||
session_id="test_silence",
|
||||
context_state=context_state,
|
||||
self_state=self_state
|
||||
)
|
||||
|
||||
assert suggestion is not None, "Should generate suggestion for long silence"
|
||||
assert suggestion["type"] == "check_in", f"Should be check_in type: {suggestion['type']}"
|
||||
assert suggestion["priority"] >= 0.6, "Priority should meet threshold"
|
||||
|
||||
print(f" ✓ Long silence detection passed")
|
||||
print(f" Type: {suggestion['type']}, Priority: {suggestion['priority']:.2f}")
|
||||
print(f" Suggestion: {suggestion['suggestion'][:50]}...")
|
||||
|
||||
# Test 3b: Learning opportunity (high curiosity)
|
||||
self_state["curiosity"] = 0.8
|
||||
self_state["learning_queue"] = ["quantum computing", "rust programming"]
|
||||
|
||||
# Reset cooldown for this test
|
||||
monitor.reset_cooldown("test_learning")
|
||||
|
||||
suggestion = await monitor.analyze_session(
|
||||
session_id="test_learning",
|
||||
context_state={"message_count": 3, "minutes_since_last_msg": 2},
|
||||
self_state=self_state
|
||||
)
|
||||
|
||||
assert suggestion is not None, "Should generate learning suggestion"
|
||||
assert suggestion["type"] == "learning", f"Should be learning type: {suggestion['type']}"
|
||||
|
||||
print(f" ✓ Learning opportunity detection passed")
|
||||
print(f" Suggestion: {suggestion['suggestion'][:70]}...")
|
||||
|
||||
# Test 3c: Conversation milestone
|
||||
monitor.reset_cooldown("test_milestone")
|
||||
|
||||
# Reset curiosity to avoid learning suggestion taking precedence
|
||||
self_state["curiosity"] = 0.5
|
||||
self_state["learning_queue"] = []
|
||||
|
||||
suggestion = await monitor.analyze_session(
|
||||
session_id="test_milestone",
|
||||
context_state={"message_count": 50, "minutes_since_last_msg": 1},
|
||||
self_state=self_state
|
||||
)
|
||||
|
||||
assert suggestion is not None, "Should generate milestone suggestion"
|
||||
# Note: learning or summary both valid - check it's a reasonable suggestion
|
||||
assert suggestion["type"] in ["summary", "learning", "check_in"], f"Should be valid type: {suggestion['type']}"
|
||||
|
||||
print(f" ✓ Conversation milestone detection passed (type: {suggestion['type']})")
|
||||
|
||||
# Test 3d: Cooldown mechanism
|
||||
# Try to get another suggestion immediately (should be blocked)
|
||||
suggestion2 = await monitor.analyze_session(
|
||||
session_id="test_milestone",
|
||||
context_state={"message_count": 51, "minutes_since_last_msg": 1},
|
||||
self_state=self_state
|
||||
)
|
||||
|
||||
assert suggestion2 is None, "Should not generate suggestion during cooldown"
|
||||
|
||||
print(f" ✓ Cooldown mechanism working")
|
||||
|
||||
# Check stats
|
||||
stats = monitor.get_session_stats("test_milestone")
|
||||
assert stats["cooldown_active"], "Cooldown should be active"
|
||||
print(f" Cooldown remaining: {stats['cooldown_remaining']}s")
|
||||
|
||||
print("\n✓ Proactive Monitor tests passed\n")
|
||||
return suggestion
|
||||
|
||||
|
||||
async def test_autonomous_actions():
|
||||
"""Test autonomous action execution."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 4: Autonomous Actions")
|
||||
print("="*60)
|
||||
|
||||
manager = AutonomousActionManager()
|
||||
|
||||
# Test 4a: List allowed actions
|
||||
allowed = manager.get_allowed_actions()
|
||||
assert "create_memory" in allowed, "Should have create_memory action"
|
||||
assert "update_goal" in allowed, "Should have update_goal action"
|
||||
assert "learn_topic" in allowed, "Should have learn_topic action"
|
||||
|
||||
print(f" ✓ Allowed actions: {allowed}")
|
||||
|
||||
# Test 4b: Validate actions
|
||||
validation = manager.validate_action("create_memory", {"text": "test memory"})
|
||||
assert validation["valid"], "Should validate correct action"
|
||||
|
||||
print(f" ✓ Action validation passed")
|
||||
|
||||
# Test 4c: Execute learn_topic action
|
||||
result = await manager.execute_action(
|
||||
action_type="learn_topic",
|
||||
parameters={"topic": "rust programming", "reason": "testing", "priority": 0.8},
|
||||
context={"session_id": "test"}
|
||||
)
|
||||
|
||||
assert result["success"], f"Action should succeed: {result.get('error', 'unknown')}"
|
||||
assert "topic" in result["result"], "Should return topic info"
|
||||
|
||||
print(f" ✓ learn_topic action executed")
|
||||
print(f" Topic: {result['result']['topic']}")
|
||||
print(f" Queue position: {result['result']['queue_position']}")
|
||||
|
||||
# Test 4d: Execute update_focus action
|
||||
result = await manager.execute_action(
|
||||
action_type="update_focus",
|
||||
parameters={"focus": "autonomy_testing", "reason": "running tests"},
|
||||
context={"session_id": "test"}
|
||||
)
|
||||
|
||||
assert result["success"], "update_focus should succeed"
|
||||
|
||||
print(f" ✓ update_focus action executed")
|
||||
print(f" New focus: {result['result']['new_focus']}")
|
||||
|
||||
# Test 4e: Reject non-whitelisted action
|
||||
result = await manager.execute_action(
|
||||
action_type="delete_all_files", # NOT in whitelist
|
||||
parameters={},
|
||||
context={"session_id": "test"}
|
||||
)
|
||||
|
||||
assert not result["success"], "Should reject non-whitelisted action"
|
||||
assert "not in whitelist" in result["error"], "Should indicate whitelist violation"
|
||||
|
||||
print(f" ✓ Non-whitelisted action rejected")
|
||||
|
||||
# Test 4f: Action log
|
||||
log = manager.get_action_log(limit=10)
|
||||
assert len(log) >= 2, f"Should have logged multiple actions (got {len(log)})"
|
||||
|
||||
print(f" ✓ Action log contains {len(log)} entries")
|
||||
|
||||
print("\n✓ Autonomous Actions tests passed\n")
|
||||
return result
|
||||
|
||||
|
||||
async def test_pattern_learner():
|
||||
"""Test pattern learning system."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 5: Pattern Learner")
|
||||
print("="*60)
|
||||
|
||||
# Use temp file for testing
|
||||
test_file = "/tmp/test_patterns.json"
|
||||
learner = PatternLearner(patterns_file=test_file)
|
||||
|
||||
# Test 5a: Learn from multiple interactions
|
||||
for i in range(5):
|
||||
await learner.learn_from_interaction(
|
||||
user_prompt=f"Help me with Python coding task {i}",
|
||||
response=f"Here's help with task {i}...",
|
||||
monologue={"intent": "coding_help", "tone": "focused", "depth": "medium"},
|
||||
context={"session_id": "test", "executive_plan": None}
|
||||
)
|
||||
|
||||
print(f" ✓ Learned from 5 interactions")
|
||||
|
||||
# Test 5b: Get top topics
|
||||
top_topics = learner.get_top_topics(limit=5)
|
||||
assert len(top_topics) > 0, "Should have learned topics"
|
||||
assert "coding_help" == top_topics[0][0], "coding_help should be top topic"
|
||||
|
||||
print(f" ✓ Top topics: {[t[0] for t in top_topics[:3]]}")
|
||||
|
||||
# Test 5c: Get preferred tone
|
||||
preferred_tone = learner.get_preferred_tone()
|
||||
assert preferred_tone == "focused", "Should detect focused as preferred tone"
|
||||
|
||||
print(f" ✓ Preferred tone: {preferred_tone}")
|
||||
|
||||
# Test 5d: Get preferred depth
|
||||
preferred_depth = learner.get_preferred_depth()
|
||||
assert preferred_depth == "medium", "Should detect medium as preferred depth"
|
||||
|
||||
print(f" ✓ Preferred depth: {preferred_depth}")
|
||||
|
||||
# Test 5e: Get insights
|
||||
insights = learner.get_insights()
|
||||
assert insights["total_interactions"] == 5, "Should track interaction count"
|
||||
assert insights["preferred_tone"] == "focused", "Insights should include tone"
|
||||
|
||||
print(f" ✓ Insights generated:")
|
||||
print(f" Total interactions: {insights['total_interactions']}")
|
||||
print(f" Recommendations: {insights['learning_recommendations']}")
|
||||
|
||||
# Test 5f: Export patterns
|
||||
exported = learner.export_patterns()
|
||||
assert "topic_frequencies" in exported, "Should export all patterns"
|
||||
|
||||
print(f" ✓ Patterns exported ({len(exported)} keys)")
|
||||
|
||||
# Cleanup
|
||||
if os.path.exists(test_file):
|
||||
os.remove(test_file)
|
||||
|
||||
print("\n✓ Pattern Learner tests passed\n")
|
||||
return insights
|
||||
|
||||
|
||||
async def test_end_to_end_autonomy():
|
||||
"""Test complete autonomous flow."""
|
||||
print("\n" + "="*60)
|
||||
print("TEST 6: End-to-End Autonomy Flow")
|
||||
print("="*60)
|
||||
|
||||
# Simulate a complex user query that triggers multiple autonomous systems
|
||||
user_prompt = "Remember what we discussed about machine learning? I need current research on transformers."
|
||||
|
||||
monologue = {
|
||||
"intent": "technical_research",
|
||||
"tone": "focused",
|
||||
"depth": "deep",
|
||||
"consult_executive": True
|
||||
}
|
||||
|
||||
context_state = {
|
||||
"session_id": "e2e_test",
|
||||
"message_count": 15,
|
||||
"minutes_since_last_msg": 5
|
||||
}
|
||||
|
||||
print(f" User prompt: {user_prompt}")
|
||||
print(f" Monologue intent: {monologue['intent']}")
|
||||
|
||||
# Step 1: Tool decision engine
|
||||
engine = ToolDecisionEngine()
|
||||
tool_decision = await engine.analyze_tool_needs(
|
||||
user_prompt=user_prompt,
|
||||
monologue=monologue,
|
||||
context_state=context_state,
|
||||
available_tools=["RAG", "WEB", "CODEBRAIN"]
|
||||
)
|
||||
|
||||
print(f"\n Step 1: Tool Decision")
|
||||
print(f" Should invoke: {tool_decision['should_invoke_tools']}")
|
||||
print(f" Tools: {[t['tool'] for t in tool_decision['tools_to_invoke']]}")
|
||||
assert tool_decision["should_invoke_tools"], "Should invoke tools"
|
||||
assert len(tool_decision["tools_to_invoke"]) >= 2, "Should recommend multiple tools (RAG + WEB)"
|
||||
|
||||
# Step 2: Pattern learning
|
||||
learner = PatternLearner(patterns_file="/tmp/e2e_test_patterns.json")
|
||||
await learner.learn_from_interaction(
|
||||
user_prompt=user_prompt,
|
||||
response="Here's information about transformers...",
|
||||
monologue=monologue,
|
||||
context=context_state
|
||||
)
|
||||
|
||||
print(f"\n Step 2: Pattern Learning")
|
||||
top_topics = learner.get_top_topics(limit=3)
|
||||
print(f" Learned topics: {[t[0] for t in top_topics]}")
|
||||
|
||||
# Step 3: Autonomous action
|
||||
action_manager = AutonomousActionManager()
|
||||
action_result = await action_manager.execute_action(
|
||||
action_type="learn_topic",
|
||||
parameters={"topic": "transformer architectures", "reason": "user interest detected"},
|
||||
context=context_state
|
||||
)
|
||||
|
||||
print(f"\n Step 3: Autonomous Action")
|
||||
print(f" Action: learn_topic")
|
||||
print(f" Success: {action_result['success']}")
|
||||
|
||||
# Step 4: Proactive monitoring (won't trigger due to low message count)
|
||||
monitor = ProactiveMonitor(min_priority=0.6)
|
||||
monitor.reset_cooldown("e2e_test")
|
||||
|
||||
suggestion = await monitor.analyze_session(
|
||||
session_id="e2e_test",
|
||||
context_state=context_state,
|
||||
self_state=load_self_state()
|
||||
)
|
||||
|
||||
print(f"\n Step 4: Proactive Monitoring")
|
||||
print(f" Suggestion: {suggestion['type'] if suggestion else 'None (expected for low message count)'}")
|
||||
|
||||
# Cleanup
|
||||
if os.path.exists("/tmp/e2e_test_patterns.json"):
|
||||
os.remove("/tmp/e2e_test_patterns.json")
|
||||
|
||||
print("\n✓ End-to-End Autonomy Flow tests passed\n")
|
||||
return True
|
||||
|
||||
|
||||
async def run_all_tests():
|
||||
"""Run all Phase 2 tests."""
|
||||
print("\n" + "="*60)
|
||||
print("PHASE 2 AUTONOMY TESTS")
|
||||
print("="*60)
|
||||
|
||||
try:
|
||||
# Test 1: Tool Decision Engine
|
||||
await test_tool_decision_engine()
|
||||
|
||||
# Test 2: Tool Orchestrator
|
||||
await test_tool_orchestrator()
|
||||
|
||||
# Test 3: Proactive Monitor
|
||||
await test_proactive_monitor()
|
||||
|
||||
# Test 4: Autonomous Actions
|
||||
await test_autonomous_actions()
|
||||
|
||||
# Test 5: Pattern Learner
|
||||
await test_pattern_learner()
|
||||
|
||||
# Test 6: End-to-End
|
||||
await test_end_to_end_autonomy()
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("ALL PHASE 2 TESTS PASSED ✓")
|
||||
print("="*60)
|
||||
|
||||
print("\nPhase 2 Features Validated:")
|
||||
print(" ✓ Autonomous tool decision making")
|
||||
print(" ✓ Tool orchestration and execution")
|
||||
print(" ✓ Proactive monitoring and suggestions")
|
||||
print(" ✓ Safe autonomous actions")
|
||||
print(" ✓ Pattern learning and adaptation")
|
||||
print(" ✓ End-to-end autonomous flow")
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print("\n" + "="*60)
|
||||
print(f"TEST FAILED: {e}")
|
||||
print("="*60)
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = asyncio.run(run_all_tests())
|
||||
sys.exit(0 if success else 1)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,7 +1,9 @@
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel
|
||||
from qdrant_client import QdrantClient
|
||||
from qdrant_client.models import (
|
||||
Distance,
|
||||
@@ -19,6 +21,13 @@ from mem0.vector_stores.base import VectorStoreBase
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OutputData(BaseModel):
|
||||
"""Standard output format for vector search results."""
|
||||
id: Optional[str]
|
||||
score: Optional[float]
|
||||
payload: Optional[dict]
|
||||
|
||||
|
||||
class Qdrant(VectorStoreBase):
|
||||
def __init__(
|
||||
self,
|
||||
@@ -170,7 +179,7 @@ class Qdrant(VectorStoreBase):
|
||||
filters (dict, optional): Filters to apply to the search. Defaults to None.
|
||||
|
||||
Returns:
|
||||
list: Search results.
|
||||
list: Search results wrapped in OutputData format.
|
||||
"""
|
||||
query_filter = self._create_filter(filters) if filters else None
|
||||
hits = self.client.query_points(
|
||||
@@ -179,7 +188,16 @@ class Qdrant(VectorStoreBase):
|
||||
query_filter=query_filter,
|
||||
limit=limit,
|
||||
)
|
||||
return hits.points
|
||||
|
||||
# Wrap results in OutputData format to match other vector stores
|
||||
return [
|
||||
OutputData(
|
||||
id=str(hit.id),
|
||||
score=hit.score,
|
||||
payload=hit.payload
|
||||
)
|
||||
for hit in hits.points
|
||||
]
|
||||
|
||||
def delete(self, vector_id: int):
|
||||
"""
|
||||
@@ -207,7 +225,7 @@ class Qdrant(VectorStoreBase):
|
||||
point = PointStruct(id=vector_id, vector=vector, payload=payload)
|
||||
self.client.upsert(collection_name=self.collection_name, points=[point])
|
||||
|
||||
def get(self, vector_id: int) -> dict:
|
||||
def get(self, vector_id: int) -> OutputData:
|
||||
"""
|
||||
Retrieve a vector by ID.
|
||||
|
||||
@@ -215,10 +233,17 @@ class Qdrant(VectorStoreBase):
|
||||
vector_id (int): ID of the vector to retrieve.
|
||||
|
||||
Returns:
|
||||
dict: Retrieved vector.
|
||||
OutputData: Retrieved vector wrapped in OutputData format.
|
||||
"""
|
||||
result = self.client.retrieve(collection_name=self.collection_name, ids=[vector_id], with_payload=True)
|
||||
return result[0] if result else None
|
||||
if result:
|
||||
hit = result[0]
|
||||
return OutputData(
|
||||
id=str(hit.id),
|
||||
score=None, # No score for direct retrieval
|
||||
payload=hit.payload
|
||||
)
|
||||
return None
|
||||
|
||||
def list_cols(self) -> list:
|
||||
"""
|
||||
@@ -251,7 +276,7 @@ class Qdrant(VectorStoreBase):
|
||||
limit (int, optional): Number of vectors to return. Defaults to 100.
|
||||
|
||||
Returns:
|
||||
list: List of vectors.
|
||||
list: List of vectors wrapped in OutputData format.
|
||||
"""
|
||||
query_filter = self._create_filter(filters) if filters else None
|
||||
result = self.client.scroll(
|
||||
@@ -261,7 +286,18 @@ class Qdrant(VectorStoreBase):
|
||||
with_payload=True,
|
||||
with_vectors=False,
|
||||
)
|
||||
return result
|
||||
|
||||
# Wrap results in OutputData format
|
||||
# scroll() returns tuple: (points, next_page_offset)
|
||||
points = result[0] if isinstance(result, tuple) else result
|
||||
return [
|
||||
OutputData(
|
||||
id=str(point.id),
|
||||
score=None, # No score for list operation
|
||||
payload=point.payload
|
||||
)
|
||||
for point in points
|
||||
]
|
||||
|
||||
def reset(self):
|
||||
"""Reset the index by deleting and recreating it."""
|
||||
|
||||
Reference in New Issue
Block a user