# Logging System Migration Complete

## ✅ What Changed

The old `VERBOSE_DEBUG` logging system has been completely replaced with the new structured `LOG_DETAIL_LEVEL` system.

### Files Modified

1. **[.env](.env)** - Removed `VERBOSE_DEBUG`, cleaned up duplicate `LOG_DETAIL_LEVEL` settings
2. **[cortex/.env](cortex/.env)** - Removed `VERBOSE_DEBUG` from cortex config
3. **[cortex/router.py](cortex/router.py)** - Replaced `VERBOSE_DEBUG` checks with `LOG_DETAIL_LEVEL`
4. **[cortex/context.py](cortex/context.py)** - Replaced `VERBOSE_DEBUG` with `LOG_DETAIL_LEVEL`, removed verbose file logging setup

## 🎯 New Logging Configuration

### Single Environment Variable

Set `LOG_DETAIL_LEVEL` in your `.env` file:

```bash
LOG_DETAIL_LEVEL=detailed
```

### Logging Levels

| Level | Lines/Message | What You See |
|-------|---------------|--------------|
| **minimal** | 1-2 | Only errors and critical events |
| **summary** | 5-7 | Pipeline completion, errors, warnings (production mode) |
| **detailed** | 30-50 | LLM outputs, timing breakdowns, context (debugging mode) |
| **verbose** | 100+ | Everything including raw JSON dumps (deep debugging) |

## 📊 What You Get at Each Level

### Summary Mode (Production)
```
📊 Context | Session: abc123 | Messages: 42 | Last: 5.2min | RAG: 3 results
🧠 Monologue | question | Tone: curious

====================================================================================================
✨ PIPELINE COMPLETE | Session: abc123 | Total: 1250ms
====================================================================================================
📤 Output: 342 characters
====================================================================================================
```

### Detailed Mode (Debugging - RECOMMENDED)
```
====================================================================================================
🚀 PIPELINE START | Session: abc123 | 14:23:45.123
====================================================================================================
📝 User: What is the meaning of life?
────────────────────────────────────────────────────────────────────────────────────────────────────

────────────────────────────────────────────────────────────────────────────────────────────────────
🧠 LLM CALL | Backend: PRIMARY | 14:23:45.234
────────────────────────────────────────────────────────────────────────────────────────────────────
📝 Prompt: You are Lyra, analyzing the user's question...
💬 Reply: Based on the context provided, here's my analysis...
────────────────────────────────────────────────────────────────────────────────────────────────────

📊 Context | Session: abc123 | Messages: 42 | Last: 5.2min | RAG: 3 results
────────────────────────────────────────────────────────────────────────────────────────────────────
[CONTEXT] Session abc123 | User: What is the meaning of life?
────────────────────────────────────────────────────────────────────────────────────────────────────
  Mode: default | Mood: neutral | Project: None
  Tools: RAG, WEB, WEATHER, CODEBRAIN, POKERBRAIN

  ╭─ INTAKE SUMMARIES ────────────────────────────────────────────────
  │ L1  : Last message discussed philosophy...
  │ L5  : Recent 5 messages covered existential topics...
  │ L10 : Past 10 messages showed curiosity pattern...
  ╰───────────────────────────────────────────────────────────────────

  ╭─ RAG RESULTS (3) ──────────────────────────────────────────────
  │ [1] 0.923 | Previous discussion about purpose...
  │ [2] 0.891 | Note about existential philosophy...
  │ [3] 0.867 | Memory of Viktor Frankl discussion...
  ╰───────────────────────────────────────────────────────────────────
────────────────────────────────────────────────────────────────────────────────────────────────────

🧠 Monologue | question | Tone: curious

====================================================================================================
✨ PIPELINE COMPLETE | Session: abc123 | Total: 1250ms
====================================================================================================
⏱️  Stage Timings:
   context        :   150ms ( 12.0%)
   identity       :    10ms (  0.8%)
   monologue      :   200ms ( 16.0%)
   tools          :     0ms (  0.0%)
   reflection     :    50ms (  4.0%)
   reasoning      :   450ms ( 36.0%)  ← BOTTLENECK!
   refinement     :   300ms ( 24.0%)
   persona        :   140ms ( 11.2%)
   learning       :    50ms (  4.0%)
📤 Output: 342 characters
====================================================================================================
```

### Verbose Mode (Maximum Debug)
Same as detailed, plus:
- Full raw JSON responses from LLMs (50-line boxes)
- Complete intake data structures
- Stack traces on errors

## 🚀 How to Use

### For Finding Weak Links (Your Use Case)
```bash
# In .env:
LOG_DETAIL_LEVEL=detailed

# Restart services:
docker-compose restart cortex relay
```

You'll now see:
- ✅ Which LLM backend is used
- ✅ What prompts are sent to each LLM
- ✅ What each LLM responds with
- ✅ Timing breakdown showing which stage is slow
- ✅ Context being used (RAG, intake summaries)
- ✅ Clean, hierarchical structure

### For Production
```bash
LOG_DETAIL_LEVEL=summary
```

### For Deep Debugging
```bash
LOG_DETAIL_LEVEL=verbose
```

## 🔍 Finding Performance Bottlenecks

With `detailed` mode, look for:

1. **Slow stages in timing breakdown:**
   ```
   reasoning      :  3450ms ( 76.0%)  ← THIS IS YOUR BOTTLENECK!
   ```

2. **Backend failures:**
   ```
   ⚠️  [LLM] PRIMARY failed | 14:23:45.234 | Connection timeout
   ✅ [LLM] SECONDARY | Reply: Based on...  ← Fell back to secondary
   ```

3. **Loop detection:**
   ```
   ⚠️  DUPLICATE MESSAGE DETECTED | Session: abc123
   🔁 LOOP DETECTED - Returning cached context
   ```

## 📁 Removed Features

The following old logging features have been removed:

- ❌ `VERBOSE_DEBUG` environment variable (replaced with `LOG_DETAIL_LEVEL`)
- ❌ File logging to `/app/logs/cortex_verbose_debug.log` (use `docker logs` instead)
- ❌ Separate verbose handlers in Python logging
- ❌ Per-module verbose flags

## ✨ New Features

- ✅ Single unified logging configuration
- ✅ Hierarchical, scannable output
- ✅ Collapsible data sections (boxes)
- ✅ Stage timing always shown in detailed mode
- ✅ Performance profiling built-in
- ✅ Loop detection and warnings
- ✅ Clean error formatting

---

**The logging is now clean, concise, and gives you exactly what you need to find weak links!** 🎯