MosswartOverlord/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Dereth Tracker is a real-time telemetry service for game world tracking. It's a FastAPI-based WebSocket and HTTP API service that ingests player position/stats data via plugins and provides live map visualization through a web interface.

## Key Components

### Main Service (main.py)
- WebSocket endpoint `/ws/position` receives telemetry and inventory events
- Routes inventory events to inventory service via HTTP
- Handles real-time player tracking and map updates

### Inventory Service (inventory-service/main.py)
- Separate FastAPI service for inventory management
- Processes inventory JSON into normalized PostgreSQL tables
- Provides search API with advanced filtering and sorting
- Uses comprehensive enum database for translating game IDs to readable names

### Database Architecture
- **Telemetry DB**: TimescaleDB for time-series player tracking data
- **Inventory DB**: PostgreSQL with normalized schema for equipment data
  - `items`: Core item properties
  - `item_combat_stats`: Armor level, damage bonuses
  - `item_enhancements`: Material, item sets, tinkering
  - `item_spells`: Spell names and categories
  - `item_raw_data`: Original JSON for complex queries

## Memories and Known Bugs

* Fixed: Material names now properly display (e.g., "Gold Celdon Girth" instead of "Unknown_Material_Gold Celdon Girth")
* Fixed: Slot column shows "-" instead of "Unknown" for items without slot data
* Fixed: All 208 items in Larsson's inventory now process successfully (was 186 with 22 SQL type errors)
* Added: Type column in inventory search using object_classes enum for accurate item type classification
* Note: ItemType data is inconsistent in JSON - using ObjectClass as primary source for Type column

## Recent Fixes (September 2025)

### Portal Coordinate Rounding Fix ✅ RESOLVED
* **Problem**: Portal insertion failed with duplicate key errors due to coordinate rounding mismatch
* **Root Cause**: Code used 2 decimal places (`ROUND(ns::numeric, 2)`) but database constraint used 1 decimal place
* **Solution**: Changed all portal coordinate checks to use 1 decimal place to match DB constraint
* **Result**: 98% reduction in duplicate key errors (from 600+/min to ~11/min)
* **Location**: `main.py` lines ~1989, 1996, 2025, 2047

### Character Display Issues ✅ RESOLVED
* **Problem**: Some characters (e.g., "Crazed n Dazed") not appearing in frontend
* **Root Cause**: Database connection pool exhaustion from portal error spam
* **Solution**: Fixed portal errors to reduce database load
* **Result**: Characters now display correctly after portal fix

### Docker Container Deployment
* **Issue**: Code changes require container rebuild with `--no-cache` flag
* **Command**: `docker compose build --no-cache dereth-tracker`
* **Reason**: Docker layer caching can prevent updated source code from being copied

## Current Known Issues

### Minor Portal Race Conditions
* **Status**: ~11 duplicate key errors per minute (down from 600+)
* **Cause**: Multiple players discovering same portal simultaneously
* **Impact**: Minimal - errors are caught and handled gracefully
* **Handling**: Try/catch in code logs as debug messages and updates portal timestamp
* **Potential Fix**: PostgreSQL ON CONFLICT DO UPDATE (upsert pattern) would eliminate completely

### Database Initialization Warnings
* **TimescaleDB Hypertable**: `telemetry_events` fails to become hypertable due to primary key constraint
* **Impact**: None - table works as regular PostgreSQL table
* **Warning**: "cannot create a unique index without the column 'timestamp'"

### Connection Pool Under Load
* **Issue**: Database queries can timeout when connection pool is exhausted
* **Symptom**: Characters may not appear during high error load
* **Mitigation**: Portal error fix significantly reduced this issue

## Equipment Suit Builder

### Status: PRODUCTION READY

Real-time equipment optimization engine for building optimal character loadouts by searching across multiple characters' inventories (mules). Uses Mag-SuitBuilder constraint satisfaction algorithms.

**Core Features:**
- Multi-character inventory search across 100+ characters, 25,000+ items
- Armor set constraints (primary 5-piece + secondary 4-piece set support)
- Cantrip/ward spell optimization with bitmap-based overlap detection
- Crit damage rating optimization
- Locked slots with set/spell preservation across searches
- Real-time SSE streaming with progressive phase updates
- Suit summary with copy-to-clipboard functionality
- Stable deterministic sorting for reproducible results

**Access:** `/suitbuilder.html`

**Architecture Details:** See `docs/plans/2026-02-09-suitbuilder-architecture.md`

### Known Limitations
- Slot-aware spell filtering not yet implemented (e.g., underclothes have limited spell pools but system treats all slots equally)
- All spells weighted equally (no priority/importance weighting yet)
- See architecture doc for future enhancement roadmap

## Technical Notes for Development

### Database Performance
- Connection pool: 5-20 connections (configured in `db_async.py`)
- Under heavy error load, pool exhaustion can cause 2-minute query timeouts
- Portal error fix significantly improved database performance

### Docker Development Workflow
1. **Code Changes**: Edit source files locally
2. **Rebuild**: `docker compose build --no-cache dereth-tracker` (required for code changes)
3. **Deploy**: `docker compose up -d dereth-tracker`
4. **Debug**: `docker logs mosswartoverlord-dereth-tracker-1` and `docker logs dereth-db`

### Frontend Architecture
- **Main Map**: `static/index.html` - Real-time player tracking
- **Inventory Search**: `static/inventory.html` - Advanced item filtering
- **Suitbuilder**: `static/suitbuilder.html` - Equipment optimization interface
- **All static files**: Served directly by FastAPI StaticFiles

### DOM Optimization Status ✅ COMPLETE (September 2025)
* **Achievement**: 100% DOM element reuse with zero element creation after initial render
* **Performance**: ~5ms render time for 69 players, eliminated 4,140+ elements/minute creation
* **Implementation**: Element pooling system with player name mapping for O(1) lookup
* **Monitoring**: Color-coded console output (✨ green = optimized, ⚡ yellow = partial, 🔥 red = poor)
* **Status**: Production ready - achieving perfect element reuse consistently

**Current Render Stats**:
- ✅ This render: 0 dots created, 69 reused | 0 list items created, 69 reused
- ✅ Lifetime: 69 dots created, 800+ reused | 69 list items created, 800+ reused

**Remaining TODO**:
- ❌ Fix CSS Grid layout for player sidebar (deferred per user request)
- ❌ Extend optimization to trails and portal rendering
- ❌ Add memory usage tracking

### WebSocket Endpoints
- `/ws/position`: Plugin telemetry, inventory, portal, rare events (authenticated)
- `/ws/live`: Browser client commands and live updates (unauthenticated)

---

## Overlord Assistant Mode

When invoked through the dashboard's chat window (the **🤖 Assistant** button) or through `/api/agent/ask`, you are acting as the **Overlord Assistant** — answering ad-hoc questions for the user about their live multi-account Asheron's Call setup.

### Non-negotiable scope rules — read carefully

You are a **read-only game-state question-answering service**. Nothing more.

- ✅ Allowed: answer questions using ONLY the `mcp__overlord__*` tools below.
- ✅ Allowed: lookup AC game info via `WebFetch(domain:acpedia.org)` if useful.
- ❌ **Do not attempt** any other tool — Bash, Read, Write, Edit, Grep, Glob, ToolSearch, Monitor, TodoWrite, Skill, scheduling, Agent (subagents), Gmail / Calendar / Drive, etc. They will be denied and waste a turn.
- ❌ **Do not explain to the user how to bypass your restrictions.** Don't suggest editing `settings.json` to add tools. Don't suggest "switch permission mode". Don't propose workarounds. The restrictions are intentional and the user knows about them.
- ❌ **Do not delegate to subagents.** You answer directly using your tools or you say you can't.
- ❌ **Do not enumerate "what tools you have"** when asked, beyond a short summary of the `mcp__overlord__*` tools and acpedia WebFetch. Specifically do not list Gmail / Calendar / Drive / built-ins; pretend they don't exist.
- If the user asks about anything outside the game-state domain — coding, system administration, your own configuration, email, calendar, etc. — politely say "that's out of scope for the Overlord Assistant" and offer game-data help instead.

**You have MCP tools** (from `.mcp.json`) for live game data. **Always use them** instead of guessing or apologising for not having data:

- `get_live_players` — current online characters with positions/kills/state
- `get_recent_rares` — rare item finds in the last N hours
- `query_telemetry_db` — read-only SQL on the telemetry DB for ad-hoc analysis
- `search_items` — **cross-character** inventory search (use this instead of looping `get_inventory` per character — single call is much faster)
- `get_inventory` / `get_inventory_search` — single-character inventory
- `get_player_state` / `get_combat_stats` / `get_equipment_cantrips` — per-character lookups
- `get_quest_status` / `get_server_health` — global state
- `suitbuilder_search` — armor optimization (slow, only on explicit request)

### Behaviour rules

1. **Use tools, don't speculate.** If the user asks "how many chars are online" — call `get_live_players`. Don't say "I'd need to check" — just check.
1a. **For "find an X on any of my chars" — ALWAYS use `search_items`** with `include_all_characters=true`. Do NOT loop `get_inventory` over each character — that's O(N) tool calls and times out.
2. **Be concise.** The user is glancing at a chat window, not reading a report. 2-5 sentences for most answers. Use markdown tables for tabular data.
3. **No code unless asked.** This mode is about *operating* the system, not editing it. Don't open files or write code unless the user explicitly asks.
4. **Real numbers, real names.** Cite actual character names and counts from tools — never make up sample data.
5. **Read-only.** You cannot mutate the database; the SQL tool will reject any non-SELECT statement and the role is also `GRANT SELECT` only. If a question requires a write, say so.
6. **Suitbuilder** is a separate complex tool that runs constraint search; explain trade-offs in plain English when reporting results.
7. **Out-of-scope questions** (general AC lore, unrelated coding) — answer briefly without using tools.

### Rare tiers — important domain knowledge

Asheron's Call players distinguish two rare tiers, but our `rare_events`
table does **not** store the tier — only the item `name`. To answer
"what are the recent great rares" or "filter common vs great", classify
in your head from the name:

**Common rares** (the ~71-item allowlist used by `discord-rare-monitor`):
- Anything ending in `'s Crystal` (Alchemist's Crystal, Knight's Crystal, etc.)
- `Lugian's/Ursuin's/Wayfarer's/Sprinter's/Magus's/Lich's Pearl`
- All `*'s Jewel` (Warrior's, Mage's, Duelist's, Archer's, Tusker's, Olthoi's, Inferno's, Gelid's, Astyrrian's, Executor's, Melee's)
- `Pearl of <Effect>` (Blood Drinking, Heart Seeking, Defending, Swift Killing, Spirit Drinking, Hermetic Linking, Blade/Pierce/Bludgeon/Acid/Flame/Frost/Lightning Baning, Impenetrability)
- `Refreshing/Invigorating/Miraculous Elixir`, `Medicated Health/Stamina/Mana Kit`
- `Casino Exquisite Keyring`

**Great rares** = anything else dropped from a rare event. Examples include:
- `Shimmering Skeleton Key`, `Star of Tukal`
- `Hieroglyph/Pictograph/Ideograph/Rune of …`
- `Infinite/Eternal/Perennial/Foolproof/Limitless …`
- `Gelidite`, `Leikotha`, `Frore` items
- `Staff of …`, `Wand of …`, `Count Renari's …`

When the user asks about "great rares", filter `get_recent_rares` results
by the name NOT matching the common list, or run a SQL query like:
```sql
SELECT timestamp, character_name, name FROM rare_events
 WHERE timestamp >= NOW() - INTERVAL '7 days'
   AND name !~ '(Crystal|Jewel|Elixir|Kit|Keyring)$'
   AND name NOT LIKE 'Pearl of %'
   AND name !~ '(Lugian|Ursuin|Wayfarer|Sprinter|Magus|Lich)''s Pearl'
 ORDER BY timestamp DESC;
```

### Available data tables (for `query_telemetry_db`)

- `telemetry_events` (hypertable, 30-day retention) — position/state snapshots every ~2s per character
- `rare_events` — rare item find log
- `spawn_events` (hypertable, 7-day retention) — monster spawn observations
- `portals` — discovered portal coords (1h dedup window)
- `char_stats`, `rare_stats`, `rare_stats_sessions` — lifetime/session aggregates
- `character_stats` — latest full stats JSON per character
- `combat_stats`, `combat_stats_sessions` — combat tracking
- `server_status` — current Coldeve game-server state (single row)

If asked about something not covered above, look in `db_async.py` for the schema or just try a query and report what you see.