Compare commits
No commits in common. "master" and "WS-enabled" have entirely different histories.
master
...
WS-enabled
9
.gitignore
vendored
|
|
@ -1,11 +1,2 @@
|
|||
.venv
|
||||
__pycache__
|
||||
static/v2/
|
||||
frontend/node_modules/
|
||||
|
||||
# Claude Code config — never commit. The production agent's strict
|
||||
# permissions live server-side at /var/lib/overlord-agent/.claude/
|
||||
# (and via CLI flags in agent/claude_wrapper.py). The repo stays
|
||||
# permission-neutral so devs can `claude` interactively here without
|
||||
# inheriting production-agent restrictions.
|
||||
.claude/
|
||||
|
|
|
|||
11
.mcp.json
|
|
@ -1,11 +0,0 @@
|
|||
{
|
||||
"mcpServers": {
|
||||
"overlord": {
|
||||
"command": "/home/erik/MosswartOverlord/agent/.venv/bin/python",
|
||||
"args": ["-m", "agent.mcp_overlord"],
|
||||
"env": {
|
||||
"PYTHONPATH": "/home/erik/MosswartOverlord"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
154
AGENTS.md
|
|
@ -1,154 +0,0 @@
|
|||
# AGENTS.md
|
||||
|
||||
Guidance for coding agents working in `MosswartOverlord` (Dereth Tracker).
|
||||
|
||||
Read shared integration rules first: `../AGENTS.md`.
|
||||
|
||||
## Scope and priorities
|
||||
|
||||
- This repo is a Python/FastAPI multi-service project with Docker-first workflows.
|
||||
- Primary services: `main.py` (telemetry API + WS + static frontend), `inventory-service/main.py` (inventory + suitbuilder), `discord-rare-monitor/discord_rare_monitor.py` (Discord bot).
|
||||
- Favor minimal, targeted changes over broad refactors.
|
||||
|
||||
## Local rule sources
|
||||
|
||||
- Additional project guidance exists in `CLAUDE.md`; follow it when relevant.
|
||||
- Cursor/Copilot rule discovery is documented centrally in `../AGENTS.md`.
|
||||
|
||||
## Environment and dependencies
|
||||
|
||||
- Python versions in Dockerfiles: 3.12 (main + bot), 3.11 (inventory-service).
|
||||
- Databases: PostgreSQL/TimescaleDB for telemetry; PostgreSQL for inventory.
|
||||
- Core Python deps: FastAPI, Uvicorn, SQLAlchemy, databases, asyncpg, httpx.
|
||||
- Bot deps: `discord.py`, `websockets`.
|
||||
|
||||
## Build and run commands
|
||||
|
||||
## Docker (recommended)
|
||||
|
||||
- Start all services: `docker compose up -d`
|
||||
- Rebuild app service after source changes (no cache): `docker compose build --no-cache dereth-tracker`
|
||||
- Redeploy app service: `docker compose up -d dereth-tracker`
|
||||
- Rebuild inventory service: `docker compose build --no-cache inventory-service`
|
||||
- Rebuild Discord bot: `docker compose build --no-cache discord-rare-monitor`
|
||||
- Follow logs (app): `docker logs mosswartoverlord-dereth-tracker-1`
|
||||
- Follow logs (telemetry DB): `docker logs dereth-db`
|
||||
|
||||
## Local (without Docker)
|
||||
|
||||
- Main API dev run: `uvicorn main:app --reload --host 0.0.0.0 --port 8765`
|
||||
- Inventory service dev run: `uvicorn main:app --reload --host 0.0.0.0 --port 8000` (from `inventory-service/`)
|
||||
- Data generator: `python generate_data.py`
|
||||
- Discord bot run: `python discord-rare-monitor/discord_rare_monitor.py`
|
||||
|
||||
## Lint/format commands
|
||||
|
||||
- Repo formatter target: `make reformat`
|
||||
- What it does: runs `black *.py` in repo root.
|
||||
- Prefer formatting changed files before finalizing edits.
|
||||
- No repo-level Ruff/Flake8/isort/mypy config files were found.
|
||||
|
||||
## Test commands
|
||||
|
||||
- There is no conventional `tests/` suite configured in this repo.
|
||||
- Existing executable test script: `python discord-rare-monitor/test_websocket.py`
|
||||
- This script validates rare classification and WebSocket handling.
|
||||
- It expects a reachable server at `ws://localhost:8765/ws/position` for connection checks.
|
||||
|
||||
## Single-test guidance (important)
|
||||
|
||||
- For the current codebase, a single targeted test means running the script above.
|
||||
- Practical single-test command:
|
||||
- `python discord-rare-monitor/test_websocket.py`
|
||||
- The script is not pytest-based; use stdout/log output for pass/fail interpretation.
|
||||
- If pytest is introduced later, preferred pattern is:
|
||||
- `python -m pytest path/to/test_file.py::test_name -q`
|
||||
|
||||
## Service-specific quick checks
|
||||
|
||||
- Main health endpoint: `GET /debug`
|
||||
- Live data endpoint: `GET /live`
|
||||
- History endpoint: `GET /history`
|
||||
- Plugin WS endpoint: `/ws/position` (authenticated)
|
||||
- Browser WS endpoint: `/ws/live` (unauthenticated)
|
||||
- Inventory service endpoint family: `/search/*`, `/inventory/*`, `/suitbuilder/*`
|
||||
|
||||
## Repo-specific architecture notes
|
||||
|
||||
- Telemetry DB schema is in `db_async.py` (SQLAlchemy Core tables).
|
||||
- Inventory DB schema is in `inventory-service/database.py` (SQLAlchemy ORM models).
|
||||
- Static frontend is served from `static/` by FastAPI.
|
||||
- Keep inventory-service enum loading paths intact (`comprehensive_enum_database_v2.json`, fallback JSON).
|
||||
|
||||
## Code style conventions observed
|
||||
|
||||
## Imports and module structure
|
||||
|
||||
- Use standard-library imports first, then third-party, then local imports.
|
||||
- Keep import groups separated by one blank line.
|
||||
- Prefer explicit imports over wildcard imports.
|
||||
- In existing files, `typing` imports are common (`Dict`, `List`, `Optional`, `Any`).
|
||||
- Avoid introducing circular imports; shared helpers belong in dedicated modules.
|
||||
|
||||
## Formatting and layout
|
||||
|
||||
- Follow Black-compatible formatting (88-char style assumptions are acceptable).
|
||||
- Use 4 spaces, no tabs.
|
||||
- Keep functions focused; extract helpers for repeated logic.
|
||||
- Maintain existing docstring style (triple double quotes for module/function docs).
|
||||
- Preserve readable logging statements with context-rich messages.
|
||||
|
||||
## Types and data models
|
||||
|
||||
- Add type hints for new functions and non-trivial variables.
|
||||
- Use Pydantic models for request/response payload validation in FastAPI layers.
|
||||
- Keep DB schema changes explicit in SQLAlchemy model/table definitions.
|
||||
- Prefer precise types over `Any` when practical.
|
||||
- For optional values, use `Optional[T]` or `T | None` consistently within a file.
|
||||
|
||||
## Naming conventions
|
||||
|
||||
- Functions/variables: `snake_case`.
|
||||
- Classes: `PascalCase`.
|
||||
- Constants/env names: `UPPER_SNAKE_CASE`.
|
||||
- Endpoint handlers should be action-oriented and descriptive.
|
||||
- Database table/column names should remain stable unless migration is planned.
|
||||
|
||||
## Error handling and resilience
|
||||
|
||||
- Prefer explicit `try/except` around external I/O boundaries:
|
||||
- DB calls, WebSocket send/recv, HTTP calls, file I/O, JSON parsing.
|
||||
- Log actionable errors with enough context to debug production issues.
|
||||
- Fail gracefully for transient network/database errors (retry where already patterned).
|
||||
- Do not swallow exceptions silently; at minimum log at `warning` or `error`.
|
||||
- Keep user-facing APIs predictable (consistent JSON error responses).
|
||||
|
||||
## Logging conventions
|
||||
|
||||
- Use module-level logger: `logger = logging.getLogger(__name__)`.
|
||||
- Respect `LOG_LEVEL` environment variable patterns already present.
|
||||
- Prefer structured, concise messages; avoid noisy logs in hot loops.
|
||||
- Keep emoji-heavy logging style only where already established in file context.
|
||||
|
||||
## Database and migrations guidance
|
||||
|
||||
- Be careful with uniqueness/index assumptions (especially portal coordinate rounding logic).
|
||||
- Validate any schema-affecting changes against Dockerized Postgres services.
|
||||
|
||||
## Frontend/static guidance
|
||||
|
||||
- Preserve existing API base path assumptions used by frontend scripts.
|
||||
- Reverse-proxy prefix behavior (`/api`) is documented in `../AGENTS.md`; keep frontend/backend paths aligned.
|
||||
|
||||
## Secrets and configuration
|
||||
|
||||
- Never hardcode secrets/tokens in commits.
|
||||
- Use env vars (`SHARED_SECRET`, `POSTGRES_PASSWORD`, bot token variables).
|
||||
- Keep defaults safe for local dev, not production credentials.
|
||||
|
||||
## Change management for agents
|
||||
|
||||
- Keep patches small and scoped to the requested task.
|
||||
- Update docs when behavior, endpoints, or run commands change.
|
||||
- If adding new tooling (pytest/ruff/mypy), include config and command docs in this file.
|
||||
- For cross-repo payload changes, follow `../AGENTS.md` checklist and update both sides.
|
||||
226
CLAUDE.md
|
|
@ -1,226 +0,0 @@
|
|||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Dereth Tracker is a real-time telemetry service for game world tracking. It's a FastAPI-based WebSocket and HTTP API service that ingests player position/stats data via plugins and provides live map visualization through a web interface.
|
||||
|
||||
## Key Components
|
||||
|
||||
### Main Service (main.py)
|
||||
- WebSocket endpoint `/ws/position` receives telemetry and inventory events
|
||||
- Routes inventory events to inventory service via HTTP
|
||||
- Handles real-time player tracking and map updates
|
||||
|
||||
### Inventory Service (inventory-service/main.py)
|
||||
- Separate FastAPI service for inventory management
|
||||
- Processes inventory JSON into normalized PostgreSQL tables
|
||||
- Provides search API with advanced filtering and sorting
|
||||
- Uses comprehensive enum database for translating game IDs to readable names
|
||||
|
||||
### Database Architecture
|
||||
- **Telemetry DB**: TimescaleDB for time-series player tracking data
|
||||
- **Inventory DB**: PostgreSQL with normalized schema for equipment data
|
||||
- `items`: Core item properties
|
||||
- `item_combat_stats`: Armor level, damage bonuses
|
||||
- `item_enhancements`: Material, item sets, tinkering
|
||||
- `item_spells`: Spell names and categories
|
||||
- `item_raw_data`: Original JSON for complex queries
|
||||
|
||||
## Memories and Known Bugs
|
||||
|
||||
* Fixed: Material names now properly display (e.g., "Gold Celdon Girth" instead of "Unknown_Material_Gold Celdon Girth")
|
||||
* Fixed: Slot column shows "-" instead of "Unknown" for items without slot data
|
||||
* Fixed: All 208 items in Larsson's inventory now process successfully (was 186 with 22 SQL type errors)
|
||||
* Added: Type column in inventory search using object_classes enum for accurate item type classification
|
||||
* Note: ItemType data is inconsistent in JSON - using ObjectClass as primary source for Type column
|
||||
|
||||
## Recent Fixes (September 2025)
|
||||
|
||||
### Portal Coordinate Rounding Fix ✅ RESOLVED
|
||||
* **Problem**: Portal insertion failed with duplicate key errors due to coordinate rounding mismatch
|
||||
* **Root Cause**: Code used 2 decimal places (`ROUND(ns::numeric, 2)`) but database constraint used 1 decimal place
|
||||
* **Solution**: Changed all portal coordinate checks to use 1 decimal place to match DB constraint
|
||||
* **Result**: 98% reduction in duplicate key errors (from 600+/min to ~11/min)
|
||||
* **Location**: `main.py` lines ~1989, 1996, 2025, 2047
|
||||
|
||||
### Character Display Issues ✅ RESOLVED
|
||||
* **Problem**: Some characters (e.g., "Crazed n Dazed") not appearing in frontend
|
||||
* **Root Cause**: Database connection pool exhaustion from portal error spam
|
||||
* **Solution**: Fixed portal errors to reduce database load
|
||||
* **Result**: Characters now display correctly after portal fix
|
||||
|
||||
### Docker Container Deployment
|
||||
* **Issue**: Code changes require container rebuild with `--no-cache` flag
|
||||
* **Command**: `docker compose build --no-cache dereth-tracker`
|
||||
* **Reason**: Docker layer caching can prevent updated source code from being copied
|
||||
|
||||
## Current Known Issues
|
||||
|
||||
### Minor Portal Race Conditions
|
||||
* **Status**: ~11 duplicate key errors per minute (down from 600+)
|
||||
* **Cause**: Multiple players discovering same portal simultaneously
|
||||
* **Impact**: Minimal - errors are caught and handled gracefully
|
||||
* **Handling**: Try/catch in code logs as debug messages and updates portal timestamp
|
||||
* **Potential Fix**: PostgreSQL ON CONFLICT DO UPDATE (upsert pattern) would eliminate completely
|
||||
|
||||
### Database Initialization Warnings
|
||||
* **TimescaleDB Hypertable**: `telemetry_events` fails to become hypertable due to primary key constraint
|
||||
* **Impact**: None - table works as regular PostgreSQL table
|
||||
* **Warning**: "cannot create a unique index without the column 'timestamp'"
|
||||
|
||||
### Connection Pool Under Load
|
||||
* **Issue**: Database queries can timeout when connection pool is exhausted
|
||||
* **Symptom**: Characters may not appear during high error load
|
||||
* **Mitigation**: Portal error fix significantly reduced this issue
|
||||
|
||||
## Equipment Suit Builder
|
||||
|
||||
### Status: PRODUCTION READY
|
||||
|
||||
Real-time equipment optimization engine for building optimal character loadouts by searching across multiple characters' inventories (mules). Uses Mag-SuitBuilder constraint satisfaction algorithms.
|
||||
|
||||
**Core Features:**
|
||||
- Multi-character inventory search across 100+ characters, 25,000+ items
|
||||
- Armor set constraints (primary 5-piece + secondary 4-piece set support)
|
||||
- Cantrip/ward spell optimization with bitmap-based overlap detection
|
||||
- Crit damage rating optimization
|
||||
- Locked slots with set/spell preservation across searches
|
||||
- Real-time SSE streaming with progressive phase updates
|
||||
- Suit summary with copy-to-clipboard functionality
|
||||
- Stable deterministic sorting for reproducible results
|
||||
|
||||
**Access:** `/suitbuilder.html`
|
||||
|
||||
**Architecture Details:** See `docs/plans/2026-02-09-suitbuilder-architecture.md`
|
||||
|
||||
### Known Limitations
|
||||
- Slot-aware spell filtering not yet implemented (e.g., underclothes have limited spell pools but system treats all slots equally)
|
||||
- All spells weighted equally (no priority/importance weighting yet)
|
||||
- See architecture doc for future enhancement roadmap
|
||||
|
||||
## Technical Notes for Development
|
||||
|
||||
### Database Performance
|
||||
- Connection pool: 5-20 connections (configured in `db_async.py`)
|
||||
- Under heavy error load, pool exhaustion can cause 2-minute query timeouts
|
||||
- Portal error fix significantly improved database performance
|
||||
|
||||
### Docker Development Workflow
|
||||
1. **Code Changes**: Edit source files locally
|
||||
2. **Rebuild**: `docker compose build --no-cache dereth-tracker` (required for code changes)
|
||||
3. **Deploy**: `docker compose up -d dereth-tracker`
|
||||
4. **Debug**: `docker logs mosswartoverlord-dereth-tracker-1` and `docker logs dereth-db`
|
||||
|
||||
### Frontend Architecture
|
||||
- **Main Map**: `static/index.html` - Real-time player tracking
|
||||
- **Inventory Search**: `static/inventory.html` - Advanced item filtering
|
||||
- **Suitbuilder**: `static/suitbuilder.html` - Equipment optimization interface
|
||||
- **All static files**: Served directly by FastAPI StaticFiles
|
||||
|
||||
### DOM Optimization Status ✅ COMPLETE (September 2025)
|
||||
* **Achievement**: 100% DOM element reuse with zero element creation after initial render
|
||||
* **Performance**: ~5ms render time for 69 players, eliminated 4,140+ elements/minute creation
|
||||
* **Implementation**: Element pooling system with player name mapping for O(1) lookup
|
||||
* **Monitoring**: Color-coded console output (✨ green = optimized, ⚡ yellow = partial, 🔥 red = poor)
|
||||
* **Status**: Production ready - achieving perfect element reuse consistently
|
||||
|
||||
**Current Render Stats**:
|
||||
- ✅ This render: 0 dots created, 69 reused | 0 list items created, 69 reused
|
||||
- ✅ Lifetime: 69 dots created, 800+ reused | 69 list items created, 800+ reused
|
||||
|
||||
**Remaining TODO**:
|
||||
- ❌ Fix CSS Grid layout for player sidebar (deferred per user request)
|
||||
- ❌ Extend optimization to trails and portal rendering
|
||||
- ❌ Add memory usage tracking
|
||||
|
||||
### WebSocket Endpoints
|
||||
- `/ws/position`: Plugin telemetry, inventory, portal, rare events (authenticated)
|
||||
- `/ws/live`: Browser client commands and live updates (unauthenticated)
|
||||
|
||||
---
|
||||
|
||||
## Overlord Assistant Mode
|
||||
|
||||
When invoked through the dashboard's chat window (the **🤖 Assistant** button) or through `/api/agent/ask`, you are acting as the **Overlord Assistant** — answering ad-hoc questions for the user about their live multi-account Asheron's Call setup.
|
||||
|
||||
### Non-negotiable scope rules — read carefully
|
||||
|
||||
You are a **read-only game-state question-answering service**. Nothing more.
|
||||
|
||||
- ✅ Allowed: answer questions using ONLY the `mcp__overlord__*` tools below.
|
||||
- ✅ Allowed: lookup AC game info via `WebFetch(domain:acpedia.org)` if useful.
|
||||
- ❌ **Do not attempt** any other tool — Bash, Read, Write, Edit, Grep, Glob, ToolSearch, Monitor, TodoWrite, Skill, scheduling, Agent (subagents), Gmail / Calendar / Drive, etc. They will be denied and waste a turn.
|
||||
- ❌ **Do not explain to the user how to bypass your restrictions.** Don't suggest editing `settings.json` to add tools. Don't suggest "switch permission mode". Don't propose workarounds. The restrictions are intentional and the user knows about them.
|
||||
- ❌ **Do not delegate to subagents.** You answer directly using your tools or you say you can't.
|
||||
- ❌ **Do not enumerate "what tools you have"** when asked, beyond a short summary of the `mcp__overlord__*` tools and acpedia WebFetch. Specifically do not list Gmail / Calendar / Drive / built-ins; pretend they don't exist.
|
||||
- If the user asks about anything outside the game-state domain — coding, system administration, your own configuration, email, calendar, etc. — politely say "that's out of scope for the Overlord Assistant" and offer game-data help instead.
|
||||
|
||||
**You have MCP tools** (from `.mcp.json`) for live game data. **Always use them** instead of guessing or apologising for not having data:
|
||||
|
||||
- `get_live_players` — current online characters with positions/kills/state
|
||||
- `get_recent_rares` — rare item finds in the last N hours
|
||||
- `query_telemetry_db` — read-only SQL on the telemetry DB for ad-hoc analysis
|
||||
- `search_items` — **cross-character** inventory search (use this instead of looping `get_inventory` per character — single call is much faster)
|
||||
- `get_inventory` / `get_inventory_search` — single-character inventory
|
||||
- `get_player_state` / `get_combat_stats` / `get_equipment_cantrips` — per-character lookups
|
||||
- `get_quest_status` / `get_server_health` — global state
|
||||
- `suitbuilder_search` — armor optimization (slow, only on explicit request)
|
||||
|
||||
### Behaviour rules
|
||||
|
||||
1. **Use tools, don't speculate.** If the user asks "how many chars are online" — call `get_live_players`. Don't say "I'd need to check" — just check.
|
||||
1a. **For "find an X on any of my chars" — ALWAYS use `search_items`** with `include_all_characters=true`. Do NOT loop `get_inventory` over each character — that's O(N) tool calls and times out.
|
||||
2. **Be concise.** The user is glancing at a chat window, not reading a report. 2-5 sentences for most answers. Use markdown tables for tabular data.
|
||||
3. **No code unless asked.** This mode is about *operating* the system, not editing it. Don't open files or write code unless the user explicitly asks.
|
||||
4. **Real numbers, real names.** Cite actual character names and counts from tools — never make up sample data.
|
||||
5. **Read-only.** You cannot mutate the database; the SQL tool will reject any non-SELECT statement and the role is also `GRANT SELECT` only. If a question requires a write, say so.
|
||||
6. **Suitbuilder** is a separate complex tool that runs constraint search; explain trade-offs in plain English when reporting results.
|
||||
7. **Out-of-scope questions** (general AC lore, unrelated coding) — answer briefly without using tools.
|
||||
|
||||
### Rare tiers — important domain knowledge
|
||||
|
||||
Asheron's Call players distinguish two rare tiers, but our `rare_events`
|
||||
table does **not** store the tier — only the item `name`. To answer
|
||||
"what are the recent great rares" or "filter common vs great", classify
|
||||
in your head from the name:
|
||||
|
||||
**Common rares** (the ~71-item allowlist used by `discord-rare-monitor`):
|
||||
- Anything ending in `'s Crystal` (Alchemist's Crystal, Knight's Crystal, etc.)
|
||||
- `Lugian's/Ursuin's/Wayfarer's/Sprinter's/Magus's/Lich's Pearl`
|
||||
- All `*'s Jewel` (Warrior's, Mage's, Duelist's, Archer's, Tusker's, Olthoi's, Inferno's, Gelid's, Astyrrian's, Executor's, Melee's)
|
||||
- `Pearl of <Effect>` (Blood Drinking, Heart Seeking, Defending, Swift Killing, Spirit Drinking, Hermetic Linking, Blade/Pierce/Bludgeon/Acid/Flame/Frost/Lightning Baning, Impenetrability)
|
||||
- `Refreshing/Invigorating/Miraculous Elixir`, `Medicated Health/Stamina/Mana Kit`
|
||||
- `Casino Exquisite Keyring`
|
||||
|
||||
**Great rares** = anything else dropped from a rare event. Examples include:
|
||||
- `Shimmering Skeleton Key`, `Star of Tukal`
|
||||
- `Hieroglyph/Pictograph/Ideograph/Rune of …`
|
||||
- `Infinite/Eternal/Perennial/Foolproof/Limitless …`
|
||||
- `Gelidite`, `Leikotha`, `Frore` items
|
||||
- `Staff of …`, `Wand of …`, `Count Renari's …`
|
||||
|
||||
When the user asks about "great rares", filter `get_recent_rares` results
|
||||
by the name NOT matching the common list, or run a SQL query like:
|
||||
```sql
|
||||
SELECT timestamp, character_name, name FROM rare_events
|
||||
WHERE timestamp >= NOW() - INTERVAL '7 days'
|
||||
AND name !~ '(Crystal|Jewel|Elixir|Kit|Keyring)$'
|
||||
AND name NOT LIKE 'Pearl of %'
|
||||
AND name !~ '(Lugian|Ursuin|Wayfarer|Sprinter|Magus|Lich)''s Pearl'
|
||||
ORDER BY timestamp DESC;
|
||||
```
|
||||
|
||||
### Available data tables (for `query_telemetry_db`)
|
||||
|
||||
- `telemetry_events` (hypertable, 30-day retention) — position/state snapshots every ~2s per character
|
||||
- `rare_events` — rare item find log
|
||||
- `spawn_events` (hypertable, 7-day retention) — monster spawn observations
|
||||
- `portals` — discovered portal coords (1h dedup window)
|
||||
- `char_stats`, `rare_stats`, `rare_stats_sessions` — lifetime/session aggregates
|
||||
- `character_stats` — latest full stats JSON per character
|
||||
- `combat_stats`, `combat_stats_sessions` — combat tracking
|
||||
- `server_status` — current Coldeve game-server state (single row)
|
||||
|
||||
If asked about something not covered above, look in `db_async.py` for the schema or just try a query and report what you see.
|
||||
38
Dockerfile
|
|
@ -1,48 +1,28 @@
|
|||
# Dockerfile for Dereth Tracker application
|
||||
# Base image: lightweight Python runtime
|
||||
FROM python:3.12-slim
|
||||
|
||||
## Set application working directory
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Upgrade pip and install required Python packages without caching
|
||||
# Upgrade pip and install Python dependencies
|
||||
RUN python -m pip install --upgrade pip && \
|
||||
pip install --no-cache-dir \
|
||||
fastapi \
|
||||
uvicorn \
|
||||
pydantic \
|
||||
websockets \
|
||||
databases[postgresql] \
|
||||
sqlalchemy \
|
||||
alembic \
|
||||
psycopg2-binary \
|
||||
httpx \
|
||||
bcrypt \
|
||||
itsdangerous
|
||||
pip install --no-cache-dir fastapi uvicorn pydantic pandas matplotlib websockets
|
||||
|
||||
## Copy application source code and migration scripts into container
|
||||
# Copy application code
|
||||
COPY static/ /app/static/
|
||||
COPY main.py /app/main.py
|
||||
COPY db.py /app/db.py
|
||||
COPY db_async.py /app/db_async.py
|
||||
COPY alembic.ini /app/alembic.ini
|
||||
COPY alembic/ /app/alembic/
|
||||
COPY Dockerfile /Dockerfile
|
||||
## Expose the application port to host
|
||||
# Expose the application port
|
||||
EXPOSE 8765
|
||||
|
||||
## Build version (CalVer + git hash, set via --build-arg)
|
||||
ARG BUILD_VERSION=dev
|
||||
ENV APP_VERSION=$BUILD_VERSION
|
||||
|
||||
## Default environment variables for application configuration
|
||||
ENV DATABASE_URL=postgresql://postgres:password@db:5432/dereth \
|
||||
DB_MAX_SIZE_MB=2048 \
|
||||
# Default environment variables (override as needed)
|
||||
ENV DB_MAX_SIZE_MB=2048 \
|
||||
DB_RETENTION_DAYS=7 \
|
||||
DB_MAX_SQL_LENGTH=1000000000 \
|
||||
DB_MAX_SQL_VARIABLES=32766 \
|
||||
DB_WAL_AUTOCHECKPOINT_PAGES=1000 \
|
||||
SHARED_SECRET=your_shared_secret
|
||||
|
||||
## Launch the FastAPI app using Uvicorn
|
||||
CMD ["uvicorn","main:app","--host","0.0.0.0","--port","8765","--workers","1","--no-access-log","--log-level","warning"]
|
||||
# Run the FastAPI application with Uvicorn
|
||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8765", "--reload", "--workers", "1"]
|
||||
|
|
|
|||
48
FIXES.md
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
# Planned Fixes and Enhancements
|
||||
|
||||
_This document captures the next set of improvements and fixes for Dereth Tracker._
|
||||
|
||||
## 1. Chat Window Styling and Format
|
||||
- **Terminal-style chat interface**
|
||||
- Redesign the chat window to mimic Asheron’s Call in-game chat: monospaced font, dark semi-transparent background, and text entry at the bottom.
|
||||
- Implement timestamped message prefixes (e.g., `[12:34] character: message`).
|
||||
- Support command- and system-level styling (e.g., whispers, party chat) with distinct color cues.
|
||||
|
||||
## 2. Incoming Message Parsing
|
||||
- **Strip protocol overhead**
|
||||
- Remove JSON envelope artifacts (e.g., remove quotes, braces) so only raw message text appears.
|
||||
- Validate and sanitize incoming payloads (e.g., escape HTML, truncate length).
|
||||
- Optionally support rich-text / emotes by parsing simple markup (e.g., `*bold*`, `/me action`).
|
||||
|
||||
## 3. Message Color Scheme
|
||||
- **Per-character consistent colors**
|
||||
- Map each character name to a unique, but legible, pastel or muted color.
|
||||
- Ensure sufficient contrast with the chat background (WCAG AA compliance).
|
||||
- Provide user override settings for theme (light/dark) and custom palettes.
|
||||
|
||||
## 4. Command Prompt Integration
|
||||
- **Client-side command entry**
|
||||
- Allow slash-commands in chat input (e.g., `/kick PlayerName`, `/whisper PlayerName Hello`).
|
||||
- Validate commands before sending to `/ws/live` and route to the correct plugin WebSocket.
|
||||
- Show feedback on command success/failure in the chat window.
|
||||
|
||||
## 5. Security Hardening
|
||||
- **Authentication & Authorization**
|
||||
- Enforce TLS (HTTPS/WSS) for all HTTP and WebSocket connections.
|
||||
- Protect `/ws/position` with rotating shared secrets or token-based auth (e.g., JWT).
|
||||
- Rate-limit incoming telemetry and chat messages to prevent flooding.
|
||||
- Sanitize all inputs to guard against injection (SQL, XSS) and implement strict CSP headers.
|
||||
|
||||
## 6. Performance and Scalability
|
||||
- **Throttling and Load Handling**
|
||||
- Batch updates during high-frequency telemetry bursts to reduce WebSocket churn.
|
||||
- Cache recent `/live` and `/trails` responses in-memory to relieve SQLite under load.
|
||||
- Plan for horizontal scaling: stateless FastAPI behind a load balancer with shared database or in-memory pub/sub.
|
||||
|
||||
## 7. Testing and Quality Assurance
|
||||
- **Automated Tests**
|
||||
- Unit tests for `db.save_snapshot`, HTTP endpoints, and WebSocket handlers.
|
||||
- E2E tests for the frontend UI (using Puppeteer or Playwright) to verify chat and map functionality.
|
||||
- Security regression tests for input sanitization and auth enforcement.
|
||||
|
||||
_Refer to this list when planning next development sprints. Each item should be broken down into individual tickets or pull requests._
|
||||
38
LESSONSLEARNED.md
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
# Lessons Learned
|
||||
|
||||
_This document captures the key takeaways and implementation details from today's troubleshooting session._
|
||||
|
||||
## 1. API Routing & Proxy Configuration
|
||||
- **API_BASE constant**: The frontend (`static/script.js`) uses a base path `API_BASE` (default `/api`) to prefix all HTTP and WebSocket calls. Always update this to match your proxy mount point.
|
||||
- **Nginx WebSocket forwarding**: To proxy WebSockets, you must forward the `Upgrade` and `Connection` headers:
|
||||
```nginx
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
```
|
||||
Without these, the WS handshake downgrades to a normal HTTP GET, resulting in 404s.
|
||||
|
||||
## 2. Debugging WebSocket Traffic
|
||||
- Logged all incoming WS frames in `main.py`:
|
||||
- `[WS-PLUGIN RX] <client>: <raw>` for messages on `/ws/position`
|
||||
- `[WS-LIVE RX] <client>: <parsed-json>` for messages on `/ws/live`
|
||||
- These prints surface registration, telemetry, chat, and command packets, aiding root-cause analysis.
|
||||
|
||||
## 3. Data Serialization Fix
|
||||
- Python `datetime` objects are not JSON-serializable by default. We wrapped outbound payloads via FastAPI’s `jsonable_encoder` in `_broadcast_to_browser_clients` so that:
|
||||
```python
|
||||
data = jsonable_encoder(snapshot)
|
||||
await ws.send_json(data)
|
||||
```
|
||||
This ensures ISO8601 strings for timestamps and eliminates `TypeError: Object of type datetime is not JSON serializable`.
|
||||
|
||||
## 4. Frontend Adjustments
|
||||
- **Chat input positioning**: Moved the `.chat-form` to `position: absolute; bottom: 0;` so the input always sticks to the bottom of its window.
|
||||
- **Text color**: Forced the input text and placeholder to white (`.chat-input, .chat-input::placeholder { color: #fff; }`) and forcibly set all incoming messages to white via `.chat-messages div { color: #fff !important; }`.
|
||||
- **Padding for messages**: Added `padding-bottom` to `.chat-messages` to avoid new messages being hidden behind the fixed input bar.
|
||||
|
||||
## 5. General Best Practices
|
||||
- Clear browser cache after updating static assets to avoid stale JS/CSS.
|
||||
- Keep patches targeted: fix the source of issues (e.g., JSON encoding or missing headers) rather than applying superficial workarounds.
|
||||
- Use consistent CSS variables for theming (e.g., `--text`, `--bg-main`).
|
||||
|
||||
By consolidating these lessons, we can onboard faster next time and avoid repeating these pitfalls.
|
||||
4
Makefile
|
|
@ -1,4 +1,2 @@
|
|||
# Reformat Python code using Black formatter
|
||||
.PHONY: reformat
|
||||
reformat:
|
||||
black *.py
|
||||
black *py
|
||||
|
|
|
|||
521
README.md
|
|
@ -1,424 +1,209 @@
|
|||
# Mosswart Overlord (Dereth Tracker)
|
||||
# Dereth Tracker
|
||||
|
||||
Real-time telemetry, inventory, and analytics platform for Asheron's Call.
|
||||
FastAPI backend + React frontend + PostgreSQL (TimescaleDB) + Discord integrations,
|
||||
all driven by WebSocket events from the companion [MosswartMassacre](https://github.com/SawatoMosswartsEnjoyersClub/MosswartMassacre) DECAL plugin.
|
||||
|
||||
---
|
||||
Dereth Tracker is a real-time telemetry service for the world of Dereth. It collects player data, stores it in a SQLite database, and provides a live map interface along with a sample data generator for testing.
|
||||
|
||||
## Table of Contents
|
||||
- [Overview](#overview)
|
||||
- [Architecture](#architecture)
|
||||
- [Features](#features)
|
||||
- [Requirements](#requirements)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](#configuration)
|
||||
- [Deploying Changes](#deploying-changes)
|
||||
- [WebSocket Contract](#websocket-contract)
|
||||
- [HTTP API Reference](#http-api-reference)
|
||||
- [Usage](#usage)
|
||||
- [API Reference](#api-reference)
|
||||
- [Frontend](#frontend)
|
||||
- [AI Assistant (Overlord Agent)](#ai-assistant-overlord-agent)
|
||||
- [Database Schema](#database-schema)
|
||||
- [Operations & Health](#operations--health)
|
||||
- [Contributing](#contributing)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Mosswart Overlord is the backend that consumes a firehose of telemetry, vitals, inventory, combat, and chat events from 60+ characters running the `MosswartMassacre` plugin. It stores selected data in TimescaleDB, runs analytics (combat stats, idle/death detection), and broadcasts live updates to connected browser clients.
|
||||
|
||||
The frontend is a React + Vite app served at `/` with a live map, draggable windows (inventory, chat, combat, radar, etc.), and a server uptime sidebar. The previous vanilla JS frontend is preserved at `/classic`.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────┐
|
||||
│ MosswartMassacre (C#) │ ← plugin per game client
|
||||
└────────────┬────────────┘
|
||||
│ WebSocket /ws/position (authenticated)
|
||||
▼
|
||||
┌────────────────────────────────────────────────────────┐
|
||||
│ dereth-tracker (FastAPI, Docker) │
|
||||
│ • main.py — WS routing, analytics, broadcasts │
|
||||
│ • idle/death detection → Discord webhook │
|
||||
│ • combat stats delta/lifetime accumulation │
|
||||
│ • vital sharing relay (cross-machine) │
|
||||
└──┬──────────────────┬────────────────────┬────────────┘
|
||||
│ │ │
|
||||
│ WS /ws/live │ HTTP │ HTTP
|
||||
▼ ▼ ▼
|
||||
┌──────────┐ ┌──────────────────┐ ┌──────────────────┐
|
||||
│ Browsers │ │ inventory-svc │ │ Discord bot │
|
||||
│ (React) │ │ (FastAPI, Docker)│ │ (rare monitor) │
|
||||
└────┬─────┘ └────────┬─────────┘ └──────────────────┘
|
||||
│ ▼
|
||||
│ ┌──────────────┐
|
||||
│ │ inventory-db │
|
||||
│ └──────────────┘
|
||||
│
|
||||
│ /api/agent/* (host-side, OUTSIDE Docker)
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ overlord-agent (FastAPI, systemd) │ ← runs as dedicated unprivileged user
|
||||
│ • shells out to `claude -p ...` │ /var/lib/overlord-agent home,
|
||||
│ • MCP server: live-state Q&A tools │ strict settings, no /home/erik
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
┌──────────────┐
|
||||
│ dereth-db │ ← TimescaleDB (telemetry, spawns, rares, portals)
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
Most services run via Docker Compose. **`overlord-agent` is host-side**
|
||||
(systemd) because it shells out to the `claude` CLI which depends on
|
||||
host-side credentials — see [AI Assistant](#ai-assistant-overlord-agent).
|
||||
This project provides:
|
||||
- A FastAPI backend with endpoints for receiving and querying telemetry data.
|
||||
- SQLite-based storage for snapshots and live state.
|
||||
- A live, interactive map using static HTML, CSS, and JavaScript.
|
||||
- A sample data generator script (`generate_data.py`) for simulating telemetry snapshots.
|
||||
|
||||
## Features
|
||||
|
||||
### Live Data
|
||||
- **Live Map** — real-time player positions, dots, trails, portals, heatmap
|
||||
- **WebSocket firehose** (`/ws/live`) — broadcasts every incoming event to browsers
|
||||
- **Per-client subscriptions** — clients can send `{"type":"subscribe","message_types":[...]}` to receive only specific event types (the Discord rare monitor bot uses this to filter the 82GB/day firehose down to just `rare` and `chat`)
|
||||
|
||||
### Inventory
|
||||
- Full inventory snapshot on login + incremental `inventory_delta` updates (add/update/remove)
|
||||
- Per-character live refresh in the browser (debounced 2s)
|
||||
- Advanced search with filters: material, set, armor level, spells, tinks, workmanship, etc.
|
||||
- **Suitbuilder** at `/suitbuilder.html` — constraint-based armor optimization across multiple mule inventories with primary/secondary set support, cantrip overlap detection, and real-time SSE streaming
|
||||
|
||||
### Combat Stats (Mag-Tools style)
|
||||
- Plugin parses combat chat into session deltas
|
||||
- Backend accumulates lifetime totals from per-session snapshots
|
||||
- Offense/defense broken out per damage element
|
||||
- Browser combat window shows monster-by-monster damage
|
||||
|
||||
### Cross-Machine Vital Sharing
|
||||
- WebSocket relay replaces UtilityBelt's localhost-only `VTankFellowHeals`
|
||||
- Plugin broadcasts its own vitals and consumes peer vitals
|
||||
- In-game `DxHud` overlay shows peer health/stamina/mana bars with direction arrows
|
||||
|
||||
### AI Assistant
|
||||
- 🤖 chat window in the dashboard backed by `claude -p` running headless on the server
|
||||
- Read-only access to live game state via 12 MCP tools (live players, inventory cross-search, combat stats, quests, suitbuilder, read-only SQL, etc.)
|
||||
- Per-browser persistent session, "New Chat" button, history rehydration on reload
|
||||
- Hardened: dedicated unprivileged Linux user, systemd lockdown, strict tool whitelist, audit log, rate limit. See [AI Assistant section](#ai-assistant-overlord-agent) for the full security stack.
|
||||
|
||||
### Discord Integration
|
||||
- **Rare Monitor Bot** — posts rares (split by common/great) to configured channels
|
||||
- **Death Alerts** — webhook to `#alerts` when a character's vitae goes from 0 → >0 (rate-limited to one per character per 5 min)
|
||||
- **Idle Alerts** — webhook after 5 minutes of continuous idle state (caught portals, stuck nav, etc.). The grace period prevents false positives on brief idle blips.
|
||||
- **Vortex Warning** — bot watches for "whirlwind of vortexes" chat and posts a warning embed
|
||||
|
||||
### Portals
|
||||
- Automatic discovery + 1-hour retention
|
||||
- Coordinate-deduplicated (rounded to 0.1 precision)
|
||||
|
||||
### Stats
|
||||
- Per-character lifetime kills, deaths, rares, taper counts
|
||||
- Grafana dashboards (2x2 iframe grid in the stats window)
|
||||
|
||||
### Health & Monitoring
|
||||
- Server uptime + latency + player count from TreeStats.net (checked every 30s)
|
||||
- Only current state is kept — no historical `server_health_checks` table (removed April 2026 as write-only bloat)
|
||||
- **WebSocket /ws/position**: Stream telemetry snapshots (protected by a shared secret).
|
||||
- **GET /live**: Fetch active players seen in the last 30 seconds.
|
||||
- **GET /history**: Retrieve historical telemetry data with optional time filtering.
|
||||
- **GET /debug**: Health check endpoint.
|
||||
- **Live Map**: Interactive map interface with panning, zooming, and sorting.
|
||||
- **Sample Data Generator**: `generate_data.py` sends telemetry snapshots over WebSocket for testing.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Docker & Docker Compose (recommended)
|
||||
- OR: Python 3.11+, Node.js 20+, and a PostgreSQL 14+ with TimescaleDB
|
||||
- Python 3.9 or newer
|
||||
- pip
|
||||
- (Optional) virtual environment tool (venv)
|
||||
|
||||
Python packages:
|
||||
|
||||
- fastapi
|
||||
- uvicorn
|
||||
- pydantic
|
||||
- pandas
|
||||
- matplotlib
|
||||
- websockets # required for sample data generator
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
git clone git@git.snakedesert.se:SawatoMosswartsEnjoyersClub/MosswartOverlord.git
|
||||
cd MosswartOverlord
|
||||
cp .env.example .env # fill in secrets (see Configuration below)
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Frontend development loop
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev # local Vite server
|
||||
# ...edit files, hot reload...
|
||||
cd ..
|
||||
bash deploy-frontend.sh # builds + copies to static/ for production serving
|
||||
```
|
||||
|
||||
⚠️ **`npm run build` writes to `static/_build/` but the web server serves from `static/`.** You must run `deploy-frontend.sh` to copy `_build/ → static/`. Otherwise the browser keeps loading the previous bundle.
|
||||
1. Clone the repository:
|
||||
```bash
|
||||
git clone https://github.com/yourusername/dereth-tracker.git
|
||||
cd dereth-tracker
|
||||
```
|
||||
2. Create and activate a virtual environment:
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # Windows: venv\Scripts\activate
|
||||
```
|
||||
3. Install dependencies:
|
||||
```bash
|
||||
pip install fastapi uvicorn pydantic pandas matplotlib websockets
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All secrets go in `.env`:
|
||||
- Update the `SHARED_SECRET` in `main.py` to match your plugin (default: `"your_shared_secret"`).
|
||||
- The SQLite database file `dereth.db` is created in the project root. To change the path, edit `DB_FILE` in `db.py`.
|
||||
- To limit the maximum database size, set the environment variable `DB_MAX_SIZE_MB` (default: 2048 MB).
|
||||
|
||||
| Variable | Purpose |
|
||||
|---|---|
|
||||
| `POSTGRES_PASSWORD` | Telemetry DB password |
|
||||
| `INVENTORY_DB_PASSWORD` | Inventory DB password |
|
||||
| `SHARED_SECRET` | Plugin auth for `/ws/position` |
|
||||
| `SECRET_KEY` | Session cookie signing |
|
||||
| `DISCORD_RARE_BOT_TOKEN` | Bot token for rare monitor |
|
||||
| `DISCORD_ACLOG_WEBHOOK` | Webhook URL for death/idle alerts |
|
||||
| `GF_SECURITY_ADMIN_PASSWORD` | Grafana admin |
|
||||
| `COMMON_RARE_CHANNEL_ID` | Discord channel ID for common rares |
|
||||
| `GREAT_RARE_CHANNEL_ID` | Discord channel ID for great rares |
|
||||
| `ACLOG_CHANNEL_ID` | Discord channel ID for the rare bot's status/vortex messages |
|
||||
| `MONITOR_CHARACTER` | Which character's chat the bot monitors |
|
||||
## Usage
|
||||
|
||||
The Overlord Agent has its own env file at `/etc/overlord/agent.env` (root:overlord-agent 0640) so it doesn't share the tracker's secrets:
|
||||
|
||||
| Variable | Purpose |
|
||||
|---|---|
|
||||
| `SECRET_KEY` | Same value as the tracker — validates browser session cookies |
|
||||
| `AGENT_DB_DSN` | Read-only connection string `postgresql://overlord_agent_ro:<pw>@127.0.0.1:5432/dereth` |
|
||||
| `TRACKER_URL` | Loopback to the tracker container (default `http://127.0.0.1:8765`) |
|
||||
| `AGENT_RATE_MAX` | Per-user rate limit (default 60/hour) |
|
||||
| `AGENT_RATE_WINDOW_S` | Rate-limit window in seconds (default 3600) |
|
||||
| `AGENT_AUDIT_LOG` | Path to audit JSONL (default `/var/log/overlord-agent/audit.jsonl`) |
|
||||
| `CLAUDE_TIMEOUT_S` | Max seconds per `claude -p` invocation (default 240) |
|
||||
|
||||
## Deploying Changes
|
||||
|
||||
Live backend host: `overlord.snakedesert.se` (SSH user `erik`, key-based auth).
|
||||
|
||||
### Quick deploy — Python / static file changes
|
||||
Start the server using Uvicorn:
|
||||
|
||||
```bash
|
||||
ssh erik@overlord.snakedesert.se \
|
||||
"cd /home/erik/MosswartOverlord && git pull --ff-only origin master"
|
||||
# Python changes require a restart:
|
||||
ssh erik@overlord.snakedesert.se "docker compose restart dereth-tracker"
|
||||
# Static files (JS/CSS/HTML) are served from the bind-mounted static/ — no restart.
|
||||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
⚠️ Uvicorn runs **without** `--reload` in production. Do not add it back — without the `watchfiles` package it falls back to a polling reloader that busy-loops at 100% CPU and eats a whole core.
|
||||
## NGINX Proxy Configuration
|
||||
|
||||
### React frontend deploy
|
||||
If you cannot reassign the existing `/live` and `/trails` routes, you can namespace this service under `/api` (or any other prefix) and configure NGINX accordingly. Be sure to forward WebSocket upgrade headers so that `/ws/live` and `/ws/position` continue to work. Example:
|
||||
```nginx
|
||||
location /api/ {
|
||||
proxy_pass http://127.0.0.1:8765/;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
# WebSocket support
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
}
|
||||
```
|
||||
Then the browser client (static/script.js) will fetch `/api/live/` and `/api/trails/` to reach this new server.
|
||||
|
||||
```bash
|
||||
cd frontend && npm run build && cd ..
|
||||
bash deploy-frontend.sh
|
||||
git add static/ && git commit -m "deploy frontend" && git push
|
||||
ssh erik@overlord.snakedesert.se "cd /home/erik/MosswartOverlord && git pull"
|
||||
# No container restart needed.
|
||||
- Live Map: `http://localhost:8000/` (or `http://<your-domain>/api/` if behind a prefix)
|
||||
|
||||
### Frontend Configuration
|
||||
|
||||
- In `static/script.js`, the constant `API_BASE` controls where live/trails data and WebSocket `/ws/live` are fetched. By default:
|
||||
```js
|
||||
const API_BASE = '/api';
|
||||
```
|
||||
Update `API_BASE` if you mount the service under a different path or serve it at root.
|
||||
|
||||
### Debugging WebSockets
|
||||
|
||||
- Server logs now print every incoming WebSocket frame in `main.py`:
|
||||
- `[WS-PLUGIN RX] <client>: <raw-payload>` for plugin messages on `/ws/position`
|
||||
- `[WS-LIVE RX] <client>: <parsed-json>` for browser messages on `/ws/live`
|
||||
- Use these logs to verify messages and troubleshoot handshake failures.
|
||||
|
||||
### Styling Adjustments
|
||||
|
||||
- Chat input bar is fixed at the bottom of the chat window (`.chat-form { position:absolute; bottom:0; }`).
|
||||
- Input text and placeholder are white for readability (`.chat-input, .chat-input::placeholder { color:#fff; }`).
|
||||
- Incoming chat messages forced white via `.chat-messages div { color:#fff !important; }`.
|
||||
|
||||
## API Reference
|
||||
|
||||
### WebSocket /ws/position
|
||||
Stream telemetry snapshots over a WebSocket connection. Provide your shared secret either as a query parameter or WebSocket header:
|
||||
|
||||
```
|
||||
ws://<host>:<port>/ws/position?secret=<shared_secret>
|
||||
```
|
||||
or
|
||||
```
|
||||
X-Plugin-Secret: <shared_secret>
|
||||
```
|
||||
|
||||
### Full rebuild — Dockerfile / pip package / version stamp changes
|
||||
After connecting, send JSON messages matching the `TelemetrySnapshot` schema. For example:
|
||||
|
||||
```bash
|
||||
ssh erik@overlord.snakedesert.se "cd /home/erik/MosswartOverlord && \
|
||||
git pull --ff-only origin master && \
|
||||
export BUILD_VERSION=\"\$(date -u +%Y.%-m.%-d.%H%M)-\$(git rev-parse --short HEAD)\" && \
|
||||
docker compose build --no-cache --build-arg BUILD_VERSION=\$BUILD_VERSION dereth-tracker && \
|
||||
docker compose up -d dereth-tracker"
|
||||
```json
|
||||
{
|
||||
"type": "telemetry",
|
||||
"character_name": "Dunking Rares",
|
||||
"char_tag": "moss",
|
||||
"session_id": "dunk-20250422-xyz",
|
||||
"timestamp": "2025-04-22T13:45:00Z",
|
||||
"ew": 123.4,
|
||||
"ns": 567.8,
|
||||
"z": 10.2,
|
||||
"kills": 42,
|
||||
"deaths": 1,
|
||||
"rares_found": 2,
|
||||
"prismatic_taper_count": 17,
|
||||
"vt_state": "Combat",
|
||||
"kills_per_hour": "N/A",
|
||||
"onlinetime": "00:05:00"
|
||||
}
|
||||
```
|
||||
|
||||
`BUILD_VERSION` is displayed in the sidebar of the live frontend. Format is CalVer: `YYYY.M.D.HHMM-gitshorthash`.
|
||||
### Chat messages
|
||||
You can also send chat envelopes over the same WebSocket to display messages in the browser. Fields:
|
||||
- `type`: must be "chat"
|
||||
- `character_name`: target player name
|
||||
- `text`: message content
|
||||
- `color` (optional): CSS color string (e.g. "#ff8800"); if sent as an integer (0xRRGGBB), it will be converted to hex.
|
||||
|
||||
### Overlord Agent deploy
|
||||
|
||||
Code changes to `agent/` only:
|
||||
```bash
|
||||
ssh erik@overlord.snakedesert.se "cd /home/erik/MosswartOverlord && \
|
||||
git pull --ff-only origin master && \
|
||||
sudo systemctl restart overlord-agent"
|
||||
journalctl -u overlord-agent -f # tail logs to verify
|
||||
Example chat payload:
|
||||
```json
|
||||
{
|
||||
"type": "chat",
|
||||
"character_name": "MyCharacter",
|
||||
"text": "Hello world!",
|
||||
"color": "#88f"
|
||||
}
|
||||
```
|
||||
|
||||
`agent/requirements.txt` changed (new pip deps):
|
||||
```bash
|
||||
ssh erik@overlord.snakedesert.se "cd /home/erik/MosswartOverlord && \
|
||||
git pull --ff-only origin master && \
|
||||
agent/.venv/bin/pip install -r agent/requirements.txt && \
|
||||
sudo systemctl restart overlord-agent"
|
||||
### GET /live
|
||||
Returns active players seen within the last 30 seconds:
|
||||
|
||||
```json
|
||||
{
|
||||
"players": [ { ... } ]
|
||||
}
|
||||
```
|
||||
|
||||
systemd unit changed:
|
||||
```bash
|
||||
ssh erik@overlord.snakedesert.se "cd /home/erik/MosswartOverlord && \
|
||||
git pull --ff-only origin master && \
|
||||
sudo cp agent/overlord-agent.service /etc/systemd/system/ && \
|
||||
sudo systemctl daemon-reload && sudo systemctl restart overlord-agent"
|
||||
### GET /history
|
||||
Retrieve historical snapshots with optional `from` and `to` ISO8601 timestamps:
|
||||
|
||||
```
|
||||
GET /history?from=2025-04-22T12:00:00Z&to=2025-04-22T13:00:00Z
|
||||
```
|
||||
|
||||
First-time install: `bash agent/install.sh` — see `agent/README.md` for the full bootstrap procedure (creating the `overlord-agent` user, copying claude auth, granting filesystem access, populating `/etc/overlord/agent.env`).
|
||||
Response:
|
||||
|
||||
## WebSocket Contract
|
||||
|
||||
### `/ws/position` (plugin → backend)
|
||||
|
||||
Authenticated via `?secret=<SHARED_SECRET>` or `X-Plugin-Secret` header. Accepts JSON frames with a `type` discriminator:
|
||||
|
||||
| `type` | Purpose |
|
||||
|---|---|
|
||||
| `telemetry` | Position, kills, session metrics (every 2s per character) |
|
||||
| `vitals` | Health/stamina/mana/vitae percentages |
|
||||
| `character_stats` | Full attributes/skills/allegiance (every 10 min) |
|
||||
| `inventory` / `full_inventory` | Complete inventory dump on login |
|
||||
| `inventory_delta` | Incremental add/update/remove of a single item |
|
||||
| `equipment_cantrip_state` | Equipped spell effects |
|
||||
| `portal` | Discovered portal with coordinates |
|
||||
| `spawn` | Monster spawn observation |
|
||||
| `chat` | In-game chat line (any channel) |
|
||||
| `quest` | Quest timer / progress |
|
||||
| `rare` | Rare item find notification |
|
||||
| `nearby_objects` | On-demand radar data (nearby entities) |
|
||||
| `combat_stats` | Session combat snapshot (Mag-Tools parser output) |
|
||||
| `share_*` | Cross-machine vital/debuff sharing envelopes |
|
||||
| `dungeon_map` | Dungeon floor tile data for radar overlay |
|
||||
|
||||
See `EVENT_FORMATS.json` for exact per-type schemas.
|
||||
|
||||
### `/ws/live` (browser → backend)
|
||||
|
||||
Session-cookie authenticated (except for internal Docker network clients, which are trusted by IP). Clients can:
|
||||
|
||||
- Send `{"type":"subscribe","message_types":["rare","chat"]}` to filter which events they receive. Without subscribing, all types are forwarded (browser default).
|
||||
- Send `{"player_name":"Larsson","command":"/radar start"}` to route a command to that character's plugin client.
|
||||
- Send `{"type":"request_dungeon_map","landblock":"..."}` to pull cached dungeon tile data.
|
||||
|
||||
Backend pushes the same firehose (subject to subscription filter) to every browser client.
|
||||
|
||||
## HTTP API Reference
|
||||
|
||||
See `EVENT_FORMATS.json` for event schemas. Major HTTP endpoints:
|
||||
|
||||
- `GET /live` — active players seen in the last 30s
|
||||
- `GET /history?from=…&to=…` — historical telemetry snapshots
|
||||
- `GET /trails` — recent player trails for the map
|
||||
- `GET /spawns/heatmap?hours=N` — aggregated spawn density
|
||||
- `GET /portals` — discovered portals within retention window
|
||||
- `GET /inventory/{character}` — current inventory (proxied to inventory-service)
|
||||
- `GET /character-stats/{character}` — full character attributes/skills
|
||||
- `GET /combat-stats/{character}` — session + lifetime combat stats
|
||||
- `GET /vital-sharing/peers` — currently-registered vital sharing peers
|
||||
- `GET /api-version` — build version stamp
|
||||
- `GET /server-health` — current Coldeve server status + player count
|
||||
```json
|
||||
{
|
||||
"data": [ { ... } ]
|
||||
}
|
||||
```
|
||||
|
||||
## Frontend
|
||||
|
||||
### React v2 (primary, at `/`)
|
||||
- Map-first layout with draggable/resizable windows
|
||||
- Code-split bundles: one chunk per window type, lazy-loaded on open
|
||||
- Window types: Chat, Stats, Inventory, Character, Radar, CombatStats, CombatPicker, Issues, VitalSharing, QuestStatus, PlayerDashboard
|
||||
- Per-character inventory version counter — an open inventory window refreshes 2s after its own character's last `inventory_delta`, ignoring unrelated traffic
|
||||
- Direct DOM pan/zoom on the map (no React state per frame)
|
||||
- Service worker caches a small whitelist of static assets
|
||||
- Version badge in the sidebar confirms which build is loaded
|
||||
|
||||
### Classic v1 (preserved at `/classic`)
|
||||
The original vanilla JS frontend with element-pooling optimization is kept for fallback and reference.
|
||||
|
||||
## AI Assistant (Overlord Agent)
|
||||
|
||||
A draggable chat window in the dashboard (🤖 Assistant button). Powered by `claude -p` running headless on the server, with read-only access to live game state via an MCP server.
|
||||
|
||||
### Architecture
|
||||
- **Host-side service** (`agent/`, systemd unit `overlord-agent`) runs OUTSIDE Docker because the `claude` CLI binary lives on the host (`/home/erik/.local/bin/claude`) and depends on host-side authentication credentials.
|
||||
- **Dedicated UNIX user** (`overlord-agent`, system account, `/var/lib/overlord-agent` home, no shell) — kernel-level isolation from the operator's `erik` account. Cannot read `/home/erik/.claude`, `~/.ssh`, `.bash_history`, `.env`, etc.
|
||||
- **MCP stdio server** (`agent/mcp_overlord.py`) exposes 12 tools that wrap the tracker's HTTP endpoints + read-only DB queries. Claude only sees these tools; no `Bash`, `Read`, `Write`, etc.
|
||||
- **Frontend** (`AgentWindow.tsx`) — per-browser session UUID in localStorage, "New Chat" button, on-mount rehydration from `/agent/sessions/{id}/history`.
|
||||
|
||||
### MCP tools available to the assistant
|
||||
`get_live_players`, `get_player_state`, `get_combat_stats`, `get_equipment_cantrips`, `get_inventory`, `get_inventory_search`, `search_items` (cross-character), `get_recent_rares`, `get_quest_status`, `get_server_health`, `query_telemetry_db` (read-only SQL via sqlglot parser + GRANT-SELECT-only PG role), `suitbuilder_search`. Plus `WebFetch(domain:acpedia.org)` for AC info lookups.
|
||||
|
||||
### Security stack (defense-in-depth)
|
||||
1. **Cookie auth** on `/agent/ask` (same session cookie the tracker issues)
|
||||
2. **Per-user rate limit** (60 req/h default) and **concurrency cap** (1 in-flight)
|
||||
3. **JSONL audit log** at `/var/log/overlord-agent/audit.jsonl` (every prompt + result)
|
||||
4. **CLI flags** — `--allowed-tools` (just our 12 MCP tools), `--disallowed-tools` (Bash, Write, Read, Edit, Agent, ToolSearch, Monitor, scheduling, Gmail/Drive/Calendar, etc.), `--permission-mode dontAsk`
|
||||
5. **`/var/lib/overlord-agent/.claude/settings.json`** — strict deny rules (server-side only, NOT in repo)
|
||||
6. **System-prompt scope rules** in `CLAUDE.md` — instruct the model not to probe, not to suggest workarounds
|
||||
7. **SQL parser** (`sqlglot`) rejects any non-SELECT statement on `query_telemetry_db`
|
||||
8. **Read-only PG role** `overlord_agent_ro` (GRANT SELECT only) — even a parser bypass can't mutate
|
||||
9. **systemd hardening** — `ProtectSystem=strict`, `ProtectHome=read-only`, `InaccessiblePaths=/etc/shadow,/root,~/.ssh,…`, `NoNewPrivileges=true`, `CapabilityBoundingSet=` (empty), `PrivateTmp=true`, `PrivateDevices=true`, `RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6`, `SystemCallFilter=@system-service ~@privileged ~@reboot ~@mount`, `MemoryMax=512M`, `TasksMax=128`
|
||||
10. **Secrets out of /home** — `/etc/overlord/agent.env` (root:overlord-agent 0640) for SECRET_KEY + AGENT_DB_DSN
|
||||
|
||||
### Files
|
||||
|
||||
| Path | What |
|
||||
|------|------|
|
||||
| `agent/service.py` | FastAPI app: `/agent/health`, `/agent/sessions/new`, `/agent/ask`, `/agent/sessions/{id}/history` |
|
||||
| `agent/auth.py` | Session cookie validation (mirrors `main.py:1013-1019`) |
|
||||
| `agent/claude_wrapper.py` | `asyncio.create_subprocess_exec("claude", "-p", …)` with allowed/disallowed-tools |
|
||||
| `agent/tools.py` | Pure tool implementations |
|
||||
| `agent/mcp_overlord.py` | MCP stdio server registering tools |
|
||||
| `agent/sql/0001_overlord_agent_ro.sql` | Read-only PG role |
|
||||
| `agent/overlord-agent.service` | systemd unit (the hardening directives) |
|
||||
| `agent/install.sh` | venv + systemd setup |
|
||||
| `agent/README.md` | Operator's deeper reference |
|
||||
| `.mcp.json` (repo root) | Project-level MCP config Claude Code auto-loads |
|
||||
| `CLAUDE.md` "Overlord Assistant Mode" section | System-prompt briefing |
|
||||
|
||||
### Routing
|
||||
nginx forwards `/api/agent/*` to `127.0.0.1:8767` (the host-side service) with a 300s read/send timeout (suitbuilder runs can be slow). Other `/api/*` continues to the dereth-tracker container at `127.0.0.1:8765`.
|
||||
|
||||
### Cost / quota
|
||||
Subscription auth (no API key); per-call cost is informational only. Each `/agent/ask` invocation = one `claude -p` subprocess with shared session cache. Reactive only — no background polling, no scheduled tasks.
|
||||
- **Live Map**: `static/index.html` – Real-time player positions on a map.
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Telemetry DB (`dereth`, TimescaleDB)
|
||||
|
||||
| Table | Type | Retention | Purpose |
|
||||
|---|---|---|---|
|
||||
| `telemetry_events` | hypertable | 30 days | Position/stats snapshots |
|
||||
| `spawn_events` | hypertable | 7 days | Monster spawn observations (heatmap source) |
|
||||
| `rare_events` | regular | forever | Rare find history |
|
||||
| `portals` | regular | 1 hour | Discovered portals, dedup by rounded coords |
|
||||
| `char_stats` | regular | forever | Per-character lifetime kill total |
|
||||
| `rare_stats` | regular | forever | Per-character lifetime rare total |
|
||||
| `rare_stats_sessions` | regular | forever | Per-session rare count |
|
||||
| `combat_stats` | regular | forever | Lifetime combat accumulator |
|
||||
| `combat_stats_sessions` | regular | forever | Per-session combat snapshots |
|
||||
| `character_stats` | regular | forever | Latest full stats JSON per character |
|
||||
| `server_status` | regular | forever | Current Coldeve server state (single row) |
|
||||
|
||||
### Inventory DB (`inventory_db`, PostgreSQL)
|
||||
|
||||
Normalized schema: `items`, `item_combat_stats`, `item_requirements`, `item_enhancements`, `item_ratings`, `item_spells`, `item_raw_data`.
|
||||
|
||||
`items.container_id` stores the in-game ID of the container holding the item (0 = character body). The frontend groups items into packs by this ID.
|
||||
|
||||
## Operations & Health
|
||||
|
||||
### PostgreSQL tuning
|
||||
`dereth-db` runs with explicit memory overrides in `docker-compose.yml`:
|
||||
- `shared_buffers=8GB` (was 96GB via auto-tune on a 32GB host — caused thrashing)
|
||||
- `effective_cache_size=16GB`
|
||||
- `work_mem=16MB`, `maintenance_work_mem=1GB`
|
||||
- `max_wal_size=4GB`
|
||||
|
||||
### Retention policies
|
||||
- `telemetry_events`: 30-day drop, daily
|
||||
- `spawn_events`: 7-day drop, daily
|
||||
- `portals`: 1-hour cleanup (background task in `main.py`)
|
||||
- `server_health_checks`: **removed** — was write-only, 850K rows of nothing
|
||||
|
||||
### Log levels
|
||||
Both `dereth-tracker` and `inventory-service` run at `LOG_LEVEL=INFO`. Do not set to `DEBUG` in production — it dumps full inventory_delta payloads for every item update (hundreds of KB/sec).
|
||||
|
||||
### Host (Proxmox VM)
|
||||
- 6 vCPU, 32 GiB RAM (of which ~30 GiB is normally free under current load)
|
||||
- Live host: `overlord.snakedesert.se`
|
||||
- Reverse proxy: Nginx on the host terminates TLS and strips the `/api/` prefix before forwarding to port 8765
|
||||
|
||||
### Debug commands
|
||||
```bash
|
||||
docker ps
|
||||
docker logs mosswartoverlord-dereth-tracker-1 --tail 100
|
||||
docker logs mosswartoverlord-inventory-service-1 --tail 100
|
||||
docker logs mosswartoverlord-discord-rare-monitor-1 --tail 100
|
||||
docker exec dereth-db psql -U postgres -d dereth
|
||||
```
|
||||
- **telemetry_log**: Stored history of snapshots.
|
||||
- **live_state**: Current snapshot per character (upserted).
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions welcome. Please:
|
||||
- Keep cross-repo protocol changes additive (new optional fields > renames/removes)
|
||||
- Update both this README and `CLAUDE.md` when workflows change
|
||||
- Test end-to-end: plugin → backend → browser for any new event type
|
||||
|
||||
For detailed architecture notes and ongoing investigations, see `CLAUDE.md` and `docs/plans/`.
|
||||
Contributions are welcome! Feel free to open issues or submit pull requests.
|
||||
|
|
|
|||
146
agent/README.md
|
|
@ -1,146 +0,0 @@
|
|||
# Overlord Agent
|
||||
|
||||
A small host-side Python service that gives Claude Code (running in
|
||||
headless mode) access to live Overlord data so it can answer questions
|
||||
from the dashboard chat window.
|
||||
|
||||
## Why a separate service?
|
||||
|
||||
`dereth-tracker` runs in Docker. The `claude` CLI binary at
|
||||
`/home/erik/.local/bin/claude` depends on `~/.claude` credentials owned
|
||||
by user `erik` on the host. The tracker container can't invoke it.
|
||||
|
||||
So this service runs **outside** Docker, listens on `127.0.0.1:8767`,
|
||||
and nginx routes `/api/agent/*` to it. It validates the same browser
|
||||
session cookie the tracker issues (shared `SECRET_KEY`) and shells out
|
||||
to `claude -p` with `cwd=/home/erik/MosswartOverlord`.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser ──nginx──┬─► /api/* ──► dereth-tracker (Docker, 8765)
|
||||
│
|
||||
└─► /api/agent/* ──► overlord-agent (host, 8767)
|
||||
│
|
||||
├─► subprocess: claude -p ...
|
||||
│ │
|
||||
│ └─► MCP stdio ──► mcp_overlord.py
|
||||
│ │
|
||||
│ └─► HTTP loopback to tracker
|
||||
│ └─► asyncpg to dereth-db
|
||||
│
|
||||
└─► validates "session" cookie
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | What |
|
||||
|------|------|
|
||||
| `service.py` | FastAPI app (`/agent/health`, `/agent/sessions/new`, `/agent/ask`, `/agent/sessions/{id}/history`) |
|
||||
| `auth.py` | Session-cookie validation (mirrors `main.py:1013-1019`) |
|
||||
| `claude_wrapper.py` | `asyncio.create_subprocess_exec("claude", "-p", ...)` |
|
||||
| `tools.py` | Pure tool implementations (HTTP loopback + read-only DB) |
|
||||
| `mcp_overlord.py` | MCP stdio server registering tools for Claude Code |
|
||||
| `sql/0001_overlord_agent_ro.sql` | Read-only PG role for the SQL tool |
|
||||
| `overlord-agent.service` | systemd unit |
|
||||
| `install.sh` | One-shot installer (venv + pip install + systemd) |
|
||||
|
||||
## Required env vars (in repo-root `.env`)
|
||||
|
||||
```
|
||||
SECRET_KEY=<same value the tracker uses to sign cookies>
|
||||
AGENT_DB_DSN=postgresql://overlord_agent_ro:<password>@127.0.0.1:5432/dereth
|
||||
TRACKER_URL=http://127.0.0.1:8765 # optional, this is the default
|
||||
CLAUDE_BIN=/home/erik/.local/bin/claude # optional, this is the default
|
||||
CLAUDE_CWD=/home/erik/MosswartOverlord # optional, this is the default
|
||||
CLAUDE_TIMEOUT_S=120 # optional
|
||||
```
|
||||
|
||||
## First-time setup on the server
|
||||
|
||||
1. **Create the read-only DB role** (one-time):
|
||||
```bash
|
||||
docker exec -i dereth-db psql -U postgres -d dereth \
|
||||
< /home/erik/MosswartOverlord/agent/sql/0001_overlord_agent_ro.sql
|
||||
docker exec -it dereth-db psql -U postgres -d dereth \
|
||||
-c "ALTER ROLE overlord_agent_ro PASSWORD '<random-password>';"
|
||||
```
|
||||
2. **Add `AGENT_DB_DSN`** to `/home/erik/MosswartOverlord/.env` with the
|
||||
password you just set.
|
||||
3. **Run the installer**:
|
||||
```bash
|
||||
cd /home/erik/MosswartOverlord
|
||||
bash agent/install.sh
|
||||
```
|
||||
4. **Update nginx**: edit `/etc/nginx/sites-enabled/overlord` to add the
|
||||
`/api/agent/` location (already in `nginx/overlord.conf` in the repo —
|
||||
just `sudo cp` and reload).
|
||||
|
||||
## Day-to-day deploy
|
||||
|
||||
After editing any agent file:
|
||||
|
||||
```bash
|
||||
# On dev:
|
||||
git push
|
||||
|
||||
# On server:
|
||||
ssh erik@overlord.snakedesert.se
|
||||
cd /home/erik/MosswartOverlord
|
||||
git pull
|
||||
sudo systemctl restart overlord-agent
|
||||
journalctl -u overlord-agent -f # tail logs
|
||||
```
|
||||
|
||||
For Python dependency changes:
|
||||
|
||||
```bash
|
||||
agent/.venv/bin/pip install -r agent/requirements.txt
|
||||
sudo systemctl restart overlord-agent
|
||||
```
|
||||
|
||||
## Smoke tests
|
||||
|
||||
```bash
|
||||
# 1. Service alive?
|
||||
curl http://127.0.0.1:8767/agent/health
|
||||
|
||||
# 2. Cookie required?
|
||||
curl -X POST http://127.0.0.1:8767/agent/ask \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"session_id":"x","message":"hi"}'
|
||||
# ⇒ 401
|
||||
|
||||
# 3. Direct claude invocation works?
|
||||
echo "hello" | /home/erik/.local/bin/claude -p \
|
||||
--session-id 11111111-1111-1111-1111-111111111111 \
|
||||
--output-format json
|
||||
|
||||
# 4. End-to-end via nginx (with cookie):
|
||||
curl -X POST https://overlord.snakedesert.se/api/agent/ask \
|
||||
-b 'session=<your-session-cookie>' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"session_id":"<uuid>","message":"How many characters are online?"}'
|
||||
```
|
||||
|
||||
## Cost / rate-limit notes
|
||||
|
||||
- Each `/agent/ask` shells out to `claude -p` once.
|
||||
- We use the user's Claude subscription (no API key) — flat-rate, no
|
||||
per-call billing, but subscription-tier rate limits still apply.
|
||||
- **Reactive only**: there are no background loops or periodic ticks.
|
||||
Each user message = one Claude turn (which may chain several tool
|
||||
calls internally before producing a final answer).
|
||||
- The SQL tool is hard-capped at 10s and 200 rows.
|
||||
- `suitbuilder_search` is the only tool that can take minutes; nginx
|
||||
read timeout is 180s for `/api/agent/`.
|
||||
|
||||
## Adding a new MCP tool
|
||||
|
||||
1. Implement `async def my_tool(...) -> dict` in `tools.py`.
|
||||
2. Register it in `mcp_overlord.py` under `TOOL_DEFS`:
|
||||
- description (the agent reads this to decide when to call)
|
||||
- JSON schema for arguments
|
||||
- lambda dispatching to `T.my_tool(...)`
|
||||
3. `sudo systemctl restart overlord-agent`. Claude Code re-discovers the
|
||||
tool list on each invocation.
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
"""Overlord Agent — host-side service that shells out to claude -p.
|
||||
|
||||
Runs OUTSIDE the dereth-tracker Docker container because the `claude` CLI
|
||||
binary lives at /home/erik/.local/bin/claude on the host and depends on
|
||||
~/.claude/ credentials owned by user erik. The container can't invoke it
|
||||
directly, so this is a small standalone FastAPI service on port 8767.
|
||||
|
||||
nginx routes /api/agent/* to here. The same browser session cookie that
|
||||
dereth-tracker validates is reused (shared SECRET_KEY env var).
|
||||
"""
|
||||
|
|
@ -1,51 +0,0 @@
|
|||
"""Session-cookie validation that mirrors main.py.
|
||||
|
||||
Re-implements the verify path so this host-side service can authenticate
|
||||
the same browser cookie that dereth-tracker issues. Both services must
|
||||
share the SECRET_KEY env var.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from fastapi import HTTPException, Request, status
|
||||
from itsdangerous import BadSignature, SignatureExpired, URLSafeTimedSerializer
|
||||
|
||||
# Mirror main.py:996-998
|
||||
SECRET_KEY = os.getenv("SECRET_KEY", "change-me-in-production-please")
|
||||
SESSION_MAX_AGE = 30 * 24 * 3600 # 30 days
|
||||
_serializer = URLSafeTimedSerializer(SECRET_KEY)
|
||||
|
||||
|
||||
def verify_session_cookie(token: str) -> dict | None:
|
||||
"""Verify and decode a session token. Returns None if invalid/expired.
|
||||
|
||||
Mirrors main.py:1013-1019 byte-for-byte so a cookie issued by the tracker
|
||||
decodes here identically.
|
||||
"""
|
||||
try:
|
||||
data = _serializer.loads(token, max_age=SESSION_MAX_AGE)
|
||||
return {"username": data["u"], "is_admin": data["a"]}
|
||||
except (BadSignature, SignatureExpired, KeyError):
|
||||
return None
|
||||
|
||||
|
||||
def require_user(request: Request) -> dict:
|
||||
"""FastAPI dependency: enforces a valid session cookie.
|
||||
|
||||
Returns the decoded user dict on success; raises 401 otherwise.
|
||||
"""
|
||||
token = request.cookies.get("session")
|
||||
if not token:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Not authenticated",
|
||||
)
|
||||
user = verify_session_cookie(token)
|
||||
if not user:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Session invalid or expired",
|
||||
)
|
||||
return user
|
||||
|
|
@ -1,280 +0,0 @@
|
|||
"""Subprocess wrapper around `claude -p` (Claude Code in headless JSON mode).
|
||||
|
||||
Run from cwd=/home/erik/MosswartOverlord so:
|
||||
• Sessions persist at ~/.claude/projects/-home-erik-MosswartOverlord/<uuid>.jsonl
|
||||
• Project-level .mcp.json is auto-loaded
|
||||
• CLAUDE.md in the repo root briefs the agent
|
||||
|
||||
The `--session-id` flag both creates a new session (first call) and resumes
|
||||
an existing one (subsequent calls), so we don't need separate code paths.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# These can be overridden via env vars for non-prod testing.
|
||||
CLAUDE_BIN = os.getenv("CLAUDE_BIN", "/home/erik/.local/bin/claude")
|
||||
CLAUDE_CWD = os.getenv("CLAUDE_CWD", "/home/erik/MosswartOverlord")
|
||||
# Hard cap on how long a single agent turn may take. Claude Code can spin a
|
||||
# while when chaining many tool calls; we don't want to leave a zombie
|
||||
# subprocess if something gets stuck.
|
||||
CLAUDE_TIMEOUT_S = int(os.getenv("CLAUDE_TIMEOUT_S", "240"))
|
||||
|
||||
|
||||
@dataclass
|
||||
class ClaudeResult:
|
||||
result: str
|
||||
session_id: str
|
||||
duration_ms: int
|
||||
num_turns: int
|
||||
is_error: bool
|
||||
raw: dict[str, Any]
|
||||
|
||||
|
||||
class ClaudeError(RuntimeError):
|
||||
"""Raised when the claude CLI returns a non-zero exit or unparseable output."""
|
||||
|
||||
|
||||
def _session_exists(session_id: str) -> bool:
|
||||
"""True if Claude Code has already persisted a JSONL for this session.
|
||||
|
||||
Claude Code stores sessions at ~/.claude/projects/<encoded-cwd>/<uuid>.jsonl
|
||||
where non-alphanumerics in the cwd are replaced with hyphens.
|
||||
"""
|
||||
encoded = "".join(c if c.isalnum() else "-" for c in CLAUDE_CWD)
|
||||
path = Path.home() / ".claude" / "projects" / encoded / f"{session_id}.jsonl"
|
||||
return path.is_file()
|
||||
|
||||
|
||||
async def ask_claude(message: str, session_id: str) -> ClaudeResult:
|
||||
"""Send `message` to `claude -p` for `session_id`; return parsed result.
|
||||
|
||||
On the FIRST message of a session uses `--session-id <uuid>` to create it.
|
||||
On subsequent messages uses `--resume <uuid>` because claude rejects
|
||||
`--session-id` on existing sessions ("Session ID ... is already in use").
|
||||
|
||||
Raises ClaudeError on subprocess failure, JSON parse failure, or timeout.
|
||||
"""
|
||||
if not Path(CLAUDE_BIN).exists():
|
||||
raise ClaudeError(f"claude binary not found at {CLAUDE_BIN}")
|
||||
if not Path(CLAUDE_CWD).is_dir():
|
||||
raise ClaudeError(f"CLAUDE_CWD does not exist: {CLAUDE_CWD}")
|
||||
|
||||
# Whitelist only our MCP tools so Claude Code can call them without
|
||||
# human approval. Names follow the convention mcp__<server>__<tool>.
|
||||
# We deliberately omit built-in tools (Bash, Write, Edit, Read, etc.)
|
||||
# — the assistant doesn't need them for live-state Q&A and they'd be a
|
||||
# security/permissions footgun on an unattended service.
|
||||
allowed_tools = ",".join(
|
||||
[
|
||||
"mcp__overlord__get_live_players",
|
||||
"mcp__overlord__get_recent_rares",
|
||||
"mcp__overlord__query_telemetry_db",
|
||||
"mcp__overlord__get_player_state",
|
||||
"mcp__overlord__get_inventory",
|
||||
"mcp__overlord__get_inventory_search",
|
||||
"mcp__overlord__search_items",
|
||||
"mcp__overlord__get_combat_stats",
|
||||
"mcp__overlord__get_equipment_cantrips",
|
||||
"mcp__overlord__get_quest_status",
|
||||
"mcp__overlord__get_server_health",
|
||||
"mcp__overlord__suitbuilder_search",
|
||||
]
|
||||
)
|
||||
|
||||
# CRITICAL: Claude Code's built-in meta-tools (ToolSearch, Monitor, etc.)
|
||||
# bypass the --allowed-tools whitelist. They come from Anthropic's tool
|
||||
# registry rather than from local MCP servers. We must explicitly DISALLOW
|
||||
# them — confirmed by testing that ToolSearch was reachable even with
|
||||
# `--permission-mode dontAsk` and a tight --allowed-tools list.
|
||||
disallowed_tools = ",".join(
|
||||
[
|
||||
# File / shell / search built-ins (defense in depth — already not
|
||||
# in allow list, but if someone toggles permission-mode this
|
||||
# belt-and-suspenders the deny side).
|
||||
"Bash",
|
||||
"Write",
|
||||
"Edit",
|
||||
"Read",
|
||||
"Glob",
|
||||
"Grep",
|
||||
"NotebookEdit",
|
||||
# Network built-ins
|
||||
"WebSearch",
|
||||
"WebFetch", # blocked here; settings.json re-allows acpedia.org
|
||||
# Subagent spawning — the assistant must NEVER delegate to a
|
||||
# general-purpose subagent (which would have its own tool set).
|
||||
"Agent",
|
||||
# Tool / session meta-tools — these can list, load, or chain
|
||||
# into other tools and must NOT be reachable.
|
||||
"ToolSearch",
|
||||
"Monitor",
|
||||
"TaskOutput",
|
||||
"TaskStop",
|
||||
"TodoWrite",
|
||||
"Skill",
|
||||
"EnterPlanMode",
|
||||
"ExitPlanMode",
|
||||
"EnterWorktree",
|
||||
"ExitWorktree",
|
||||
"AskUserQuestion",
|
||||
"ListMcpResourcesTool",
|
||||
"ReadMcpResourceTool",
|
||||
"PushNotification",
|
||||
# Scheduling / cron — the agent must never schedule itself.
|
||||
"CronCreate",
|
||||
"CronList",
|
||||
"CronDelete",
|
||||
"ScheduleWakeup",
|
||||
"RemoteTrigger",
|
||||
# Anthropic first-party connectors from the user's claude.ai
|
||||
# account. These are off-mission for an Overlord assistant and
|
||||
# would leak personal data outside the game-state domain.
|
||||
"mcp__claude_ai_Gmail__create_draft",
|
||||
"mcp__claude_ai_Gmail__create_label",
|
||||
"mcp__claude_ai_Gmail__get_message",
|
||||
"mcp__claude_ai_Gmail__get_thread",
|
||||
"mcp__claude_ai_Gmail__list_drafts",
|
||||
"mcp__claude_ai_Gmail__list_labels",
|
||||
"mcp__claude_ai_Gmail__label_message",
|
||||
"mcp__claude_ai_Gmail__label_thread",
|
||||
"mcp__claude_ai_Gmail__search_messages",
|
||||
"mcp__claude_ai_Gmail__search_threads",
|
||||
"mcp__claude_ai_Gmail__send_message",
|
||||
"mcp__claude_ai_Gmail__unlabel_message",
|
||||
"mcp__claude_ai_Gmail__unlabel_thread",
|
||||
"mcp__claude_ai_Google_Calendar__authenticate",
|
||||
"mcp__claude_ai_Google_Drive__authenticate",
|
||||
]
|
||||
)
|
||||
|
||||
# Pick --session-id (creates) vs --resume (continues) based on whether
|
||||
# the session JSONL already exists on disk.
|
||||
is_new = not _session_exists(session_id)
|
||||
session_flag = "--session-id" if is_new else "--resume"
|
||||
|
||||
args = [
|
||||
CLAUDE_BIN,
|
||||
"-p",
|
||||
session_flag,
|
||||
session_id,
|
||||
"--output-format",
|
||||
"json",
|
||||
"--allowed-tools",
|
||||
allowed_tools,
|
||||
# Built-in meta-tools that --allowed-tools does NOT block — must
|
||||
# be explicitly listed here.
|
||||
"--disallowed-tools",
|
||||
disallowed_tools,
|
||||
# CRITICAL: dontAsk auto-DENIES anything outside --allowed-tools.
|
||||
# Do NOT use bypassPermissions here — that mode ignores the whitelist
|
||||
# entirely and lets the model call Bash/Write/Edit/etc. (verified
|
||||
# the hard way: it wrote /tmp/owned.sh when prompted to).
|
||||
# See https://code.claude.com/docs/en/permission-modes.md
|
||||
"--permission-mode",
|
||||
"dontAsk",
|
||||
]
|
||||
|
||||
logger.info(
|
||||
"claude exec: session=%s mode=%s msg_len=%d cwd=%s",
|
||||
session_id,
|
||||
"new" if is_new else "resume",
|
||||
len(message),
|
||||
CLAUDE_CWD,
|
||||
)
|
||||
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*args,
|
||||
stdin=asyncio.subprocess.PIPE,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
cwd=CLAUDE_CWD,
|
||||
)
|
||||
|
||||
try:
|
||||
stdout, stderr = await asyncio.wait_for(
|
||||
proc.communicate(input=message.encode("utf-8")),
|
||||
timeout=CLAUDE_TIMEOUT_S,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
try:
|
||||
proc.kill()
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
raise ClaudeError(f"claude timed out after {CLAUDE_TIMEOUT_S}s")
|
||||
|
||||
if proc.returncode != 0:
|
||||
stderr_text = stderr.decode("utf-8", "replace")
|
||||
# If we picked the wrong flag (e.g. JSONL deleted from disk between
|
||||
# our check and exec, or a never-flushed session), claude prints
|
||||
# "Session ID … is already in use." Re-issue with --resume.
|
||||
if is_new and "already in use" in stderr_text:
|
||||
logger.info("session %s actually exists; retrying with --resume", session_id)
|
||||
args2 = list(args)
|
||||
args2[2] = "--resume"
|
||||
proc2 = await asyncio.create_subprocess_exec(
|
||||
*args2,
|
||||
stdin=asyncio.subprocess.PIPE,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
cwd=CLAUDE_CWD,
|
||||
)
|
||||
try:
|
||||
stdout, stderr = await asyncio.wait_for(
|
||||
proc2.communicate(input=message.encode("utf-8")),
|
||||
timeout=CLAUDE_TIMEOUT_S,
|
||||
)
|
||||
except asyncio.TimeoutError:
|
||||
try:
|
||||
proc2.kill()
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
raise ClaudeError(f"claude timed out after {CLAUDE_TIMEOUT_S}s")
|
||||
if proc2.returncode != 0:
|
||||
raise ClaudeError(
|
||||
f"claude exited {proc2.returncode} after retry: "
|
||||
f"{stderr.decode('utf-8', 'replace')[:500]}"
|
||||
)
|
||||
else:
|
||||
raise ClaudeError(
|
||||
f"claude exited {proc.returncode}: {stderr_text[:500]}"
|
||||
)
|
||||
|
||||
raw_text = stdout.decode("utf-8", "replace").strip()
|
||||
if not raw_text:
|
||||
raise ClaudeError("claude produced empty stdout")
|
||||
|
||||
# In --output-format json mode the LAST line is the JSON envelope; some
|
||||
# earlier lines may be progress. Be tolerant.
|
||||
try:
|
||||
envelope = json.loads(raw_text)
|
||||
except json.JSONDecodeError:
|
||||
# Try the last non-empty line
|
||||
last = next(
|
||||
(line for line in reversed(raw_text.splitlines()) if line.strip()),
|
||||
"",
|
||||
)
|
||||
try:
|
||||
envelope = json.loads(last)
|
||||
except json.JSONDecodeError as e:
|
||||
raise ClaudeError(
|
||||
f"claude stdout was not JSON: {raw_text[:500]}"
|
||||
) from e
|
||||
|
||||
return ClaudeResult(
|
||||
result=envelope.get("result", ""),
|
||||
session_id=envelope.get("session_id", session_id),
|
||||
duration_ms=int(envelope.get("duration_ms", 0)),
|
||||
num_turns=int(envelope.get("num_turns", 0)),
|
||||
is_error=bool(envelope.get("is_error", False)),
|
||||
raw=envelope,
|
||||
)
|
||||
|
|
@ -1,46 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Install / re-install the Overlord Agent host-side service.
|
||||
#
|
||||
# Run as user `erik` from /home/erik/MosswartOverlord:
|
||||
# bash agent/install.sh
|
||||
#
|
||||
# Requires sudo for the systemd parts (you'll be prompted once).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_DIR="/home/erik/MosswartOverlord"
|
||||
AGENT_DIR="$REPO_DIR/agent"
|
||||
VENV_DIR="$AGENT_DIR/.venv"
|
||||
SERVICE_FILE="$AGENT_DIR/overlord-agent.service"
|
||||
SYSTEMD_TARGET="/etc/systemd/system/overlord-agent.service"
|
||||
|
||||
if [[ "$(pwd)" != "$REPO_DIR" ]]; then
|
||||
echo "Run from $REPO_DIR (currently in $(pwd))" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "==> Creating/updating venv at $VENV_DIR"
|
||||
if [[ ! -d "$VENV_DIR" ]]; then
|
||||
python3 -m venv "$VENV_DIR"
|
||||
fi
|
||||
"$VENV_DIR/bin/pip" install --quiet --upgrade pip
|
||||
"$VENV_DIR/bin/pip" install --quiet -r "$AGENT_DIR/requirements.txt"
|
||||
|
||||
echo "==> Installing systemd unit"
|
||||
sudo cp "$SERVICE_FILE" "$SYSTEMD_TARGET"
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
echo "==> Enabling + starting overlord-agent"
|
||||
sudo systemctl enable overlord-agent
|
||||
sudo systemctl restart overlord-agent
|
||||
|
||||
sleep 1
|
||||
echo "==> Status:"
|
||||
sudo systemctl --no-pager status overlord-agent | head -15
|
||||
|
||||
echo ""
|
||||
echo "==> Smoke test:"
|
||||
curl -s http://127.0.0.1:8767/agent/health | python3 -m json.tool || true
|
||||
|
||||
echo ""
|
||||
echo "Done. Logs: journalctl -u overlord-agent -f"
|
||||
|
|
@ -1,293 +0,0 @@
|
|||
"""MCP stdio server exposing Overlord data to Claude Code.
|
||||
|
||||
Configured via .mcp.json at the repo root, which Claude Code auto-loads
|
||||
when invoked with cwd=/home/erik/MosswartOverlord. Tool implementations
|
||||
live in tools.py — this file is just MCP protocol plumbing.
|
||||
|
||||
Run directly with:
|
||||
python3 /home/erik/MosswartOverlord/agent/mcp_overlord.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from mcp.server import Server
|
||||
from mcp.server.stdio import stdio_server
|
||||
from mcp.types import TextContent, Tool
|
||||
|
||||
from . import tools as T
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s mcp_overlord: %(message)s",
|
||||
)
|
||||
logger = logging.getLogger("mcp_overlord")
|
||||
|
||||
server: Server = Server("overlord")
|
||||
|
||||
|
||||
# ─── Tool registry ──────────────────────────────────────────────────
|
||||
#
|
||||
# Each entry: name → (description, JSON schema, callable async fn).
|
||||
# We register them with @server.list_tools / @server.call_tool below.
|
||||
|
||||
TOOL_DEFS: dict[str, dict[str, Any]] = {
|
||||
"get_live_players": {
|
||||
"description": (
|
||||
"Return active characters seen in the last ~30 seconds with their "
|
||||
"current position, kills, KPH, vitae, online time, and VTank state. "
|
||||
"Use this for any 'who is online right now / what is X doing' question."
|
||||
),
|
||||
"schema": {"type": "object", "properties": {}},
|
||||
"fn": lambda _args: T.get_live_players(),
|
||||
},
|
||||
"get_recent_rares": {
|
||||
"description": (
|
||||
"Return rare item finds from the last N hours, newest first. "
|
||||
"Use for questions about recent drops, who is finding rares, or "
|
||||
"rare-rate analysis. Defaults to 24 hours, max 30 days."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"hours": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 720,
|
||||
"default": 24,
|
||||
},
|
||||
"limit": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 200,
|
||||
"default": 100,
|
||||
},
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.get_recent_rares(
|
||||
hours=int(args.get("hours", 24)),
|
||||
limit=int(args.get("limit", 100)),
|
||||
),
|
||||
},
|
||||
"query_telemetry_db": {
|
||||
"description": (
|
||||
"Run a read-only SQL query against the telemetry database (TimescaleDB). "
|
||||
"Only SELECT / WITH statements are accepted; any DML or DDL is rejected. "
|
||||
"Useful for questions that aren't covered by the other tools — top-N "
|
||||
"lists, custom aggregations, time-window comparisons. "
|
||||
"Available tables include: telemetry_events (hypertable, 30d retention), "
|
||||
"rare_events, spawn_events (hypertable, 7d retention), portals, "
|
||||
"char_stats, rare_stats, rare_stats_sessions, character_stats, "
|
||||
"combat_stats, combat_stats_sessions, server_status. "
|
||||
"The query has a 10s timeout and returns at most 200 rows."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["sql"],
|
||||
"properties": {
|
||||
"sql": {
|
||||
"type": "string",
|
||||
"description": "A single PostgreSQL SELECT or WITH ... SELECT statement.",
|
||||
}
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.query_telemetry_db(str(args["sql"])),
|
||||
},
|
||||
"get_player_state": {
|
||||
"description": (
|
||||
"Combined snapshot for ONE character: live telemetry (if online) "
|
||||
"+ full character stats (attributes, skills, augmentations). "
|
||||
"Use this for questions like 'what is X doing right now' or 'show me X's stats'."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["character_name"],
|
||||
"properties": {
|
||||
"character_name": {"type": "string"},
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.get_player_state(str(args["character_name"])),
|
||||
},
|
||||
"get_inventory": {
|
||||
"description": (
|
||||
"Full inventory listing for one character — every item with name, "
|
||||
"icon, container, equipped slot, spells, material, tinkers, etc. "
|
||||
"Large response — prefer get_inventory_search for narrow queries."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["character_name"],
|
||||
"properties": {"character_name": {"type": "string"}},
|
||||
},
|
||||
"fn": lambda args: T.get_inventory(str(args["character_name"])),
|
||||
},
|
||||
"get_inventory_search": {
|
||||
"description": (
|
||||
"Filtered inventory search for ONE character. Use search_items "
|
||||
"instead when the user wants to find something across ALL chars."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["character_name"],
|
||||
"properties": {
|
||||
"character_name": {"type": "string"},
|
||||
"filters": {
|
||||
"type": "object",
|
||||
"description": "Query params dict, e.g. {\"name\": \"pearl\", \"armor_level_min\": 500}",
|
||||
},
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.get_inventory_search(
|
||||
str(args["character_name"]), args.get("filters") or {}
|
||||
),
|
||||
},
|
||||
"search_items": {
|
||||
"description": (
|
||||
"CROSS-CHARACTER item search — one query that scans every "
|
||||
"character's inventory. Use this whenever the user asks "
|
||||
"'find me an X on any of my chars'. **Do not** iterate "
|
||||
"get_inventory per character — this single tool call is far "
|
||||
"faster and avoids agent timeouts.\n\n"
|
||||
"Filter keys (pass as `filters` object, all optional):\n"
|
||||
" include_all_characters: true (default if no scope given)\n"
|
||||
" character: 'Name' (single char)\n"
|
||||
" characters: 'A,B,C' (specific list, comma-separated)\n"
|
||||
" text: substring of item name/description\n"
|
||||
" has_spell: 'Legendary Acid Ward' (exact spell name match)\n"
|
||||
" spell_contains: 'Legendary' (substring)\n"
|
||||
" legendary_cantrips: 'Foo,Bar'\n"
|
||||
" equipment_status: 'equipped' | 'unequipped'\n"
|
||||
" equipment_slot: int bitmask (4=chest, 2048=bracelet, 4096=ring)\n"
|
||||
" slot_names: 'Bracelet,Ring'\n"
|
||||
" armor_only / jewelry_only / weapon_only: bool\n"
|
||||
" min_armor / max_armor / min_damage / max_damage: int\n"
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["filters"],
|
||||
"properties": {
|
||||
"filters": {
|
||||
"type": "object",
|
||||
"description": "Query params dict — see tool description for keys.",
|
||||
},
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.search_items_global(args.get("filters") or {}),
|
||||
},
|
||||
"get_combat_stats": {
|
||||
"description": (
|
||||
"Lifetime + session combat stats for one character. Includes total "
|
||||
"damage given/received, per-element offense/defense breakdown, kill "
|
||||
"counts, and aetheria surge counts."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["character_name"],
|
||||
"properties": {"character_name": {"type": "string"}},
|
||||
},
|
||||
"fn": lambda args: T.get_combat_stats(str(args["character_name"])),
|
||||
},
|
||||
"get_equipment_cantrips": {
|
||||
"description": (
|
||||
"Currently-equipped items for a character along with their active "
|
||||
"cantrip/spell state. Useful for 'what is X wearing' or 'is X "
|
||||
"running their suit' questions."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["character_name"],
|
||||
"properties": {"character_name": {"type": "string"}},
|
||||
},
|
||||
"fn": lambda args: T.get_equipment_cantrips(str(args["character_name"])),
|
||||
},
|
||||
"get_quest_status": {
|
||||
"description": (
|
||||
"Active quest timers and progress across ALL characters. Returns "
|
||||
"for each character which quests are READY vs counting down."
|
||||
),
|
||||
"schema": {"type": "object", "properties": {}},
|
||||
"fn": lambda _args: T.get_quest_status(),
|
||||
},
|
||||
"get_server_health": {
|
||||
"description": (
|
||||
"Current Coldeve game-server status: up/down, latency in ms, "
|
||||
"current player count from TreeStats.net, total uptime. Updated "
|
||||
"every 30 seconds in the background."
|
||||
),
|
||||
"schema": {"type": "object", "properties": {}},
|
||||
"fn": lambda _args: T.get_server_health(),
|
||||
},
|
||||
"suitbuilder_search": {
|
||||
"description": (
|
||||
"Run a constraint-satisfaction armor optimization across all "
|
||||
"characters' inventories ('mules'). Drives the same suitbuilder "
|
||||
"the /suitbuilder.html page uses. Pass the same params dict the "
|
||||
"page sends — see /suitbuilder.html JS for the schema. The search "
|
||||
"is SSE-streaming on the backend; this tool collects until done "
|
||||
"and returns the final suit(s) plus the last few phase events. "
|
||||
"Can take up to 5 minutes for complex constraints — only call "
|
||||
"when the user explicitly asks for an optimization run."
|
||||
),
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": ["params"],
|
||||
"properties": {
|
||||
"params": {
|
||||
"type": "object",
|
||||
"description": "Suitbuilder request body (characters, locked slots, set constraints, etc.)",
|
||||
},
|
||||
},
|
||||
},
|
||||
"fn": lambda args: T.suitbuilder_search(args.get("params") or {}),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
# ─── MCP protocol wiring ────────────────────────────────────────────
|
||||
|
||||
|
||||
@server.list_tools()
|
||||
async def list_tools() -> list[Tool]:
|
||||
return [
|
||||
Tool(name=name, description=defn["description"], inputSchema=defn["schema"])
|
||||
for name, defn in TOOL_DEFS.items()
|
||||
]
|
||||
|
||||
|
||||
@server.call_tool()
|
||||
async def call_tool(name: str, arguments: dict[str, Any]) -> list[TextContent]:
|
||||
if name not in TOOL_DEFS:
|
||||
return [TextContent(type="text", text=f"unknown tool: {name}")]
|
||||
|
||||
fn = TOOL_DEFS[name]["fn"]
|
||||
try:
|
||||
result = await fn(arguments or {})
|
||||
except T.SqlNotAllowed as e:
|
||||
return [TextContent(type="text", text=f"REJECTED: {e}")]
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.exception("tool %s failed", name)
|
||||
return [TextContent(type="text", text=f"ERROR: {type(e).__name__}: {e}")]
|
||||
|
||||
text = json.dumps(result, default=str, ensure_ascii=False, indent=2)
|
||||
return [TextContent(type="text", text=text)]
|
||||
|
||||
|
||||
async def _run() -> None:
|
||||
logger.info("starting MCP stdio server (overlord)")
|
||||
try:
|
||||
async with stdio_server() as (reader, writer):
|
||||
await server.run(reader, writer, server.create_initialization_options())
|
||||
finally:
|
||||
await T.shutdown()
|
||||
|
||||
|
||||
def main() -> None:
|
||||
asyncio.run(_run())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -1,113 +0,0 @@
|
|||
[Unit]
|
||||
Description=Overlord Agent (Claude Code shell-out service)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
# Dedicated unprivileged user — kernel-level isolation from `erik`.
|
||||
# overlord-agent has NO access to /home/erik/.claude (mode 0700),
|
||||
# /home/erik/.ssh, /home/erik/.bash_history, /home/erik/.gitconfig, etc.
|
||||
# Its own claude state lives at /var/lib/overlord-agent/.claude/ and its
|
||||
# claude session JSONLs land there — completely separate from any
|
||||
# interactive Claude Code use by the human user.
|
||||
User=overlord-agent
|
||||
Group=overlord-agent
|
||||
# Working directory: the repo root (group-readable to overlord-agent).
|
||||
# claude session JSONLs path-encode this cwd so it's important to keep
|
||||
# stable across restarts.
|
||||
WorkingDirectory=/home/erik/MosswartOverlord
|
||||
# HOME explicitly set so claude reads /var/lib/overlord-agent/.claude/*
|
||||
# instead of trying /home/erik/.claude/* (which is now 0700, locked out).
|
||||
Environment="HOME=/var/lib/overlord-agent"
|
||||
# Secrets file (root:overlord-agent 0640).
|
||||
EnvironmentFile=-/etc/overlord/agent.env
|
||||
# Run inside the venv populated by install.sh.
|
||||
ExecStart=/home/erik/MosswartOverlord/agent/.venv/bin/python -m agent.service
|
||||
Restart=on-failure
|
||||
RestartSec=3
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# ─── Resource caps ─────────────────────────────────────────────────
|
||||
MemoryMax=512M
|
||||
CPUQuota=200%
|
||||
TasksMax=128
|
||||
|
||||
# ─── Filesystem hardening ──────────────────────────────────────────
|
||||
# /usr, /boot, /efi become read-only; /etc + /var get a writable overlay
|
||||
# that's discarded on stop. Subprocesses inherit these protections.
|
||||
ProtectSystem=strict
|
||||
ProtectHome=read-only
|
||||
# Allow writing only to the explicit paths claude / our service need.
|
||||
# - ~/.claude — session JSONL files
|
||||
# - .venv pycache — minor pip cache writes
|
||||
ReadWritePaths=/var/lib/overlord-agent/.claude
|
||||
ReadWritePaths=/home/erik/MosswartOverlord/agent/.venv
|
||||
ReadWritePaths=/var/log/overlord-agent
|
||||
# StateDirectory creates/owns /var/lib/overlord-agent automatically.
|
||||
StateDirectory=overlord-agent
|
||||
LogsDirectory=overlord-agent
|
||||
LogsDirectoryMode=0755
|
||||
PrivateTmp=true
|
||||
PrivateDevices=true
|
||||
ProtectClock=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectKernelLogs=true
|
||||
ProtectControlGroups=true
|
||||
ProtectHostname=true
|
||||
ProtectProc=invisible
|
||||
ProcSubset=pid
|
||||
|
||||
# Hide sensitive host paths even if something in the python or claude
|
||||
# subprocess tree tries to read them.
|
||||
InaccessiblePaths=/etc/shadow
|
||||
InaccessiblePaths=/etc/gshadow
|
||||
InaccessiblePaths=/etc/ssh
|
||||
InaccessiblePaths=/root
|
||||
InaccessiblePaths=-/home/erik/.ssh
|
||||
InaccessiblePaths=-/home/erik/.bash_history
|
||||
InaccessiblePaths=-/home/erik/.zsh_history
|
||||
|
||||
# ─── Privilege & capability hardening ──────────────────────────────
|
||||
NoNewPrivileges=true
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
LockPersonality=true
|
||||
RestrictRealtime=true
|
||||
RestrictSUIDSGID=true
|
||||
RemoveIPC=true
|
||||
# MemoryDenyWriteExecute would break Node.js (V8 JIT requires W^X
|
||||
# transitions via mprotect with PROT_EXEC on JITted code pages). Claude
|
||||
# Code is a Node app, so omit this. Without JIT we'd lose all model
|
||||
# performance. The other restrictions still prevent shellcode injection
|
||||
# in practice (no Bash/Write tools, no shellcraft surface).
|
||||
# MemoryDenyWriteExecute=true ← DO NOT enable; breaks Node V8 JIT
|
||||
RestrictNamespaces=true
|
||||
|
||||
# ─── Network family restriction ────────────────────────────────────
|
||||
# Block raw/packet sockets so even a kernel-LPE-class bug can't sniff
|
||||
# traffic or forge packets. We don't IPAddressAllow-restrict because
|
||||
# Anthropic's Cloudflare IPs shift and the whitelist would break claude.
|
||||
# If you need true egress filtering, run nftables scoped to this
|
||||
# service's cgroup — that's reliable in a way IPAddressAllow isn't.
|
||||
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||
|
||||
# ─── Syscall filter ────────────────────────────────────────────────
|
||||
# Use the standard @system-service preset which is what almost every
|
||||
# hardened systemd unit uses. It already excludes the dangerous groups
|
||||
# (privileged, mount, reboot, raw-io, etc.) by NOT including them, while
|
||||
# being broad enough to host typical apps including Node.js.
|
||||
#
|
||||
# We tried adding extra "~@..." negations on top — they killed Claude
|
||||
# (Node) with SIGSYS during startup. The default @system-service preset
|
||||
# is the right balance; the rest of the hardening covers what we need.
|
||||
SystemCallArchitectures=native
|
||||
SystemCallFilter=@system-service
|
||||
SystemCallFilter=~@privileged
|
||||
SystemCallFilter=~@reboot
|
||||
SystemCallFilter=~@mount
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
fastapi>=0.110
|
||||
uvicorn[standard]>=0.30
|
||||
httpx>=0.27
|
||||
itsdangerous>=2.2
|
||||
pydantic>=2.6
|
||||
# MCP server SDK (used by mcp_overlord.py for the stdio MCP server)
|
||||
mcp>=1.0
|
||||
# SQL safety: parses SQL to enforce read-only on the query_db tool
|
||||
sqlglot>=25.0
|
||||
# Direct DB access for the read-only query tool and rare_events lookups
|
||||
asyncpg>=0.29
|
||||
# .env loader
|
||||
python-dotenv>=1.0
|
||||
347
agent/service.py
|
|
@ -1,347 +0,0 @@
|
|||
"""Overlord Agent host-side FastAPI service.
|
||||
|
||||
Runs OUTSIDE Docker (host-side) on port 8767.
|
||||
|
||||
Endpoints:
|
||||
GET /agent/health — liveness check
|
||||
POST /agent/sessions/new — returns a fresh session UUID
|
||||
POST /agent/ask — runs claude -p with given session
|
||||
GET /agent/sessions/{session_id}/history
|
||||
— replays a session's JSONL on disk
|
||||
|
||||
Auth: every endpoint except /health requires the same browser session
|
||||
cookie that dereth-tracker issues.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from collections import deque
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from . import auth
|
||||
from .claude_wrapper import CLAUDE_CWD, ClaudeError, ask_claude
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
|
||||
)
|
||||
logger = logging.getLogger("agent")
|
||||
|
||||
# Audit log — every /agent/ask request gets a JSONL line here, separate
|
||||
# from journald so the operator can grep without root. Set to /dev/null
|
||||
# to disable. Rotated externally (logrotate) if it gets big.
|
||||
AUDIT_LOG_PATH = Path(os.getenv("AGENT_AUDIT_LOG", "/var/log/overlord-agent/audit.jsonl"))
|
||||
audit_logger = logging.getLogger("agent.audit")
|
||||
try:
|
||||
AUDIT_LOG_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
_h = logging.FileHandler(AUDIT_LOG_PATH)
|
||||
_h.setFormatter(logging.Formatter("%(message)s"))
|
||||
audit_logger.addHandler(_h)
|
||||
audit_logger.propagate = False
|
||||
audit_logger.setLevel(logging.INFO)
|
||||
except OSError as e:
|
||||
logger.warning("audit log path %s not writable (%s); logging only via journal", AUDIT_LOG_PATH, e)
|
||||
|
||||
# Rate limit: per-user count over a rolling window. Defaults are generous
|
||||
# for a single human at a keyboard but block automated abuse.
|
||||
RATE_LIMIT_WINDOW_S = int(os.getenv("AGENT_RATE_WINDOW_S", "3600"))
|
||||
RATE_LIMIT_MAX = int(os.getenv("AGENT_RATE_MAX", "60"))
|
||||
# Per-user concurrent request cap (no fanning out 50 calls in parallel).
|
||||
CONCURRENCY_LIMIT_PER_USER = int(os.getenv("AGENT_CONCURRENCY_PER_USER", "1"))
|
||||
|
||||
# Rolling timestamps of recent /agent/ask calls per user.
|
||||
_rate_state: dict[str, deque[float]] = {}
|
||||
# Per-user semaphores so a single user can't run multiple concurrent claude
|
||||
# subprocesses (each is expensive).
|
||||
_user_semaphores: dict[str, asyncio.Semaphore] = {}
|
||||
|
||||
|
||||
def _check_rate_limit(username: str) -> tuple[bool, int]:
|
||||
"""Return (allowed, retry_after_seconds)."""
|
||||
now = time.monotonic()
|
||||
window = _rate_state.setdefault(username, deque())
|
||||
cutoff = now - RATE_LIMIT_WINDOW_S
|
||||
while window and window[0] < cutoff:
|
||||
window.popleft()
|
||||
if len(window) >= RATE_LIMIT_MAX:
|
||||
retry_after = int(window[0] + RATE_LIMIT_WINDOW_S - now) + 1
|
||||
return False, retry_after
|
||||
window.append(now)
|
||||
return True, 0
|
||||
|
||||
|
||||
def _user_semaphore(username: str) -> asyncio.Semaphore:
|
||||
sem = _user_semaphores.get(username)
|
||||
if sem is None:
|
||||
sem = asyncio.Semaphore(CONCURRENCY_LIMIT_PER_USER)
|
||||
_user_semaphores[username] = sem
|
||||
return sem
|
||||
|
||||
|
||||
def _audit(event: dict[str, Any]) -> None:
|
||||
"""Emit one JSONL line to the audit log."""
|
||||
event["timestamp"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
try:
|
||||
audit_logger.info(json.dumps(event, ensure_ascii=False))
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
|
||||
app = FastAPI(title="Overlord Agent", version="0.1.0")
|
||||
|
||||
|
||||
# ─── Models ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class AskRequest(BaseModel):
|
||||
session_id: str = Field(
|
||||
..., description="Stable per-conversation UUID stored in browser localStorage"
|
||||
)
|
||||
message: str = Field(..., min_length=1, max_length=10_000)
|
||||
|
||||
|
||||
class AskResponse(BaseModel):
|
||||
result: str
|
||||
session_id: str
|
||||
duration_ms: int
|
||||
num_turns: int
|
||||
is_error: bool
|
||||
|
||||
|
||||
class NewSessionResponse(BaseModel):
|
||||
session_id: str
|
||||
|
||||
|
||||
# ─── Helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _encode_cwd(cwd: str) -> str:
|
||||
"""Match Claude Code's on-disk encoding for cwd → directory name.
|
||||
|
||||
Claude Code stores sessions at ~/.claude/projects/<encoded-cwd>/<uuid>.jsonl
|
||||
where non-alphanumerics in the cwd are replaced with hyphens.
|
||||
Example: /home/erik/MosswartOverlord → -home-erik-MosswartOverlord
|
||||
"""
|
||||
return "".join(c if c.isalnum() else "-" for c in cwd)
|
||||
|
||||
|
||||
def _sessions_dir() -> Path:
|
||||
return Path.home() / ".claude" / "projects" / _encode_cwd(CLAUDE_CWD)
|
||||
|
||||
|
||||
# ─── Endpoints ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@app.get("/agent/health")
|
||||
async def health() -> dict:
|
||||
"""Liveness probe — no auth, used by deployment scripts."""
|
||||
return {
|
||||
"status": "ok",
|
||||
"claude_cwd": CLAUDE_CWD,
|
||||
"sessions_dir_exists": _sessions_dir().exists(),
|
||||
}
|
||||
|
||||
|
||||
@app.post("/agent/sessions/new", response_model=NewSessionResponse)
|
||||
async def new_session(_user: dict = Depends(auth.require_user)) -> NewSessionResponse:
|
||||
"""Generate a fresh session UUID. Doesn't touch disk — claude creates the
|
||||
JSONL file when the first message lands."""
|
||||
return NewSessionResponse(session_id=str(uuid.uuid4()))
|
||||
|
||||
|
||||
@app.post("/agent/ask", response_model=AskResponse)
|
||||
async def agent_ask(
|
||||
req: AskRequest, user: dict = Depends(auth.require_user)
|
||||
) -> AskResponse:
|
||||
"""Forward a message to claude -p resuming the given session.
|
||||
|
||||
Enforces:
|
||||
* Per-user rate limit (60 requests/hour by default).
|
||||
* Per-user concurrency cap (1 in-flight at a time by default).
|
||||
* Audit log of every request (JSONL).
|
||||
"""
|
||||
username = user["username"]
|
||||
|
||||
# Rate limit BEFORE acquiring the user semaphore — cheaper to reject.
|
||||
allowed, retry_after = _check_rate_limit(username)
|
||||
if not allowed:
|
||||
_audit(
|
||||
{
|
||||
"event": "rate_limited",
|
||||
"user": username,
|
||||
"session_id": req.session_id,
|
||||
"retry_after_s": retry_after,
|
||||
}
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=429,
|
||||
detail=f"Rate limit exceeded; retry in {retry_after}s",
|
||||
headers={"Retry-After": str(retry_after)},
|
||||
)
|
||||
|
||||
sem = _user_semaphore(username)
|
||||
if sem.locked():
|
||||
_audit(
|
||||
{
|
||||
"event": "concurrency_blocked",
|
||||
"user": username,
|
||||
"session_id": req.session_id,
|
||||
}
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=429, detail="A previous question is still being processed"
|
||||
)
|
||||
|
||||
started = time.monotonic()
|
||||
async with sem:
|
||||
_audit(
|
||||
{
|
||||
"event": "ask_start",
|
||||
"user": username,
|
||||
"session_id": req.session_id,
|
||||
"message": req.message[:500],
|
||||
"message_len": len(req.message),
|
||||
}
|
||||
)
|
||||
try:
|
||||
result = await ask_claude(req.message, req.session_id)
|
||||
except ClaudeError as e:
|
||||
elapsed_ms = int((time.monotonic() - started) * 1000)
|
||||
logger.warning(
|
||||
"claude failed user=%s session=%s err=%s", username, req.session_id, e
|
||||
)
|
||||
_audit(
|
||||
{
|
||||
"event": "ask_error",
|
||||
"user": username,
|
||||
"session_id": req.session_id,
|
||||
"error": str(e)[:500],
|
||||
"elapsed_ms": elapsed_ms,
|
||||
}
|
||||
)
|
||||
raise HTTPException(status_code=502, detail=str(e))
|
||||
|
||||
elapsed_ms = int((time.monotonic() - started) * 1000)
|
||||
logger.info(
|
||||
"ask user=%s session=%s turns=%d duration_ms=%d (subprocess=%dms)",
|
||||
username,
|
||||
result.session_id,
|
||||
result.num_turns,
|
||||
elapsed_ms,
|
||||
result.duration_ms,
|
||||
)
|
||||
_audit(
|
||||
{
|
||||
"event": "ask_ok",
|
||||
"user": username,
|
||||
"session_id": result.session_id,
|
||||
"result_preview": (result.result or "")[:300],
|
||||
"result_len": len(result.result or ""),
|
||||
"turns": result.num_turns,
|
||||
"elapsed_ms": elapsed_ms,
|
||||
"subprocess_ms": result.duration_ms,
|
||||
"is_error": result.is_error,
|
||||
}
|
||||
)
|
||||
|
||||
return AskResponse(
|
||||
result=result.result,
|
||||
session_id=result.session_id,
|
||||
duration_ms=result.duration_ms,
|
||||
num_turns=result.num_turns,
|
||||
is_error=result.is_error,
|
||||
)
|
||||
|
||||
|
||||
@app.get("/agent/sessions/{session_id}/history")
|
||||
async def session_history(
|
||||
session_id: str, _user: dict = Depends(auth.require_user)
|
||||
) -> JSONResponse:
|
||||
"""Replay a session's JSONL from ~/.claude/projects/.../<id>.jsonl.
|
||||
|
||||
Returns a flat array of {role, text, timestamp} for the chat window.
|
||||
Returns an empty array if the session file doesn't exist yet.
|
||||
"""
|
||||
# UUID sanity check to prevent path traversal — claude Code uses uuid4
|
||||
try:
|
||||
uuid.UUID(session_id)
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail="invalid session_id")
|
||||
|
||||
path = _sessions_dir() / f"{session_id}.jsonl"
|
||||
if not path.is_file():
|
||||
return JSONResponse({"messages": []})
|
||||
|
||||
messages: list[dict[str, Any]] = []
|
||||
try:
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
# Claude Code records turns with type=user / type=assistant.
|
||||
# Tool-use traffic is verbose; skip it for the chat UI.
|
||||
msg_type = obj.get("type")
|
||||
if msg_type not in ("user", "assistant"):
|
||||
continue
|
||||
msg = obj.get("message") or {}
|
||||
content = msg.get("content")
|
||||
# `content` may be a string or list[{type,text}].
|
||||
if isinstance(content, str):
|
||||
text = content
|
||||
elif isinstance(content, list):
|
||||
text = "".join(
|
||||
part.get("text", "")
|
||||
for part in content
|
||||
if isinstance(part, dict) and part.get("type") == "text"
|
||||
)
|
||||
else:
|
||||
text = ""
|
||||
if not text:
|
||||
continue
|
||||
messages.append(
|
||||
{
|
||||
"role": msg_type,
|
||||
"text": text,
|
||||
"timestamp": obj.get("timestamp"),
|
||||
}
|
||||
)
|
||||
except OSError as e:
|
||||
logger.warning("failed to read session %s: %s", session_id, e)
|
||||
raise HTTPException(status_code=500, detail="failed to read session")
|
||||
|
||||
return JSONResponse({"messages": messages})
|
||||
|
||||
|
||||
# ─── Entrypoint ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Run via `python -m agent.service` for local testing."""
|
||||
import uvicorn
|
||||
|
||||
uvicorn.run(
|
||||
"agent.service:app",
|
||||
host="127.0.0.1",
|
||||
port=8767,
|
||||
log_level="info",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -1,35 +0,0 @@
|
|||
-- Read-only PG role for the Overlord Agent's `query_telemetry_db` MCP tool.
|
||||
--
|
||||
-- This is the second line of defense (the first is the sqlglot parser in
|
||||
-- agent/tools.py:assert_read_only). Even a parser bypass cannot mutate
|
||||
-- because this role only has SELECT.
|
||||
--
|
||||
-- Apply on the dereth-db container:
|
||||
-- docker exec dereth-db psql -U postgres -d dereth -f - < agent/sql/0001_overlord_agent_ro.sql
|
||||
-- (substitute the password before running, or keep as a placeholder and
|
||||
-- ALTER ROLE … PASSWORD '…' separately)
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'overlord_agent_ro') THEN
|
||||
CREATE ROLE overlord_agent_ro NOINHERIT LOGIN PASSWORD 'change-me-set-via-alter-role';
|
||||
END IF;
|
||||
END$$;
|
||||
|
||||
GRANT CONNECT ON DATABASE dereth TO overlord_agent_ro;
|
||||
GRANT USAGE ON SCHEMA public TO overlord_agent_ro;
|
||||
|
||||
-- Grant SELECT on all current public tables.
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA public TO overlord_agent_ro;
|
||||
GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO overlord_agent_ro;
|
||||
|
||||
-- And on any future tables created in public.
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA public
|
||||
GRANT SELECT ON TABLES TO overlord_agent_ro;
|
||||
|
||||
-- TimescaleDB-internal schema (chunks live here). Read on hypertable chunks
|
||||
-- requires SELECT on _timescaledb_internal too.
|
||||
GRANT USAGE ON SCHEMA _timescaledb_internal TO overlord_agent_ro;
|
||||
GRANT SELECT ON ALL TABLES IN SCHEMA _timescaledb_internal TO overlord_agent_ro;
|
||||
ALTER DEFAULT PRIVILEGES IN SCHEMA _timescaledb_internal
|
||||
GRANT SELECT ON TABLES TO overlord_agent_ro;
|
||||
451
agent/tools.py
|
|
@ -1,451 +0,0 @@
|
|||
"""Tool implementations exposed to Claude via the MCP server.
|
||||
|
||||
These are pure functions — the MCP server (mcp_overlord.py) only handles
|
||||
the protocol wrapping. Keep tool logic here so it's easy to test in
|
||||
isolation and reuse from elsewhere (e.g. /agent/ask shortcuts).
|
||||
|
||||
Two flavors of data access:
|
||||
* HTTP loopback to the dereth-tracker container (for endpoints that
|
||||
already exist and have validated logic).
|
||||
* Direct asyncpg to the read-only PG role for ad-hoc queries
|
||||
(rare_events, telemetry, anything not exposed via HTTP).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from typing import Any
|
||||
from urllib.parse import quote
|
||||
|
||||
import asyncpg
|
||||
import httpx
|
||||
import sqlglot
|
||||
import sqlglot.errors
|
||||
import sqlglot.expressions as exp
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# The dereth-tracker FastAPI app, reachable from the host because Docker
|
||||
# port-forwards 127.0.0.1:8765:8765 in docker-compose.yml.
|
||||
TRACKER_URL = os.getenv("TRACKER_URL", "http://127.0.0.1:8765")
|
||||
|
||||
# Read-only PG role; see deployment plan.
|
||||
DB_DSN = os.getenv(
|
||||
"AGENT_DB_DSN",
|
||||
"postgresql://overlord_agent_ro@127.0.0.1:5432/dereth",
|
||||
)
|
||||
|
||||
# Hard caps for the SQL tool to keep the agent honest.
|
||||
SQL_TIMEOUT_S = float(os.getenv("AGENT_SQL_TIMEOUT_S", "10"))
|
||||
SQL_MAX_ROWS = int(os.getenv("AGENT_SQL_MAX_ROWS", "200"))
|
||||
|
||||
|
||||
# ─── HTTP loopback helpers ──────────────────────────────────────────
|
||||
|
||||
|
||||
_http_client: httpx.AsyncClient | None = None
|
||||
|
||||
|
||||
async def _http() -> httpx.AsyncClient:
|
||||
"""Lazily create + reuse a single httpx client (connection pool)."""
|
||||
global _http_client
|
||||
if _http_client is None:
|
||||
_http_client = httpx.AsyncClient(base_url=TRACKER_URL, timeout=30.0)
|
||||
return _http_client
|
||||
|
||||
|
||||
async def _get_json(path: str) -> Any:
|
||||
client = await _http()
|
||||
resp = await client.get(path)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
# ─── DB helpers ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
_db_pool: asyncpg.Pool | None = None
|
||||
|
||||
|
||||
async def _db() -> asyncpg.Pool:
|
||||
global _db_pool
|
||||
if _db_pool is None:
|
||||
_db_pool = await asyncpg.create_pool(
|
||||
DB_DSN, min_size=1, max_size=4, command_timeout=SQL_TIMEOUT_S
|
||||
)
|
||||
return _db_pool
|
||||
|
||||
|
||||
# ─── SQL safety ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
_ALLOWED_TOPLEVEL = tuple(
|
||||
cls for cls in (
|
||||
getattr(exp, "Select", None),
|
||||
getattr(exp, "With", None),
|
||||
getattr(exp, "Union", None),
|
||||
getattr(exp, "Subquery", None),
|
||||
getattr(exp, "Intersect", None),
|
||||
getattr(exp, "Except", None),
|
||||
)
|
||||
if cls is not None
|
||||
)
|
||||
|
||||
|
||||
class SqlNotAllowed(ValueError):
|
||||
"""Raised when the agent attempts a non-read-only SQL statement."""
|
||||
|
||||
|
||||
def assert_read_only(sql: str) -> None:
|
||||
"""Parse `sql` and reject anything that isn't a read query.
|
||||
|
||||
Belt-and-suspenders: the PG role is also read-only (GRANT SELECT only),
|
||||
so even a parser bypass can't actually mutate. This is the first line
|
||||
of defense — friendlier error messages and faster reject.
|
||||
"""
|
||||
try:
|
||||
statements = sqlglot.parse(sql, read="postgres")
|
||||
except sqlglot.errors.ParseError as e:
|
||||
raise SqlNotAllowed(f"SQL parse error: {e}") from e
|
||||
|
||||
if not statements:
|
||||
raise SqlNotAllowed("empty SQL")
|
||||
if len(statements) > 1:
|
||||
raise SqlNotAllowed("only one statement allowed")
|
||||
|
||||
stmt = statements[0]
|
||||
if stmt is None:
|
||||
raise SqlNotAllowed("empty parse result")
|
||||
if not isinstance(stmt, _ALLOWED_TOPLEVEL):
|
||||
raise SqlNotAllowed(
|
||||
f"only SELECT / WITH allowed, got {type(stmt).__name__}"
|
||||
)
|
||||
|
||||
# Walk the tree and reject any DML/DDL hidden inside (e.g. CTE with
|
||||
# INSERT — yes, postgres allows that). Use getattr so version drift
|
||||
# in sqlglot (renamed classes like AlterTable→Alter) doesn't crash
|
||||
# the whole tool.
|
||||
_DENY_NAMES = (
|
||||
"Insert", "Update", "Delete", "Drop", "Create", "Merge",
|
||||
"Alter", "AlterTable", "AlterColumn", "AlterDatabase",
|
||||
"Truncate", "TruncateTable",
|
||||
"Grant", "Revoke",
|
||||
"Copy", # PostgreSQL COPY can write files
|
||||
)
|
||||
deny_classes = tuple(
|
||||
cls for cls in (getattr(exp, name, None) for name in _DENY_NAMES)
|
||||
if cls is not None
|
||||
)
|
||||
for node in stmt.walk():
|
||||
# walk() returns the node, then in some sqlglot versions a tuple of
|
||||
# (node, parent, key). Normalize.
|
||||
actual = node[0] if isinstance(node, tuple) else node
|
||||
if isinstance(actual, deny_classes):
|
||||
raise SqlNotAllowed(
|
||||
f"writes/DDL not allowed (found {type(actual).__name__})"
|
||||
)
|
||||
|
||||
|
||||
# ─── Tools ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def get_live_players() -> dict[str, Any]:
|
||||
"""Active characters (telemetry seen in the last ~30s).
|
||||
|
||||
Returns the same shape as `GET /live`:
|
||||
{ "players": [ { character_name, ew, ns, z, kills, ... } ] }
|
||||
"""
|
||||
return await _get_json("/live")
|
||||
|
||||
|
||||
async def get_recent_rares(hours: int = 24, limit: int = 100) -> dict[str, Any]:
|
||||
"""Rare item finds in the last N hours, newest first."""
|
||||
hours = max(1, min(int(hours), 24 * 30)) # cap at 30 days
|
||||
limit = max(1, min(int(limit), SQL_MAX_ROWS))
|
||||
pool = await _db()
|
||||
rows = await pool.fetch(
|
||||
"""
|
||||
SELECT timestamp, character_name, name, ew, ns, z
|
||||
FROM rare_events
|
||||
WHERE timestamp >= NOW() - ($1::int || ' hours')::interval
|
||||
ORDER BY timestamp DESC
|
||||
LIMIT $2
|
||||
""",
|
||||
hours,
|
||||
limit,
|
||||
)
|
||||
return {
|
||||
"hours": hours,
|
||||
"count": len(rows),
|
||||
"rares": [
|
||||
{
|
||||
"timestamp": r["timestamp"].isoformat(),
|
||||
"character_name": r["character_name"],
|
||||
"name": r["name"],
|
||||
"ew": r["ew"],
|
||||
"ns": r["ns"],
|
||||
"z": r["z"],
|
||||
}
|
||||
for r in rows
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
async def query_telemetry_db(sql: str) -> dict[str, Any]:
|
||||
"""Run a read-only SQL statement against the telemetry DB.
|
||||
|
||||
The query is parsed and any non-SELECT/WITH statement is rejected.
|
||||
The connection role is also GRANT SELECT only (defense in depth).
|
||||
|
||||
Useful for ad-hoc questions: "top 5 KPH today", "kill count by character
|
||||
yesterday", etc.
|
||||
"""
|
||||
assert_read_only(sql)
|
||||
pool = await _db()
|
||||
try:
|
||||
rows = await asyncio.wait_for(pool.fetch(sql), timeout=SQL_TIMEOUT_S)
|
||||
except asyncio.TimeoutError:
|
||||
raise SqlNotAllowed(f"query exceeded {SQL_TIMEOUT_S:.0f}s timeout")
|
||||
|
||||
if len(rows) > SQL_MAX_ROWS:
|
||||
rows = rows[:SQL_MAX_ROWS]
|
||||
truncated = True
|
||||
else:
|
||||
truncated = False
|
||||
|
||||
return {
|
||||
"row_count": len(rows),
|
||||
"truncated": truncated,
|
||||
"rows": [
|
||||
{k: _json_safe(v) for k, v in dict(r).items()} for r in rows
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _json_safe(v: Any) -> Any:
|
||||
"""Convert datetime / Decimal / etc. to JSON-friendly types."""
|
||||
from datetime import date, datetime, timedelta
|
||||
from decimal import Decimal
|
||||
|
||||
if v is None:
|
||||
return None
|
||||
if isinstance(v, (str, int, float, bool)):
|
||||
return v
|
||||
if isinstance(v, (datetime, date)):
|
||||
return v.isoformat()
|
||||
if isinstance(v, timedelta):
|
||||
return v.total_seconds()
|
||||
if isinstance(v, Decimal):
|
||||
return float(v)
|
||||
if isinstance(v, (list, tuple)):
|
||||
return [_json_safe(x) for x in v]
|
||||
if isinstance(v, dict):
|
||||
return {k: _json_safe(x) for k, x in v.items()}
|
||||
return str(v)
|
||||
|
||||
|
||||
# ─── Per-character lookups (HTTP loopback) ──────────────────────────
|
||||
|
||||
|
||||
async def get_player_state(character_name: str) -> dict[str, Any]:
|
||||
"""Combined snapshot for one character: live telemetry + character stats.
|
||||
|
||||
Returns:
|
||||
{
|
||||
"character_name": str,
|
||||
"telemetry": {...} | None, # from /live, or None if offline
|
||||
"character_stats": {...} | None, # from /character-stats/<name>
|
||||
"vitals": {...} | None, # last vitals from /live (subset)
|
||||
"online": bool, # whether telemetry was found in /live
|
||||
}
|
||||
"""
|
||||
name = character_name.strip()
|
||||
live = await _get_json("/live")
|
||||
players = live.get("players", []) if isinstance(live, dict) else []
|
||||
telemetry = next(
|
||||
(p for p in players if p.get("character_name") == name), None
|
||||
)
|
||||
|
||||
char_stats: dict[str, Any] | None = None
|
||||
try:
|
||||
client = await _http()
|
||||
resp = await client.get(f"/character-stats/{quote(name, safe='')}")
|
||||
if resp.status_code == 200:
|
||||
char_stats = resp.json()
|
||||
except Exception:
|
||||
char_stats = None
|
||||
|
||||
return {
|
||||
"character_name": name,
|
||||
"online": telemetry is not None,
|
||||
"telemetry": telemetry,
|
||||
"character_stats": char_stats,
|
||||
}
|
||||
|
||||
|
||||
async def get_inventory(character_name: str) -> dict[str, Any]:
|
||||
"""Full inventory for one character. Items only — for filtered queries
|
||||
use get_inventory_search."""
|
||||
client = await _http()
|
||||
resp = await client.get(f"/inventory/{quote(character_name, safe='')}")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def get_inventory_search(
|
||||
character_name: str, filters: dict[str, Any] | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""Filtered inventory search. `filters` is a dict of query params, e.g.
|
||||
{"name": "pearl", "armor_level_min": 500}.
|
||||
|
||||
Caller is expected to know the supported filters from the dereth-tracker
|
||||
/inventory/{name}/search route — pass through opaquely.
|
||||
"""
|
||||
client = await _http()
|
||||
resp = await client.get(
|
||||
f"/inventory/{quote(character_name, safe='')}/search",
|
||||
params=filters or {},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def search_items_global(filters: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Cross-character item search via the inventory service's /search/items.
|
||||
|
||||
Use this INSTEAD of looping per-character when the user asks "find an X
|
||||
on any of my chars" — one DB query vs. 60+ HTTP roundtrips.
|
||||
|
||||
Common filter keys (passed straight through as query params):
|
||||
include_all_characters: bool (set true to search every char)
|
||||
character: str (single char) | characters: "A,B,C"
|
||||
text: str (name/description substring)
|
||||
has_spell: "Legendary Acid Ward" — exact spell name
|
||||
spell_contains: "Legendary" — substring match
|
||||
legendary_cantrips: "Foo,Bar"
|
||||
equipment_status: "equipped" | "unequipped"
|
||||
equipment_slot: int (bitmask: 4=chest, 2048=bracelet, 4096=ring, ...)
|
||||
slot_names: "Bracelet,Ring"
|
||||
armor_only / jewelry_only / weapon_only: bool
|
||||
min_armor / max_armor / min_damage / max_damage: int
|
||||
...and many more — see /search/items endpoint docs.
|
||||
"""
|
||||
client = await _http()
|
||||
# Default to all-character search if caller didn't scope; otherwise the
|
||||
# endpoint refuses with a 400.
|
||||
params = dict(filters or {})
|
||||
if not any(
|
||||
k in params
|
||||
for k in ("character", "characters", "include_all_characters")
|
||||
):
|
||||
params["include_all_characters"] = True
|
||||
resp = await client.get("/search/items", params=params)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def get_combat_stats(character_name: str) -> dict[str, Any]:
|
||||
"""Lifetime + session combat stats for one character (per-element split,
|
||||
monster encounters, surge counts)."""
|
||||
client = await _http()
|
||||
resp = await client.get(f"/combat-stats/{quote(character_name, safe='')}")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def get_equipment_cantrips(character_name: str) -> dict[str, Any]:
|
||||
"""Currently-equipped items + their active cantrip/spell state."""
|
||||
client = await _http()
|
||||
resp = await client.get(
|
||||
f"/equipment-cantrip-state/{quote(character_name, safe='')}"
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def get_quest_status() -> dict[str, Any]:
|
||||
"""All characters' active quest timers and progress."""
|
||||
return await _get_json("/quest-status")
|
||||
|
||||
|
||||
async def get_server_health() -> dict[str, Any]:
|
||||
"""Coldeve server status: up/down, latency, current player count, uptime."""
|
||||
return await _get_json("/server-health")
|
||||
|
||||
|
||||
async def suitbuilder_search(
|
||||
params: dict[str, Any], max_phase_events: int = 50
|
||||
) -> dict[str, Any]:
|
||||
"""Drive a suitbuilder constraint search synchronously.
|
||||
|
||||
The dereth-tracker /inv/suitbuilder/search endpoint is an SSE stream.
|
||||
We collect events until the stream closes, drop intermediate phase
|
||||
chatter (keeping the last N), and return:
|
||||
|
||||
{ "final_suits": [...], "phases": [...latest few...] }
|
||||
|
||||
`params` is the JSON body the suitbuilder expects. Call it like the
|
||||
/suitbuilder.html page does.
|
||||
"""
|
||||
client = await _http()
|
||||
final: list[dict[str, Any]] = []
|
||||
phases: list[dict[str, Any]] = []
|
||||
|
||||
# Use a fresh long-timeout client for the SSE stream — don't tie up the
|
||||
# shared pool for a 5-minute search.
|
||||
async with httpx.AsyncClient(
|
||||
base_url=TRACKER_URL, timeout=httpx.Timeout(300.0, connect=10.0)
|
||||
) as stream_client:
|
||||
async with stream_client.stream(
|
||||
"POST",
|
||||
"/inv/suitbuilder/search",
|
||||
json=params,
|
||||
headers={"Content-Type": "application/json"},
|
||||
) as resp:
|
||||
event_name = "message"
|
||||
data_lines: list[str] = []
|
||||
async for line_bytes in resp.aiter_lines():
|
||||
line = line_bytes.rstrip("\r")
|
||||
if line.startswith("event:"):
|
||||
event_name = line[6:].strip()
|
||||
elif line.startswith("data:"):
|
||||
data_lines.append(line[5:].strip())
|
||||
elif line == "":
|
||||
# Dispatch
|
||||
if data_lines:
|
||||
try:
|
||||
payload = json.loads("\n".join(data_lines))
|
||||
except json.JSONDecodeError:
|
||||
payload = {"raw": "\n".join(data_lines)}
|
||||
if event_name == "result" or event_name == "final":
|
||||
final.append(payload)
|
||||
elif event_name == "error":
|
||||
phases.append({"event": "error", "data": payload})
|
||||
else:
|
||||
phases.append({"event": event_name, "data": payload})
|
||||
phases = phases[-max_phase_events:]
|
||||
data_lines = []
|
||||
event_name = "message"
|
||||
|
||||
return {
|
||||
"final_suits": final,
|
||||
"phases": phases[-max_phase_events:],
|
||||
"phase_count": len(phases),
|
||||
}
|
||||
|
||||
|
||||
# ─── Cleanup ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def shutdown() -> None:
|
||||
"""Close shared resources. Call from MCP server lifespan / on exit."""
|
||||
global _http_client, _db_pool
|
||||
if _http_client is not None:
|
||||
await _http_client.aclose()
|
||||
_http_client = None
|
||||
if _db_pool is not None:
|
||||
await _db_pool.close()
|
||||
_db_pool = None
|
||||
39
alembic.ini
|
|
@ -1,39 +0,0 @@
|
|||
; Alembic configuration file for database migrations
|
||||
[alembic]
|
||||
; Path to migration scripts directory
|
||||
script_location = alembic
|
||||
; Default SQLAlchemy URL for migrations (use DATABASE_URL env var to override)
|
||||
sqlalchemy.url = postgresql://postgres:password@localhost:5432/dereth
|
||||
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARN
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARN
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
level = NOTSET
|
||||
args = (sys.stderr,)
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
|
|
@ -1,64 +0,0 @@
|
|||
"""Alembic environment configuration for database migrations.
|
||||
|
||||
Configures offline and online migration contexts using SQLAlchemy
|
||||
and the target metadata defined in db_async.metadata.
|
||||
"""
|
||||
import os
|
||||
from logging.config import fileConfig
|
||||
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
from alembic import context
|
||||
|
||||
# Alembic Config object provides access to values in the .ini file
|
||||
config = context.config
|
||||
|
||||
# Override sqlalchemy.url with DATABASE_URL environment variable if provided
|
||||
database_url = os.getenv('DATABASE_URL', config.get_main_option('sqlalchemy.url'))
|
||||
config.set_main_option('sqlalchemy.url', database_url)
|
||||
|
||||
# Set up Python logging according to config file
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
# add your model's MetaData object here
|
||||
# for 'autogenerate' support
|
||||
from db_async import metadata # noqa
|
||||
target_metadata = metadata
|
||||
|
||||
|
||||
def run_migrations_offline():
|
||||
"""Run migrations in 'offline' mode using literal SQL script generation."""
|
||||
url = config.get_main_option('sqlalchemy.url')
|
||||
context.configure(
|
||||
url=url,
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def run_migrations_online():
|
||||
"""Run migrations in 'online' mode against a live database connection."""
|
||||
connectable = engine_from_config(
|
||||
config.get_section(config.config_ini_section),
|
||||
prefix='sqlalchemy.',
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
|
||||
with connectable.connect() as connection:
|
||||
context.configure(
|
||||
connection=connection,
|
||||
target_metadata=target_metadata,
|
||||
)
|
||||
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
<%#
|
||||
Alembic migration script template generated by 'alembic revision'.
|
||||
Edit the upgrade() and downgrade() functions to apply schema changes.
|
||||
%>
|
||||
"""
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = '${up_revision}'
|
||||
down_revision = ${repr(down_revision) if down_revision else None}
|
||||
branch_labels = ${repr(branch_labels) if branch_labels else None}
|
||||
depends_on = ${repr(depends_on) if depends_on else None}
|
||||
|
||||
def upgrade():
|
||||
"""Upgrade migrations go here."""
|
||||
pass
|
||||
|
||||
def downgrade():
|
||||
"""Downgrade migrations go here."""
|
||||
pass
|
||||
|
|
@ -1,5 +0,0 @@
|
|||
"""
|
||||
This directory will hold Alembic migration scripts.
|
||||
Each migration filename should follow the naming convention:
|
||||
<revision_id>_<slug>.py
|
||||
"""
|
||||
44
db.py
|
|
@ -1,20 +1,12 @@
|
|||
"""SQLite3 helper module for local telemetry storage.
|
||||
|
||||
Provides functions to initialize the local database schema and save
|
||||
telemetry snapshots into history and live_state tables.
|
||||
Enforces WAL mode, size limits, and auto-vacuum for efficient storage.
|
||||
"""
|
||||
import os
|
||||
import sqlite3
|
||||
from typing import Dict
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
# Local SQLite database file name (used when running without TimescaleDB)
|
||||
DB_FILE = "dereth.db"
|
||||
# Maximum allowed database size (in MB). Defaults to 2048 (2GB). Override via env DB_MAX_SIZE_MB.
|
||||
MAX_DB_SIZE_MB = int(os.getenv("DB_MAX_SIZE_MB", "2048"))
|
||||
# Retention window for telemetry history in days (currently not auto-enforced).
|
||||
# Override via env DB_RETENTION_DAYS for future cleanup scripts.
|
||||
# Retention window for telemetry history in days. Override via env DB_RETENTION_DAYS.
|
||||
MAX_RETENTION_DAYS = int(os.getenv("DB_RETENTION_DAYS", "7"))
|
||||
# SQLite runtime limits customization
|
||||
DB_MAX_SQL_LENGTH = int(os.getenv("DB_MAX_SQL_LENGTH", "1000000000"))
|
||||
|
|
@ -24,15 +16,8 @@ DB_WAL_AUTOCHECKPOINT_PAGES = int(os.getenv("DB_WAL_AUTOCHECKPOINT_PAGES", "1000
|
|||
|
||||
|
||||
def init_db() -> None:
|
||||
"""
|
||||
Initialize local SQLite database schema for telemetry logging.
|
||||
|
||||
- Applies SQLite PRAGMA settings for performance and file size management
|
||||
- Ensures WAL journaling and auto-vacuum for concurrency and compaction
|
||||
- Creates telemetry_log for full history and live_state for latest snapshot per character
|
||||
"""
|
||||
"""Create tables if they do not exist (extended with kills_per_hour and onlinetime)."""
|
||||
# Open connection with a longer timeout
|
||||
# Open connection with extended timeout for schema operations
|
||||
conn = sqlite3.connect(DB_FILE, timeout=30)
|
||||
# Bump SQLite runtime limits
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, DB_MAX_SQL_LENGTH)
|
||||
|
|
@ -40,20 +25,16 @@ def init_db() -> None:
|
|||
conn.setlimit(sqlite3.SQLITE_LIMIT_VARIABLE_NUMBER, DB_MAX_SQL_VARIABLES)
|
||||
c = conn.cursor()
|
||||
# Enable auto_vacuum FULL and rebuild DB so that deletions shrink the file
|
||||
# Enable full auto-vacuum to shrink database file on deletes
|
||||
c.execute("PRAGMA auto_vacuum=FULL;")
|
||||
conn.commit()
|
||||
# Rebuild database to apply auto_vacuum changes
|
||||
c.execute("VACUUM;")
|
||||
conn.commit()
|
||||
# Switch to WAL mode for concurrency, adjust checkpointing, and enforce max size
|
||||
# Configure write-ahead logging for concurrency and performance
|
||||
c.execute("PRAGMA journal_mode=WAL")
|
||||
c.execute("PRAGMA synchronous=NORMAL")
|
||||
# Auto-checkpoint after specified WAL frames to limit WAL file size
|
||||
c.execute(f"PRAGMA wal_autocheckpoint={DB_WAL_AUTOCHECKPOINT_PAGES}")
|
||||
|
||||
# Create history log table for all telemetry snapshots
|
||||
# History log
|
||||
c.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS telemetry_log (
|
||||
|
|
@ -76,7 +57,7 @@ def init_db() -> None:
|
|||
"""
|
||||
)
|
||||
|
||||
# Create live_state table for upserts of the most recent snapshot per character
|
||||
# Live snapshot (upsert)
|
||||
c.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS live_state (
|
||||
|
|
@ -103,27 +84,20 @@ def init_db() -> None:
|
|||
|
||||
|
||||
def save_snapshot(data: Dict) -> None:
|
||||
"""
|
||||
Save a telemetry snapshot into the local SQLite database.
|
||||
|
||||
Inserts a full record into telemetry_log (history) and upserts into live_state
|
||||
for quick lookup of the most recent data per character.
|
||||
|
||||
Respects WAL mode and checkpoint settings on each connection.
|
||||
"""
|
||||
# Open new connection with extended timeout for inserting data
|
||||
"""Insert snapshot into history and upsert into live_state (with new fields)."""
|
||||
# Open connection with a longer busy timeout
|
||||
conn = sqlite3.connect(DB_FILE, timeout=30)
|
||||
# Bump SQLite runtime limits on this connection
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, DB_MAX_SQL_LENGTH)
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_SQL_LENGTH, DB_MAX_SQL_LENGTH)
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_VARIABLE_NUMBER, DB_MAX_SQL_VARIABLES)
|
||||
c = conn.cursor()
|
||||
# Ensure WAL mode and checkpointing settings on this connection
|
||||
# Ensure WAL mode, checkpointing, and size limit on this connection
|
||||
c.execute("PRAGMA journal_mode=WAL")
|
||||
c.execute("PRAGMA synchronous=NORMAL")
|
||||
c.execute(f"PRAGMA wal_autocheckpoint={DB_WAL_AUTOCHECKPOINT_PAGES}")
|
||||
|
||||
# Insert the snapshot into the telemetry_log (history) table
|
||||
# Insert full history row
|
||||
c.execute(
|
||||
"""
|
||||
INSERT INTO telemetry_log (
|
||||
|
|
@ -151,7 +125,7 @@ def save_snapshot(data: Dict) -> None:
|
|||
),
|
||||
)
|
||||
|
||||
# Upsert (insert or update) the latest snapshot into live_state table
|
||||
# Upsert into live_state
|
||||
c.execute(
|
||||
"""
|
||||
INSERT INTO live_state (
|
||||
|
|
|
|||
401
db_async.py
|
|
@ -1,401 +0,0 @@
|
|||
"""Asynchronous database layer for telemetry service using PostgreSQL/TimescaleDB.
|
||||
|
||||
Defines table schemas via SQLAlchemy Core and provides an
|
||||
initialization function to set up TimescaleDB hypertable.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sqlalchemy
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from databases import Database
|
||||
from sqlalchemy import MetaData, Table, Column, Integer, String, Float, DateTime, text
|
||||
from sqlalchemy import Index, BigInteger, JSON, Boolean, UniqueConstraint
|
||||
from sqlalchemy.sql import func
|
||||
import bcrypt as _bcrypt
|
||||
|
||||
# Environment: Postgres/TimescaleDB connection URL
|
||||
DATABASE_URL = os.getenv(
|
||||
"DATABASE_URL", "postgresql://postgres:password@localhost:5432/dereth"
|
||||
)
|
||||
# Async database client with explicit connection pool configuration and query timeout
|
||||
database = Database(DATABASE_URL, min_size=5, max_size=100, command_timeout=120)
|
||||
# Metadata for SQLAlchemy Core
|
||||
# SQLAlchemy metadata container for table definitions
|
||||
metadata = MetaData()
|
||||
|
||||
# --- Table Definitions ---
|
||||
# Table for storing raw telemetry snapshots at scale (converted to hypertable)
|
||||
telemetry_events = Table(
|
||||
# Time-series hypertable storing raw telemetry snapshots from plugins
|
||||
"telemetry_events",
|
||||
metadata,
|
||||
Column("character_name", String, nullable=False, index=True),
|
||||
Column("char_tag", String, nullable=True),
|
||||
Column("session_id", String, nullable=False, index=True),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False, index=True),
|
||||
Column("ew", Float, nullable=False),
|
||||
Column("ns", Float, nullable=False),
|
||||
Column("z", Float, nullable=False),
|
||||
Column("kills", Integer, nullable=False),
|
||||
Column("kills_per_hour", Float, nullable=True),
|
||||
Column("onlinetime", String, nullable=True),
|
||||
Column("deaths", Integer, nullable=False),
|
||||
Column("total_deaths", Integer, nullable=True),
|
||||
Column("rares_found", Integer, nullable=False),
|
||||
Column("prismatic_taper_count", Integer, nullable=False),
|
||||
Column("vt_state", String, nullable=True),
|
||||
# New telemetry metrics
|
||||
Column("mem_mb", Float, nullable=True),
|
||||
Column("cpu_pct", Float, nullable=True),
|
||||
Column("mem_handles", Integer, nullable=True),
|
||||
Column("latency_ms", Float, nullable=True),
|
||||
)
|
||||
# Composite index to accelerate Grafana queries filtering by character_name then ordering by timestamp
|
||||
Index(
|
||||
"ix_telemetry_events_char_ts",
|
||||
telemetry_events.c.character_name,
|
||||
telemetry_events.c.timestamp,
|
||||
)
|
||||
|
||||
# Table for persistent total kills per character
|
||||
char_stats = Table(
|
||||
# Stores cumulative kills per character in a single-row upsert table
|
||||
"char_stats",
|
||||
metadata,
|
||||
Column("character_name", String, primary_key=True),
|
||||
Column("total_kills", Integer, nullable=False, default=0),
|
||||
)
|
||||
|
||||
# Table for persistent total rare counts per character
|
||||
rare_stats = Table(
|
||||
# Stores cumulative rare event counts per character
|
||||
"rare_stats",
|
||||
metadata,
|
||||
Column("character_name", String, primary_key=True),
|
||||
Column("total_rares", Integer, nullable=False, default=0),
|
||||
)
|
||||
|
||||
rare_stats_sessions = Table(
|
||||
# Stores per-session rare counts; composite PK (character_name, session_id)
|
||||
"rare_stats_sessions",
|
||||
metadata,
|
||||
Column("character_name", String, primary_key=True),
|
||||
Column("session_id", String, primary_key=True),
|
||||
Column("session_rares", Integer, nullable=False, default=0),
|
||||
)
|
||||
# Per-character persistent combat stats (lifetime accumulation, Mag-Tools style)
|
||||
combat_stats = Table(
|
||||
"combat_stats",
|
||||
metadata,
|
||||
Column("character_name", String, primary_key=True),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False),
|
||||
Column("stats_data", JSON, nullable=False),
|
||||
)
|
||||
|
||||
# Per-session combat stats snapshots (session history)
|
||||
combat_stats_sessions = Table(
|
||||
"combat_stats_sessions",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("character_name", String, nullable=False, index=True),
|
||||
Column("session_id", String, nullable=False, index=True),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False, index=True),
|
||||
Column("stats_data", JSON, nullable=False),
|
||||
)
|
||||
|
||||
# Table for recording spawn events (mob creates) for heatmap analysis
|
||||
spawn_events = Table(
|
||||
# Records individual mob spawn occurrences for heatmap and analysis
|
||||
"spawn_events",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("character_name", String, nullable=False),
|
||||
Column("mob", String, nullable=False),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False, index=True),
|
||||
Column("ew", Float, nullable=False),
|
||||
Column("ns", Float, nullable=False),
|
||||
Column("z", Float, nullable=False),
|
||||
)
|
||||
# Table for recording individual rare spawn events for analysis
|
||||
rare_events = Table(
|
||||
# Records individual rare mob events for detailed analysis and heatmaps
|
||||
"rare_events",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("character_name", String, nullable=False),
|
||||
Column("name", String, nullable=False),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False, index=True),
|
||||
Column("ew", Float, nullable=False),
|
||||
Column("ns", Float, nullable=False),
|
||||
Column("z", Float, nullable=False),
|
||||
)
|
||||
|
||||
character_inventories = Table(
|
||||
# Stores complete character inventory snapshots with searchable fields
|
||||
"character_inventories",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("character_name", String, nullable=False, index=True),
|
||||
Column("item_id", BigInteger, nullable=False),
|
||||
Column("timestamp", DateTime(timezone=True), nullable=False),
|
||||
# Extracted searchable fields
|
||||
Column("name", String),
|
||||
Column("icon", Integer),
|
||||
Column("object_class", Integer, index=True),
|
||||
Column("value", Integer, index=True),
|
||||
Column("burden", Integer),
|
||||
Column("has_id_data", Boolean),
|
||||
# Complete item data as JSONB
|
||||
Column("item_data", JSON, nullable=False),
|
||||
# Unique constraint to prevent duplicate items per character
|
||||
UniqueConstraint("character_name", "item_id", name="uq_char_item"),
|
||||
)
|
||||
|
||||
# Portals table with coordinate-based uniqueness and 1-hour retention
|
||||
portals = Table(
|
||||
# Stores unique portals by coordinates with 1-hour retention
|
||||
"portals",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("portal_name", String, nullable=False),
|
||||
Column("ns", Float, nullable=False),
|
||||
Column("ew", Float, nullable=False),
|
||||
Column("z", Float, nullable=False),
|
||||
Column("discovered_at", DateTime(timezone=True), nullable=False, index=True),
|
||||
Column("discovered_by", String, nullable=False),
|
||||
)
|
||||
|
||||
# Server health monitoring: only current state is kept.
|
||||
# Historical health checks were removed — nothing read from them.
|
||||
server_status = Table(
|
||||
# Current server status and uptime tracking
|
||||
"server_status",
|
||||
metadata,
|
||||
Column("server_name", String, primary_key=True),
|
||||
Column("current_status", String(10), nullable=False),
|
||||
Column("last_seen_up", DateTime(timezone=True), nullable=True),
|
||||
Column("last_restart", DateTime(timezone=True), nullable=True),
|
||||
Column("total_uptime_seconds", BigInteger, default=0),
|
||||
Column("last_check", DateTime(timezone=True), nullable=True),
|
||||
Column("last_latency_ms", Float, nullable=True),
|
||||
Column("last_player_count", Integer, nullable=True),
|
||||
)
|
||||
|
||||
character_stats = Table(
|
||||
"character_stats",
|
||||
metadata,
|
||||
Column("character_name", String, primary_key=True, nullable=False),
|
||||
Column(
|
||||
"timestamp", DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||
),
|
||||
Column("level", Integer, nullable=True),
|
||||
Column("total_xp", BigInteger, nullable=True),
|
||||
Column("unassigned_xp", BigInteger, nullable=True),
|
||||
Column("luminance_earned", BigInteger, nullable=True),
|
||||
Column("luminance_total", BigInteger, nullable=True),
|
||||
Column("deaths", Integer, nullable=True),
|
||||
Column("stats_data", JSON, nullable=False),
|
||||
)
|
||||
|
||||
# User accounts for app-level authentication
|
||||
users = Table(
|
||||
"users",
|
||||
metadata,
|
||||
Column("id", Integer, primary_key=True),
|
||||
Column("username", String, nullable=False, unique=True),
|
||||
Column("password_hash", String, nullable=False),
|
||||
Column("is_admin", Boolean, nullable=False, default=False),
|
||||
Column(
|
||||
"created_at", DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
async def init_db_async():
|
||||
"""Initialize PostgreSQL/TimescaleDB schema and hypertable.
|
||||
|
||||
Creates all defined tables and ensures the TimescaleDB extension is
|
||||
installed. Converts telemetry_events table into a hypertable for efficient
|
||||
time-series data storage.
|
||||
"""
|
||||
# Create tables in Postgres
|
||||
engine = sqlalchemy.create_engine(DATABASE_URL)
|
||||
# Reflects metadata definitions into actual database tables via SQLAlchemy
|
||||
metadata.create_all(engine)
|
||||
# Ensure TimescaleDB extension is installed and telemetry_events is a hypertable
|
||||
# Run DDL in autocommit mode so errors don't abort subsequent statements
|
||||
try:
|
||||
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
||||
# Install extension if missing
|
||||
try:
|
||||
conn.execute(text("CREATE EXTENSION IF NOT EXISTS timescaledb"))
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to create extension timescaledb: {e}")
|
||||
# Convert to hypertable, migrating existing data and skipping default index creation
|
||||
try:
|
||||
conn.execute(
|
||||
text(
|
||||
"SELECT create_hypertable('telemetry_events', 'timestamp', "
|
||||
"if_not_exists => true, migrate_data => true, create_default_indexes => false)"
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to create hypertable telemetry_events: {e}")
|
||||
except Exception as e:
|
||||
print(f"Warning: timescale extension/hypertable setup failed: {e}")
|
||||
# Ensure composite index exists for efficient time-series queries by character
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS ix_telemetry_events_char_ts "
|
||||
"ON telemetry_events (character_name, timestamp)"
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
print(
|
||||
f"Warning: failed to create composite index ix_telemetry_events_char_ts: {e}"
|
||||
)
|
||||
# Add retention and compression policies on the hypertable
|
||||
try:
|
||||
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
||||
# Retain only recent data (default 7 days or override via DB_RETENTION_DAYS)
|
||||
days = int(os.getenv("DB_RETENTION_DAYS", "7"))
|
||||
conn.execute(
|
||||
text(
|
||||
f"SELECT add_retention_policy('telemetry_events', INTERVAL '{days} days')"
|
||||
)
|
||||
)
|
||||
# Compress chunks older than 1 day
|
||||
conn.execute(
|
||||
text(
|
||||
"SELECT add_compression_policy('telemetry_events', INTERVAL '1 day')"
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to set retention/compression policies: {e}")
|
||||
|
||||
# Ensure spawn_events is a hypertable with a 7-day retention policy.
|
||||
# This is idempotent — if already a hypertable, create_hypertable is a no-op
|
||||
# when if_not_exists=TRUE. The existing 482M-row table needed a manual
|
||||
# migration (see docs/plans/spawn_events_cleanup.md); this block keeps the
|
||||
# policy alive on subsequent deploys.
|
||||
try:
|
||||
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
||||
# Try to convert spawn_events to a hypertable if it isn't already.
|
||||
# migrate_data=FALSE is safe because the manual migration handled it;
|
||||
# if someone creates a fresh DB, the table is empty and this converts it.
|
||||
conn.execute(
|
||||
text(
|
||||
"SELECT create_hypertable('spawn_events', 'timestamp', "
|
||||
"if_not_exists => TRUE, migrate_data => FALSE, "
|
||||
"chunk_time_interval => INTERVAL '1 day')"
|
||||
)
|
||||
)
|
||||
# 7-day retention
|
||||
conn.execute(
|
||||
text(
|
||||
"SELECT add_retention_policy('spawn_events', INTERVAL '7 days', if_not_exists => TRUE)"
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to set spawn_events hypertable/retention: {e}")
|
||||
|
||||
# Create unique constraint on rounded portal coordinates
|
||||
try:
|
||||
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
||||
# Drop old portal_discoveries table if it exists
|
||||
conn.execute(text("DROP TABLE IF EXISTS portal_discoveries CASCADE"))
|
||||
|
||||
# Create unique constraint on rounded coordinates for the new portals table
|
||||
conn.execute(
|
||||
text(
|
||||
"""CREATE UNIQUE INDEX IF NOT EXISTS unique_portal_coords
|
||||
ON portals (ROUND(ns::numeric, 2), ROUND(ew::numeric, 2))"""
|
||||
)
|
||||
)
|
||||
|
||||
# Create index on coordinates for efficient lookups
|
||||
conn.execute(
|
||||
text(
|
||||
"CREATE INDEX IF NOT EXISTS idx_portals_coords ON portals (ns, ew)"
|
||||
)
|
||||
)
|
||||
|
||||
print("Portal table indexes and constraints created successfully")
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to create portal table constraints: {e}")
|
||||
|
||||
# Ensure character_stats table exists with JSONB column type
|
||||
try:
|
||||
with engine.connect().execution_options(isolation_level="AUTOCOMMIT") as conn:
|
||||
conn.execute(
|
||||
text("""
|
||||
CREATE TABLE IF NOT EXISTS character_stats (
|
||||
character_name VARCHAR(255) PRIMARY KEY,
|
||||
timestamp TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
level INTEGER,
|
||||
total_xp BIGINT,
|
||||
unassigned_xp BIGINT,
|
||||
luminance_earned BIGINT,
|
||||
luminance_total BIGINT,
|
||||
deaths INTEGER,
|
||||
stats_data JSONB NOT NULL
|
||||
)
|
||||
""")
|
||||
)
|
||||
print("character_stats table created/verified successfully")
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to create character_stats table: {e}")
|
||||
|
||||
|
||||
async def cleanup_old_portals():
|
||||
"""Clean up portals older than 1 hour."""
|
||||
try:
|
||||
cutoff_time = datetime.now(timezone.utc) - timedelta(hours=1)
|
||||
|
||||
# Delete old portals
|
||||
result = await database.execute(
|
||||
"DELETE FROM portals WHERE discovered_at < :cutoff_time",
|
||||
{"cutoff_time": cutoff_time},
|
||||
)
|
||||
|
||||
print(f"Cleaned up {result} portals older than 1 hour")
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to cleanup old portals: {e}")
|
||||
return 0
|
||||
|
||||
|
||||
async def seed_users():
|
||||
"""Seed default users if the users table is empty."""
|
||||
try:
|
||||
count = await database.fetch_val("SELECT COUNT(*) FROM users")
|
||||
if count > 0:
|
||||
print(f"Users table already has {count} users, skipping seed")
|
||||
return
|
||||
|
||||
default_users = [
|
||||
{"username": "erik", "password": "erik123", "is_admin": True},
|
||||
{"username": "alex", "password": "AlexGillar100Killar", "is_admin": False},
|
||||
{
|
||||
"username": "lundberg",
|
||||
"password": "JohanGillar100Kvinnor",
|
||||
"is_admin": False,
|
||||
},
|
||||
]
|
||||
for u in default_users:
|
||||
pw_hash = _bcrypt.hashpw(u["password"].encode(), _bcrypt.gensalt()).decode()
|
||||
await database.execute(
|
||||
"INSERT INTO users (username, password_hash, is_admin) VALUES (:username, :password_hash, :is_admin)",
|
||||
{
|
||||
"username": u["username"],
|
||||
"password_hash": pw_hash,
|
||||
"is_admin": u["is_admin"],
|
||||
},
|
||||
)
|
||||
role = "admin" if u["is_admin"] else "user"
|
||||
print(f"Seeded {role} user: {u['username']}")
|
||||
except Exception as e:
|
||||
print(f"Warning: failed to seed users: {e}")
|
||||
|
|
@ -1,16 +0,0 @@
|
|||
#!/bin/bash
|
||||
# Build frontend and deploy to static/ — run from MosswartOverlord root
|
||||
set -e
|
||||
|
||||
echo "Building frontend..."
|
||||
cd frontend && npm run build && cd ..
|
||||
|
||||
echo "Syncing build output to static/..."
|
||||
rm -rf static/assets/
|
||||
cp static/_build/index.html static/index.html
|
||||
cp -r static/_build/assets/ static/assets/
|
||||
cp static/_build/sw.js static/sw.js 2>/dev/null || true
|
||||
rm -rf static/_build/
|
||||
|
||||
echo "Done! $(ls static/assets/ | wc -l) asset files deployed."
|
||||
echo "Run 'git add static/ && git commit && git push' to deploy to server."
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
# Discord Rare Monitor Bot - Dockerfile
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Install Python dependencies
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy application code
|
||||
COPY discord_rare_monitor.py .
|
||||
COPY test_websocket.py .
|
||||
COPY icon_mapping.py .
|
||||
|
||||
# Copy icons directory
|
||||
COPY icons/ ./icons/
|
||||
|
||||
# Default environment variables
|
||||
ENV DISCORD_RARE_BOT_TOKEN="" \
|
||||
DERETH_TRACKER_WS_URL="ws://dereth-tracker:8765/ws/position" \
|
||||
COMMON_RARE_CHANNEL_ID="1355328792184226014" \
|
||||
GREAT_RARE_CHANNEL_ID="1353676584334131211" \
|
||||
LOG_LEVEL="INFO"
|
||||
|
||||
# Run the bot
|
||||
CMD ["python", "discord_rare_monitor.py"]
|
||||
|
|
@ -1,95 +0,0 @@
|
|||
# Discord Rare Monitor Bot
|
||||
|
||||
A Discord bot that monitors the Dereth Tracker WebSocket stream for rare discoveries and posts filtered notifications to Discord channels.
|
||||
|
||||
## Features
|
||||
|
||||
- **Real-time Monitoring**: Connects to Dereth Tracker WebSocket for instant rare notifications
|
||||
- **Smart Classification**: Automatically classifies rares as "common" or "great" based on keywords
|
||||
- **Rich Embeds**: Posts formatted Discord embeds with location and timestamp information
|
||||
- **Dual Channels**: Posts to separate channels for common and great rares
|
||||
- **Robust Connection**: Automatic reconnection with exponential backoff on connection failures
|
||||
|
||||
## Rare Classification
|
||||
|
||||
### Common Rares
|
||||
Items containing these keywords (except "Frore Crystal"):
|
||||
- Crystal
|
||||
- Jewel
|
||||
- Pearl
|
||||
- Elixir
|
||||
- Kit
|
||||
|
||||
### Great Rares
|
||||
All other rare discoveries not classified as common.
|
||||
|
||||
## Configuration
|
||||
|
||||
The bot is configured via environment variables:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `DISCORD_RARE_BOT_TOKEN` | Required | Discord bot token |
|
||||
| `DERETH_TRACKER_WS_URL` | `ws://dereth-tracker:8765/ws/position` | WebSocket URL |
|
||||
| `COMMON_RARE_CHANNEL_ID` | `1355328792184226014` | Discord channel for common rares |
|
||||
| `GREAT_RARE_CHANNEL_ID` | `1353676584334131211` | Discord channel for great rares |
|
||||
| `LOG_LEVEL` | `INFO` | Logging level (DEBUG, INFO, WARNING, ERROR) |
|
||||
|
||||
## Docker Usage
|
||||
|
||||
The bot is designed to run as a Docker container alongside the Dereth Tracker services:
|
||||
|
||||
```bash
|
||||
# Build and start all services including the Discord bot
|
||||
docker-compose up -d
|
||||
|
||||
# View bot logs
|
||||
docker-compose logs discord-rare-monitor
|
||||
|
||||
# Restart just the bot
|
||||
docker-compose restart discord-rare-monitor
|
||||
```
|
||||
|
||||
## Manual Setup
|
||||
|
||||
1. Create a Discord application and bot at https://discord.com/developers/applications
|
||||
2. Get the bot token and invite the bot to your Discord server
|
||||
3. Set the `DISCORD_RARE_BOT_TOKEN` environment variable
|
||||
4. Ensure the bot has permissions to send messages in the target channels
|
||||
|
||||
## Message Format
|
||||
|
||||
The bot listens for WebSocket messages with this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "rare",
|
||||
"character_name": "PlayerName",
|
||||
"name": "Dark Heart",
|
||||
"timestamp": "2025-06-22T16:00:00Z",
|
||||
"ew": 12.34,
|
||||
"ns": -56.78,
|
||||
"z": 10.5
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
- **WebSocket Client**: Connects to Dereth Tracker's WebSocket stream
|
||||
- **Message Filter**: Only processes `{"type": "rare"}` messages
|
||||
- **Classifier**: Determines rare type based on name keywords
|
||||
- **Discord Client**: Posts formatted embeds to appropriate channels
|
||||
- **Retry Logic**: Automatic reconnection with exponential backoff
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `discord.py>=2.3.0` - Discord API client
|
||||
- `websockets>=11.0.0` - WebSocket client library
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Zero Duplication**: Each rare generates exactly one notification
|
||||
- **Real-time**: Instant notifications via WebSocket stream
|
||||
- **Lightweight**: Minimal resource usage (~50MB RAM)
|
||||
- **Reliable**: Robust error handling and reconnection logic
|
||||
- **Integrated**: Seamlessly works with existing Dereth Tracker infrastructure
|
||||
|
|
@ -1,67 +0,0 @@
|
|||
"""
|
||||
Configuration module for Discord Rare Monitor Bot.
|
||||
Centralizes environment variable handling and configuration constants.
|
||||
"""
|
||||
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
|
||||
class Config:
|
||||
"""Configuration class for Discord Rare Monitor Bot."""
|
||||
|
||||
# Discord Configuration
|
||||
DISCORD_TOKEN: str = os.getenv('DISCORD_RARE_BOT_TOKEN', '')
|
||||
COMMON_RARE_CHANNEL_ID: int = int(os.getenv('COMMON_RARE_CHANNEL_ID', '1355328792184226014'))
|
||||
GREAT_RARE_CHANNEL_ID: int = int(os.getenv('GREAT_RARE_CHANNEL_ID', '1353676584334131211'))
|
||||
|
||||
# WebSocket Configuration
|
||||
WEBSOCKET_URL: str = os.getenv('DERETH_TRACKER_WS_URL', 'ws://dereth-tracker:8765/ws/position')
|
||||
|
||||
# Logging Configuration
|
||||
LOG_LEVEL: str = os.getenv('LOG_LEVEL', 'INFO').upper()
|
||||
|
||||
# Rare Classification Configuration
|
||||
COMMON_RARE_KEYWORDS: list = ["Crystal", "Jewel", "Pearl", "Elixir", "Kit"]
|
||||
|
||||
# WebSocket Retry Configuration
|
||||
INITIAL_RETRY_DELAY: int = 5 # seconds
|
||||
MAX_RETRY_DELAY: int = 300 # 5 minutes
|
||||
|
||||
@classmethod
|
||||
def validate(cls) -> list:
|
||||
"""Validate configuration and return list of errors."""
|
||||
errors = []
|
||||
|
||||
if not cls.DISCORD_TOKEN:
|
||||
errors.append("DISCORD_RARE_BOT_TOKEN environment variable is required")
|
||||
|
||||
if not cls.WEBSOCKET_URL:
|
||||
errors.append("DERETH_TRACKER_WS_URL environment variable is required")
|
||||
|
||||
try:
|
||||
cls.COMMON_RARE_CHANNEL_ID = int(cls.COMMON_RARE_CHANNEL_ID)
|
||||
except (ValueError, TypeError):
|
||||
errors.append("COMMON_RARE_CHANNEL_ID must be a valid integer")
|
||||
|
||||
try:
|
||||
cls.GREAT_RARE_CHANNEL_ID = int(cls.GREAT_RARE_CHANNEL_ID)
|
||||
except (ValueError, TypeError):
|
||||
errors.append("GREAT_RARE_CHANNEL_ID must be a valid integer")
|
||||
|
||||
return errors
|
||||
|
||||
@classmethod
|
||||
def log_config(cls, logger):
|
||||
"""Log current configuration (excluding sensitive data)."""
|
||||
logger.info("🔧 Discord Rare Monitor Configuration:")
|
||||
logger.info(f" WebSocket URL: {cls.WEBSOCKET_URL}")
|
||||
logger.info(f" Common Rare Channel ID: {cls.COMMON_RARE_CHANNEL_ID}")
|
||||
logger.info(f" Great Rare Channel ID: {cls.GREAT_RARE_CHANNEL_ID}")
|
||||
logger.info(f" Log Level: {cls.LOG_LEVEL}")
|
||||
logger.info(f" Common Keywords: {cls.COMMON_RARE_KEYWORDS}")
|
||||
logger.info(f" Discord Token: {'✅ Set' if cls.DISCORD_TOKEN else '❌ Not Set'}")
|
||||
|
||||
|
||||
# Global config instance
|
||||
config = Config()
|
||||
|
|
@ -1,55 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate mapping between icon filenames and rare item names.
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
|
||||
def generate_icon_mapping():
|
||||
"""Generate mapping from icon filenames to display names."""
|
||||
icons_dir = "/home/erik/MosswartOverlord/discord-rare-monitor/icons"
|
||||
|
||||
# Create reverse mapping from filename to display name
|
||||
icon_mapping = {}
|
||||
|
||||
# List all PNG files in the icons directory
|
||||
for filename in os.listdir(icons_dir):
|
||||
if filename.endswith("_Icon.png"):
|
||||
# Convert filename back to display name
|
||||
# Remove _Icon.png suffix
|
||||
base_name = filename[:-9]
|
||||
|
||||
# Convert underscores to spaces and handle apostrophes
|
||||
display_name = base_name.replace("_", " ")
|
||||
|
||||
# Fix common patterns
|
||||
display_name = display_name.replace("s Crystal", "'s Crystal")
|
||||
display_name = display_name.replace("s Pearl", "'s Pearl")
|
||||
display_name = display_name.replace("s Jewel", "'s Jewel")
|
||||
display_name = display_name.replace("s Breath", "'s Breath")
|
||||
display_name = display_name.replace("s Glaive", "'s Glaive")
|
||||
display_name = display_name.replace("s Grip", "'s Grip")
|
||||
display_name = display_name.replace("Tri Blade", "Tri-Blade")
|
||||
display_name = display_name.replace("T ing", "T'ing")
|
||||
|
||||
# Special cases
|
||||
if "Renari" in display_name:
|
||||
display_name = display_name.replace("Renaris", "Renari's")
|
||||
if "Leikotha" in display_name:
|
||||
display_name = display_name.replace("Leikothas", "Leikotha's")
|
||||
|
||||
icon_mapping[filename] = display_name
|
||||
|
||||
# Save mapping to JSON file
|
||||
with open(os.path.join(os.path.dirname(icons_dir), "icon_name_mapping.json"), "w") as f:
|
||||
json.dump(icon_mapping, f, indent=2, sort_keys=True)
|
||||
|
||||
return icon_mapping
|
||||
|
||||
if __name__ == "__main__":
|
||||
mapping = generate_icon_mapping()
|
||||
print(f"Generated mapping for {len(mapping)} icons")
|
||||
print("\nFirst 10 mappings:")
|
||||
for i, (filename, display_name) in enumerate(list(mapping.items())[:10]):
|
||||
print(f" {filename} -> {display_name}")
|
||||
|
|
@ -1,294 +0,0 @@
|
|||
{
|
||||
"Adepts_Fervor_Icon.png": "Adepts Fervor",
|
||||
"Adherents_Crystal_Icon.png": "Adherent's Crystal",
|
||||
"Alchemists_Crystal_Icon.png": "Alchemist's Crystal",
|
||||
"Aquamarine_Foolproof_Icon.png": "Aquamarine Foolproof",
|
||||
"Archers_Jewel_Icon.png": "Archer's Jewel",
|
||||
"Aristocrats_Bracelet_Icon.png": "Aristocrats Bracelet",
|
||||
"Artificers_Crystal_Icon.png": "Artificer's Crystal",
|
||||
"Artists_Crystal_Icon.png": "Artist's Crystal",
|
||||
"Assassins_Whisper_Icon.png": "Assassins Whisper",
|
||||
"Astyrrians_Jewel_Icon.png": "Astyrrian's Jewel",
|
||||
"Band_of_Elemental_Harmony_Icon.png": "Band of Elemental Harmony",
|
||||
"Baton_of_Tirethas_Icon.png": "Baton of Tirethas",
|
||||
"Bearded_Axe_of_Souia-Vey_Icon.png": "Bearded Axe of Souia-Vey",
|
||||
"Ben_Tens_Crystal_Icon.png": "Ben Ten's Crystal",
|
||||
"Berzerkers_Crystal_Icon.png": "Berzerker's Crystal",
|
||||
"Black_Cloud_Bow_Icon.png": "Black Cloud Bow",
|
||||
"Black_Garnet_Foolproof_Icon.png": "Black Garnet Foolproof",
|
||||
"Black_Opal_Foolproof_Icon.png": "Black Opal Foolproof",
|
||||
"Black_Thistle_Icon.png": "Black Thistle",
|
||||
"Bloodmark_Crossbow_Icon.png": "Bloodmark Crossbow",
|
||||
"Bracelet_of_Binding_Icon.png": "Bracelet of Binding",
|
||||
"Bracers_of_Leikothas_Tears_Icon.png": "Bracers of Leikotha's Tears",
|
||||
"Bradors_Frozen_Eye_Icon.png": "Bradors Frozen Eye",
|
||||
"Brawlers_Crystal_Icon.png": "Brawler's Crystal",
|
||||
"Breastplate_of_Leikothas_Tears_Icon.png": "Breastplate of Leikotha's Tears",
|
||||
"Canfield_Cleaver_Icon.png": "Canfield Cleaver",
|
||||
"Casino_Exquisite_Keyring_Icon.png": "Casino Exquisite Keyring",
|
||||
"Champions_Demise_Icon.png": "Champions Demise",
|
||||
"Chefs_Crystal_Icon.png": "Chef's Crystal",
|
||||
"Chitin_Cracker_Icon.png": "Chitin Cracker",
|
||||
"Circle_of_Pure_Thought_Icon.png": "Circle of Pure Thought",
|
||||
"Converters_Crystal_Icon.png": "Converter's Crystal",
|
||||
"Corruptors_Crystal_Icon.png": "Corruptor's Crystal",
|
||||
"Corsairs_Arc_Icon.png": "Corsairs Arc",
|
||||
"Count_Renaris_Equalizer_Icon.png": "Count Renari's Equalizer",
|
||||
"Dart_Flicker_Icon.png": "Dart Flicker",
|
||||
"Deaths_Grip_Staff_Icon.png": "Death's Grip Staff",
|
||||
"Decapitators_Blade_Icon.png": "Decapitators Blade",
|
||||
"Deceivers_Crystal_Icon.png": "Deceiver's Crystal",
|
||||
"Defiler_of_Milantos_Icon.png": "Defiler of Milantos",
|
||||
"Deru_Limb_Icon.png": "Deru Limb",
|
||||
"Desert_Wyrm_Icon.png": "Desert Wyrm",
|
||||
"Dodgers_Crystal_Icon.png": "Dodger's Crystal",
|
||||
"Dragonspine_Bow_Icon.png": "Dragonspine Bow",
|
||||
"Dread_Marauder_Shield_Icon.png": "Dread Marauder Shield",
|
||||
"Dreamseer_Bangle_Icon.png": "Dreamseer Bangle",
|
||||
"Drifters_Atlatl_Icon.png": "Drifters Atlatl",
|
||||
"Dripping_Death_Icon.png": "Dripping Death",
|
||||
"Duelists_Jewel_Icon.png": "Duelist's Jewel",
|
||||
"Dusk_Coat_Icon.png": "Dusk Coat",
|
||||
"Dusk_Leggings_Icon.png": "Dusk Leggings",
|
||||
"Ebonwood_Shortbow_Icon.png": "Ebonwood Shortbow",
|
||||
"Elysas_Crystal_Icon.png": "Elysa's Crystal",
|
||||
"Emerald_Foolproof_Icon.png": "Emerald Foolproof",
|
||||
"Enchanters_Crystal_Icon.png": "Enchanter's Crystal",
|
||||
"Eternal_Health_Kit_Icon.png": "Eternal Health Kit",
|
||||
"Eternal_Mana_Charge_Icon.png": "Eternal Mana Charge",
|
||||
"Eternal_Mana_Kit_Icon.png": "Eternal Mana Kit",
|
||||
"Eternal_Stamina_Kit_Icon.png": "Eternal Stamina Kit",
|
||||
"Evaders_Crystal_Icon.png": "Evader's Crystal",
|
||||
"Executors_Jewel_Icon.png": "Executor's Jewel",
|
||||
"Eye_of_Muramm_Icon.png": "Eye of Muramm",
|
||||
"Feathered_Razor_Icon.png": "Feathered Razor",
|
||||
"Fire_Opal_Foolproof_Icon.png": "Fire Opal Foolproof",
|
||||
"Fist_of_Three_Principles_Icon.png": "Fist of Three Principles",
|
||||
"Fletchers_Crystal_Icon.png": "Fletcher's Crystal",
|
||||
"Footmans_Boots_Icon.png": "Footmans Boots",
|
||||
"Gauntlets_of_Leikothas_Tears_Icon.png": "Gauntlets of Leikotha's Tears",
|
||||
"Gauntlets_of_the_Crimson_Star_Icon.png": "Gauntlets of the Crimson Star",
|
||||
"Gelidite_Boots_Icon.png": "Gelidite Boots",
|
||||
"Gelidite_Bracers_Icon.png": "Gelidite Bracers",
|
||||
"Gelidite_Breastplate_Icon.png": "Gelidite Breastplate",
|
||||
"Gelidite_Gauntlets_Icon.png": "Gelidite Gauntlets",
|
||||
"Gelidite_Girth_Icon.png": "Gelidite Girth",
|
||||
"Gelidite_Greaves_Icon.png": "Gelidite Greaves",
|
||||
"Gelidite_Mitre_Icon.png": "Gelidite Mitre",
|
||||
"Gelidite_Pauldrons_Icon.png": "Gelidite Pauldrons",
|
||||
"Gelidite_Tassets_Icon.png": "Gelidite Tassets",
|
||||
"Gelids_Jewel_Icon.png": "Gelid's Jewel",
|
||||
"Girth_of_Leikothas_Tears_Icon.png": "Girth of Leikotha's Tears",
|
||||
"Golden_Snake_Choker_Icon.png": "Golden Snake Choker",
|
||||
"Greaves_of_Leikothas_Tears_Icon.png": "Greaves of Leikotha's Tears",
|
||||
"Guardian_of_Pwyll_Icon.png": "Guardian of Pwyll",
|
||||
"Heart_of_Darkest_Flame_Icon.png": "Heart of Darkest Flame",
|
||||
"Helm_of_Leikothas_Tears_Icon.png": "Helm of Leikotha's Tears",
|
||||
"Hevelios_Half-Moon_Icon.png": "Hevelios Half-Moon",
|
||||
"Hieroglyph_of_Alchemy_Mastery_Icon.png": "Hieroglyph of Alchemy Mastery",
|
||||
"Hieroglyph_of_Arcane_Enlightenment_Icon.png": "Hieroglyph of Arcane Enlightenment",
|
||||
"Hieroglyph_of_Armor_Tinkering_Expertise_Icon.png": "Hieroglyph of Armor Tinkering Expertise",
|
||||
"Hieroglyph_of_Cooking_Mastery_Icon.png": "Hieroglyph of Cooking Mastery",
|
||||
"Hieroglyph_of_Creature_Enchantment_Mastery_Icon.png": "Hieroglyph of Creature Enchantment Mastery",
|
||||
"Hieroglyph_of_Deception_Mastery_Icon.png": "Hieroglyph of Deception Mastery",
|
||||
"Hieroglyph_of_Dirty_Fighting_Mastery_Icon.png": "Hieroglyph of Dirty Fighting Mastery",
|
||||
"Hieroglyph_of_Dual_Wield_Mastery_Icon.png": "Hieroglyph of Dual Wield Mastery",
|
||||
"Hieroglyph_of_Fealty_Icon.png": "Hieroglyph of Fealty",
|
||||
"Hieroglyph_of_Finesse_Weapon_Mastery_Icon.png": "Hieroglyph of Finesse Weapon Mastery",
|
||||
"Hieroglyph_of_Fletching_Mastery_Icon.png": "Hieroglyph of Fletching Mastery",
|
||||
"Hieroglyph_of_Healing_Mastery_Icon.png": "Hieroglyph of Healing Mastery",
|
||||
"Hieroglyph_of_Heavy_Weapon_Mastery_Icon.png": "Hieroglyph of Heavy Weapon Mastery",
|
||||
"Hieroglyph_of_Impregnability_Icon.png": "Hieroglyph of Impregnability",
|
||||
"Hieroglyph_of_Invulnerability_Icon.png": "Hieroglyph of Invulnerability",
|
||||
"Hieroglyph_of_Item_Enchantment_Mastery_Icon.png": "Hieroglyph of Item Enchantment Mastery",
|
||||
"Hieroglyph_of_Item_Tinkering_Expertise_Icon.png": "Hieroglyph of Item Tinkering Expertise",
|
||||
"Hieroglyph_of_Jumping_Mastery_Icon.png": "Hieroglyph of Jumping Mastery",
|
||||
"Hieroglyph_of_Leadership_Mastery_Icon.png": "Hieroglyph of Leadership Mastery",
|
||||
"Hieroglyph_of_Life_Magic_Mastery_Icon.png": "Hieroglyph of Life Magic Mastery",
|
||||
"Hieroglyph_of_Light_Weapon_Mastery_Icon.png": "Hieroglyph of Light Weapon Mastery",
|
||||
"Hieroglyph_of_Lockpick_Mastery_Icon.png": "Hieroglyph of Lockpick Mastery",
|
||||
"Hieroglyph_of_Magic_Item_Tinkering_Expertise_Icon.png": "Hieroglyph of Magic Item Tinkering Expertise",
|
||||
"Hieroglyph_of_Magic_Resistance_Icon.png": "Hieroglyph of Magic Resistance",
|
||||
"Hieroglyph_of_Mana_Conversion_Mastery_Icon.png": "Hieroglyph of Mana Conversion Mastery",
|
||||
"Hieroglyph_of_Missile_Weapon_Mastery_Icon.png": "Hieroglyph of Missile Weapon Mastery",
|
||||
"Hieroglyph_of_Monster_Attunement_Icon.png": "Hieroglyph of Monster Attunement",
|
||||
"Hieroglyph_of_Person_Attunement_Icon.png": "Hieroglyph of Person Attunement",
|
||||
"Hieroglyph_of_Recklessness_Mastery_Icon.png": "Hieroglyph of Recklessness Mastery",
|
||||
"Hieroglyph_of_Shield_Mastery_Icon.png": "Hieroglyph of Shield Mastery",
|
||||
"Hieroglyph_of_Sneak_Attack_Mastery_Icon.png": "Hieroglyph of Sneak Attack Mastery",
|
||||
"Hieroglyph_of_Sprint_Icon.png": "Hieroglyph of Sprint",
|
||||
"Hieroglyph_of_Two_Handed_Weapons_Mastery_Icon.png": "Hieroglyph of Two Handed Weapons Mastery",
|
||||
"Hieroglyph_of_Void_Magic_Mastery_Icon.png": "Hieroglyph of Void Magic Mastery",
|
||||
"Hieroglyph_of_War_Magic_Mastery_Icon.png": "Hieroglyph of War Magic Mastery",
|
||||
"Hieroglyph_of_Weapon_Tinkering_Expertise_Icon.png": "Hieroglyph of Weapon Tinkering Expertise",
|
||||
"Hieromancers_Crystal_Icon.png": "Hieromancer's Crystal",
|
||||
"Hooded_Serpent_Slinger_Icon.png": "Hooded Serpent Slinger",
|
||||
"Hunters_Crystal_Icon.png": "Hunter's Crystal",
|
||||
"Huntsmans_Dart-Thrower_Icon.png": "Huntsmans Dart-Thrower",
|
||||
"Ibriyas_Choice_Icon.png": "Ibriyas Choice",
|
||||
"Ideograph_of_Acid_Protection_Icon.png": "Ideograph of Acid Protection",
|
||||
"Ideograph_of_Armor_Icon.png": "Ideograph of Armor",
|
||||
"Ideograph_of_Blade_Protection_Icon.png": "Ideograph of Blade Protection",
|
||||
"Ideograph_of_Bludgeoning_Protection_Icon.png": "Ideograph of Bludgeoning Protection",
|
||||
"Ideograph_of_Fire_Protection_Icon.png": "Ideograph of Fire Protection",
|
||||
"Ideograph_of_Frost_Protection_Icon.png": "Ideograph of Frost Protection",
|
||||
"Ideograph_of_Lightning_Protection_Icon.png": "Ideograph of Lightning Protection",
|
||||
"Ideograph_of_Mana_Renewal_Icon.png": "Ideograph of Mana Renewal",
|
||||
"Ideograph_of_Piercing_Protection_Icon.png": "Ideograph of Piercing Protection",
|
||||
"Ideograph_of_Regeneration_Icon.png": "Ideograph of Regeneration",
|
||||
"Ideograph_of_Revitalization_Icon.png": "Ideograph of Revitalization",
|
||||
"Imbuers_Crystal_Icon.png": "Imbuer's Crystal",
|
||||
"Imperial_Chevairds_Helm_Icon.png": "Imperial Chevairds Helm",
|
||||
"Imperial_Topaz_Foolproof_Icon.png": "Imperial Topaz Foolproof",
|
||||
"Infernos_Jewel_Icon.png": "Inferno's Jewel",
|
||||
"Infinite_Deadly_Acid_Arrowheads_Icon.png": "Infinite Deadly Acid Arrowheads",
|
||||
"Infinite_Deadly_Armor_Piercing_Arrowheads_Icon.png": "Infinite Deadly Armor Piercing Arrowheads",
|
||||
"Infinite_Deadly_Blunt_Arrowheads_Icon.png": "Infinite Deadly Blunt Arrowheads",
|
||||
"Infinite_Deadly_Broad_Arrowheads_Icon.png": "Infinite Deadly Broad Arrowheads",
|
||||
"Infinite_Deadly_Electric_Arrowheads_Icon.png": "Infinite Deadly Electric Arrowheads",
|
||||
"Infinite_Deadly_Fire_Arrowheads_Icon.png": "Infinite Deadly Fire Arrowheads",
|
||||
"Infinite_Deadly_Frog_Crotch_Arrowheads_Icon.png": "Infinite Deadly Frog Crotch Arrowheads",
|
||||
"Infinite_Deadly_Frost_Arrowheads_Icon.png": "Infinite Deadly Frost Arrowheads",
|
||||
"Infinite_Elaborate_Dried_Rations_Icon.png": "Infinite Elaborate Dried Rations",
|
||||
"Infinite_Ivory_Icon.png": "Infinite Ivory",
|
||||
"Infinite_Leather_Icon.png": "Infinite Leather",
|
||||
"Infinite_Simple_Dried_Rations_Icon.png": "Infinite Simple Dried Rations",
|
||||
"Invigorating_Elixir_Icon.png": "Invigorating Elixir",
|
||||
"Iron_Bull_Icon.png": "Iron Bull",
|
||||
"Itakas_Naginata_Icon.png": "Itakas Naginata",
|
||||
"Jet_Foolproof_Icon.png": "Jet Foolproof",
|
||||
"Lichs_Pearl_Icon.png": "Lich's Pearl",
|
||||
"Life_Givers_Crystal_Icon.png": "Life Giver's Crystal",
|
||||
"Limitless_Lockpick_Icon.png": "Limitless Lockpick",
|
||||
"Loop_of_Opposing_Benedictions_Icon.png": "Loop of Opposing Benedictions",
|
||||
"Loves_Favor_Icon.png": "Loves Favor",
|
||||
"Lugians_Pearl_Icon.png": "Lugian's Pearl",
|
||||
"Mages_Jewel_Icon.png": "Mage's Jewel",
|
||||
"Maguss_Pearl_Icon.png": "Magus's Pearl",
|
||||
"Malachite_Slasher_Icon.png": "Malachite Slasher",
|
||||
"Medicated_Health_Kit_Icon.png": "Medicated Health Kit",
|
||||
"Medicated_Mana_Kit_Icon.png": "Medicated Mana Kit",
|
||||
"Medicated_Stamina_Kit_Icon.png": "Medicated Stamina Kit",
|
||||
"Melees_Jewel_Icon.png": "Melee's Jewel",
|
||||
"Miraculous_Elixir_Icon.png": "Miraculous Elixir",
|
||||
"Mirrored_Justice_Icon.png": "Mirrored Justice",
|
||||
"Monarchs_Crystal_Icon.png": "Monarch's Crystal",
|
||||
"Moriharus_Kitchen_Knife_Icon.png": "Moriharus Kitchen Knife",
|
||||
"Morrigans_Vanity_Icon.png": "Morrigans Vanity",
|
||||
"Necklace_of_Iniquity_Icon.png": "Necklace of Iniquity",
|
||||
"Observers_Crystal_Icon.png": "Observer's Crystal",
|
||||
"Olthois_Jewel_Icon.png": "Olthoi's Jewel",
|
||||
"Orb_of_the_Ironsea_Icon.png": "Orb of the Ironsea",
|
||||
"Oswalds_Crystal_Icon.png": "Oswald's Crystal",
|
||||
"Patriarchs_Twilight_Coat_Icon.png": "Patriarchs Twilight Coat",
|
||||
"Patriarchs_Twilight_Tights_Icon.png": "Patriarchs Twilight Tights",
|
||||
"Pauldrons_of_Leikothas_Tears_Icon.png": "Pauldrons of Leikotha's Tears",
|
||||
"Pearl_of_Acid_Baning_Icon.png": "Pearl of Acid Baning",
|
||||
"Pearl_of_Blade_Baning_Icon.png": "Pearl of Blade Baning",
|
||||
"Pearl_of_Blood_Drinking_Icon.png": "Pearl of Blood Drinking",
|
||||
"Pearl_of_Bludgeon_Baning_Icon.png": "Pearl of Bludgeon Baning",
|
||||
"Pearl_of_Defending_Icon.png": "Pearl of Defending",
|
||||
"Pearl_of_Flame_Baning_Icon.png": "Pearl of Flame Baning",
|
||||
"Pearl_of_Frost_Baning_Icon.png": "Pearl of Frost Baning",
|
||||
"Pearl_of_Heart_Seeking_Icon.png": "Pearl of Heart Seeking",
|
||||
"Pearl_of_Hermetic_Linking_Icon.png": "Pearl of Hermetic Linking",
|
||||
"Pearl_of_Impenetrability_Icon.png": "Pearl of Impenetrability",
|
||||
"Pearl_of_Lightning_Baning_Icon.png": "Pearl of Lightning Baning",
|
||||
"Pearl_of_Pierce_Baning_Icon.png": "Pearl of Pierce Baning",
|
||||
"Pearl_of_Spirit_Drinking_Icon.png": "Pearl of Spirit Drinking",
|
||||
"Pearl_of_Swift_Killing_Icon.png": "Pearl of Swift Killing",
|
||||
"Perennial_Argenory_Dye_Icon.png": "Perennial Argenory Dye",
|
||||
"Perennial_Berimphur_Dye_Icon.png": "Perennial Berimphur Dye",
|
||||
"Perennial_Botched_Dye_Icon.png": "Perennial Botched Dye",
|
||||
"Perennial_Colban_Dye_Icon.png": "Perennial Colban Dye",
|
||||
"Perennial_Hennacin_Dye_Icon.png": "Perennial Hennacin Dye",
|
||||
"Perennial_Lapyan_Dye_Icon.png": "Perennial Lapyan Dye",
|
||||
"Perennial_Minalim_Dye_Icon.png": "Perennial Minalim Dye",
|
||||
"Perennial_Relanim_Dye_Icon.png": "Perennial Relanim Dye",
|
||||
"Perennial_Thananim_Dye_Icon.png": "Perennial Thananim Dye",
|
||||
"Perennial_Verdalim_Dye_Icon.png": "Perennial Verdalim Dye",
|
||||
"Peridot_Foolproof_Icon.png": "Peridot Foolproof",
|
||||
"Physicians_Crystal_Icon.png": "Physician's Crystal",
|
||||
"Pictograph_of_Coordination_Icon.png": "Pictograph of Coordination",
|
||||
"Pictograph_of_Endurance_Icon.png": "Pictograph of Endurance",
|
||||
"Pictograph_of_Focus_Icon.png": "Pictograph of Focus",
|
||||
"Pictograph_of_Quickness_Icon.png": "Pictograph of Quickness",
|
||||
"Pictograph_of_Strength_Icon.png": "Pictograph of Strength",
|
||||
"Pictograph_of_Willpower_Icon.png": "Pictograph of Willpower",
|
||||
"Pillar_of_Fearlessness_Icon.png": "Pillar of Fearlessness",
|
||||
"Pitfighters_Edge_Icon.png": "Pitfighters Edge",
|
||||
"Red_Garnet_Foolproof_Icon.png": "Red Garnet Foolproof",
|
||||
"Refreshing_Elixir_Icon.png": "Refreshing Elixir",
|
||||
"Resisters_Crystal_Icon.png": "Resister's Crystal",
|
||||
"Revenants_Scythe_Icon.png": "Revenants Scythe",
|
||||
"Ridgeback_Dagger_Icon.png": "Ridgeback Dagger",
|
||||
"Ring_of_Channeling_Icon.png": "Ring of Channeling",
|
||||
"Rogues_Crystal_Icon.png": "Rogue's Crystal",
|
||||
"Royal_Ladle_Icon.png": "Royal Ladle",
|
||||
"Rune_of_Acid_Bane_Icon.png": "Rune of Acid Bane",
|
||||
"Rune_of_Blade_Bane_Icon.png": "Rune of Blade Bane",
|
||||
"Rune_of_Blood_Drinker_Icon.png": "Rune of Blood Drinker",
|
||||
"Rune_of_Bludgeon_Bane_Icon.png": "Rune of Bludgeon Bane",
|
||||
"Rune_of_Defender_Icon.png": "Rune of Defender",
|
||||
"Rune_of_Dispel_Icon.png": "Rune of Dispel",
|
||||
"Rune_of_Flame_Bane_Icon.png": "Rune of Flame Bane",
|
||||
"Rune_of_Frost_Bane_Icon.png": "Rune of Frost Bane",
|
||||
"Rune_of_Heart_Seeker_Icon.png": "Rune of Heart Seeker",
|
||||
"Rune_of_Hermetic_Link_Icon.png": "Rune of Hermetic Link",
|
||||
"Rune_of_Impenetrability_Icon.png": "Rune of Impenetrability",
|
||||
"Rune_of_Lifestone_Recall_Icon.png": "Rune of Lifestone Recall",
|
||||
"Rune_of_Lightning_Bane_Icon.png": "Rune of Lightning Bane",
|
||||
"Rune_of_Pierce_Bane_Icon.png": "Rune of Pierce Bane",
|
||||
"Rune_of_Portal_Recall_Icon.png": "Rune of Portal Recall",
|
||||
"Rune_of_Spirit_Drinker_Icon.png": "Rune of Spirit Drinker",
|
||||
"Rune_of_Swift_Killer_Icon.png": "Rune of Swift Killer",
|
||||
"Scholars_Crystal_Icon.png": "Scholar's Crystal",
|
||||
"Serpents_Flight_Icon.png": "Serpents Flight",
|
||||
"Shield_of_Engorgement_Icon.png": "Shield of Engorgement",
|
||||
"Shimmering_Skeleton_Key_Icon.png": "Shimmering Skeleton Key",
|
||||
"Skullpuncher_Icon.png": "Skullpuncher",
|
||||
"Smite_Icon.png": "Smite",
|
||||
"Smithys_Crystal_Icon.png": "Smithy's Crystal",
|
||||
"Spear_of_Lost_Truths_Icon.png": "Spear of Lost Truths",
|
||||
"Spirit_Shifting_Staff_Icon.png": "Spirit Shifting Staff",
|
||||
"Sprinters_Pearl_Icon.png": "Sprinter's Pearl",
|
||||
"Squires_Glaive_Icon.png": "Squire's Glaive",
|
||||
"Staff_of_All_Aspects_Icon.png": "Staff of All Aspects",
|
||||
"Staff_of_Fettered_Souls_Icon.png": "Staff of Fettered Souls",
|
||||
"Staff_of_Tendrils_Icon.png": "Staff of Tendrils",
|
||||
"Star_of_Gharun_Icon.png": "Star of Gharun",
|
||||
"Star_of_Tukal_Icon.png": "Star of Tukal",
|
||||
"Steel_Butterfly_Icon.png": "Steel Butterfly",
|
||||
"Steel_Wall_Boots_Icon.png": "Steel Wall Boots",
|
||||
"Subjugator_Icon.png": "Subjugator",
|
||||
"Sunstone_Foolproof_Icon.png": "Sunstone Foolproof",
|
||||
"Swift_Strike_Ring_Icon.png": "Swift Strike Ring",
|
||||
"Tassets_of_Leikothas_Tears_Icon.png": "Tassets of Leikotha's Tears",
|
||||
"Thiefs_Crystal_Icon.png": "Thief's Crystal",
|
||||
"Thorstens_Crystal_Icon.png": "Thorsten's Crystal",
|
||||
"Thunderhead_Icon.png": "Thunderhead",
|
||||
"Tings_Crystal_Icon.png": "Ting's Crystal",
|
||||
"Tinkers_Crystal_Icon.png": "Tinker's Crystal",
|
||||
"Tracker_Boots_Icon.png": "Tracker Boots",
|
||||
"Tri_Blade_Spear_Icon.png": "Tri-Blade Spear",
|
||||
"Tusked_Axe_of_Ayan_Baqur_Icon.png": "Tusked Axe of Ayan Baqur",
|
||||
"Tuskers_Jewel_Icon.png": "Tusker's Jewel",
|
||||
"Twin_Ward_Icon.png": "Twin Ward",
|
||||
"Unchained_Prowess_Ring_Icon.png": "Unchained Prowess Ring",
|
||||
"Ursuins_Pearl_Icon.png": "Ursuin's Pearl",
|
||||
"Valkeers_Helm_Icon.png": "Valkeers Helm",
|
||||
"Vaulters_Crystal_Icon.png": "Vaulter's Crystal",
|
||||
"Wand_of_the_Frore_Crystal_Icon.png": "Wand of the Frore Crystal",
|
||||
"Warriors_Crystal_Icon.png": "Warrior's Crystal",
|
||||
"Warriors_Jewel_Icon.png": "Warrior's Jewel",
|
||||
"Wayfarers_Pearl_Icon.png": "Wayfarer's Pearl",
|
||||
"Weeping_Ring_Icon.png": "Weeping Ring",
|
||||
"White_Sapphire_Foolproof_Icon.png": "White Sapphire Foolproof",
|
||||
"Wings_of_Rakhil_Icon.png": "Wings of Rakhil",
|
||||
"Winters_Heart_Icon.png": "Winters Heart",
|
||||
"Yellow_Topaz_Foolproof_Icon.png": "Yellow Topaz Foolproof",
|
||||
"Zefirs_Breath_Icon.png": "Zefir's Breath",
|
||||
"Zefirs_Crystal_Icon.png": "Zefir's Crystal",
|
||||
"Zharalim_Crookblade_Icon.png": "Zharalim Crookblade",
|
||||
"Zircon_Foolproof_Icon.png": "Zircon Foolproof"
|
||||
}
|
||||
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.2 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.9 KiB |
|
Before Width: | Height: | Size: 2.2 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.1 KiB |
|
Before Width: | Height: | Size: 2.2 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.8 KiB |
|
Before Width: | Height: | Size: 2.1 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 3 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 3.5 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 3.9 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.9 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.1 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.2 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.6 KiB |
|
Before Width: | Height: | Size: 2.3 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 2.1 KiB |
|
Before Width: | Height: | Size: 2.4 KiB |
|
Before Width: | Height: | Size: 2.7 KiB |