Update suitbuilder documentation to reflect production-ready status
- Replace outdated CLAUDE.md suitbuilder section with concise production-ready summary - Remove incorrect claims about spell system not being implemented - Create detailed architecture document at docs/plans/2026-02-09-suitbuilder-architecture.md - Document spell bitmap system, locked slots, suit summary, and SSE streaming - Add future enhancement roadmap (spell improvements, new builders, advanced features) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
erik
parent
e0265e261c
commit
850cd62d7b
2 changed files with 373 additions and 0 deletions
140
CLAUDE.md
Normal file
140
CLAUDE.md
Normal file
|
|
@ -0,0 +1,140 @@
|
||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
Dereth Tracker is a real-time telemetry service for game world tracking. It's a FastAPI-based WebSocket and HTTP API service that ingests player position/stats data via plugins and provides live map visualization through a web interface.
|
||||||
|
|
||||||
|
## Key Components
|
||||||
|
|
||||||
|
### Main Service (main.py)
|
||||||
|
- WebSocket endpoint `/ws/position` receives telemetry and inventory events
|
||||||
|
- Routes inventory events to inventory service via HTTP
|
||||||
|
- Handles real-time player tracking and map updates
|
||||||
|
|
||||||
|
### Inventory Service (inventory-service/main.py)
|
||||||
|
- Separate FastAPI service for inventory management
|
||||||
|
- Processes inventory JSON into normalized PostgreSQL tables
|
||||||
|
- Provides search API with advanced filtering and sorting
|
||||||
|
- Uses comprehensive enum database for translating game IDs to readable names
|
||||||
|
|
||||||
|
### Database Architecture
|
||||||
|
- **Telemetry DB**: TimescaleDB for time-series player tracking data
|
||||||
|
- **Inventory DB**: PostgreSQL with normalized schema for equipment data
|
||||||
|
- `items`: Core item properties
|
||||||
|
- `item_combat_stats`: Armor level, damage bonuses
|
||||||
|
- `item_enhancements`: Material, item sets, tinkering
|
||||||
|
- `item_spells`: Spell names and categories
|
||||||
|
- `item_raw_data`: Original JSON for complex queries
|
||||||
|
|
||||||
|
## Memories and Known Bugs
|
||||||
|
|
||||||
|
* Fixed: Material names now properly display (e.g., "Gold Celdon Girth" instead of "Unknown_Material_Gold Celdon Girth")
|
||||||
|
* Fixed: Slot column shows "-" instead of "Unknown" for items without slot data
|
||||||
|
* Fixed: All 208 items in Larsson's inventory now process successfully (was 186 with 22 SQL type errors)
|
||||||
|
* Added: Type column in inventory search using object_classes enum for accurate item type classification
|
||||||
|
* Note: ItemType data is inconsistent in JSON - using ObjectClass as primary source for Type column
|
||||||
|
|
||||||
|
## Recent Fixes (September 2025)
|
||||||
|
|
||||||
|
### Portal Coordinate Rounding Fix ✅ RESOLVED
|
||||||
|
* **Problem**: Portal insertion failed with duplicate key errors due to coordinate rounding mismatch
|
||||||
|
* **Root Cause**: Code used 2 decimal places (`ROUND(ns::numeric, 2)`) but database constraint used 1 decimal place
|
||||||
|
* **Solution**: Changed all portal coordinate checks to use 1 decimal place to match DB constraint
|
||||||
|
* **Result**: 98% reduction in duplicate key errors (from 600+/min to ~11/min)
|
||||||
|
* **Location**: `main.py` lines ~1989, 1996, 2025, 2047
|
||||||
|
|
||||||
|
### Character Display Issues ✅ RESOLVED
|
||||||
|
* **Problem**: Some characters (e.g., "Crazed n Dazed") not appearing in frontend
|
||||||
|
* **Root Cause**: Database connection pool exhaustion from portal error spam
|
||||||
|
* **Solution**: Fixed portal errors to reduce database load
|
||||||
|
* **Result**: Characters now display correctly after portal fix
|
||||||
|
|
||||||
|
### Docker Container Deployment
|
||||||
|
* **Issue**: Code changes require container rebuild with `--no-cache` flag
|
||||||
|
* **Command**: `docker compose build --no-cache dereth-tracker`
|
||||||
|
* **Reason**: Docker layer caching can prevent updated source code from being copied
|
||||||
|
|
||||||
|
## Current Known Issues
|
||||||
|
|
||||||
|
### Minor Portal Race Conditions
|
||||||
|
* **Status**: ~11 duplicate key errors per minute (down from 600+)
|
||||||
|
* **Cause**: Multiple players discovering same portal simultaneously
|
||||||
|
* **Impact**: Minimal - errors are caught and handled gracefully
|
||||||
|
* **Handling**: Try/catch in code logs as debug messages and updates portal timestamp
|
||||||
|
* **Potential Fix**: PostgreSQL ON CONFLICT DO UPDATE (upsert pattern) would eliminate completely
|
||||||
|
|
||||||
|
### Database Initialization Warnings
|
||||||
|
* **TimescaleDB Hypertable**: `telemetry_events` fails to become hypertable due to primary key constraint
|
||||||
|
* **Impact**: None - table works as regular PostgreSQL table
|
||||||
|
* **Warning**: "cannot create a unique index without the column 'timestamp'"
|
||||||
|
|
||||||
|
### Connection Pool Under Load
|
||||||
|
* **Issue**: Database queries can timeout when connection pool is exhausted
|
||||||
|
* **Symptom**: Characters may not appear during high error load
|
||||||
|
* **Mitigation**: Portal error fix significantly reduced this issue
|
||||||
|
|
||||||
|
## Equipment Suit Builder
|
||||||
|
|
||||||
|
### Status: PRODUCTION READY
|
||||||
|
|
||||||
|
Real-time equipment optimization engine for building optimal character loadouts by searching across multiple characters' inventories (mules). Uses Mag-SuitBuilder constraint satisfaction algorithms.
|
||||||
|
|
||||||
|
**Core Features:**
|
||||||
|
- Multi-character inventory search across 100+ characters, 25,000+ items
|
||||||
|
- Armor set constraints (primary 5-piece + secondary 4-piece set support)
|
||||||
|
- Cantrip/ward spell optimization with bitmap-based overlap detection
|
||||||
|
- Crit damage rating optimization
|
||||||
|
- Locked slots with set/spell preservation across searches
|
||||||
|
- Real-time SSE streaming with progressive phase updates
|
||||||
|
- Suit summary with copy-to-clipboard functionality
|
||||||
|
- Stable deterministic sorting for reproducible results
|
||||||
|
|
||||||
|
**Access:** `/suitbuilder.html`
|
||||||
|
|
||||||
|
**Architecture Details:** See `docs/plans/2026-02-09-suitbuilder-architecture.md`
|
||||||
|
|
||||||
|
### Known Limitations
|
||||||
|
- Slot-aware spell filtering not yet implemented (e.g., underclothes have limited spell pools but system treats all slots equally)
|
||||||
|
- All spells weighted equally (no priority/importance weighting yet)
|
||||||
|
- See architecture doc for future enhancement roadmap
|
||||||
|
|
||||||
|
## Technical Notes for Development
|
||||||
|
|
||||||
|
### Database Performance
|
||||||
|
- Connection pool: 5-20 connections (configured in `db_async.py`)
|
||||||
|
- Under heavy error load, pool exhaustion can cause 2-minute query timeouts
|
||||||
|
- Portal error fix significantly improved database performance
|
||||||
|
|
||||||
|
### Docker Development Workflow
|
||||||
|
1. **Code Changes**: Edit source files locally
|
||||||
|
2. **Rebuild**: `docker compose build --no-cache dereth-tracker` (required for code changes)
|
||||||
|
3. **Deploy**: `docker compose up -d dereth-tracker`
|
||||||
|
4. **Debug**: `docker logs mosswartoverlord-dereth-tracker-1` and `docker logs dereth-db`
|
||||||
|
|
||||||
|
### Frontend Architecture
|
||||||
|
- **Main Map**: `static/index.html` - Real-time player tracking
|
||||||
|
- **Inventory Search**: `static/inventory.html` - Advanced item filtering
|
||||||
|
- **Suitbuilder**: `static/suitbuilder.html` - Equipment optimization interface
|
||||||
|
- **All static files**: Served directly by FastAPI StaticFiles
|
||||||
|
|
||||||
|
### DOM Optimization Status ✅ COMPLETE (September 2025)
|
||||||
|
* **Achievement**: 100% DOM element reuse with zero element creation after initial render
|
||||||
|
* **Performance**: ~5ms render time for 69 players, eliminated 4,140+ elements/minute creation
|
||||||
|
* **Implementation**: Element pooling system with player name mapping for O(1) lookup
|
||||||
|
* **Monitoring**: Color-coded console output (✨ green = optimized, ⚡ yellow = partial, 🔥 red = poor)
|
||||||
|
* **Status**: Production ready - achieving perfect element reuse consistently
|
||||||
|
|
||||||
|
**Current Render Stats**:
|
||||||
|
- ✅ This render: 0 dots created, 69 reused | 0 list items created, 69 reused
|
||||||
|
- ✅ Lifetime: 69 dots created, 800+ reused | 69 list items created, 800+ reused
|
||||||
|
|
||||||
|
**Remaining TODO**:
|
||||||
|
- ❌ Fix CSS Grid layout for player sidebar (deferred per user request)
|
||||||
|
- ❌ Extend optimization to trails and portal rendering
|
||||||
|
- ❌ Add memory usage tracking
|
||||||
|
|
||||||
|
### WebSocket Endpoints
|
||||||
|
- `/ws/position`: Plugin telemetry, inventory, portal, rare events (authenticated)
|
||||||
|
- `/ws/live`: Browser client commands and live updates (unauthenticated)
|
||||||
233
docs/plans/2026-02-09-suitbuilder-architecture.md
Normal file
233
docs/plans/2026-02-09-suitbuilder-architecture.md
Normal file
|
|
@ -0,0 +1,233 @@
|
||||||
|
# Suitbuilder Architecture & Implementation Status
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The Equipment Suit Builder is a real-time optimization engine that finds optimal equipment loadouts for characters by searching across multiple characters' inventories (mules). It uses constraint satisfaction algorithms derived from the original Mag-SuitBuilder (C#) plugin, reimplemented in Python as part of the inventory-service.
|
||||||
|
|
||||||
|
**Status:** Production Ready
|
||||||
|
**Access:** `/suitbuilder.html`
|
||||||
|
**Backend:** `inventory-service/suitbuilder.py`
|
||||||
|
**Frontend:** `static/suitbuilder.js`, `static/suitbuilder.html`, `static/suitbuilder.css`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Achievements (2025-2026)
|
||||||
|
|
||||||
|
- Multi-character search across 100+ characters and 25,000+ items
|
||||||
|
- Mag-SuitBuilder constraint satisfaction algorithms with branch pruning
|
||||||
|
- Real-time SSE streaming with phase updates and progress tracking
|
||||||
|
- Spell bitmap-based overlap detection and scoring
|
||||||
|
- Locked slot constraints with set/spell preservation across searches
|
||||||
|
- Stable deterministic sorting for reproducible results
|
||||||
|
- Comprehensive UI with suit summary and copy-to-clipboard
|
||||||
|
- Individual slot clear buttons for granular control
|
||||||
|
- Full cantrip/ward selection UI (attributes, weapon skills, magic skills, defenses, wards)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
1. **ConstraintSatisfactionSolver** - Main search engine. Performs recursive depth-first search through equipment buckets, applying constraints and scoring at each level.
|
||||||
|
|
||||||
|
2. **ItemBucket** - Slot-based item organization. Groups items by equipment slot, sorts by priority (armor level, ratings, spells), and handles armor reduction variants.
|
||||||
|
|
||||||
|
3. **SpellBitmapIndex** - O(1) spell overlap detection using bitwise operations. Each spell is assigned a unique bit position, enabling instant duplicate detection.
|
||||||
|
|
||||||
|
4. **SuitState** - Mutable state during recursive search. Tracks equipped items, aggregated spell bitmap, set counts, total armor, and ratings.
|
||||||
|
|
||||||
|
5. **SearchConstraints** - User-defined requirements including character selection, set preferences, required spells, locked slots, and scoring weights.
|
||||||
|
|
||||||
|
6. **ItemPreFilter** - Pre-processing step that removes strictly dominated items (items surpassed in all metrics by another item in the same slot).
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
User (suitbuilder.html)
|
||||||
|
|
|
||||||
|
| POST /inv/suitbuilder/search (constraints JSON)
|
||||||
|
|
|
||||||
|
v
|
||||||
|
Main Service (main.py)
|
||||||
|
|
|
||||||
|
| Streaming proxy (dedicated SSE endpoint)
|
||||||
|
|
|
||||||
|
v
|
||||||
|
Inventory Service (inventory-service/main.py)
|
||||||
|
|
|
||||||
|
| Routes to suitbuilder module
|
||||||
|
|
|
||||||
|
v
|
||||||
|
ConstraintSatisfactionSolver
|
||||||
|
|
|
||||||
|
| 1. Load items from inventory DB
|
||||||
|
| 2. Create equipment buckets per slot
|
||||||
|
| 3. Apply armor reduction rules
|
||||||
|
| 4. Sort buckets (smallest first for pruning)
|
||||||
|
| 5. Recursive search with constraint checking
|
||||||
|
| 6. Score and yield suits via SSE stream
|
||||||
|
|
|
||||||
|
v
|
||||||
|
User (real-time suit results appear as found)
|
||||||
|
```
|
||||||
|
|
||||||
|
### SSE Streaming Architecture
|
||||||
|
|
||||||
|
The suitbuilder search is a long-running operation (up to 5 minutes). Results stream in real-time via Server-Sent Events (SSE).
|
||||||
|
|
||||||
|
**Main service** (`main.py`) has a dedicated streaming proxy endpoint at `/inv/suitbuilder/search` that uses `httpx.AsyncClient.stream()` to forward chunks without buffering. The generic `/inv/{path:path}` proxy buffers entire responses, which would cause timeouts for SSE streams.
|
||||||
|
|
||||||
|
**Event types streamed:**
|
||||||
|
- `phase` - Search phase updates (loading, buckets, reducing, sorting, searching)
|
||||||
|
- `suit` - New suit found with items and score
|
||||||
|
- `log` - Verbose debug messages
|
||||||
|
- `stats` - Search statistics (branches explored, suits found, elapsed time)
|
||||||
|
- `complete` - Search finished
|
||||||
|
- `error` - Error occurred
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Spell System Implementation
|
||||||
|
|
||||||
|
### Spell Bitmap Index
|
||||||
|
|
||||||
|
Uses bitwise operations for O(1) spell overlap detection:
|
||||||
|
- Each spell is assigned a unique bit position in an integer bitmap
|
||||||
|
- `spell_bitmap |= item.spell_bitmap` aggregates spells across items in a suit
|
||||||
|
- `new_spells = item_bitmap & needed_bitmap & ~current_bitmap` detects novel spell contributions
|
||||||
|
- Prevents duplicate spell slots across items in strict mode
|
||||||
|
|
||||||
|
### Spell Constraint Flow
|
||||||
|
|
||||||
|
1. **UI Collection**: Checkboxes for legendary cantrips and wards
|
||||||
|
- **Attributes**: Strength, Endurance, Coordination, Quickness, Focus, Willpower
|
||||||
|
- **Weapon Skills**: Finesse, Heavy, Light, Missile, Two-handed
|
||||||
|
- **Magic Skills**: War Magic, Void Magic, Creature, Item, Life Magic
|
||||||
|
- **Defenses**: Magic Resistance, Invulnerability
|
||||||
|
- **Wards**: Flame, Frost, Acid, Storm, Slashing, Piercing, Bludgeoning, Armor
|
||||||
|
|
||||||
|
2. **Backend Processing**:
|
||||||
|
- `needed_spell_bitmap = spell_index.get_bitmap(required_spells)` pre-computes target
|
||||||
|
- During item loading: `item.spell_bitmap = spell_index.get_bitmap(item.spell_names)`
|
||||||
|
- During search: `_can_get_beneficial_spell_from()` rejects items with only duplicate spells
|
||||||
|
- Items with spells that contribute nothing new to the suit are pruned
|
||||||
|
|
||||||
|
3. **Scoring**:
|
||||||
|
- Each fulfilled required spell: +100 points
|
||||||
|
- Duplicate spells prevented by constraint (not scored negatively, just blocked)
|
||||||
|
- Encourages maximum spell diversity across the suit
|
||||||
|
|
||||||
|
### Jewelry Special Handling
|
||||||
|
|
||||||
|
Jewelry (rings, bracelets, necklaces, trinkets) receives special treatment:
|
||||||
|
- Only added to suits if they contribute new required spells
|
||||||
|
- Prevents jewelry spam when no spell constraints are specified
|
||||||
|
- Prioritizes spell contribution over raw stats for jewelry slots
|
||||||
|
- Items with zero beneficial spell contribution are rejected
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Recent Features (February 2026)
|
||||||
|
|
||||||
|
### Locked Slots System
|
||||||
|
|
||||||
|
Allows users to lock specific equipment slots with set and spell constraints. Locked slots are preserved when selecting different suit results.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- Lock populated slots automatically (extracts set + spells from the item)
|
||||||
|
- Lock empty slots manually (configure set + spell requirements via form)
|
||||||
|
- Visual distinction (red border, lock icon)
|
||||||
|
- Constraints sent to backend (locked slots filtered from search buckets)
|
||||||
|
- Preserved when selecting different suits from results
|
||||||
|
|
||||||
|
**Use Cases:**
|
||||||
|
- "I already have this Legendary Empyrean Helm, find suits around it"
|
||||||
|
- "Lock Head slot to Empyrean set with L. Strength + L. Flame Ward"
|
||||||
|
- Iterative refinement: lock good items, re-search for better accessories
|
||||||
|
|
||||||
|
### Suit Summary Panel
|
||||||
|
|
||||||
|
Copy-paste friendly summary below equipment slots showing all items with their source characters.
|
||||||
|
|
||||||
|
**Format:**
|
||||||
|
```
|
||||||
|
Slot: Item Name [Spell1, Spell2] - Character Name
|
||||||
|
```
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- Auto-updates when suit selected or slots cleared
|
||||||
|
- Copy to clipboard button with visual feedback
|
||||||
|
- Monospace font for readability
|
||||||
|
- Shows all equipped items with their source characters
|
||||||
|
|
||||||
|
### Clear Slot Button
|
||||||
|
|
||||||
|
Small "x" button on populated slots for granular control:
|
||||||
|
- Clears individual slots without affecting locked state
|
||||||
|
- Removes item from selected suit state
|
||||||
|
- Updates summary automatically
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Known Limitations
|
||||||
|
|
||||||
|
### Slot-Aware Spell Filtering
|
||||||
|
The system currently treats all slots equally when checking spell availability. In reality, certain cantrips and wards never appear on underclothes (shirt/pants). The search could be optimized by pre-filtering based on realistic spell pools per slot type.
|
||||||
|
|
||||||
|
### Spell Priority Weighting
|
||||||
|
All required spells are weighted equally (+100 points each). There is no mechanism to mark certain spells as "critical" vs "nice to have."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Future Enhancements Roadmap
|
||||||
|
|
||||||
|
### Phase 1: Spell System Improvements
|
||||||
|
|
||||||
|
#### Spell Priority Weighting
|
||||||
|
Allow users to mark certain spells as critical vs optional:
|
||||||
|
- Critical spells: +200 points (must have)
|
||||||
|
- High priority: +150 points
|
||||||
|
- Normal priority: +100 points (current default)
|
||||||
|
- Low priority: +50 points
|
||||||
|
|
||||||
|
#### Slot-Aware Spell Filtering
|
||||||
|
Pre-filter items based on realistic spell pools per slot type:
|
||||||
|
- **Underclothes** (Shirt/Pants): Primarily damage rating, rarely cantrips/wards
|
||||||
|
- **Armor** (Head/Chest/Arms/Legs/Hands/Feet): Full cantrip/ward pool
|
||||||
|
- **Jewelry** (Rings/Bracelets/Neck): Full cantrip/ward pool
|
||||||
|
- **Trinket**: Limited or no spells
|
||||||
|
|
||||||
|
Benefits: Faster search (smaller search space), more realistic suit expectations, better UX.
|
||||||
|
|
||||||
|
### Phase 2: New Builders
|
||||||
|
|
||||||
|
#### Weapon Suit Builder
|
||||||
|
Optimize weapon + shield/offhand combinations:
|
||||||
|
- Weapon skill requirements (Heavy, Light, Finesse, etc.)
|
||||||
|
- Damage type optimization
|
||||||
|
- Special property constraints (Crushing Blow, Armor Rending, etc.)
|
||||||
|
|
||||||
|
#### Accessory-Only Builder
|
||||||
|
Find best jewelry/trinket combinations when armor is already decided:
|
||||||
|
- Smaller search space = faster results
|
||||||
|
- Focus on spell coverage and rating optimization
|
||||||
|
- Use case: "I have my armor set, find best accessories"
|
||||||
|
|
||||||
|
#### Multi-Suit Batch Generation
|
||||||
|
Generate multiple viable suits in one search:
|
||||||
|
- "Show me top 10 suits for different playstyles"
|
||||||
|
- Variation in set combinations
|
||||||
|
- Export/compare multiple options
|
||||||
|
|
||||||
|
### Phase 3: Advanced Features
|
||||||
|
|
||||||
|
#### Saved Searches
|
||||||
|
- Save constraint configurations as named presets
|
||||||
|
- Quick re-run with updated inventory
|
||||||
|
- Presets like "Mage Build", "Tank Build", etc.
|
||||||
|
|
||||||
|
#### Suit Comparison
|
||||||
|
- Side-by-side comparison of 2-3 suits
|
||||||
|
- Highlight differences in items, spells, and ratings
|
||||||
|
- Visual scoring breakdown
|
||||||
Loading…
Add table
Add a link
Reference in a new issue