Update suitbuilder documentation to reflect production-ready status
- Replace outdated CLAUDE.md suitbuilder section with concise production-ready summary - Remove incorrect claims about spell system not being implemented - Create detailed architecture document at docs/plans/2026-02-09-suitbuilder-architecture.md - Document spell bitmap system, locked slots, suit summary, and SSE streaming - Add future enhancement roadmap (spell improvements, new builders, advanced features) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
erik
parent
e0265e261c
commit
850cd62d7b
2 changed files with 373 additions and 0 deletions
140
CLAUDE.md
Normal file
140
CLAUDE.md
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Dereth Tracker is a real-time telemetry service for game world tracking. It's a FastAPI-based WebSocket and HTTP API service that ingests player position/stats data via plugins and provides live map visualization through a web interface.
|
||||
|
||||
## Key Components
|
||||
|
||||
### Main Service (main.py)
|
||||
- WebSocket endpoint `/ws/position` receives telemetry and inventory events
|
||||
- Routes inventory events to inventory service via HTTP
|
||||
- Handles real-time player tracking and map updates
|
||||
|
||||
### Inventory Service (inventory-service/main.py)
|
||||
- Separate FastAPI service for inventory management
|
||||
- Processes inventory JSON into normalized PostgreSQL tables
|
||||
- Provides search API with advanced filtering and sorting
|
||||
- Uses comprehensive enum database for translating game IDs to readable names
|
||||
|
||||
### Database Architecture
|
||||
- **Telemetry DB**: TimescaleDB for time-series player tracking data
|
||||
- **Inventory DB**: PostgreSQL with normalized schema for equipment data
|
||||
- `items`: Core item properties
|
||||
- `item_combat_stats`: Armor level, damage bonuses
|
||||
- `item_enhancements`: Material, item sets, tinkering
|
||||
- `item_spells`: Spell names and categories
|
||||
- `item_raw_data`: Original JSON for complex queries
|
||||
|
||||
## Memories and Known Bugs
|
||||
|
||||
* Fixed: Material names now properly display (e.g., "Gold Celdon Girth" instead of "Unknown_Material_Gold Celdon Girth")
|
||||
* Fixed: Slot column shows "-" instead of "Unknown" for items without slot data
|
||||
* Fixed: All 208 items in Larsson's inventory now process successfully (was 186 with 22 SQL type errors)
|
||||
* Added: Type column in inventory search using object_classes enum for accurate item type classification
|
||||
* Note: ItemType data is inconsistent in JSON - using ObjectClass as primary source for Type column
|
||||
|
||||
## Recent Fixes (September 2025)
|
||||
|
||||
### Portal Coordinate Rounding Fix ✅ RESOLVED
|
||||
* **Problem**: Portal insertion failed with duplicate key errors due to coordinate rounding mismatch
|
||||
* **Root Cause**: Code used 2 decimal places (`ROUND(ns::numeric, 2)`) but database constraint used 1 decimal place
|
||||
* **Solution**: Changed all portal coordinate checks to use 1 decimal place to match DB constraint
|
||||
* **Result**: 98% reduction in duplicate key errors (from 600+/min to ~11/min)
|
||||
* **Location**: `main.py` lines ~1989, 1996, 2025, 2047
|
||||
|
||||
### Character Display Issues ✅ RESOLVED
|
||||
* **Problem**: Some characters (e.g., "Crazed n Dazed") not appearing in frontend
|
||||
* **Root Cause**: Database connection pool exhaustion from portal error spam
|
||||
* **Solution**: Fixed portal errors to reduce database load
|
||||
* **Result**: Characters now display correctly after portal fix
|
||||
|
||||
### Docker Container Deployment
|
||||
* **Issue**: Code changes require container rebuild with `--no-cache` flag
|
||||
* **Command**: `docker compose build --no-cache dereth-tracker`
|
||||
* **Reason**: Docker layer caching can prevent updated source code from being copied
|
||||
|
||||
## Current Known Issues
|
||||
|
||||
### Minor Portal Race Conditions
|
||||
* **Status**: ~11 duplicate key errors per minute (down from 600+)
|
||||
* **Cause**: Multiple players discovering same portal simultaneously
|
||||
* **Impact**: Minimal - errors are caught and handled gracefully
|
||||
* **Handling**: Try/catch in code logs as debug messages and updates portal timestamp
|
||||
* **Potential Fix**: PostgreSQL ON CONFLICT DO UPDATE (upsert pattern) would eliminate completely
|
||||
|
||||
### Database Initialization Warnings
|
||||
* **TimescaleDB Hypertable**: `telemetry_events` fails to become hypertable due to primary key constraint
|
||||
* **Impact**: None - table works as regular PostgreSQL table
|
||||
* **Warning**: "cannot create a unique index without the column 'timestamp'"
|
||||
|
||||
### Connection Pool Under Load
|
||||
* **Issue**: Database queries can timeout when connection pool is exhausted
|
||||
* **Symptom**: Characters may not appear during high error load
|
||||
* **Mitigation**: Portal error fix significantly reduced this issue
|
||||
|
||||
## Equipment Suit Builder
|
||||
|
||||
### Status: PRODUCTION READY
|
||||
|
||||
Real-time equipment optimization engine for building optimal character loadouts by searching across multiple characters' inventories (mules). Uses Mag-SuitBuilder constraint satisfaction algorithms.
|
||||
|
||||
**Core Features:**
|
||||
- Multi-character inventory search across 100+ characters, 25,000+ items
|
||||
- Armor set constraints (primary 5-piece + secondary 4-piece set support)
|
||||
- Cantrip/ward spell optimization with bitmap-based overlap detection
|
||||
- Crit damage rating optimization
|
||||
- Locked slots with set/spell preservation across searches
|
||||
- Real-time SSE streaming with progressive phase updates
|
||||
- Suit summary with copy-to-clipboard functionality
|
||||
- Stable deterministic sorting for reproducible results
|
||||
|
||||
**Access:** `/suitbuilder.html`
|
||||
|
||||
**Architecture Details:** See `docs/plans/2026-02-09-suitbuilder-architecture.md`
|
||||
|
||||
### Known Limitations
|
||||
- Slot-aware spell filtering not yet implemented (e.g., underclothes have limited spell pools but system treats all slots equally)
|
||||
- All spells weighted equally (no priority/importance weighting yet)
|
||||
- See architecture doc for future enhancement roadmap
|
||||
|
||||
## Technical Notes for Development
|
||||
|
||||
### Database Performance
|
||||
- Connection pool: 5-20 connections (configured in `db_async.py`)
|
||||
- Under heavy error load, pool exhaustion can cause 2-minute query timeouts
|
||||
- Portal error fix significantly improved database performance
|
||||
|
||||
### Docker Development Workflow
|
||||
1. **Code Changes**: Edit source files locally
|
||||
2. **Rebuild**: `docker compose build --no-cache dereth-tracker` (required for code changes)
|
||||
3. **Deploy**: `docker compose up -d dereth-tracker`
|
||||
4. **Debug**: `docker logs mosswartoverlord-dereth-tracker-1` and `docker logs dereth-db`
|
||||
|
||||
### Frontend Architecture
|
||||
- **Main Map**: `static/index.html` - Real-time player tracking
|
||||
- **Inventory Search**: `static/inventory.html` - Advanced item filtering
|
||||
- **Suitbuilder**: `static/suitbuilder.html` - Equipment optimization interface
|
||||
- **All static files**: Served directly by FastAPI StaticFiles
|
||||
|
||||
### DOM Optimization Status ✅ COMPLETE (September 2025)
|
||||
* **Achievement**: 100% DOM element reuse with zero element creation after initial render
|
||||
* **Performance**: ~5ms render time for 69 players, eliminated 4,140+ elements/minute creation
|
||||
* **Implementation**: Element pooling system with player name mapping for O(1) lookup
|
||||
* **Monitoring**: Color-coded console output (✨ green = optimized, ⚡ yellow = partial, 🔥 red = poor)
|
||||
* **Status**: Production ready - achieving perfect element reuse consistently
|
||||
|
||||
**Current Render Stats**:
|
||||
- ✅ This render: 0 dots created, 69 reused | 0 list items created, 69 reused
|
||||
- ✅ Lifetime: 69 dots created, 800+ reused | 69 list items created, 800+ reused
|
||||
|
||||
**Remaining TODO**:
|
||||
- ❌ Fix CSS Grid layout for player sidebar (deferred per user request)
|
||||
- ❌ Extend optimization to trails and portal rendering
|
||||
- ❌ Add memory usage tracking
|
||||
|
||||
### WebSocket Endpoints
|
||||
- `/ws/position`: Plugin telemetry, inventory, portal, rare events (authenticated)
|
||||
- `/ws/live`: Browser client commands and live updates (unauthenticated)
|
||||
233
docs/plans/2026-02-09-suitbuilder-architecture.md
Normal file
233
docs/plans/2026-02-09-suitbuilder-architecture.md
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
# Suitbuilder Architecture & Implementation Status
|
||||
|
||||
## Overview
|
||||
|
||||
The Equipment Suit Builder is a real-time optimization engine that finds optimal equipment loadouts for characters by searching across multiple characters' inventories (mules). It uses constraint satisfaction algorithms derived from the original Mag-SuitBuilder (C#) plugin, reimplemented in Python as part of the inventory-service.
|
||||
|
||||
**Status:** Production Ready
|
||||
**Access:** `/suitbuilder.html`
|
||||
**Backend:** `inventory-service/suitbuilder.py`
|
||||
**Frontend:** `static/suitbuilder.js`, `static/suitbuilder.html`, `static/suitbuilder.css`
|
||||
|
||||
---
|
||||
|
||||
## Key Achievements (2025-2026)
|
||||
|
||||
- Multi-character search across 100+ characters and 25,000+ items
|
||||
- Mag-SuitBuilder constraint satisfaction algorithms with branch pruning
|
||||
- Real-time SSE streaming with phase updates and progress tracking
|
||||
- Spell bitmap-based overlap detection and scoring
|
||||
- Locked slot constraints with set/spell preservation across searches
|
||||
- Stable deterministic sorting for reproducible results
|
||||
- Comprehensive UI with suit summary and copy-to-clipboard
|
||||
- Individual slot clear buttons for granular control
|
||||
- Full cantrip/ward selection UI (attributes, weapon skills, magic skills, defenses, wards)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Core Components
|
||||
|
||||
1. **ConstraintSatisfactionSolver** - Main search engine. Performs recursive depth-first search through equipment buckets, applying constraints and scoring at each level.
|
||||
|
||||
2. **ItemBucket** - Slot-based item organization. Groups items by equipment slot, sorts by priority (armor level, ratings, spells), and handles armor reduction variants.
|
||||
|
||||
3. **SpellBitmapIndex** - O(1) spell overlap detection using bitwise operations. Each spell is assigned a unique bit position, enabling instant duplicate detection.
|
||||
|
||||
4. **SuitState** - Mutable state during recursive search. Tracks equipped items, aggregated spell bitmap, set counts, total armor, and ratings.
|
||||
|
||||
5. **SearchConstraints** - User-defined requirements including character selection, set preferences, required spells, locked slots, and scoring weights.
|
||||
|
||||
6. **ItemPreFilter** - Pre-processing step that removes strictly dominated items (items surpassed in all metrics by another item in the same slot).
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
User (suitbuilder.html)
|
||||
|
|
||||
| POST /inv/suitbuilder/search (constraints JSON)
|
||||
|
|
||||
v
|
||||
Main Service (main.py)
|
||||
|
|
||||
| Streaming proxy (dedicated SSE endpoint)
|
||||
|
|
||||
v
|
||||
Inventory Service (inventory-service/main.py)
|
||||
|
|
||||
| Routes to suitbuilder module
|
||||
|
|
||||
v
|
||||
ConstraintSatisfactionSolver
|
||||
|
|
||||
| 1. Load items from inventory DB
|
||||
| 2. Create equipment buckets per slot
|
||||
| 3. Apply armor reduction rules
|
||||
| 4. Sort buckets (smallest first for pruning)
|
||||
| 5. Recursive search with constraint checking
|
||||
| 6. Score and yield suits via SSE stream
|
||||
|
|
||||
v
|
||||
User (real-time suit results appear as found)
|
||||
```
|
||||
|
||||
### SSE Streaming Architecture
|
||||
|
||||
The suitbuilder search is a long-running operation (up to 5 minutes). Results stream in real-time via Server-Sent Events (SSE).
|
||||
|
||||
**Main service** (`main.py`) has a dedicated streaming proxy endpoint at `/inv/suitbuilder/search` that uses `httpx.AsyncClient.stream()` to forward chunks without buffering. The generic `/inv/{path:path}` proxy buffers entire responses, which would cause timeouts for SSE streams.
|
||||
|
||||
**Event types streamed:**
|
||||
- `phase` - Search phase updates (loading, buckets, reducing, sorting, searching)
|
||||
- `suit` - New suit found with items and score
|
||||
- `log` - Verbose debug messages
|
||||
- `stats` - Search statistics (branches explored, suits found, elapsed time)
|
||||
- `complete` - Search finished
|
||||
- `error` - Error occurred
|
||||
|
||||
---
|
||||
|
||||
## Spell System Implementation
|
||||
|
||||
### Spell Bitmap Index
|
||||
|
||||
Uses bitwise operations for O(1) spell overlap detection:
|
||||
- Each spell is assigned a unique bit position in an integer bitmap
|
||||
- `spell_bitmap |= item.spell_bitmap` aggregates spells across items in a suit
|
||||
- `new_spells = item_bitmap & needed_bitmap & ~current_bitmap` detects novel spell contributions
|
||||
- Prevents duplicate spell slots across items in strict mode
|
||||
|
||||
### Spell Constraint Flow
|
||||
|
||||
1. **UI Collection**: Checkboxes for legendary cantrips and wards
|
||||
- **Attributes**: Strength, Endurance, Coordination, Quickness, Focus, Willpower
|
||||
- **Weapon Skills**: Finesse, Heavy, Light, Missile, Two-handed
|
||||
- **Magic Skills**: War Magic, Void Magic, Creature, Item, Life Magic
|
||||
- **Defenses**: Magic Resistance, Invulnerability
|
||||
- **Wards**: Flame, Frost, Acid, Storm, Slashing, Piercing, Bludgeoning, Armor
|
||||
|
||||
2. **Backend Processing**:
|
||||
- `needed_spell_bitmap = spell_index.get_bitmap(required_spells)` pre-computes target
|
||||
- During item loading: `item.spell_bitmap = spell_index.get_bitmap(item.spell_names)`
|
||||
- During search: `_can_get_beneficial_spell_from()` rejects items with only duplicate spells
|
||||
- Items with spells that contribute nothing new to the suit are pruned
|
||||
|
||||
3. **Scoring**:
|
||||
- Each fulfilled required spell: +100 points
|
||||
- Duplicate spells prevented by constraint (not scored negatively, just blocked)
|
||||
- Encourages maximum spell diversity across the suit
|
||||
|
||||
### Jewelry Special Handling
|
||||
|
||||
Jewelry (rings, bracelets, necklaces, trinkets) receives special treatment:
|
||||
- Only added to suits if they contribute new required spells
|
||||
- Prevents jewelry spam when no spell constraints are specified
|
||||
- Prioritizes spell contribution over raw stats for jewelry slots
|
||||
- Items with zero beneficial spell contribution are rejected
|
||||
|
||||
---
|
||||
|
||||
## Recent Features (February 2026)
|
||||
|
||||
### Locked Slots System
|
||||
|
||||
Allows users to lock specific equipment slots with set and spell constraints. Locked slots are preserved when selecting different suit results.
|
||||
|
||||
**Features:**
|
||||
- Lock populated slots automatically (extracts set + spells from the item)
|
||||
- Lock empty slots manually (configure set + spell requirements via form)
|
||||
- Visual distinction (red border, lock icon)
|
||||
- Constraints sent to backend (locked slots filtered from search buckets)
|
||||
- Preserved when selecting different suits from results
|
||||
|
||||
**Use Cases:**
|
||||
- "I already have this Legendary Empyrean Helm, find suits around it"
|
||||
- "Lock Head slot to Empyrean set with L. Strength + L. Flame Ward"
|
||||
- Iterative refinement: lock good items, re-search for better accessories
|
||||
|
||||
### Suit Summary Panel
|
||||
|
||||
Copy-paste friendly summary below equipment slots showing all items with their source characters.
|
||||
|
||||
**Format:**
|
||||
```
|
||||
Slot: Item Name [Spell1, Spell2] - Character Name
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Auto-updates when suit selected or slots cleared
|
||||
- Copy to clipboard button with visual feedback
|
||||
- Monospace font for readability
|
||||
- Shows all equipped items with their source characters
|
||||
|
||||
### Clear Slot Button
|
||||
|
||||
Small "x" button on populated slots for granular control:
|
||||
- Clears individual slots without affecting locked state
|
||||
- Removes item from selected suit state
|
||||
- Updates summary automatically
|
||||
|
||||
---
|
||||
|
||||
## Known Limitations
|
||||
|
||||
### Slot-Aware Spell Filtering
|
||||
The system currently treats all slots equally when checking spell availability. In reality, certain cantrips and wards never appear on underclothes (shirt/pants). The search could be optimized by pre-filtering based on realistic spell pools per slot type.
|
||||
|
||||
### Spell Priority Weighting
|
||||
All required spells are weighted equally (+100 points each). There is no mechanism to mark certain spells as "critical" vs "nice to have."
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements Roadmap
|
||||
|
||||
### Phase 1: Spell System Improvements
|
||||
|
||||
#### Spell Priority Weighting
|
||||
Allow users to mark certain spells as critical vs optional:
|
||||
- Critical spells: +200 points (must have)
|
||||
- High priority: +150 points
|
||||
- Normal priority: +100 points (current default)
|
||||
- Low priority: +50 points
|
||||
|
||||
#### Slot-Aware Spell Filtering
|
||||
Pre-filter items based on realistic spell pools per slot type:
|
||||
- **Underclothes** (Shirt/Pants): Primarily damage rating, rarely cantrips/wards
|
||||
- **Armor** (Head/Chest/Arms/Legs/Hands/Feet): Full cantrip/ward pool
|
||||
- **Jewelry** (Rings/Bracelets/Neck): Full cantrip/ward pool
|
||||
- **Trinket**: Limited or no spells
|
||||
|
||||
Benefits: Faster search (smaller search space), more realistic suit expectations, better UX.
|
||||
|
||||
### Phase 2: New Builders
|
||||
|
||||
#### Weapon Suit Builder
|
||||
Optimize weapon + shield/offhand combinations:
|
||||
- Weapon skill requirements (Heavy, Light, Finesse, etc.)
|
||||
- Damage type optimization
|
||||
- Special property constraints (Crushing Blow, Armor Rending, etc.)
|
||||
|
||||
#### Accessory-Only Builder
|
||||
Find best jewelry/trinket combinations when armor is already decided:
|
||||
- Smaller search space = faster results
|
||||
- Focus on spell coverage and rating optimization
|
||||
- Use case: "I have my armor set, find best accessories"
|
||||
|
||||
#### Multi-Suit Batch Generation
|
||||
Generate multiple viable suits in one search:
|
||||
- "Show me top 10 suits for different playstyles"
|
||||
- Variation in set combinations
|
||||
- Export/compare multiple options
|
||||
|
||||
### Phase 3: Advanced Features
|
||||
|
||||
#### Saved Searches
|
||||
- Save constraint configurations as named presets
|
||||
- Quick re-run with updated inventory
|
||||
- Presets like "Mage Build", "Tank Build", etc.
|
||||
|
||||
#### Suit Comparison
|
||||
- Side-by-side comparison of 2-3 suits
|
||||
- Highlight differences in items, spells, and ratings
|
||||
- Visual scoring breakdown
|
||||
Loading…
Add table
Add a link
Reference in a new issue