279 lines
8.8 KiB
Markdown
279 lines
8.8 KiB
Markdown
# Dereth Tracker
|
||
|
||
Dereth Tracker is a real-time telemetry service for the world of Dereth. It collects player data, stores it in a SQLite database, and provides a live map interface along with a sample data generator for testing.
|
||
|
||
## Table of Contents
|
||
- [Overview](#overview)
|
||
- [Features](#features)
|
||
- [Requirements](#requirements)
|
||
- [Installation](#installation)
|
||
- [Configuration](#configuration)
|
||
- [Usage](#usage)
|
||
- [API Reference](#api-reference)
|
||
- [Frontend](#frontend)
|
||
- [Database Schema](#database-schema)
|
||
- [Contributing](#contributing)
|
||
|
||
## Overview
|
||
|
||
This project provides:
|
||
- A FastAPI backend with endpoints for receiving and querying telemetry data.
|
||
- SQLite-based storage for snapshots and live state.
|
||
- A live, interactive map using static HTML, CSS, and JavaScript.
|
||
- A sample data generator script (`generate_data.py`) for simulating telemetry snapshots.
|
||
|
||
## Features
|
||
|
||
- **WebSocket /ws/position**: Stream telemetry snapshots (protected by a shared secret).
|
||
- **GET /live**: Fetch active players seen in the last 30 seconds.
|
||
- **GET /history**: Retrieve historical telemetry data with optional time filtering.
|
||
- **GET /debug**: Health check endpoint.
|
||
- **Live Map**: Interactive map interface with panning, zooming, and sorting.
|
||
- **Sample Data Generator**: `generate_data.py` sends telemetry snapshots over WebSocket for testing.
|
||
|
||
## Requirements
|
||
|
||
- Python 3.9 or newer (only if running without Docker)
|
||
- pip (only if running without Docker)
|
||
- Docker & Docker Compose (recommended)
|
||
|
||
Python packages (if using local virtualenv):
|
||
|
||
- fastapi
|
||
- uvicorn
|
||
- pydantic
|
||
- databases
|
||
- asyncpg
|
||
- sqlalchemy
|
||
- websockets # required for sample data generator
|
||
|
||
## Installation
|
||
|
||
1. Clone the repository:
|
||
```bash
|
||
git clone https://github.com/yourusername/dereth-tracker.git
|
||
cd dereth-tracker
|
||
```
|
||
2. Create and activate a virtual environment:
|
||
```bash
|
||
python3 -m venv venv
|
||
source venv/bin/activate # Windows: venv\Scripts\activate
|
||
```
|
||
3. Install dependencies:
|
||
```bash
|
||
pip install fastapi uvicorn pydantic websockets
|
||
```
|
||
|
||
## Configuration
|
||
|
||
- Update the `SHARED_SECRET` in `main.py` to match your plugin (default: `"your_shared_secret"`).
|
||
- The SQLite database file `dereth.db` is created in the project root. To change the path, edit `DB_FILE` in `db.py`.
|
||
- To limit the maximum database size, set the environment variable `DB_MAX_SIZE_MB` (default: 2048 MB).
|
||
|
||
## Usage
|
||
|
||
Start the server using Uvicorn:
|
||
|
||
```bash
|
||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||
```
|
||
|
||
# Grafana Dashboard UI
|
||
```nginx
|
||
location /grafana/ {
|
||
proxy_pass http://127.0.0.1:3000/;
|
||
proxy_http_version 1.1;
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Real-IP $remote_addr;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
# WebSocket support (for live panels)
|
||
proxy_set_header Upgrade $http_upgrade;
|
||
proxy_set_header Connection "upgrade";
|
||
proxy_cache_bypass $http_upgrade;
|
||
}
|
||
```
|
||
|
||
## NGINX Proxy Configuration
|
||
|
||
If you cannot reassign the existing `/live` and `/trails` routes, you can namespace this service under `/api` (or any other prefix) and configure NGINX accordingly. Be sure to forward WebSocket upgrade headers so that `/ws/live` and `/ws/position` continue to work. Example:
|
||
```nginx
|
||
location /api/ {
|
||
proxy_pass http://127.0.0.1:8765/;
|
||
proxy_http_version 1.1;
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Real-IP $remote_addr;
|
||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
# WebSocket support
|
||
proxy_set_header Upgrade $http_upgrade;
|
||
proxy_set_header Connection "upgrade";
|
||
proxy_cache_bypass $http_upgrade;
|
||
}
|
||
```
|
||
Then the browser client (static/script.js) will fetch `/api/live/` and `/api/trails/` to reach this new server.
|
||
|
||
- Live Map: `http://localhost:8000/` (or `http://<your-domain>/api/` if behind a prefix)
|
||
- Grafana UI: `http://localhost:3000/grafana/` (or `http://<your-domain>/grafana/` if proxied under that path)
|
||
|
||
### Frontend Configuration
|
||
|
||
- In `static/script.js`, the constant `API_BASE` controls where live/trails data and WebSocket `/ws/live` are fetched. By default:
|
||
```js
|
||
const API_BASE = '/api';
|
||
```
|
||
Update `API_BASE` if you mount the service under a different path or serve it at root.
|
||
|
||
### Debugging WebSockets
|
||
|
||
- Server logs now print every incoming WebSocket frame in `main.py`:
|
||
- `[WS-PLUGIN RX] <client>: <raw-payload>` for plugin messages on `/ws/position`
|
||
- `[WS-LIVE RX] <client>: <parsed-json>` for browser messages on `/ws/live`
|
||
- Use these logs to verify messages and troubleshoot handshake failures.
|
||
|
||
### Styling Adjustments
|
||
|
||
- Chat input bar is fixed at the bottom of the chat window (`.chat-form { position:absolute; bottom:0; }`).
|
||
- Input text and placeholder are white for readability (`.chat-input, .chat-input::placeholder { color:#fff; }`).
|
||
- Incoming chat messages forced white via `.chat-messages div { color:#fff !important; }`.
|
||
|
||
## API Reference
|
||
|
||
### WebSocket /ws/position
|
||
Stream telemetry snapshots over a WebSocket connection. Provide your shared secret either as a query parameter or WebSocket header:
|
||
|
||
```
|
||
ws://<host>:<port>/ws/position?secret=<shared_secret>
|
||
```
|
||
or
|
||
```
|
||
X-Plugin-Secret: <shared_secret>
|
||
```
|
||
|
||
After connecting, send JSON messages matching the `TelemetrySnapshot` schema. For example:
|
||
|
||
```json
|
||
{
|
||
"type": "telemetry",
|
||
"character_name": "Dunking Rares",
|
||
"char_tag": "moss",
|
||
"session_id": "dunk-20250422-xyz",
|
||
"timestamp": "2025-04-22T13:45:00Z",
|
||
"ew": 123.4,
|
||
"ns": 567.8,
|
||
"z": 10.2,
|
||
"kills": 42,
|
||
"deaths": 1,
|
||
"prismatic_taper_count": 17,
|
||
"vt_state": "Combat",
|
||
"kills_per_hour": "N/A",
|
||
"onlinetime": "00:05:00"
|
||
}
|
||
```
|
||
|
||
Each message above is sent as its own JSON object over the WebSocket (one frame per event). When you want to report a rare spawn, send a standalone `rare` event instead of embedding rare counts in telemetry. For example:
|
||
|
||
```json
|
||
{
|
||
"type": "rare",
|
||
"timestamp": "2025-04-22T13:48:00Z",
|
||
"character_name": "MyCharacter",
|
||
"name": "Golden Gryphon",
|
||
"ew": 150.5,
|
||
"ns": 350.7,
|
||
"z": 5.0,
|
||
"additional_info": "first sighting of the day"
|
||
}
|
||
```
|
||
|
||
### Chat messages
|
||
You can also send chat envelopes over the same WebSocket to display messages in the browser. Fields:
|
||
- `type`: must be "chat"
|
||
- `character_name`: target player name
|
||
- `text`: message content
|
||
- `color` (optional): CSS color string (e.g. "#ff8800"); if sent as an integer (0xRRGGBB), it will be converted to hex.
|
||
|
||
Example chat payload:
|
||
```json
|
||
{
|
||
"type": "chat",
|
||
"character_name": "MyCharacter",
|
||
"text": "Hello world!",
|
||
"color": "#88f"
|
||
}
|
||
```
|
||
|
||
## Event Payload Formats
|
||
|
||
For a complete reference of JSON payloads accepted by the backend (over `/ws/position`), see the file `EVENT_FORMATS.json` in the project root. It contains example schemas for:
|
||
- **Telemetry events** (`type`: "telemetry")
|
||
- **Spawn events** (`type`: "spawn")
|
||
- **Chat events** (`type`: "chat")
|
||
- **Rare events** (`type`: "rare")
|
||
|
||
Notes on payload changes:
|
||
- Spawn events no longer require the `z` coordinate; if omitted, the server defaults it to 0.0.
|
||
Coordinates (`ew`, `ns`, `z`) may be sent as JSON numbers or strings; the backend will coerce them to floats.
|
||
- Telemetry events have removed the `latency_ms` field; please omit it from your payloads.
|
||
|
||
Each entry shows all required and optional fields, their types, and example values.
|
||
|
||
### GET /live
|
||
Returns active players seen within the last 30 seconds:
|
||
|
||
```json
|
||
{
|
||
"players": [ { ... } ]
|
||
}
|
||
```
|
||
|
||
### GET /history
|
||
Retrieve historical snapshots with optional `from` and `to` ISO8601 timestamps:
|
||
|
||
```
|
||
GET /history?from=2025-04-22T12:00:00Z&to=2025-04-22T13:00:00Z
|
||
```
|
||
|
||
Response:
|
||
|
||
```json
|
||
{
|
||
"data": [ { ... } ]
|
||
}
|
||
```
|
||
|
||
## Frontend
|
||
|
||
- **Live Map**: `static/index.html` – Real-time player positions on a map.
|
||
|
||
## Database Schema
|
||
|
||
- **telemetry_log**: Stored history of snapshots.
|
||
- **live_state**: Current snapshot per character (upserted).
|
||
|
||
## Contributing
|
||
|
||
Contributions are welcome! Feel free to open issues or submit pull requests.
|
||
|
||
## Roadmap & TODO
|
||
For detailed tasks, migration steps, and future enhancements, see [TODO.md](TODO.md).
|
||
|
||
### Local Development Database
|
||
This project will migrate from SQLite to PostgreSQL/TimescaleDB. You can configure local development using Docker Compose or connect to an external instance:
|
||
|
||
1. PostgreSQL/TimescaleDB via Docker Compose (recommended):
|
||
- Pros:
|
||
- Reproducible, isolated environment out-of-the-box
|
||
- No need to install Postgres locally
|
||
- Aligns development with production setups
|
||
- Cons:
|
||
- Additional resource usage (memory, CPU)
|
||
- Slightly more complex Docker configuration
|
||
|
||
2. External PostgreSQL instance:
|
||
- Pros:
|
||
- Leverages existing infrastructure
|
||
- No Docker overhead
|
||
- Cons:
|
||
- Requires manual setup and Timescale extension
|
||
- Less portable for new contributors
|