acdream/docs/plans/2026-04-24-ui-framework.md

UI framework plan

Date: 2026-04-24 (design), shipped 2026-04-25 Status: Phase D.2a and the D.2b retained gameplay UI have shipped. ImGui remains the ACDREAM_DEVTOOLS=1 developer stack. Retail gameplay UI is the independent UiHost/UiRoot retained tree under AcDream.App/UI, built from production LayoutDesc/DAT assets. The stable cross-stack seam is game state, ViewModels, and commands — not an IPanelRenderer backend swap. Owner: lead engineer (erik) + Claude

Captures the UI strategy agreed via discussion on 2026-04-24. Documents the choices AND the alternatives considered so future sessions can re-evaluate with the same context.

2026-04-25 pivot: Hexa.NET.ImGui → ImGui.NET

The original choice (documented below) was Hexa.NET.ImGui + Hexa.NET.ImGui.Backends.OpenGL3. It did not survive first-light integration:

• First launch with Hexa's backend crashed with 0xC0000005 inside Hexa.NET.ImGui.Backends.OpenGL3.ImGuiImplOpenGL3.InitNative.
• Root cause: Hexa's native OpenGL3 backend does its own GL function resolution, looking up symbols via GLFW or SDL. Silk.NET uses neither, so the resolved function pointers were null and the native code dereferenced them on init.
• Hexa's Silk.NET examples rely on GLFW being co-loaded (its default on Hexa's own scenes) — not applicable here.

Mitigation path was already written into this doc (§"What we give up": "switching to ImGui.NET later is a one-morning operation if Hexa misbehaves") and taken:

• Packages swapped → ImGui.NET 1.91.6.1 + Silk.NET.OpenGL.Extensions.ImGui 2.23.0.
• Silk.NET.OpenGL.Extensions.ImGui.ImGuiController handles the whole integration (GL backend init against the Silk.NET GL binding, keyboard + mouse IO event subscription). No hand-written input bridge needed.
• ImGuiBootstrapper is a ~10-line IDisposable wrapping the ImGuiController instance. ImGuiPanelRenderer wraps ImGuiNET.ImGui.*.
• Boundary discipline preserved — panels never import ImGuiNET directly; they only use IPanelRenderer. The backend swap is invisible above the abstraction layer, as designed.

Sections below from §"Choice: Hexa.NET.ImGui" onward are kept as the historical design reasoning. They remain useful if we ever re-evaluate native AOT / upstream-tracking tradeoffs.

Goal

acdream needs a playable game UI: chat, vitals HUD, inventory, character panel, skills, spellbook, fellowship, allegiance, trade, options, map, quest log, tooltips — and a first-class plugin API so plugin authors can ship their own panels.

Current strategy: two coexisting stacks, shared state contracts

┌─────────────────────────────────────────┐
│  Developer UI ─ ImGui + IPanelRenderer  │
│  Gameplay UI  ─ UiRoot retained widgets │
├─────────────────────────────────────────┤
│  ViewModels + Commands  (per panel)     │  ← stable contracts
├─────────────────────────────────────────┤
│  Game state + events + net (existing)   │  ← unchanged
└─────────────────────────────────────────┘

• Developer stack: AcDream.UI.Abstractions panels render through IPanelRenderer on ImGui. This stack is permanent for diagnostics, packet inspection, settings development, and other devtools.
• Gameplay stack: UiHost owns a retained UiRoot tree. LayoutImporter builds retail windows from LayoutDesc/DAT assets and focused gm*UI-style controllers bind runtime values and actions.
• Shared seam: both stacks consume the same session state, ViewModels, and command/event services. A panel is ported to retained UI by binding those shared models to its DAT-authored tree, not by implementing IPanelRenderer a second time.

Choice: Hexa.NET.ImGui for the short-term backend

Decision: Hexa.NET.ImGui + its bundled Hexa.NET.ImGui.Backends.OpenGL3.

Why Hexa over ImGui.NET + Silk.NET.OpenGL.Extensions.ImGui

• Auto-generated from cimgui, tracks upstream ImGui closely — docking, viewports, tables current within days of release.
• Native-AOT first-class — single-file publish is painless; aligns with where we want acdream to land long-term for distribution.
• Per-RID native-lib bundling is cleaner — no separate cimgui.dll side-loading to manage.
• Ships its own Silk.NET-compatible OpenGL3 backend; the Silk.NET official extension is unnecessary.

What we give up

• Silk.NET.OpenGL.Extensions.ImGui is battle-tested with hundreds of Silk.NET sample projects. Hexa's backend is newer, slightly higher risk of edge-case bugs.
• Mitigation: keep integration tight (~50 lines) so switching to ImGui.NET later is a one-morning operation if Hexa misbehaves.

Why ImGui at all (vs. going straight to custom)

• Get game logic validated end-to-end in weeks, not months.
• ImGui stays forever as the devtools layer (ACDREAM_DEVTOOLS=1): packet trace inspector, state dump, dat browser. Having a working ImGui integration is a permanent asset even after game UI moves off it.
• Lets us design the plugin API and ViewModel contracts against a real running panel rather than designing in the abstract.

The three layers in detail

Layer 1 — Game state (unchanged)

Already exists: IGameState, IEvents (plugin-host interfaces), WorldSession, PlayerWeenie, Inventory, SpellBook, LightManager, the live-session wire code. The UI reads from these; we add nothing.

Layer 2 — ViewModels + Commands

New module: src/AcDream.UI.Abstractions/. Backend-agnostic.

ViewModels — per-panel data contracts. Example:

public sealed record VitalsVM(
    int   HpCurrent, int HpMax,
    int   StamCurrent, int StamMax,
    int   ManaCurrent, int ManaMax,
    float HpRegenRate,
    bool  Stunned,
    bool  LowHpWarning);

public sealed record InventoryVM(
    IReadOnlyList<InventoryItemVM> Items,
    int                            BurdenCurrent,
    int                            BurdenCapacity);

public sealed record ChatVM(
    IReadOnlyList<ChatLineVM> Recent,
    int                       UnreadCount,
    ChatChannel               ActiveInputChannel);

Built from IGameState each frame (cheap record allocation). Observable via IEvents subscription for panels that want push updates instead of pull.

Commands — user actions going back.

public sealed record UseItemCmd(uint ItemGuid);
public sealed record SendChatCmd(ChatChannel Channel, string Text);
public sealed record CastSpellCmd(uint SpellId, uint? TargetGuid);
public sealed record DragItemCmd(uint ItemGuid, uint DestContainerGuid, int Slot);
public sealed record EquipItemCmd(uint ItemGuid, EquipSlot Slot);

Dispatched to an ICommandBus that routes to the appropriate subsystem (WorldSession.SendSelect, ChatService.Send, etc.).

Layer 3a — ImGui developer backend

New module: src/AcDream.UI.ImGui/. References AcDream.UI.Abstractions.

• Thin adapter: each panel implements an IPanel interface — Draw(VM) method, emits commands on interaction.
• Panels registered into an IPanelHost which the backend iterates per frame.
• Keyboard / mouse / focus handled by ImGui natively.

This layer remains the permanent devtools surface. It is not the production retail gameplay renderer.

Layer 3b — retained retail gameplay UI

src/AcDream.App/UI/ contains the GL-free widget tree, LayoutDesc importer, window runtime, and panel controllers. Rendering dependencies enter through small sprite/font/viewport resolver seams. Controllers consume the same ViewModels and command services used by the ImGui panels, while binding behavior to existing retail element ids instead of procedurally redrawing the panel through IPanelRenderer.

Plugin UI API

The shipped plugin-facing gameplay UI contract is the additive BCL-only AcDream.Plugin.Abstractions.IUiRegistry.AddPanel: a plugin provides a stable window id/title/icon descriptor, KSML-style markup, and a binding object; the host builds it into the retained UiRoot tree. The API-v1 AddMarkupPanel member remains source/binary compatible and is enriched into the same first-class window route by the scoped host. IPanel/ IPanelRenderer remains a historical first-party developer-panel contract and is intentionally not referenced by Plugin.Abstractions.

This makes plugin gameplay panels presentation-assembly independent while allowing them to share the retained input, window, and DAT-sprite runtime. Registrations made before the graphical host exists are buffered. The host assigns plugin:{pluginId}:{windowId}, registers every panel with the common window manager, persists its geometry/visibility, and exposes it through the shared plugin sidepanel. Hiding/minimizing a panel does not dispose or pause the plugin. No-window hosts retain the plugin session but expose the no-op UI capability.

The retained markup vocabulary includes panels, nested groups, labels, buttons, meters, tabs, lamp-style toggles, scalar sliders, editable fields, dropdown menus, scrollable lists, and (Slice B, docs/plans/2026-09-06-plugin-shelf-and-dat-icons.md) DAT icons — a standalone <icon> element plus icon-bearing extensions of <button> and <list> — resolved from a Decal/VirindiViewService-compatible icon-id grammar (AcDream.Plugin.Abstractions.PluginIcons.Normalize) against raw RenderSurface DIDs, retail's composited spell icon, or a live object's composited item icon. Controls bind to BCL-visible properties/actions on the plugin binding object; visible controls must correspond to real behavior, never placeholders that report success. Full grammar and binding rules: docs/plugin-ui-markup.md. The Slice A movable/collapsible plugin shelf (plugin-shelf, drag grip, Shift+Ctrl+F1 hide/show) is documented there too.

The following was the original pre-D.2b proposal and remains historical context, not the shipped plugin contract:

public interface IPanel
{
    string Id { get; }
    string Title { get; }
    void   Draw(in PanelContext ctx);
}

public readonly ref struct PanelContext
{
    public readonly IGameState    State;
    public readonly ICommandBus   Commands;
    public readonly IPanelRenderer Renderer;  // drawing primitives
    // (widget calls flow through Renderer so the panel never references
    //  ImGuiNET / Hexa / our custom widget namespaces directly)
}

IPanelRenderer exposes a retail-UI-friendly primitive set: Panel, Label, Button, TextField, ScrollView, ListView, Icon, Tab, ProgressBar, DragSource, DropTarget. ImGui implementation wraps ImGui calls; custom implementation uses our retained-mode toolkit.

Key discipline: no panel references Hexa.NET.ImGui directly. If a panel needs a feature the abstraction doesn't expose, add it to IPanelRenderer, don't reach through.

Implementation order

Sprint 1 — Infrastructure + first visible panel

1. AcDream.UI.Abstractions: IPanel, IPanelHost, IPanelRenderer, ICommandBus, base ViewModels (start with VitalsVM only).
2. AcDream.UI.ImGui: Hexa.NET.ImGui wired, ImGuiPanelRenderer implementation of IPanelRenderer (Label, Button, Panel, ProgressBar is enough for vitals).
3. GameWindow: host the IPanelHost; render on top of scene when ACDREAM_DEVTOOLS=1.
4. VitalsPanel — first real panel. Reads HP/stam/mana from IGameState, renders three progress bars.

Success criteria: launch the client, see three coloured bars in the top-left that actually reflect the character's current vitals as they walk around / take damage / regen.

Sprint 2 — Interaction panels

• ChatPanel — reads ChatVM, emits SendChatCmd. ICommandBus routes to WorldSession.
• InventoryPanel — reads InventoryVM, click-to-select, double-click to equip, drag target for future move.
• CharacterPanel — attributes, skills, XP.

Sprint 3 — Plugin API hardening (superseded shape)

• Document the IPanel contract.
• The shipped route is IUiRegistry.AddPanel (with AddMarkupPanel as the compatible legacy entry), not plugin-owned IPanel implementations.
• Confirm plugins can subscribe to game events and expose retained markup bindings without referencing App or ImGui assemblies.

Sprint 4+ — More panels

Spellbook, allegiance, fellowship, trade, map, quest log, options. Continue to expand InventoryPanel with drag-drop, split, appraise.

D.2b — retained retail-look gameplay UI (shipped, expanding panel by panel)

AcDream.App/UI imports LayoutDesc trees and draws retail DAT assets through the retained UiRoot runtime. ImGui remains available for devtools; shared ViewModels and commands prevent duplicate game-state logic.

Non-goals for this first pass

• Not going to theme ImGui to look retail. Waste of effort when we'll swap the backend. Devtools aesthetic is fine.
• Not byte-porting Keystone internals that are unavailable. Observable widget behavior is recovered from the named client call sites, DAT properties, and live retail evidence.
• Not hand-authoring first-party gameplay layouts where retail LayoutDesc data exists. KSML-style markup remains the plugin/extension layout surface.

Alternatives considered

Option	Pros	Cons	Why not picked
ImGui.NET + Silk.NET.OpenGL.Extensions.ImGui	Official Silk.NET path, battle-tested	Lags upstream ImGui, AOT story has sharp edges	Hexa tracks upstream faster, cleaner AOT
Myra	Retained-mode, less-debug-y look	Needs a custom Silk.NET backend (~300 LOC), slower iteration	ImGui is faster to first pixel; aesthetics will move to custom anyway
Avalonia	Mature, XAML designer, great devtools	Hostile to Silk.NET render loop, huge dep	Integration cost too high, aesthetics wrong
NoesisGUI	Slick, XAML-like, production-quality	Commercial license, big dep	Premature optimization
RmlUi	HTML/CSS mental model	Bindings immature, own render backend needed	Too much glue
Pure custom on Silk.NET from day one	Full control, retail look immediately	Months of work before first visible panel	Can't validate game logic fast enough

Risks + mitigations

• Risk: IPanelRenderer grows to leak ImGui-isms. Mitigation: code review every addition; if a feature only exists in ImGui and the retail toolkit can't express it, don't add it.

• Risk: ImGui and retained controllers grow separate game-state truth. Mitigation: one session model/ViewModel and one command path per subsystem; each surface is only a projection.

• Risk: Plugin markup relies on App-only widget details. Mitigation: keep IUiRegistry BCL-only, resolve bindings by contract, and use a smoke plugin as the retained-runtime canary.

• Risk: Hexa.NET.ImGui stops being maintained. Mitigation: integration is small (<100 LOC), switching to ImGui.NET is a one-morning operation.

Open questions (defer to implementation)

• Where does input focus live — ImGui captures keyboard by default when a text field is active, does our game-side input system need to check "did ImGui want this event"? (Yes, standard pattern. Wire io.WantCaptureKeyboard gate.)
• Do devtools panels ship in release builds? (Yes, gated on env var, cost is negligible when disabled.)
• Modal dialogs? Drag-drop? Fleshed out in Sprint 2 when we have inventory actually working.