erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(studio): forward canvas mouse to the previewed panel (interactive preview)

Browse source 
Canvas clicks now reach the panel UiHost so buttons, tabs, and slots
respond to user interaction. Previously only UiRoot.Pick (inspector
selection) received click events; the panel itself was inert.

Key changes:
- StudioInspector.DrawCanvas now returns a CanvasInputEvent struct
  (was nullable click tuple) — carries isHovered, move position,
  leftDown/Up, and scroll delta, all in panel-local pixels.
- Coordinate mapping: panel_local = mouse_screen - GetItemRectMin()
  (1:1, no scale factor). V-flip (uv0.Y=1, uv1.Y=0) makes screen
  top = panel Y=0, so NO extra Y inversion. Documented in comments.
- StudioWindow.OnLoad: removed WireMouse — raw Silk window coords are
  offset by the canvas sub-window position and land in the wrong place.
  WireKeyboard kept (keyboard input needs no spatial remapping).
- StudioWindow.OnRender: forwards OnMouseMove always (hover states),
  plus OnMouseDown/Up/OnScroll in Interact mode. Console.WriteLine
  on each forwarded left-click for live verification.
- Interact/Inspect toggle: checkbox in the Studio toolbar (default
  Interact). Inspect mode restores old click-to-select-element behavior
  while still forwarding OnMouseMove for hover states.
- CanvasCoordMappingTests: 7 pure-math unit tests covering the
  origin, interior points, corner, chrome OOB, and the no-extra-flip
  invariant (no GL required).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Erik 2026-06-25 21:32:00 +02:00
parent eccacc59de
commit 21d8485053
3 changed files with 280 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

161
src/AcDream.App/Studio/StudioInspector.cs
View file

@ -4,49 +4,88 @@ using ImGuiNET;
namespace AcDream.App.Studio;
/// <summary>
/// All canvas mouse events gathered by <see cref="StudioInspector.DrawCanvas"/> in one frame.
/// All coordinates are already mapped to panel-local pixels (origin top-left, same as UiRoot).
/// </summary>
public readonly struct CanvasInputEvent
{
/// <summary>Mouse is currently hovering the canvas image. When false all other fields are 0 / false.</summary>
public readonly bool IsHovered;
/// <summary>Panel-local pixel coordinate of the mouse this frame (valid when <see cref="IsHovered"/>).</summary>
public readonly int MouseX;
/// <summary>Panel-local pixel coordinate of the mouse this frame (valid when <see cref="IsHovered"/>).</summary>
public readonly int MouseY;
/// <summary>Left-button went down this frame.</summary>
public readonly bool LeftDown;
/// <summary>Left-button came up this frame.</summary>
public readonly bool LeftUp;
/// <summary>Mouse-wheel scroll delta (lines, positive = up). Zero when no scroll.</summary>
public readonly int ScrollDelta;
public CanvasInputEvent(bool hovered, int mx, int my, bool ld, bool lu, int scroll)
{
IsHovered = hovered;
MouseX = mx;
MouseY = my;
LeftDown = ld;
LeftUp = lu;
ScrollDelta = scroll;
}
}
/// <summary>
/// Four-pane ImGui IDE for the acdream UI Studio:
/// <list type="bullet">
/// <item><b>Toolbar</b> — panel picker (slug combo) across the top.</item>
/// <item><b>Canvas</b> — shows the panel FBO texture; click-to-inspect returns the
/// panel-local pixel coordinate of the click; draws a green outline over
/// <see cref="Selected"/>.</item>
/// <item><b>Canvas</b> — shows the panel FBO texture; in Interact mode mouse events
/// are forwarded to the panel UiHost (buttons/tabs respond); in Inspect mode a
/// left-click hit-tests and selects the element under the cursor.</item>
/// <item><b>Tree</b> — recursive ImGui tree of the element hierarchy; clicking a node
/// sets <see cref="Selected"/>.</item>
/// <item><b>Properties</b> — shows the <see cref="Selected"/> element's geometry,
/// anchors, and z-order.</item>
/// </list>
///
/// <para>Coordinate mapping for the canvas:
/// The FBO is rendered at the same logical size as the window, so a click at image-local
/// pixel (ix, iy) maps directly to panel coord (ix, iy) — no scale factor needed when
/// the image is drawn 1:1. We draw it 1:1 if it fits; if the Canvas ImGui window is
/// smaller we let ImGui clip it (the click coordinates are still image-local pixels so
/// no scale correction is needed in v1).</para>
/// <para><b>Coordinate mapping for the canvas:</b>
/// The FBO is rendered at the full window size and displayed 1:1 inside the Canvas ImGui
/// sub-window. After <c>ImGui.Image</c> we call <c>ImGui.GetItemRectMin()</c> to get the
/// screen-space top-left of the drawn image (accounting for the sub-window's title bar,
/// padding, and any scrolling). Subtracting that from the raw mouse screen position gives
/// panel-local pixels directly — no additional scale factor is needed because the image is
/// drawn 1:1.</para>
///
/// <para>V-flip: the FBO origin is bottom-left (GL convention), but ImGui images use
/// top-left. We pass uv0=(0,1), uv1=(1,0) to <c>ImGui.Image</c>, which flips V so
/// the panel renders right-side-up in the canvas. Because V is flipped, image-local
/// pixel y = 0 is the TOP of the panel, matching the UiRoot's top-left origin — so
/// click y maps directly without further inversion.</para>
/// <para><b>V-flip — no extra Y inversion needed:</b>
/// The FBO origin is bottom-left (GL convention), so we pass uv0=(0,1), uv1=(1,0) to
/// <c>ImGui.Image</c> to flip V. After this flip, displayed row 0 (top of the image on
/// screen) corresponds to panel Y=0 (the top of the UI panel), matching UiRoot's
/// top-left origin. Therefore the panel-local Y computed above maps directly into UiRoot
/// without further inversion — do NOT flip Y again.</para>
///
/// <para>Layout: <see cref="DrawToolbar"/> / <see cref="DrawCanvas"/> /
/// <see cref="DrawTree"/> / <see cref="DrawProperties"/> each call
/// <c>SetNextWindowPos</c> + <c>SetNextWindowSize</c> with
/// <c>ImGuiCond.FirstUseEver</c> so the panes start in a docked-style arrangement
/// that the user can freely drag from.</para>
/// <para>Layout: the four panes call <c>SetNextWindowPos</c> + <c>SetNextWindowSize</c>
/// with <c>ImGuiCond.FirstUseEver</c> so they start docked but can be freely dragged.</para>
/// </summary>
public sealed class StudioInspector
{
/// <summary>Currently selected element (set by tree-click or canvas-click).</summary>
/// <summary>Currently selected element (set by tree-click or canvas-click in Inspect mode).</summary>
public UiElement? Selected { get; set; }
/// <summary>
/// When true (default) canvas mouse events are forwarded to the panel UiHost so elements
/// respond to clicks. When false a canvas click hit-tests and selects an element in the
/// inspector tree instead. Toggle via the "Interact / Inspect" checkbox in the toolbar.
/// </summary>
public bool InteractMode { get; set; } = true;
// ── Toolbar ───────────────────────────────────────────────────────────────────
/// <summary>
/// Draw the "Studio" toolbar window (top strip) containing a slug combo-box.
/// Returns the newly-selected slug when the user picks a different panel, or
/// null when the selection is unchanged.
/// Draw the "Studio" toolbar window (top strip) containing a slug combo-box and
/// the Interact / Inspect mode toggle. Returns the newly-selected slug when the
/// user picks a different panel, or null when unchanged.
/// <para>The mode toggle sets <see cref="InteractMode"/>: checked = Interact (panel
/// elements respond to clicks), unchecked = Inspect (clicks select elements in the
/// tree).</para>
/// </summary>
/// <param name="slugs">All available panel slugs (from <c>UiDumpModel.ListSlugs</c>).</param>
/// <param name="current">The slug of the panel currently loaded.</param>
@ -74,6 +113,13 @@ public sealed class StudioInspector
ImGui.EndCombo();
}
ImGui.SameLine();
bool interact = InteractMode;
if (ImGui.Checkbox("Interact", ref interact))
InteractMode = interact;
if (ImGui.IsItemHovered())
ImGui.SetTooltip("Interact: canvas clicks reach the panel (buttons/tabs respond).\nUncheck to Inspect: clicks select elements in the tree.");
ImGui.End();
return result;
}
@ -81,20 +127,24 @@ public sealed class StudioInspector
// ── Canvas ────────────────────────────────────────────────────────────────────
/// <summary>
/// Draw the "Canvas" ImGui window containing the panel FBO texture.
/// If the user left-clicks inside the image, returns the panel-local pixel
/// coordinate of the click (origin top-left, matching UiRoot's space); else null.
/// Draw the "Canvas" ImGui window containing the panel FBO texture and return all
/// canvas mouse events for this frame as a <see cref="CanvasInputEvent"/>.
///
/// <para>The texture is drawn with uv0=(0,1) / uv1=(1,0) to flip V, correcting
/// the GL bottom-left FBO origin to the ImGui top-left convention.</para>
/// <para><b>Coordinate mapping:</b> After <c>ImGui.Image</c>, <c>GetItemRectMin()</c>
/// returns the actual screen-space top-left of the drawn image (accounting for the
/// sub-window title bar, padding, and scrolling). Subtracting that from the raw ImGui
/// mouse position gives panel-local pixels directly — no scale factor because the
/// image is drawn 1:1.</para>
///
/// <para>If <see cref="Selected"/> is non-null and has non-zero size, a bright-green
/// 2-pixel outline is drawn over the element's screen-space rect using the window
/// draw list. <see cref="UiElement.ScreenPosition"/> gives the absolute top-left in
/// root/panel space, which maps 1:1 to FBO pixels (the image is drawn 1:1 and both
/// the element tree and the FBO share the same top-left origin).</para>
/// <para><b>V-flip — no extra Y inversion:</b> we pass uv0=(0,1) / uv1=(1,0) so the
/// GL bottom-left origin is flipped to top-left on screen. After the flip, screen
/// row 0 = panel Y 0 (top of the UI), so the computed Y already matches UiRoot's
/// top-left origin — do NOT flip Y again.</para>
///
/// <para>If <see cref="Selected"/> is non-null a bright-green 2-pixel outline is
/// drawn over it using the window draw list.</para>
/// </summary>
public (int x, int y)? DrawCanvas(nint panelTex, int width, int height,
public CanvasInputEvent DrawCanvas(nint panelTex, int width, int height,
int windowX, int windowW, int windowY, int windowH)
{
ImGui.SetNextWindowPos(new Vector2(windowX, windowY), ImGuiCond.FirstUseEver);
@ -109,12 +159,11 @@ public sealed class StudioInspector
var uv1 = new Vector2(1f, 0f);
ImGui.Image(panelTex, imageSize, uv0, uv1);
// ── Selection highlight ───────────────────────────────────────────────
// After ImGui.Image the item rect gives us the screen-space top-left of
// the drawn texture. ScreenPosition is in root/panel space (top-left origin,
// same coordinate system the FBO was drawn in), so offsetting by rectMin
// converts directly to screen space for the draw list.
// rectMin: screen-space top-left of the image AFTER ImGui.Image + any chrome offset.
// This is what lets us translate raw mouse screen coords into panel-local pixels.
var rectMin = ImGui.GetItemRectMin();
// ── Selection highlight ───────────────────────────────────────────────
var el = Selected;
if (el is not null && el.Width > 0f && el.Height > 0f)
{
@ -127,25 +176,39 @@ public sealed class StudioInspector
0f, ImDrawFlags.None, 2f);
}
(int x, int y)? result = null;
// ── Gather canvas mouse events ────────────────────────────────────────
// IsItemHovered is true when the mouse is over the Image item (not just the window).
bool hovered = ImGui.IsItemHovered();
int mx = 0, my = 0;
bool leftDown = false, leftUp = false;
int scroll = 0;
// Detect a left-click on the image.
if (ImGui.IsItemHovered() && ImGui.IsMouseClicked(ImGuiMouseButton.Left))
if (hovered)
{
var mousePos = ImGui.GetMousePos();
// Panel-local pixel = mouse offset from image top-left.
// Because we flipped V (uv0.Y=1, uv1.Y=0), image row 0 IS the top of the panel,
// so y needs no further inversion.
// Panel-local pixel = mouse offset from the image's screen-space top-left.
// Scale is 1:1 (image drawn at full FBO size). Y needs no extra flip — see summary.
int ix = (int)(mousePos.X - rectMin.X);
int iy = (int)(mousePos.Y - rectMin.Y);
// Clamp to image bounds.
// Clamp to image bounds (mouse can be on the image edge pixel).
if (ix >= 0 && ix < width && iy >= 0 && iy < height)
result = (ix, iy);
{
mx = ix;
my = iy;
leftDown = ImGui.IsMouseClicked(ImGuiMouseButton.Left);
leftUp = ImGui.IsMouseReleased(ImGuiMouseButton.Left);
float wheelY = ImGui.GetIO().MouseWheel;
scroll = (int)wheelY; // positive = scroll up
}
else
{
// Mouse is over ImGui chrome (title bar, padding) adjacent to image — not over the panel.
hovered = false;
}
}
ImGui.End();
return result;
return new CanvasInputEvent(hovered, mx, my, leftDown, leftUp, scroll);
}
// ── Tree ──────────────────────────────────────────────────────────────────────

63
src/AcDream.App/Studio/StudioWindow.cs
View file

@ -183,9 +183,10 @@ public sealed class StudioWindow : IDisposable
if (_opts.ScreenshotPath is null)
{
var input = _window!.CreateInput();
// Wire input into UiHost (interactive only).
foreach (var mouse in input.Mice)
_stack.UiHost.WireMouse(mouse);
// Wire KEYBOARD input into UiHost (keyboard coords are not spatial, so no remapping needed).
// Do NOT wire mouse here — raw Silk window coords would be offset by the canvas sub-window's
// position (tree pane width + ImGui chrome) and land in the wrong panel-local location.
// Mouse is forwarded manually from DrawCanvas with correct panel-local mapping below.
foreach (var kb in input.Keyboards)
_stack.UiHost.WireKeyboard(kb);
_imgui = new AcDream.UI.ImGui.ImGuiBootstrapper(gl, _window!, input);
@ -319,21 +320,57 @@ public sealed class StudioWindow : IDisposable
if (pickedSlug is not null)
LoadDumpPanel(pickedSlug);
// 6. Canvas pane — show the FBO texture; detect clicks.
(int x, int y)? click = null;
// 6. Canvas pane — show the FBO texture; gather canvas mouse events.
var canvasEvt = default(CanvasInputEvent);
if (panelTex != 0)
click = _inspector.DrawCanvas(
canvasEvt = _inspector.DrawCanvas(
(nint)panelTex, iw, ih,
canvasX, canvasW, paneY, paneH);
// 7. If the user clicked inside the canvas, hit-test the whole UI tree (UiRoot.Pick honors
// Z-order + modal exclusivity) and select the topmost element. The canvas click coord is
// in the same root-space the FBO was drawn in, so it maps 1:1.
if (click is { } c)
// 7. Forward canvas mouse events to the panel UiHost or the inspector tree.
//
// Coordinate mapping (see StudioInspector.DrawCanvas summary):
// panel-local pixel = raw_mouse - ImGui.GetItemRectMin() (1:1 scale, no Y flip)
// The image is drawn V-flipped (uv0.Y=1, uv1.Y=0) so screen top = panel Y=0.
//
// Interact mode (default): canvas mouse events go directly to UiRoot so elements respond.
// OnMouseMove + OnMouseDown/Up + OnScroll are all forwarded.
// A Console.WriteLine confirms each forwarded left-click for live verification.
//
// Inspect mode: left-click hit-tests and selects the element in the tree (old behavior).
// OnMouseMove is still forwarded so hover/tooltip state in the panel stays live.
if (canvasEvt.IsHovered)
{
var hit = _stack.UiHost.Root.Pick(c.x, c.y);
if (hit is not null)
_inspector.Selected = hit;
int mx = canvasEvt.MouseX;
int my = canvasEvt.MouseY;
var root = _stack.UiHost.Root;
// Always forward mouse-move so hover highlights / tooltips in the panel work.
root.OnMouseMove(mx, my);
if (_inspector.InteractMode)
{
// ── Interact: live panel interaction ──────────────────────────────
if (canvasEvt.LeftDown)
{
Console.WriteLine($"[studio] canvas click → panel ({mx}, {my})");
root.OnMouseDown(UiMouseButton.Left, mx, my);
}
if (canvasEvt.LeftUp)
root.OnMouseUp(UiMouseButton.Left, mx, my);
if (canvasEvt.ScrollDelta != 0)
root.OnScroll(canvasEvt.ScrollDelta);
}
else
{
// ── Inspect: click selects an element in the tree ─────────────────
if (canvasEvt.LeftDown)
{
var hit = root.Pick(mx, my);
if (hit is not null)
_inspector.Selected = hit;
}
}
}
// 8. Element tree pane.