erik/acdream
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(studio): forward canvas mouse to the previewed panel (interactive preview)

Browse source 
Canvas clicks now reach the panel UiHost so buttons, tabs, and slots
respond to user interaction. Previously only UiRoot.Pick (inspector
selection) received click events; the panel itself was inert.

Key changes:
- StudioInspector.DrawCanvas now returns a CanvasInputEvent struct
  (was nullable click tuple) — carries isHovered, move position,
  leftDown/Up, and scroll delta, all in panel-local pixels.
- Coordinate mapping: panel_local = mouse_screen - GetItemRectMin()
  (1:1, no scale factor). V-flip (uv0.Y=1, uv1.Y=0) makes screen
  top = panel Y=0, so NO extra Y inversion. Documented in comments.
- StudioWindow.OnLoad: removed WireMouse — raw Silk window coords are
  offset by the canvas sub-window position and land in the wrong place.
  WireKeyboard kept (keyboard input needs no spatial remapping).
- StudioWindow.OnRender: forwards OnMouseMove always (hover states),
  plus OnMouseDown/Up/OnScroll in Interact mode. Console.WriteLine
  on each forwarded left-click for live verification.
- Interact/Inspect toggle: checkbox in the Studio toolbar (default
  Interact). Inspect mode restores old click-to-select-element behavior
  while still forwarding OnMouseMove for hover states.
- CanvasCoordMappingTests: 7 pure-math unit tests covering the
  origin, interior points, corner, chrome OOB, and the no-extra-flip
  invariant (no GL required).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Erik 2026-06-25 21:32:00 +02:00
parent eccacc59de
commit 21d8485053
3 changed files with 280 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

161
src/AcDream.App/Studio/StudioInspector.cs
View file

@ -4,49 +4,88 @@ using ImGuiNET;
namespace AcDream.App.Studio;
/// <summary>
/// All canvas mouse events gathered by <see cref="StudioInspector.DrawCanvas"/> in one frame.
/// All coordinates are already mapped to panel-local pixels (origin top-left, same as UiRoot).
/// </summary>
public readonly struct CanvasInputEvent
{
/// <summary>Mouse is currently hovering the canvas image. When false all other fields are 0 / false.</summary>
public readonly bool IsHovered;
/// <summary>Panel-local pixel coordinate of the mouse this frame (valid when <see cref="IsHovered"/>).</summary>
public readonly int MouseX;
/// <summary>Panel-local pixel coordinate of the mouse this frame (valid when <see cref="IsHovered"/>).</summary>
public readonly int MouseY;
/// <summary>Left-button went down this frame.</summary>
public readonly bool LeftDown;
/// <summary>Left-button came up this frame.</summary>
public readonly bool LeftUp;
/// <summary>Mouse-wheel scroll delta (lines, positive = up). Zero when no scroll.</summary>
public readonly int ScrollDelta;
public CanvasInputEvent(bool hovered, int mx, int my, bool ld, bool lu, int scroll)
{
IsHovered = hovered;
MouseX = mx;
MouseY = my;
LeftDown = ld;
LeftUp = lu;
ScrollDelta = scroll;
}
}
/// <summary>
/// Four-pane ImGui IDE for the acdream UI Studio:
/// <list type="bullet">
/// <item><b>Toolbar</b> — panel picker (slug combo) across the top.</item>
/// <item><b>Canvas</b> — shows the panel FBO texture; click-to-inspect returns the
/// panel-local pixel coordinate of the click; draws a green outline over
/// <see cref="Selected"/>.</item>
/// <item><b>Canvas</b> — shows the panel FBO texture; in Interact mode mouse events
/// are forwarded to the panel UiHost (buttons/tabs respond); in Inspect mode a
/// left-click hit-tests and selects the element under the cursor.</item>
/// <item><b>Tree</b> — recursive ImGui tree of the element hierarchy; clicking a node
/// sets <see cref="Selected"/>.</item>
/// <item><b>Properties</b> — shows the <see cref="Selected"/> element's geometry,
/// anchors, and z-order.</item>
/// </list>
///
/// <para>Coordinate mapping for the canvas:
/// The FBO is rendered at the same logical size as the window, so a click at image-local
/// pixel (ix, iy) maps directly to panel coord (ix, iy) — no scale factor needed when
/// the image is drawn 1:1. We draw it 1:1 if it fits; if the Canvas ImGui window is
/// smaller we let ImGui clip it (the click coordinates are still image-local pixels so
/// no scale correction is needed in v1).</para>
/// <para><b>Coordinate mapping for the canvas:</b>
/// The FBO is rendered at the full window size and displayed 1:1 inside the Canvas ImGui
/// sub-window. After <c>ImGui.Image</c> we call <c>ImGui.GetItemRectMin()</c> to get the
/// screen-space top-left of the drawn image (accounting for the sub-window's title bar,
/// padding, and any scrolling). Subtracting that from the raw mouse screen position gives
/// panel-local pixels directly — no additional scale factor is needed because the image is
/// drawn 1:1.</para>
///
/// <para>V-flip: the FBO origin is bottom-left (GL convention), but ImGui images use
/// top-left. We pass uv0=(0,1), uv1=(1,0) to <c>ImGui.Image</c>, which flips V so
/// the panel renders right-side-up in the canvas. Because V is flipped, image-local
/// pixel y = 0 is the TOP of the panel, matching the UiRoot's top-left origin — so
/// click y maps directly without further inversion.</para>
/// <para><b>V-flip — no extra Y inversion needed:</b>
/// The FBO origin is bottom-left (GL convention), so we pass uv0=(0,1), uv1=(1,0) to
/// <c>ImGui.Image</c> to flip V. After this flip, displayed row 0 (top of the image on
/// screen) corresponds to panel Y=0 (the top of the UI panel), matching UiRoot's
/// top-left origin. Therefore the panel-local Y computed above maps directly into UiRoot
/// without further inversion — do NOT flip Y again.</para>
///
/// <para>Layout: <see cref="DrawToolbar"/> / <see cref="DrawCanvas"/> /
/// <see cref="DrawTree"/> / <see cref="DrawProperties"/> each call
/// <c>SetNextWindowPos</c> + <c>SetNextWindowSize</c> with
/// <c>ImGuiCond.FirstUseEver</c> so the panes start in a docked-style arrangement
/// that the user can freely drag from.</para>
/// <para>Layout: the four panes call <c>SetNextWindowPos</c> + <c>SetNextWindowSize</c>
/// with <c>ImGuiCond.FirstUseEver</c> so they start docked but can be freely dragged.</para>
/// </summary>
public sealed class StudioInspector
{
/// <summary>Currently selected element (set by tree-click or canvas-click).</summary>
/// <summary>Currently selected element (set by tree-click or canvas-click in Inspect mode).</summary>
public UiElement? Selected { get; set; }
/// <summary>
/// When true (default) canvas mouse events are forwarded to the panel UiHost so elements
/// respond to clicks. When false a canvas click hit-tests and selects an element in the
/// inspector tree instead. Toggle via the "Interact / Inspect" checkbox in the toolbar.
/// </summary>
public bool InteractMode { get; set; } = true;
// ── Toolbar ───────────────────────────────────────────────────────────────────
/// <summary>
/// Draw the "Studio" toolbar window (top strip) containing a slug combo-box.
/// Returns the newly-selected slug when the user picks a different panel, or
/// null when the selection is unchanged.
/// Draw the "Studio" toolbar window (top strip) containing a slug combo-box and
/// the Interact / Inspect mode toggle. Returns the newly-selected slug when the
/// user picks a different panel, or null when unchanged.
/// <para>The mode toggle sets <see cref="InteractMode"/>: checked = Interact (panel
/// elements respond to clicks), unchecked = Inspect (clicks select elements in the
/// tree).</para>
/// </summary>
/// <param name="slugs">All available panel slugs (from <c>UiDumpModel.ListSlugs</c>).</param>
/// <param name="current">The slug of the panel currently loaded.</param>
@ -74,6 +113,13 @@ public sealed class StudioInspector
ImGui.EndCombo();
}
ImGui.SameLine();
bool interact = InteractMode;
if (ImGui.Checkbox("Interact", ref interact))
InteractMode = interact;
if (ImGui.IsItemHovered())
ImGui.SetTooltip("Interact: canvas clicks reach the panel (buttons/tabs respond).\nUncheck to Inspect: clicks select elements in the tree.");
ImGui.End();
return result;
}
@ -81,20 +127,24 @@ public sealed class StudioInspector
// ── Canvas ────────────────────────────────────────────────────────────────────
/// <summary>
/// Draw the "Canvas" ImGui window containing the panel FBO texture.
/// If the user left-clicks inside the image, returns the panel-local pixel
/// coordinate of the click (origin top-left, matching UiRoot's space); else null.
/// Draw the "Canvas" ImGui window containing the panel FBO texture and return all
/// canvas mouse events for this frame as a <see cref="CanvasInputEvent"/>.
///
/// <para>The texture is drawn with uv0=(0,1) / uv1=(1,0) to flip V, correcting
/// the GL bottom-left FBO origin to the ImGui top-left convention.</para>
/// <para><b>Coordinate mapping:</b> After <c>ImGui.Image</c>, <c>GetItemRectMin()</c>
/// returns the actual screen-space top-left of the drawn image (accounting for the
/// sub-window title bar, padding, and scrolling). Subtracting that from the raw ImGui
/// mouse position gives panel-local pixels directly — no scale factor because the
/// image is drawn 1:1.</para>
///
/// <para>If <see cref="Selected"/> is non-null and has non-zero size, a bright-green
/// 2-pixel outline is drawn over the element's screen-space rect using the window
/// draw list. <see cref="UiElement.ScreenPosition"/> gives the absolute top-left in
/// root/panel space, which maps 1:1 to FBO pixels (the image is drawn 1:1 and both
/// the element tree and the FBO share the same top-left origin).</para>
/// <para><b>V-flip — no extra Y inversion:</b> we pass uv0=(0,1) / uv1=(1,0) so the
/// GL bottom-left origin is flipped to top-left on screen. After the flip, screen
/// row 0 = panel Y 0 (top of the UI), so the computed Y already matches UiRoot's
/// top-left origin — do NOT flip Y again.</para>
///
/// <para>If <see cref="Selected"/> is non-null a bright-green 2-pixel outline is
/// drawn over it using the window draw list.</para>
/// </summary>
public (int x, int y)? DrawCanvas(nint panelTex, int width, int height,
public CanvasInputEvent DrawCanvas(nint panelTex, int width, int height,
int windowX, int windowW, int windowY, int windowH)
{
ImGui.SetNextWindowPos(new Vector2(windowX, windowY), ImGuiCond.FirstUseEver);
@ -109,12 +159,11 @@ public sealed class StudioInspector
var uv1 = new Vector2(1f, 0f);
ImGui.Image(panelTex, imageSize, uv0, uv1);
// ── Selection highlight ───────────────────────────────────────────────
// After ImGui.Image the item rect gives us the screen-space top-left of
// the drawn texture. ScreenPosition is in root/panel space (top-left origin,
// same coordinate system the FBO was drawn in), so offsetting by rectMin
// converts directly to screen space for the draw list.
// rectMin: screen-space top-left of the image AFTER ImGui.Image + any chrome offset.
// This is what lets us translate raw mouse screen coords into panel-local pixels.
var rectMin = ImGui.GetItemRectMin();
// ── Selection highlight ───────────────────────────────────────────────
var el = Selected;
if (el is not null && el.Width > 0f && el.Height > 0f)
{
@ -127,25 +176,39 @@ public sealed class StudioInspector
0f, ImDrawFlags.None, 2f);
}
(int x, int y)? result = null;
// ── Gather canvas mouse events ────────────────────────────────────────
// IsItemHovered is true when the mouse is over the Image item (not just the window).
bool hovered = ImGui.IsItemHovered();
int mx = 0, my = 0;
bool leftDown = false, leftUp = false;
int scroll = 0;
// Detect a left-click on the image.
if (ImGui.IsItemHovered() && ImGui.IsMouseClicked(ImGuiMouseButton.Left))
if (hovered)
{
var mousePos = ImGui.GetMousePos();
// Panel-local pixel = mouse offset from image top-left.
// Because we flipped V (uv0.Y=1, uv1.Y=0), image row 0 IS the top of the panel,
// so y needs no further inversion.
// Panel-local pixel = mouse offset from the image's screen-space top-left.
// Scale is 1:1 (image drawn at full FBO size). Y needs no extra flip — see summary.
int ix = (int)(mousePos.X - rectMin.X);
int iy = (int)(mousePos.Y - rectMin.Y);
// Clamp to image bounds.
// Clamp to image bounds (mouse can be on the image edge pixel).
if (ix >= 0 && ix < width && iy >= 0 && iy < height)
result = (ix, iy);
{
mx = ix;
my = iy;
leftDown = ImGui.IsMouseClicked(ImGuiMouseButton.Left);
leftUp = ImGui.IsMouseReleased(ImGuiMouseButton.Left);
float wheelY = ImGui.GetIO().MouseWheel;
scroll = (int)wheelY; // positive = scroll up
}
else
{
// Mouse is over ImGui chrome (title bar, padding) adjacent to image — not over the panel.
hovered = false;
}
}
ImGui.End();
return result;
return new CanvasInputEvent(hovered, mx, my, leftDown, leftUp, scroll);
}
// ── Tree ──────────────────────────────────────────────────────────────────────

63
src/AcDream.App/Studio/StudioWindow.cs
View file

@ -183,9 +183,10 @@ public sealed class StudioWindow : IDisposable
if (_opts.ScreenshotPath is null)
{
var input = _window!.CreateInput();
// Wire input into UiHost (interactive only).
foreach (var mouse in input.Mice)
_stack.UiHost.WireMouse(mouse);
// Wire KEYBOARD input into UiHost (keyboard coords are not spatial, so no remapping needed).
// Do NOT wire mouse here — raw Silk window coords would be offset by the canvas sub-window's
// position (tree pane width + ImGui chrome) and land in the wrong panel-local location.
// Mouse is forwarded manually from DrawCanvas with correct panel-local mapping below.
foreach (var kb in input.Keyboards)
_stack.UiHost.WireKeyboard(kb);
_imgui = new AcDream.UI.ImGui.ImGuiBootstrapper(gl, _window!, input);
@ -319,21 +320,57 @@ public sealed class StudioWindow : IDisposable
if (pickedSlug is not null)
LoadDumpPanel(pickedSlug);
// 6. Canvas pane — show the FBO texture; detect clicks.
(int x, int y)? click = null;
// 6. Canvas pane — show the FBO texture; gather canvas mouse events.
var canvasEvt = default(CanvasInputEvent);
if (panelTex != 0)
click = _inspector.DrawCanvas(
canvasEvt = _inspector.DrawCanvas(
(nint)panelTex, iw, ih,
canvasX, canvasW, paneY, paneH);
// 7. If the user clicked inside the canvas, hit-test the whole UI tree (UiRoot.Pick honors
// Z-order + modal exclusivity) and select the topmost element. The canvas click coord is
// in the same root-space the FBO was drawn in, so it maps 1:1.
if (click is { } c)
// 7. Forward canvas mouse events to the panel UiHost or the inspector tree.
//
// Coordinate mapping (see StudioInspector.DrawCanvas summary):
// panel-local pixel = raw_mouse - ImGui.GetItemRectMin() (1:1 scale, no Y flip)
// The image is drawn V-flipped (uv0.Y=1, uv1.Y=0) so screen top = panel Y=0.
//
// Interact mode (default): canvas mouse events go directly to UiRoot so elements respond.
// OnMouseMove + OnMouseDown/Up + OnScroll are all forwarded.
// A Console.WriteLine confirms each forwarded left-click for live verification.
//
// Inspect mode: left-click hit-tests and selects the element in the tree (old behavior).
// OnMouseMove is still forwarded so hover/tooltip state in the panel stays live.
if (canvasEvt.IsHovered)
{
var hit = _stack.UiHost.Root.Pick(c.x, c.y);
if (hit is not null)
_inspector.Selected = hit;
int mx = canvasEvt.MouseX;
int my = canvasEvt.MouseY;
var root = _stack.UiHost.Root;
// Always forward mouse-move so hover highlights / tooltips in the panel work.
root.OnMouseMove(mx, my);
if (_inspector.InteractMode)
{
// ── Interact: live panel interaction ──────────────────────────────
if (canvasEvt.LeftDown)
{
Console.WriteLine($"[studio] canvas click → panel ({mx}, {my})");
root.OnMouseDown(UiMouseButton.Left, mx, my);
}
if (canvasEvt.LeftUp)
root.OnMouseUp(UiMouseButton.Left, mx, my);
if (canvasEvt.ScrollDelta != 0)
root.OnScroll(canvasEvt.ScrollDelta);
}
else
{
// ── Inspect: click selects an element in the tree ─────────────────
if (canvasEvt.LeftDown)
{
var hit = root.Pick(mx, my);
if (hit is not null)
_inspector.Selected = hit;
}
}
}
// 8. Element tree pane.

118
tests/AcDream.App.Tests/Studio/CanvasCoordMappingTests.cs Normal file
View file

@ -0,0 +1,118 @@
using AcDream.App.Studio;
namespace AcDream.App.Tests.Studio;
/// <summary>
/// Pure-math tests for the canvas → panel-local coordinate mapping used by
/// <see cref="StudioInspector.DrawCanvas"/>.
///
/// <para>No GL context required — we're just verifying the formula:
/// panel_local = (raw_mouse_screen) - (image_screen_top_left)</para>
///
/// <para>The image_screen_top_left is what ImGui.GetItemRectMin() returns after
/// ImGui.Image: the sub-window top-left + title-bar height + inner padding + any
/// scrolling. We model it as a constant offset in these tests.</para>
///
/// <para>V-flip: the image is drawn with uv0=(0,1) / uv1=(1,0) so GL's bottom-left
/// origin is flipped to top-left on screen. After the flip, screen Y=0 (top of image)
/// = panel Y=0 (top of the UI), so NO additional Y inversion is applied.</para>
/// </summary>
public class CanvasCoordMappingTests
{
// Simulates the mapping DrawCanvas performs:
// panel pixel = mouse_screen - image_rectMin
// Returns null when the result falls outside [0, width) x [0, height).
private static (int px, int py)? Map(
float mouseScreenX, float mouseScreenY,
float imageOriginX, float imageOriginY,
int panelWidth, int panelHeight)
{
int ix = (int)(mouseScreenX - imageOriginX);
int iy = (int)(mouseScreenY - imageOriginY);
if (ix < 0 || ix >= panelWidth || iy < 0 || iy >= panelHeight)
return null;
return (ix, iy);
}
[Fact]
public void TopLeft_of_image_maps_to_panel_origin()
{
// The canvas image starts at screen (300, 50) (after sub-window chrome).
// A click exactly at the image's screen top-left → panel (0, 0).
var result = Map(mouseScreenX: 300f, mouseScreenY: 50f,
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Equal((0, 0), result);
}
[Fact]
public void Interior_point_maps_correctly()
{
// Image origin at screen (300, 50). Mouse at screen (780, 230).
// Expected panel coord: (780-300, 230-50) = (480, 180).
var result = Map(mouseScreenX: 780f, mouseScreenY: 230f,
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Equal((480, 180), result);
}
[Fact]
public void Bottom_right_corner_maps_to_last_valid_pixel()
{
// Image is 1280×720, origin at screen (300, 50).
// Last pixel in bottom-right is panel (1279, 719) → screen (1579, 769).
var result = Map(mouseScreenX: 1579f, mouseScreenY: 769f,
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Equal((1279, 719), result);
}
[Fact]
public void Mouse_on_ImGui_chrome_above_image_returns_null()
{
// The ImGui window title-bar / padding is above rectMin, i.e. at screenY < 50.
// The mouse there should NOT produce a panel event.
var result = Map(mouseScreenX: 400f, mouseScreenY: 40f, // 10px above image origin
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Null(result);
}
[Fact]
public void Mouse_below_image_returns_null()
{
// screenY = 50 + 720 = 770 → iy = 720 which is >= panelHeight (720).
var result = Map(mouseScreenX: 400f, mouseScreenY: 770f,
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Null(result);
}
[Fact]
public void Y_is_not_inverted_after_vflip()
{
// Confirm the "no extra Y inversion" contract:
// The image is V-flipped in ImGui (uv0.Y=1, uv1.Y=0), so screen top row = panel Y=0.
// A click near the TOP of the image should give a SMALL panel Y, not a large one.
// Image origin at (300, 50). Click at (400, 55) → panel (100, 5). Y is small (near top).
var result = Map(mouseScreenX: 400f, mouseScreenY: 55f,
imageOriginX: 300f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Equal((100, 5), result);
// NOT (100, 715) — which would be the result if Y were incorrectly inverted.
Assert.True(result!.Value.py < 720 / 2, "Y near screen top should map to small panel Y, not near bottom");
}
[Fact]
public void LargeChrome_offset_is_fully_absorbed()
{
// Simulate a canvas sub-window with large chrome: ImGui title (20px) + padding (8px)
// puts the image origin at screenY = menuBar(22) + titleBar(20) + padding(8) = 50.
// Also a wide tree pane puts imageOriginX = 280 + padding.
// A click at screen (400, 110) with origin (290, 50) → panel (110, 60).
var result = Map(mouseScreenX: 400f, mouseScreenY: 110f,
imageOriginX: 290f, imageOriginY: 50f,
panelWidth: 1280, panelHeight: 720);
Assert.Equal((110, 60), result);
}
}