githubnext/ace2Public

Ace: a collaborative coding environment on the team's own machines

Report moved-window coordinate clicks rejected before dispatch as refused

IA

Real native validation for pixel-snapshot coordinate authority (#8, #68) found that a point click from an unused snapshot, after a verified move of its exact window, returns signed `unknown` even though the native service rejects it before input.

## Observed

Signed Ace-dev build of `codex/pixel-snapshot-authority` at `769438a`, disposable AppKit pointer fixture:

1. Inspect the exact window and keep its unused snapshot. 2. Move the window with `desktop_move` (completed, verified geometry). 3. Call `desktop_click` with a normalized point from that snapshot.

The result has `outcome: unknown` and native outcome `indeterminate` / `may_have_dispatched` / `completion_unknown` / `retry_safety: unsafe`. The message is `Snapshot is stale: Exact-window click identity changed before final dispatch; capture a fresh snapshot.` The fixture recorded no pointer events and no button activation. Fresh inventory showed the same window and process generation at the moved bounds.

This reproduces identically with a **pixels** snapshot and with an **Accessibility** snapshot (separate fresh fixture, one attempt each, never repeated). It is not caused by pixel authority.

## Cause (source)

In Peekaboo 4.8.0 plus Ace's patch stack, `ClickService.executeClick` prepares clicks with `requireCurrentTarget(..., validateExactWindow: false)`, so preparation checks only the process generation. A coordinate click then resolves no Accessibility element: `performActionClick` throws `ActionInputError.unsupported(.missingElement)` before its own target check, and the action-first executor falls back to the synthesis route. The first exact-window check runs in that route, in `performClick`, as `requireCurrentTarget(afterDispatch: false)`. It throws an untyped `PeekabooError.snapshotStale`. The bridge's canonical mutation-failure mapping treats an untyped error from a mutating operation after execution starts as `executionMayHaveStarted`, so it reports `indeterminate` / `may_have_dispatched`.

The route-level check cannot simply be typed as a refusal. A route can follow an earlier Accessibility attempt that may have dispatched: for example, a right-click `AXShowMenu` failure classified as `targetUnavailable` becomes fallback-eligible `unsupported`, and `performClick`'s check then runs after possible input. (The original description attributed the failing check to `performActionClick` before `tryClick`; that is not the path for coordinate clicks.)

## Expected

Keep `unknown` whenever input may have been dispatched. Where the native service proves that rejection happened before any input unit, return a typed pre-dispatch refusal with its evidence and receipt. Do not classify by error text, and do not weaken exact-window checks. #69 made the same kind of narrow typed correction for literal insertion. Verify with a real moved-window click that the receiver stays unchanged and the result is `refused`.

Tracked by #8 / #68; dogfooding #5.

  1. iamnbutlerreferenced this in feat(desktop): authorize coordinate actions from pixel snapshots #156last month on GitHub

  2. iamnbutlerreferenced this in [Meta] Dogfood Ace end to end #5last month on GitHub

  3. iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub

  4. iamnbutlerreferenced this in Finish native pointer delivery and validation #68last month on GitHub

  5. iamnbutlerreferenced this in Preserve native preflight refusal evidence after focus changes #69last month on GitHub

  6. iamnbutlerparent-issue-addedlast month on GitHub

  7. iamnbutlerreferenced this in fix(desktop): report clicks on an already moved window as refused #187last month on GitHub

  8. IA
    Author

    Fix in #187 (`1a09f3c`). I corrected the Cause section above: coordinate clicks resolve no element, so the failing check is in the synthesis route's `performClick`, not in `performActionClick` before `tryClick`. That route check can follow a possibly dispatched AXShowMenu fallback, so it stays conservative. The fix is a single typed exact-window check in the click's `prepare` step, before any route. Real Ace-dev validation with signed receipts: Accessibility and pixels snapshots of an already moved window → `refused` / dispatch `none` with zero receiver input; a same-process positive control produced exactly one effect; a window change after the menu action stays `unknown`. Limitations are listed in the PR.

  9. iamnbutlerclosedlast month on GitHub