githubnext/ace2Public

Ace: a collaborative coding environment on the team's own machines

fix(desktop): report clicks on an already moved window as refused #187

Mergediamnbutlermerged 1 commit intomainfromrefs/pull/187/head
  1. IA

    If an exact window has already moved or resized since its snapshot was taken, a click now reports `refused` instead of `unknown`, because nothing was sent. A window change detected after an input route has started still reports `unknown`.

    Closes #155. Related: #8, #68.

    ### Why only the prepare check

    Click preparation checked only the process generation. A coordinate click resolves no element, so the action-first executor falls back to synthesis. The first exact-window check then ran inside that route, in `performClick`, and threw an untyped stale error. The bridge mapped that to `may_have_dispatched`.

    The route check itself can't become a refusal. A route can follow an earlier Accessibility attempt that may have sent input; for example, a failed right-click `AXShowMenu` becomes fallback-eligible. Instead, `peekaboo-stale-click.patch` adds one exact-window check to the click's `prepare` step. `prepare` runs once, before any strategy route, and the fallback never reruns it. A mismatch there throws a typed `preDispatchRefusal` with `target_unavailable`, `SNAPSHOT_STALE`, a refresh hint, and the captured window target.

    Unchanged: every route and post-dispatch check, the long-press exemption, process-generation checks, and existing receipt layers. The signed outer receipt stays targetless. There is no new API or store.

    ### Validation

    The real run used a signed Ace-dev build of `1a09f3c` on its own profile, driven by a real Anthropic channel. Exported signed receipts and the loaded dylib confirm the patched build handled every action.

    - **Accessibility and pixels snapshots:** inspect, verified move, then one click with the old snapshot. Result: `refused`, dispatch `none`, safe to retry, `SNAPSHOT_STALE`, with the captured window identity. The fixture recorded no input, effect, or activation. A fresh inspect and click in the same process produced exactly one effect. - **Late menu:** right-click on the observed `Real Menu` element. The fixture entered `AXShowMenu`, moved its window, and opened a real menu. Result: `unknown`, retry unsafe, "final target identity validation failed". - **Reopen:** the channel worker exited and a different worker reopened the channel. Tool-call IDs, results, and usage were unchanged, with no new action receipts or fixture events.

    `bun run ci` and `bun types` pass. The 15-patch stack applies to pinned Peekaboo 4.8.0.

    Limitations: - The ambiguous-return fallback was not reproduced. macOS accepted an `AXShowMenu` handler that returned false, so that run only shows the menu action was entered and reported as sent, with unsafe retry. - On that first run, the existing post-dispatch check ran before the window server reported the move. Its result was "completed, unverified" rather than `unknown`. - The person using the machine caused some activations outside the click intervals.

    ### Built in Ace

    Implementation, build, real validation, and this PR were done in this Ace channel, with validation through an Ace-dev channel. The root session and the read-only source and evidence reviews ran in Codex, which Ace can't yet run (#9). Nothing else was used outside Ace.

  2. iamnbutlerreferenced this in Report moved-window coordinate clicks rejected before dispatch as refused #155last month on GitHub

  3. iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub

  4. iamnbutlermerged 4d503d312cf2last month on GitHub

  5. iamnbutlerclosedlast month on GitHub