fix(desktop): resolve containing windows for pointer input #131
- IA
Coordinate clicks on visible WebKit controls could be reported as occluded because the Accessibility leaf did not directly expose its containing window. Resolve the native AXWindow link through Peekaboo’s existing owning-window resolver before checking the pinned window. Missing or different window identities still fail; process, bounds, and final dispatch checks remain in place.
Part of #8 and #106. This fixes containing-window resolution for pointer input; it does not establish coordinate focus or paste in Ace’s composer.
Validation on signed head `c718378`: types, lint, GitHub CI, patch application, independent source review, and strict app signatures pass. A real external WKWebView button reproduced the old failure on signed `9374d8`: one point click reported occlusion and its callback counter stayed at zero. With this change, the same fixture and observed normalized point produced one background Accessibility action and exactly one button callback. Fresh inspections confirmed the target’s PID/generation/window and unchanged editor text, input/paste counters, and native button counter. No input was retried. Existing completed-action reporting remains delivery-accepted; the actual callback supplies independent effect evidence.
Both native runtimes and fixtures are stopped, with exact PID and socket checks recorded. The fixture cleanup report initially treated Bun’s null exit code on signal termination as still running; independent process checks confirmed it had exited. First inspection still needed one later read to expose the WebKit subtree (#64). Multiple-display and new real-model runs were not added for this resolver-only fix.
### Built in Ace
Nate explicitly allowed direct tools for this #8 pass. Codex shell/file tools and review agents implemented and validated the fix through Ace’s signed native bridge; GitHub CLI opened and updated this PR. The observed capability gap is #106, under #8/#5; external harness support remains #9. No installed app, OS permission, private clipboard content, or existing channel data was changed.
iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub
iamnbutlerreferenced this in Focus and paste into WebKit composers with native desktop tools #106last month on GitHub
iamnbutlerready-for-reviewlast month on GitHub
iamnbutlermerged 39a17adbb955last month on GitHub
iamnbutlerreferenced this in feat(desktop): add exact-target normal application quit #132last month on GitHub