Add targeted click variants and screenshot points #82
- IA
Extend `desktop_click` with single, double, right, middle, and triple clicks on either an observed element or a normalized screenshot point. Coordinates use the snapshot's native capture geometry, preserving exact process/window checks, single-use snapshots, background delivery, and unsafe replay semantics.
This is the click-only portion of #68 and #8. Scrolling and dragging remain separate work; this does not close #68.
Validation: repository type checks and formatting/lint pass, as do the signed desktop build, strict signature verification, Ace diff review, and independent source review. A real signed native fixture using the shared click implementation passed six groups: the existing AX click, a coordinate single click from a resized screenshot, and double/right/middle/triple button/count checks. That broader run later stopped on an uncertain scroll result; no scroll or drag pass is claimed here. A real pi/Anthropic run using this exact focused candidate’s tool schema, host adapter, signed client and GUI then inspected the fixture, selected a screenshot coordinate inside Activate, clicked once, and reinspected. The real button counter increased exactly once while the scroll position and prior board events stayed unchanged. Exact results/screenshots survived closing and reopening its private pi SQLite store with unchanged usage and no repeated input. Worker-crash and hosted end-to-end coverage remain in #8; this check does not claim them.
### Built in Ace
Used Ace's terminal for dependencies, checks, the signed build, Git, and GitHub, and its changes/patch surface for the diff. Codex source tools performed editing and independent review because the external agent/editor integration is tracked in #9 and #22. Native validation was coordinated through Ace's real host adapter and signed desktop bridge against a disposable AppKit fixture. External CUA handled validation-app setup/quit while Ace’s lifecycle tools remain incomplete under #8.
iamnbutlerready-for-reviewlast month on GitHub
iamnbutlermerged e5618c1553belast month on GitHub
iamnbutlerreferenced this in Add native targeted scrolling #70last month on GitHub
iamnbutlerreferenced this in Finish native pointer delivery and validation #68last month on GitHub
iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub
iamnbutlerreferenced this in Activate apps and recover exact windows #74last month on GitHub
iamnbutlerreferenced this in Land desktop inspection/actions and tab API from feat/github-cache #89last month on GitHub