Native button click can take effect but time out waiting for completion
iamnbutler opened ·
While dogfooding the first native-action slice of #8, a real Accessibility click can change Ace's UI and then fail to return its native result. The host times out after 30 seconds and correctly reports an uncertain outcome.
Observed in a signed development build with embedded Peekaboo 4.8.0 and all required permissions granted:
1. Inspect Ace's exact window and use the returned snapshot and literal control ID. 2. Click **Rename** in the channel rename dialog. The dialog closes and the channel name is persisted, but the native client times out. 3. The same occurs when clicking **Toggle details sidebar**, whose control remains present.
Field replacement and the exact-window basic-key path complete normally. This is not a reason to retry the click: its effect already happened.
Investigation points to the embedded native Accessibility press path. Peekaboo performs the press synchronously on its main actor and also reads properties of the target after pressing it. Those reads are especially problematic when the action removes its own control. We need to verify the precise blocking point and fix native completion without weakening application/window/control binding or declaring an uncertain action successful.
Acceptance:
- Ordinary Ace button clicks return promptly with their real native outcome and a fresh observation when available. - Controls that disappear after activation do not stall native completion. - Stop/disconnect still cancel pending work; desktop coordination remains held until any in-flight native operation actually finishes. - Lost or uncertain responses remain unsafe to replay, with real-runtime coverage.
Blocks reliable button-click dogfooding for #8.
iamnbutlerreferenced this in feat(desktop): add targeted native actions #63last month on GitHub
- IA
The fix in draft PR #63 is now verified against the real signed Ace app: clicking Rename completed promptly, closed the dialog, and persisted the channel name. A real pi/Anthropic run also completed and verified the same button workflow.
The pinned Peekaboo patch runs semantic AXPress off MainActor, retains native operation ownership until the call returns, and treats ambiguous native failures as uncertain delivery rather than falling back to a second click. Delivered-input Stop and restart checks each dispatched once and preserved uncertainty without replay.
PR #63 remains draft for a separate final hosted-path validation blocked by desktop visibility. This issue remains open until the fix lands.
iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub
- IA
Fixed in merged PR #63. Real signed Ace button clicks and the pi workflow now complete promptly; delivered-input Stop/restart checks preserve uncertainty without replay.
iamnbutlerreferenced this in [Meta] Dogfood Ace end to end #5last month on GitHub
iamnbutlerreferenced this in Land desktop inspection/actions and tab API from feat/github-cache #89last month on GitHub