Validate native text insertion on hosted channels and under interruption
iamnbutler opened ·
**Status (Oct 6):** implementation is merged (#81, #67). Remaining: hosted workspace execution of the new insertion route and broader refusal/interruption acceptance below. Clipboard ownership lifecycle is #78; public clipboard tools are #72 (draft #75 was closed as superseded).
Extend the first native-action tools from #63 with contextual text selection, insertion at the caret, and keyboard shortcuts. Tracked by #8; pointer gestures and public clipboard tools have separate PRs.
- [x] Select literal text or position the caret before/after it, using adjacent context to distinguish repeated matches and refusing ambiguous selections. Merged in #81. - [x] Navigation, letters/digits/function keys, and Command/Control/Option/Shift modifiers as complete press/release chords. Merged in #81, including the AXEnabled metadata fix (#66). - [x] Insert Unicode and multiline text at the observed caret/selection without submitting newlines or changing surrounding text. Merged in #67, including the actual Ace WKWebView defect #76 and plain-text line-break fidelity. - Preserve exact process/window/focused-control binding, single-use observations, native coordination, interruption evidence, and completed/refused/unknown outcomes. Mutations remain unsafe to replay; no global-input fallback. - Temporary paste must verify meaningful consumption before restoring prior clipboard contents, preserve newer human copies, and retain ownership while delivery is uncertain (#78). Public clipboard tools are tracked in #72. - Access follows #8 and the [current architecture](https://github.com/githubnext/ace2/blob/main/docs/architecture.md#the-team): the channel owner's agent-access and desktop-tools settings, with no participant grants or per-action approvals.
## Implementation status
**Merged #81:** contextual UTF-16 selection, caret movement, ambiguous/disabled refusal, complete shortcuts, deletion, and consumed-snapshot checks. Real AppKit and local/hosted pi checks exercised the unchanged selection/shortcut paths; persisted results replayed without repeating input. #80 separately repairs application activity/visibility metadata.
**Merged #67:** GUI-owned literal paste landed as `8fb8ccf` from signed head `eff5e49`. Actual Ace composer validation proved contextual Unicode/multiline replacement, exact surrounding text and UTF-16 caret, observed consumption, restored clipboard, released ownership, and unchanged durable history across all chats. Plain-text paste/copy preserves single, blank, and edge line breaks. Two distinct signed clients verified delayed-paste ownership, refusal without input or clipboard change, and a fresh second insertion after read-only resolution. Real pi/Anthropic completed and Stop-during-pending-paste checks preserved exact results/images and usage after channel-worker restart, without repeating input. Stop correctly retained its partial-input warning; the original paste arrived once afterward. Type/lint checks, signed build/signatures, source review, and React Doctor passed.
**Remaining validation:** keep this issue open for the new literal-paste route’s hosted workspace execution and broader refusal/interruption coverage. Earlier hosted checks exercised selection/shortcuts and do not certify the new insertion route. Native-client death, response loss, GUI crash/restart, and separate-channel ownership cases remain in #78. Public clipboard tools are tracked in #72.
Done when the merged selection/shortcut workflow and the merged literal-insertion route pass real native and pi/model checks, including ambiguous/stale/focus refusal, Stop/restart without replay, and hosted workspace execution. Unsupported delivery routes must remain explicit refusals.
iamnbutlerreferenced this in [Meta] Complete native computer use in Ace #8last month on GitHub
iamnbutlerreferenced this in Do not reject editable text areas when AXEnabled is absent #66last month on GitHub
- IA
Implementation and signed build are complete; real native validation found #66. Its fix uses Peekaboo’s knownIsEnabled property so unknown AXEnabled can proceed to native action validation while explicitly disabled controls remain refused. The signed rebuild passes; native/model/recovery checks are continuing. Issue #8 now includes the current validation status.
iamnbutlerreferenced this in Explain native inspection failure when the desktop or target window becomes unavailable #64last month on GitHub
iamnbutlerreferenced this in Add literal desktop insertion with reserved clipboard ownership #67last month on GitHub
iamnbutlerreferenced this in Finish native pointer delivery and validation #68last month on GitHub
iamnbutlerreferenced this in Preserve native preflight refusal evidence after focus changes #69last month on GitHub
iamnbutlerreferenced this in [Meta] Dogfood Ace end to end #5last month on GitHub
iamnbutlerreferenced this in Preserve literal newlines when inserting text into web composers #76last month on GitHub
iamnbutlerreferenced this in Add native text selection and keyboard shortcuts #81last month on GitHub
iamnbutlerreferenced this in fix(app): default the composer to agent mode and keep each channel's mode #88last month on GitHub
iamnbutlerreferenced this in Keep clipboard ownership while a native paste is unresolved #78last month on GitHub
iamnbutlerreferenced this in Avoid macOS Automation prompts when agents drive other apps #103last month on GitHub
iamnbutlerreferenced this in Focus and paste into WebKit composers with native desktop tools #106last month on GitHub
iamnbutlerparent-issue-addedlast month on GitHub
iamnbutlerrenamed Select native text, insert at the caret, and use keyboard shortcuts to Validate native text insertion on hosted channels and under interruptionlast month on GitHub